Abstract
Objective To obtain information from participants in the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial regarding their perception of the retention materials employed by the screening centers. Also, to determine the viability of using email or the internet as a data collection tool with an older population.
Design Three of ten PLCO screening centers queried participants towards the end of the study (2010) as to their opinions of the various retention materials and whether they would have been willing to use electronic communication for study activities, had the option been available.
Setting The questionnaires were administered by mail, and responses were returned to the originating screening center.
Participants The participants in this study consisted of all the active participants at three PLCO screening centers: the University of Colorado Anschutz Medical Campus, the University of Utah, and Henry Ford Health System.
Methods A short, self-administered questionnaire was mailed to all active participants at three PLCO centers (n=41,482). This was a one-time mailing with no follow-up, as the responses were designed to be anonymous in order to obtain the most honest responses.
Results The response rate was 62%. Of respondents, 97% reported their PLCO experience was good or excellent. Nearly 50% of respondents indicated that receipt of an annual newsletter made them more likely to participate; newsletter features they reported as most important were those that conveyed information on cancer, study findings, and how their data were being used. Results did not support study coordinators’ suppositions that receipt of a token gift or birthday card by participants was important for retention. Fewer than 30% of respondents indicated that they would have been unwilling to use a secure website to complete study forms.
Conclusion These data indicate the importance of querying participants rather than relying on impressions of study staff, and also indicate that the internet will be a viable means of data collection in future prevention studies that include older Americans.
The ability of clinical trials to rigorously examine research questions depends on many factors. Careful adherence to clinical aspects of the protocol is of the utmost importance, yet a trial cannot be successful unless participants actually participate and continue to do so throughout the trial. They must adhere to trial regimens and must provide information, typically through interviews or questionnaires at pre-specified intervals, so that trial outcomes can be tracked. Lack of participation can result in meaningful decreases in statistical power; all clinical trials must account for that possibility when calculating necessary sample size, both at the beginning of the trial and throughout its course.
Participants comply for many different reasons. Some comply because they are receiving an experimental treatment for an otherwise thought-to-be incurable illness; others comply because they receive medical care that they otherwise would be unable to afford. But many participants, especially those in prevention trials of relatively rare diseases, comply even though their chance of direct benefit is quite small, and the impact of their altruistic efforts cannot be immediately or easily measured. In such instances, reasons for compliance, both initial and continued, can be enigmatic. Researchers must carefully consider strategies that will keep participants engaged in the trial, willing to adhere to study interventions, and willing to provide information when queried. This is of particular importance in long-term prevention trials, where it can take many years to accrue the necessary number of events.
The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial was a large, long-term, multi-phasic study of cancer screening. Throughout the trial, investigators and study coordinators spent much time and effort on development and implementation of retention strategies. While the experience of other large, long-term cancer primary and secondary prevention trials would have been of value, none had published on participant reactions to retention materials prior to the end of PLCO, and to the best of our knowledge, still have not.
Near the end of the trial, three of the ten screening centers chose to query participants as to their opinions of the various retention materials, including whether they would have been willing to use electronic communication for study activities, had the option been available. In this manuscript, we report on the findings of that survey.
Methods
The PLCO Cancer Screening Trial has been described in detail elsewhere.1 In brief, this multi-phasic, randomized, controlled trial of cancer screening began in 1992, enrolled participants through mid-2001, and screened through 2006. Participants were followed until study withdrawal, death, 13 years of follow-up, or December 31, 2009 (whichever occurred first). Final results were published in 2011 and 2012.2–5 Nearly 155,000 participants, aged 55 years to 74 years at entry, were enrolled at one of ten screening centers nationwide and were randomized to either an intervention or control arm. Participants in the intervention arm received invitations to receive a prostate (men only), lung, colorectal, and ovarian (women only) cancer screening regimen, and participants in the control arm were advised to receive their usual medical care. There were six annual rounds of prostate cancer screening, three (never smokers) or four (ever smokers) annual rounds of lung cancer screening, two rounds (spaced either 3 or 5 years apart) of colorectal cancer screening, and six annual rounds of ovarian cancer screening. Follow-up was accomplished through mailed, self-administered questionnaires or, in the instance of those not returning questionnaires, by telephone interview. Medical records were collected for those participants with positive screens, those reporting prostate, lung, colorectal, or ovarian cancer diagnoses, and those chosen for death review. Death certificates were collected to verify each reported death. Nearly 96% of the 1.7 million expected questionnaires were completed. A very small percent (3.5%) of participants withdrew, were lost to follow-up, or developed medical conditions that precluded continued study participation.
While there was no study-wide protocol for participant retention, participants did receive a study-wide newsletter, prepared by the National Cancer Institute (NCI) and study center coordinators, at least once a year for most calendar years of the study. Each center was free to choose its other retention strategies, and study coordinators often consulted with one another about which methods were successful. Most centers sent birthday cards, holiday cards, and small gifts. Some also produced center-specific newsletters.
Close to the end of PLCO’s follow-up phase, three screening centers, Henry Ford Health System (HFHS), University of Colorado Anschutz Medical Campus (CU Anschutz), and the University of Utah Health Sciences Center (UUHSC), jointly developed a self-administered questionnaire (figure A1, Appendix) to query their participants about retention materials and their impact on participation. Participants also were asked to rate their overall experience with the trial and whether they would have been willing, during the trial, to provide information by way of a secure website, as this might have aided in tracking and compliance with forms completion. The questionnaire was comprised of 18 closed-ended questions and an open text comments section. Of the 18 questions, 15 asked about trial experiences and opinions of retention materials, 2 asked about computer use, and one asked about preferences for using computer systems to submit trial forms. Questionnaires were not tagged with identifiers that could link a questionnaire to a specific participant. Mailings that included the questionnaire and a postage-paid, return envelope were sent by way of the US Postal Service to active participants (that is, those who were still alive and had not withdrawn) at the three centers. CU Anschutz mailed surveys in January 2010, UUHSC in March 2010, and HFHS in May 2010. Because questionnaires were completed anonymously, non-respondents could not be identified and re-contacted, and questionnaires could not be linked to trial data. However, HFHS and UUHSC inconspicuously tagged questionnaires to distinguish intervention and control arm participants.
A Microsoft Access database was created at CU Anschutz and provided to the other two centers. Each center entered its own data using project-wide data entry guidelines provided with the database and performed data entry accuracy checks on at least 10% of the questionnaires they entered. Databases were combined and exported to SAS (SAS Institute, Version 9, Cary, NC) for analysis. Respondent characteristics were compared to those of all enrollees (rather than those to whom questionnaires were sent) due to lags in data submission to the central data repository: this delay, combined with absence of local storage of certain covariate data, precluded identifying active participants at the time of the mailings and linking their covariate information. Cross-tabulations were produced to evaluate responses. Each percentage was calculated using a denominator that reflected only questionnaires with an answer to the question at hand. Because we consider the strength of our results to be descriptive rather than inferential given that nearly all chi-square test of independence P values were less than 0.01 due to large sample size, we do not report results of statistical tests. Comments were reviewed for a 5% random sample of the 7,328 questionnaires with comment text to determine if additional information regarding participant opinions could be gleaned. Questionnaires with responses to at least two questions that were returned before October 1, 2010 were included in analyses.
Results
Questionnaires were mailed to 41,482 participants; 26,010 surveys (63%) were returned and recorded in the database, and 25,985 (62%) were analyzable. CU Anschutz’s response rate was 69%, UUHSC’s rate was 66%, and HFHS’ rate was 57%. The distribution of gender was similar to that of the enrolled participants at the three centers, although respondents were younger and, due to unequal response rates, the centers were not proportionally represented (table 1). At the two centers with trial arm data, 52% of respondents were intervention arm participants. Over 97% of respondents rated their PLCO experience as excellent or good, and this percentage did not vary by screening center, gender, or age (table 2). Of note, 70% of intervention arm but only 52% of control arm participants rated their PLCO experience as excellent. The percent responding that the individual retention materials — the newsletter, pocket calendar, and birthday card — made them more likely to participate was 49%, 24%, and 21%, respectively (table 3). These percentages did not vary meaningfully by age and gender, although 7% of participants age 80 and older stated that receipt of a birthday card made them less likely to participate.
Characteristics of survey respondents and PLCO participants.
About three-quarters of respondents read the PLCO newsletter most or all of the time, and nearly all respondents felt that the quality of the newsletter was good or excellent (table 4). Over 90% of respondents felt that it was either important or very important to include information on the PLCO cancers, and 87% felt the same about including information on other cancers. Similar levels of interest were reported for inclusion of information on published scientific articles that had used PLCO data, as well as information on how PLCO data and bio-specimens were being used. The least popular feature was the crossword/word-find puzzle: only 18% felt its inclusion was important or very important.
PLCO experience ratings by selected characteristics.a
Influence of individual retention materials on PLCO participation by trial arm.a
Of the respondents, 59% reported that they use the internet to view or use websites, and 65% reported that they use email (table 5). Also, 28% reported using websites, and 38% reported using email, 5 to 7 days a week. Over 40% of respondents indicated that they would have been willing to use a secure web site to update study information (table 5); about a quarter indicated that they would not, and the remainder of respondents was not sure. As would be expected, willingness to use a secure website was positively correlated with frequency of use of the internet and email. Willingness to use a secure website decreased with age: 51% of respondents younger than 70, but only 28% of participants 80 or over, indicated that they would be willing to do so (data not shown). Review of comments gleaned little additional information to that obtained from the multiple choice questions. Most comments that were general in nature were complementary, expressed gratitude, or stated that the trial was an important endeavor, and were in line with the positive responses presented in table 2. A number of comments expressed that receipt of retention materials made participants feel they were appreciated and, in some instances, that trial staff cared about them.
Discussion
Clinical trials cannot succeed without compliance and retention of participants. PLCO researchers recognized the importance of keeping participants engaged and pleased with their experience, and as such, made retention strategies a trial priority. Data collected by three PLCO screening centers suggest that nearly all participants felt favorably about their PLCO experiences and that retention materials were well-received. The newsletter, an important tool, appears to have been widely read, and a number of topics covered in the newsletter appear to have been of particular importance to participants.
PLCO was a unique trial. While other long-term cancer screening trials have been done, they have not been nationwide, as large as PLCO, and were not concurrently investigating modalities for multiple organ sites. Large cancer and non-cancer chemoprevention trials, which could serve as a comparison, have not published on retention materials or strategies for their study populations as a whole, although at least one has published on retention materials or strategies for study sub-populations.6 In 2007, Robinson et al7 conducted a systematic review of retention strategies to retain participants in 21 studies that required in-person follow-up, and although these studies are quite different from PLCO, the authors used them to develop a useful retention theme framework, one that encompasses twelve retention strategies. They specifically mention providing small tokens of appreciation (like a pocket calendar) under the strategy of nonfinancial incentives and newsletters under the strategy of patient contact.7 The PLCO newsletter can be classified under two other retention strategies as well: creating a project identity and highlighting the significance and potential benefits of study participation.
Throughout the course of the trial, PLCO study coordinators mentioned that they frequently received positive comments about one retention material, the pocket calendar. Nevertheless, only a quarter of respondents indicated that the pocket calendar made them more likely to participate. More than half of respondents indicated that inclusion of a word puzzle, like a crossword, did not make them more likely to participate, although some screening centers felt it was important to include one. Our findings demonstrate the importance of querying participants about retention materials and other communication tools. Had we known that persons who participated in PLCO were more interested in receiving information during the life of the trial rather than small tokens of appreciation, we might have “tweaked” our retention methods and approaches.
PLCO enrollment began in 1993. By the end of the trial in 2011, many participants had asked whether email could be used for contact and forms completed at a website. To further explore email and internet use in the PLCO study population and to gauge the magnitude of interest in using email and internet among study participants, three questions were included on the questionnaire. Our data suggest that in some segments of the US population, more than half of persons who are over age 70 and healthy are using email and the internet at least once a month. In 2004, the Pew Interest Group reported that 22% of Americans over the age of 65 use email or internet,8 but in 2012, that percentage had climbed to 53%, and among those, 86% were using internet and email every day.9 Given that this trend is likely to continue, future cancer screening trials, and probably most clinical trials, will need to provide an option to use electronic resources.
Strengths of this project include the anonymity of respondents. It is likely that respondents who were unhappy with their experiences or not enamored of retention materials answered more honestly than they would have otherwise. Our data are the first to be published that examine the effect of specific materials in long-term randomized controlled trials conducted in healthy populations and to do so in a manner that did not limit findings to special populations. Furthermore, they reflect participants’ opinions, rather than those of trial staff.
The major limitation of our data is that the questionnaire response rate was only 62%. Because unsatisfied participants would have been less likely to return questionnaires, our findings probably overestimate participant satisfaction. Nevertheless, the fact that over 96% of PLCO participants did not withdraw from the study suggests that it is unlikely that non-respondents’ overall satisfaction with the trial and retention materials differed drastically from that of respondents. It also is unlikely that experiences and opinions of participants at the seven PLCO centers that abstained from this activity would have differed in a manner that would change the interpretation of our findings, given that rates of withdrawal, form submission, and study exam attendance at those seven centers were similar to those at the three that participated. Nevertheless, higher response rates and surveying of all centers would have allowed us to be more certain of our conclusions.
Another limitation is that the questionnaire was sent out very close to the end of trial follow-up. Thus, we do not know whether usefulness of retention materials differed throughout the lifecycle of PLCO, and we did not capture the opinions of participants who withdrew due to dissatisfaction early in the study. It also would have been useful to have targeted questions for control arm participants, to determine whether certain materials appealed more to them, as control participants may be more difficult to retain in some studies. And because all study participants received retention materials, we have no measure of study satisfaction in the absence of these materials; therefore, concluding that retention materials increased participant satisfaction cannot be made without caveats.
Reactions to PLCO newsletter and importance of specific components in future newsletters by age and gender.
Preference for use of a secure website for study activities by internet and email.a,b
Conclusion
Most PLCO participants had positive experiences in the trial. About 50% of respondents indicated that receipt of a newsletter was more likely to make them participate, compared with no more than a quarter for a token gift or a birthday card. These data indicate that study participants prefer to receive information rather than small gifts or tokens of appreciation. At the time of questionnaire completion (2010), fewer than 30% of respondents indicated that they would be unwilling to use a secure website to complete study materials, suggesting that the internet should be considered as a means of data collection in older US populations.
Acknowledgements
The authors thank the PLCO participants and the hundreds of individuals who worked on the trial throughout its course. We thank Carrie N. Klabunde, PhD, for her careful review of the manuscript and helpful suggestions. We also wish to acknowledge our colleague and friend, Eduard Gamito (deceased), who was Recruitment/Retention Coordinator at the University of Colorado PLCO site. He conceived the idea for this project, directed the initial work, and collaborated with the authors. His inspiration and contributions made this paper possible. We dedicate it to his memory.
Appendix
The jointly-developed, self-administered questionnaire to query participants about retention materials and their impact on participation.
Footnotes
Funding Disclosure: Supported by contracts from the Division of Cancer Prevention, National Cancer Institute, to the coordinating center (N01-CN-25476 to Westat, Inc.), the three screening centers (N01-CN-25512 to Henry Ford Health System; N01-CN-25514 to the University of Colorado Anschutz Medical Campus; N01-CN-25524 to the University of Utah Health Sciences Center, and Information Management Services, Inc (N02-CN-55023-76 and N02-CN-35001-45).
- Received January 30, 2015.
- Revision received May 15, 2015.
- Revision received June 15, 2015.
- Accepted June 18, 2015.
References
- 1↵National Cancer Institute, Division of Cancer Prevention. Cancer Data Access System. Available at: https://biometry.nci.nih.gov/cdas/. Accessed July 18, 2013.
- 2↵BuysSSPartridgeEBlackAJohnsonCCLameratoLIsaacsCRedingDJGreenleeRTYokochiLAKesselBCrawfordEDChurchTRAndrioleGLWeissfeldJLFouadMNChiaDO’BrienBRagardLRClappJDRathmellJMRileyTLHartgePPinskyPFZhuCSIzmirlianGKramerBSMillerABXuJLProrokPCGohaganJKBergCD; PLCO Project Team. Effect of screening on ovarian cancer mortality: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Randomized Controlled Trial. JAMA 2011;305:2295–2303.
- 3OkenMMHockingWGKvalePAAndrioleGLBuysSSChurchTRCrawfordEDFouadMNIsaacsCRedingDJWeissfeldJLYokochiLAO’BrienBRagardLRRathmellJMRileyTLWrightPCaparasoNHuPIzmirlianGPinskyPFProrokPCKramerBSMillerABGohaganJKBergCD; PLCO Project Team. Screening by chest radiograph and lung cancer mortality: the Prostate, Lung, Colorectal, and Ovarian (PLCO) randomized trial. JAMA 2011;306:1865–1873.
- 4AndrioleGLCrawfordEDGrubbRL3rdBuysSSChiaDChurchTRFouadMNIsaacsCKvalePARedingDJWeissfeldJLYokochiLAO’BrienBRagardLRClappJDRathmellJMRileyTLHsingAWIzmirlianGPinskyPFKramerBSMillerABGohaganJKProrokPC; PLCO Project Team. Prostate cancer screening in the randomized Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial: mortality results after 13 years of follow-up. J Natl Cancer Inst 2012;104:125–132.
- 5↵SchoenREPinskyPFWeissfeldJLYokochiLAChurchTLaiyemoAOBresalierRAndrioleGLBuysSSCrawfordEDFouadMNIsaacsCJohnsonCCRedingDJO’BrienBCarrickDMWrightPRileyTLPurdueMPIzmirlianGKramerBSMillerABGohaganJKProrokPCBergCD; PLCO Project Team. Colorectal-cancer incidence and mortality with screening flexible sigmoidoscopy. N Engl J Med 2012;366:2345–2357.
- 6↵WilcoxSShumakerSABowenDJNaughtonMJRosalMCLudlamSEDuganEHuntJRStevensS. Promoting adherence and retention to clinical trials in special populations: a women’s health initiative workshop. Control Clin Trials 2001;22:279–289.
- 7↵RobinsonKADennisonCRPronovostPJNeedhamDM. Systematic review identifies number of strategies important for retaining study participants. J Clin Epidemiol 2007:60:757–765.
- 8↵Pew Internet and American Live Project. Older Americans and the Internet, 2004. Available at: http://www.pewinternet.org/Reports/2004/Older-Americans-and-the-Internet.aspx. Accessed July 18, 2013.
- 9↵Pew Internet and American Live Project. Older Americans and the Internet, 2012. Available at: http://www.pewinternet.org/Reports/2012/Older-adults-and-internet-use.aspx. Accessed July 18, 2013.
