Contributed Commentary By Marie McCarthy, Tim Clark and Peter Schueler

June 23, 2020 | Blinding data to remove bias is a fundamental principle of randomized controlled trials (RCTs). Does the advent of smartwatches, fitness trackers and other wearable technology and health apps mean that steps should no longer be used as an endpoint in RCTs?

eMarketer has estimated that over a quarter of adults in the U.S. use a wearable device at least once a month, and by Statistica’s estimation the global penetration of smartphones reached 41.5% last year. These figures are supported by a 2018 mHealth survey looking at the prevalence of mobile health technology among individuals irrespective of their cardiovascular disease risk. Among the population surveyed, 92% had smartphones, 55% had downloaded a wellness app and 51% had used a wearable to track their health progress in the previous 12 months, as was reported at the American Heart Association’s 2019 Scientific Sessions.

Wearables are purpose-built to engage users, with optimized interfaces, easy-to-use screens, appealing companion apps and easy-to-interpret dashboards. Users are sent regular text messages or alerts to encourage them to reach specific step targets which, when achieved, earn them personalized gamification rewards.

But the question remains: Does the widespread availability of these devices influence the physical activity (PA) behaviours of study participants and the validity of steps and other PA-derived parameters as endpoints? This is particularly pertinent in studies where a subset of participants are tracking a study outcome parameter, motivated and influenced by their mobile device and this data is unblinded.

Physical Activity as an Outcome Measure

The measurement of physical activity is considered an important outcome measure for assessing drug efficacy. In the context of clinical research there are a number of recognized assessments for physical activity, including patient self-reporting using questionnaires such as the short version of the International Physical Activity Questionnaire (IPAQ-s) or Modifiable Activity Questionnaire (MAQ); clinical assessments such as the six-minute walk test; or the use of objective measures like physical activity monitors or pedometers. However, this is a very complex area where a multitude of endpoints might be used to track changes in physical activity—e.g., steps, intensity, number of bouts, timing and duration of the episodes. 

The appropriateness of the specific endpoint will largely depend on the study population and drug under investigation. Wrist-worn devices to measure physical activity have significant potential, capturing ambulatory data in a passive, low-burden fashion, as study participants go about their daily activities.

The use of medical devices to measure physical activity in research and clinical studies is well established. These devices have functional settings that give researchers control over the blinding of PA data, so study participant can see some or all of their PA data (as appropriate) in the context of a specific study. However, the ability to blind data in commercially available wearables is limited.

Interest in the use of smartwatches and fitness trackers in clinical studies is growing, particularly among technology and software vendors seeking regulatory approval to make specific label claims about their product. Some of the better-known devices are being used in clinical studies and are part of the FDA’s Software Precertification (Pre-Cert) Program. But the use cases are limited largely to behavioural rather than drug development studies.
 

Beyond the generation of clinical study endpoints, an additional but not widely acknowledged concern is the impact of the devices as a covariate (i.e., a characteristic that may affect the outcome of a study). Participants using wearables potentially have access to data that is a key efficacy variable. If steps are used as an endpoint in a study, anywhere from 30% to 50% of participants might have visibility to this data.

Not only do such devices allow a user in-depth knowledge of their activity levels; they could potentially act as an intervention—e.g., encouraging wearers to reach their daily goals and maintaining specific targets thereafter. Use of the devices could result in “activity peaks” and “activity plateaus” driven not by drug efficacy but as a response to the smartwatch/fitness tracker targets.

Unintended Consequences

Physical activity remains a key endpoint and outcome assessment in clinical trials, but the likely impacts of smartwatches need mitigating. Potential solutions include prohibiting participants from wearing activity trackers throughout the study, or ensuring that all participants are using the same device. It may be too extreme to make device ownership an exclusion criterion, but it may be useful to explore the impact of use/non-use of trackers on statistical analysis.

Greater awareness and consideration also need to be given to the use of wearables during the screening and recruitment phase. Anecdotal information on the marginal, long-term impact of these devices on physical activity levels needs to be gathered early on, particularly if physical activity is to be a primary endpoint.

Measures can be taken to reduce the likely impact of participants’ own smartwatch/fitness tracker, such as: establishing physical activity levels during baseline data collection to determine if the devices are impacting their daily activity goals, selecting alternative physical activity endpoints that may be less influenced by the devices (e.g., duration and number of activity bouts), and educating participants about the need and rationale for reducing notifications from their device during the study.

Additional research is needed in this arena, given that the impact of smartwatches and fitness trackers on study participants and study endpoints is unclear beyond trials where wearables are being deployed as a data capture tool. More certain is that the unblinding of study data could have far-reaching if unintended consequences by introducing bias into the data analysis process.

Marie McCarthy is senior director of product innovation, information technology, at ICON and is part of its multidisciplinary Innovation Team. Seen as a key entrepreneur within the organization, she is responsibility for developing solutions in the direct-to-patient paradigm. Her focus is primarily on the use of wearables and sensors in clinical trials, with emphasis on the potential of these devices to monitor the physical behaviours of the digital patient. Prior to ICON, McCarthy was EU sales and marketing manager with Philips Respironics, building awareness of the value of actigraphy endpoints among clinicians and researchers. She can be reached at Marie.McCarthy@iconplc.com.

Peter Schueler, M.D., is senior vice president, CNS, Drug Development & Consulting, at ICON. He is board certified in neurology and in pharmaceutical medicine (Swiss Medical Association). He has experience working in the pharmaceutical industry and CRO world and joined ICON in 2007. He has been first author on over 40 publications related to drug development and drug safety. His book, “Re-engineering Clinical Trials,” was published by Elsevier in 2015. Schueler continues to lecture in pharmaceutical medicine at the University of Duisburg-Essen and at the European Center Pharmaceutical Medicine (ECPM) in Basel, Switzerland. He can be reached at Peter.Schueler@iconplc.com.

Tim Clark, Dip. Stat., Ph.D., is vice president of clinical sciences, drug development services, at ICON. He gained his Ph.D. in clinical trial methodology at the Institute for Medical Informatics, Biometry and Epidemiology at Ludwig-Maximilians-University in Munich. Clark is part of the drug development services group that provides strategic drug development advice to customers and internal teams. His specialty is clinical study design and protocol optimization. Prior to joining ICON in 2009, Clark worked in clinical research and regulatory affairs for large French and American pharmaceutical companies and as an independent consultant advising on a range of drug development issues. He has worked on small molecule and biological (including biosimilars) programs indicated for cancer as well as CNS, autoimmune, cardiovascular, and infectious diseases. He has also published on clinical study design, protocol optimization and sample size determination in peer-reviewed journals and is currently an honorary associate professor at the Institute of Clinical Trials & Methodology at University College London. He can be reached at Tim.Clark@iconplc.com.