By Deborah Borfitz 

January 17, 2025 | Digital clinical measures that are truly novel and therefore lack a good reference measure for assessing their performance have been given overdue attention by a public-private partnership between the nonprofit Digital Medicine Society (DiMe) and the U.S. Food and Drug Administration (FDA). After 15 months of concerted effort, the collaboration has resulted in a set of resources digital health technology (DHT) developers and clinical trial sponsors can reference in their interactions with regulators, reports Benjamin Vandendriessche, chief delivery officer at DiMe. 

The project is part of a broader effort to support the creation and adoption of regulatory-grade devices for collecting health metrics in clinical studies by the DiMe-hosted DATAcc (Digital Health Measurement Collaborative Community) initiative. The primary focus of the newly launched information repository is ensuring the appropriate level of analytical validation, he says.  

An interactive guide comprising introductory materials and a study builder, plus more in-depth guidance on statistical methodologies, are among the newly available resources. They’re designed to complement the Verification, Analytical Validation, and Clinical Validation (V3) Framework created by DiMe in 2020, and extended to V3+ in 2024 to include a fourth component—usability validation for assessing the quality, performance, and clinical relevance of sensor-based DHTs to ensure they’re “fit for purpose,” says Vandendriessche.  

Analytical validation is done by assessing the performance of an algorithm in transforming raw data generated by a sensor into some sort of actionable insight. “To do that, you almost invariably need a good reference measure,” he says. 

Novel measures exist on a hierarchy whereby their context of use determines the level of rigor with which they need to be analytically validated, continues Vandendriessche. If no good reference measure exists, DHT developers may have to “make one from scratch.” 

Failing that, “anchor” measures can be used, a term the FDA has used to describe an external criterion for determining if patients have experienced a meaningful change in their condition. For validation purposes, anchor measures may show “statistical association, not correlation,” which while not perfect is a steppingstone, he says. 

Taking that step often translates into a long development road, making it a hard but necessary choice to “reach the kind of rigor in your data” needed for a primary digital endpoint in a targeted patient population with an intended clinical outcome, says Vandendriessche.   

Aiding Decisions

The foundational V3+ Framework provided a “flow of work” to follow when evaluating sensor-based digital health technologies, such as cardiac patches that measures the heart’s electric activity, to ensure they meet or exceed an established measurement tool (i.e., electrocardiogram), Vandendriessche says. It is part of DiMe’s mission to accelerate the adoption of digital endpoints in clinical trials, which has also included an analysis with the Tufts Center for the Study of Drug Development (CSDD) about their potential financial net benefit to sponsors in terms of study size and duration.  

The study with Tufts CSDD found the value of shortening trial timelines and time to market equated to a return on investment of four to six times the initial investment, he notes. The estimated financial gains of incorporating digital endpoints into phase 3 trials were estimated to be between $24 and $40 million. 

A vast universe of sensor-based devices produces digital clinical measures, and they run the gamut from well-established technologies like ECG cardiac patches to fitness trackers for predicting opioid use disorder relapse—another project DiMe currently has underway with the Duke BIG IDEAs Lab. The data being generated in the study falls in the novel category and, in the absence of reference measures, investigators will be falling back on questionnaires and urine analysis to assess performance, says Vandendriessche. 

In between are established devices being deployed for a new purpose, such as an electroencephalogram monitoring device being used at home for the detection of seizures in people with a rare type of epilepsy. “The concept of novel is on a spectrum,” he stresses. 

A different approach and set of statistical methodologies are needed for the analytical validation piece when the units of what’s being measured and references are not directly comparable, or no good reference is available at all, Vandendriessche adds. The new set of resources is intended to help guide decision-making in these cases at the time analytical validation studies are being designed. 

The project was funded by Arnold Ventures and overseen by a highly diverse statistical advisory committee that included Andrew Potter, Ph.D., senior mathematical statistician in the FDA’s Center for Drug Evaluation and Research. It is far broader in scope than originally imagined, since the concept of a novel digital clinical measure extends to sensor technologies that have been around for a long time, he notes. 

Blood pressure is a recognized clinical measure, for example, but has a hierarchy of reference measures with different levels of rigor for different contexts of use, continues Vandendriessche. In-hospital blood pressure measurements might variably be taken through an arterial line in the hospital, using a manual blood pressure cuff with a healthcare professional listening for blood flow using a stethoscope, or using an at-home blood pressure cuff using electronic sensors to automatically detect blood pressure.  

An automatic blood pressure cuff might be appropriate except for a population of people with arrhythmias that can affect the accuracy of blood pressure readings, he says. In that case, it would be better to have someone taking manual measurements who can make the necessary compensatory adjustments. Likewise, from a safety perspective, an arterial line may be “completely out of scope” for measuring blood pressure in a pediatric population. 

Scenario Planning

The new set of online resources includes an interactive flipbook covering the basics of analytical validation of novel digital clinical measures that users can click through, says Vandendriessche. A longer document is also available that will walk developers step by step through the process of building a statistical plan for their analytical validation study.