By Deborah Borfitz

March 7, 2022 | The development and validation of novel digital endpoints in clinical trials was the subject of several case studies shared at the recent Summit for Clinical Ops Executives (SCOPE), including a pilot study of a speech-language assessment tool by Regeneron Pharmaceuticals and trials of at-home monitoring devices for gait speed and nocturnal scratching by Pfizer. Rapid adoption of wearables has provided opportunities to innovate and better understand patients and biomarkers of health abnormalities, according to Rinol Alaj, Regeneron’s director of clinical outcomes assessment and patient innovation.

Outcome questionnaires used in clinical trials typically evaluate sleep quality based on a 0-to-10 rating scale and invasive, in-clinic procedures are likewise subjective because the data is collected only during study visits and therefore don’t reflect real-world patient conditions, Alaj says. Digital biomarkers, on the other hand, assess physiological functions obtained through continuous monitoring by noninvasive wearable sensors that can detect signals that otherwise go unnoticed.

That translates into a smaller number of patients needed to run proof-of-concept studies, says Alaj. Program-level planning in the preclinical stage of product development is the time to decide which digital endpoint to focus on based on scientific need. From there, it’s a “step-by-step journey” to test the usability of a digital biomarker, verify it works as intended, and validate its clinical utility.

At Regeneron, digital endpoint validation happens in stages, beginning with preclinical “discovery ideation” when the technology landscape is reviewed and strategic partnerships form, Alaj shares. In the next stage, the algorithm’s performance is measured against a reference method and usability testing is done to ensure positive perceptions of the device. A clinical validation study happens in the final stage.

Speech-language assessment represents an unmet need in the neurodegenerative disease arena, he notes. At present, such measurements are mostly qualitative based on patient-reported outcomes and a clinical rating scale.

Assessing Speech

Speech abnormalities are characteristic of various diseases, including Parkinson’s and Alzheimer’s, says Bharatkumar Koyani, senior manager of patient innovation at Regeneron. Symptoms and clinical manifestations can be mapped using quantitative speech assessment to make a diagnosis as well as monitor patient progress.

In a pilot study for the use case of patients with mild Alzheimer’s disease, speech assessment was done with a mobile, tablet-based app found to compare favorably with traditional neuropsychiatric measures, he says. The technology is simple enough to be used at home for more frequent data collection than would be possible at a designated study site where subjective ratings are “semi-quantitative at best.”

The first step was to design the pilot study and evaluate the device (Winterlight) and its capabilities within the established framework for digital endpoint development, making it eligible for testing in an ongoing trial, Koyani continues. The pilot study enrolled healthy volunteers and examined feasibility and any data-related issues.

Results of the study included positive feedback from users on the speech assessment task (answering questions via the app) with 94% of data missingness related to the relevance of a speech feature (e.g., acoustic, lexical, or syntactical) to a respective activity, he says. The required parameters were therefore met.

Winterlight provides a feature extraction platform to measure speech, Koyani says. Speech disorders can be assessed using more than 500 features.

In the pilot study, a heat map was created from 230 speech samples showing data from all tasks (e.g., picture descriptions and object-naming) and their time points. The technology succeeded in generating data that was “convincing,” he says. Data will now be collected on a diseased population, making mobile-based speech assessment a potential endpoint in future clinical trials.

In concluding the session, Alaj advises against innovating “for the sake of innovating” but to instead incorporate it via early and rapid device testing with healthy volunteers. He additionally recommends integrating as many indications as possible to avoid reinventing the wheel.

Gait And Scratch Movement

Digital technology is likewise being leveraged by Pfizer, based on the recognition that it is can be difficult for participants to get to study visits, and point-in-time evaluations done at prescribed intervals don’t necessarily provide a full picture of the endpoint, according to Carrie Northcott, Ph.D., director and project lead. Greater use of sensors and wearables offers the opportunity to get more qualitative measures, which can typically be passively collected, as well as recruit a broader patient population. But use of the technology can also place new burdens on patients, caregivers, and sites.

Before deploying a new technology, Pfizer’s digital medicine and translational imaging group starts by asking itself a series of questions: Is it meaningful to patients, clinicians, and researchers? Are the data required and user-specific? How will the data be securely collected? What new analytics will be needed to evaluate the endpoints? “It takes a team to look at all these aspects,” Northcutt points out.

Pfizer’s approach to validating novel digital endpoints considers what’s important to patients, she continues, and requires that the measuring device be compared to the gold-standard reference test. It must also be “fit for purpose” and able to answer the research question.

The validation work happens in the Pfizer Innovation Research (PflRe) Lab, says Northcutt, after which the device gets tested on various populations in partnership with internal teams and academic institutions. Clinical studies, for labeling purposes, quantify therapeutic response based on changes in the novel clinical endpoint.

Among the digital endpoints under development is one for gait speed, which is considered a “sixth vital sign,” predictive of mortality, falls, and functional decline, Northcutt says. In 2019, stride velocity in the 95th percentile became the first wearable-derived digital clinical outcome assessment qualified by the European Medicines Agency for use as a secondary endpoint in trials for Duchenne muscular dystrophy.

In the PflRe Lab, the gold-standard (in-clinic walk task called GaitRITE) was compared to GaitPy, which reads accelerometry data from a lumbar-mounted sensor. GitPy “underestimates gait speed slightly but provides consistent measurements,” she notes.

Comparison testing in the two settings revealed significant differences in gait speed and cadence between age groups when testing was done at home, Northcutt reports. But GaitPy had high compliance irrespective of age, with 96% of users wearing the device for almost nine days.

Pfizer has also investigated the novel nocturnal scratch and sleep endpoint that is important with conditions such as atopic dermatitis, says Northcutt. Accelerometer-based devices worn during sleep to monitor movement were compared to ground-truth polysomnography and found to be 73% accurate, and performance has since improved further. Users also reported the device to be comfortable and were willing to wear it.