By Deborah Borfitz

August 1, 2023 | Randomized controlled trials are a fantastic first step in assessing the clinical effects of medical interventions, but they are not sufficient to transform the “busy throughput environment” of real-world healthcare involving appointment scheduling and referrals, differing practice patterns, and patients with comorbidities and social determinants of health like housing instability and low literacy skills. That’s why the National Institutes of Health (NIH) has embraced the idea of embedding pragmatic clinical trials with routine patient care, according to Andrew Boyd, M.D., associate professor of biomedical and health information sciences at the University of Illinois Chicago (UIC). 

The flipside is that embedded pragmatic clinical trials (ePCTs) can potentially leave out people who are from underrepresented and underserved groups—especially when health systems rely heavily on electronic health records (EHRs) for data collection, says Boyd. His concern is shared by a small group of NIH-funded pragmatic trialists who recently offered mitigating tactics for how to increase generalizability of ePCT (Contemporary Clinical Trials, DOI: 10.1016/j.cct.2023.107238). 

These days, nearly all health systems use an EHR but the data it warehouses are “only as good as the people who show up,” Boyd notes. Access to healthcare is not equitable, so some patient populations are poorly represented. Using patient portals for the intake of patient-reported outcomes (PROs) further exacerbates the situation since they may miss individuals with limited health literacy or people who lack smartphone and internet access or don’t have the technological wherewithal to navigate the web. 

Boyd and his peers have five general recommendations for how to lower the barriers to participation in ePCTs, so data are collected from everyone. These include the adoption of equity-based data collection methods that incorporate SDOH measures, identifying the best barrier-reducing data collection instrument (e.g., use of bidirectional text messaging), adopting community engaged approaches to EHR research, understanding the effects of interventions through the lens of different combinations of SDOH variables (including those not typically documented in EHRs), and accommodating differing reading levels to ensure PRO surveys are suitable for the population of interest. 

Hidden Biases

The idea with ePCTs is to not just measure the effects of an intervention—be it to treat a disease or simply improve overall care quality—but to learn if the approach is feasible in the context of where it will be used, says Boyd. “There is an increased realization that clinical studies... need to be conducted in pragmatic environments.” 

A dietary intervention being evaluated in a clinical trial setting might involve cooking and mailing food to individual research participants, all but ensuring they eat as instructed, he offers as an example. That is entirely different than real-world healthcare where patients are going to the grocery store to purchase food according to the dietary recommendations and may not find what they’re looking for or take a detour down the snack aisle—or maybe never get there at all because they live in a food desert. 

When conducted at a hospital, the EHR is a staple of ePCTs, Boyd says. In other healthcare settings—notably, physical and occupational clinics and long-term care facilities—patient records are oftentimes still paper-based.    

The key problem with using EHRs to identify, enroll, and engage study participants is that they invariably contain “hidden sources of bias,” he continues. This became apparent during early conversations Boyd had with his colleagues involved in the ongoing Hybrid Effectiveness-Implementation Trial of Guided Relaxation and Acupuncture for Chronic Sickle Cell Disease Pain (GRACE) trial. 

EHR-related tactics that worked at other health systems just weren’t working at institutions collaborating on the GRACE trial. At some locations, half of patients enrolled in the study had not even accessed the health system before, says Boyd. 

To overcome challenges like these, many pragmatic trials engage “citizen scientists” (aka patient representatives or community advisors) to identify and resolve at least some of the barriers. For the GRACE trial, they helped investigators describe guided meditation and acupuncture to study participants who were almost wholly unfamiliar with the interventions and understandably wary about having needles inserted into their body, he says. “How you present these concepts to research participants is critical; otherwise, they’re not going to be interested in the study.” 

Text Messaging

It is entirely possible to unintentionally exacerbate existing health inequities unless biases stemming from the use of EHR data for both research and healthcare delivery are addressed, says Boyd. He specifically references a 2019 article in Science (DOI: 10.1126/science.aax2342) finding evidence of racial bias in one widely adopted machine learning algorithm developed by the health technology company Optum to predict the severity of patients—the basis for enrolling them into high-risk care management programs. Since the algorithm was trained on EHR billing and lab data, and African Americans and Hispanic Latinos do not show up as often for care, the severity scoring system requires them to have more-severe hypertension, diabetes, renal failure, and anemia, and higher cholesterol than Caucasians. 

In the context of ePCTs, a theoretical example of this sort of bias would arise with the use of PROs, he says. To fill out these questionnaires, “you [often] need a smartphone, you need internet access, you need a username, you need to have passwords, [and you] have to have a relatively high technological sophistication.” 

Why some people log onto an EHR, and others don’t, is still an open research question, Boyd says. But at UIC, many pragmatic studies send text messages to research participants that include a link to the PRO survey so they can skip the authentication process and just answer the questions. “Because of both the health literacy inequities and the heath technology inequities, if you only use the electronic health record the research participants in these embedded trials won’t [all] be responding.” 

About 70% of people now live without a landline and almost everyone has some sort of mobile device, and most users are comfortable texting, he adds. Even if monthly minutes are being purchased, texting is free on lots of different data plans. And texting doesn’t require having the internet turned on. It can therefore be an ideal and hassle-free way of increasing study participation. 

PRO surveys themselves are often written above the health literacy level of many people, says Boyd. To reach the masses, the current recommendation from the National Academy of Medicine is for patient materials to be written at both the sixth and eighth grade reading levels. That translates into short and somewhat choppy sentences, which is admittedly difficult to do while still adequately explaining large and complex topics—and why such materials are more often written at or above a 12th grade reading level.  

Intersectionality Analyses

One of the newer concepts introduced in the published commentary is intersectionality analysis. This is a way of combining multiple different dimensions of one’s identity—e.g., race, gender, sexual orientation, disability, immigration status, housing, education, and income—and “how they are individually linked in ways that we may not always study in clinical trials,” explains Boyd. The intersectionality of these variables might have a differential effect, but studies aren’t appropriately powered to look at some of the smaller subgroups. 

Such information isn’t being collected at a fine enough grain, and for multiple reasons, he adds. EHR systems often don’t have data fields for these social categories and SDOH variables. In some parts of the country, it isn’t even legal to ask someone about their gender identity or preferred pronouns. Intersectionality in general is hard to get correct because of the “preciseness” of how people in a community might describe themselves. 

But capturing this sort of information can help promote heath equity, says Boyd, specifically citing individuals experiencing housing instability either because they keep moving from apartment to apartment, are staying with friends, or are homeless. If that variable goes unrecorded, its impact across intervention groups in an ePCT will go unrecognized. 

The problem of housing instability is also more likely to persist. At UIC, which serves a highly diverse population, Stephen Brown (UI Health’s director of preventive emergency medicine) worked with a local nonprofit to create a housing program to address the issue among its patient population, resulting in lower inappropriate utilization of the emergency room, Boyd reports.