By Deborah Borfitz 

February 4, 2020 | Clinical trial stakeholders, most notably the U.S. Food and Drug Administration (FDA), continue to grapple with how to harness the value of real-world data (RWD) and real-world evidence (RWE) for demonstrating product effectiveness. Draft guidance expected to be issued by the end of 2021 should shed light on the three key issues outlined in the agency’s Framework for FDA’s Real-World Evidence Program: what makes RWD fit for use, how to generate RWE to provide adequate scientific evidence to address regulatory questions of interest, and how to ensure the conduct of the study meets regulatory requirements (e.g., study monitoring and data collection). 

Both the framework and guidance are mandates of the 21st Century Cures Act and the sixth Prescription Drug User Fee Act (PDUFA VI), making the use of RWD and RWE for effectiveness decision-making an FDA priority, according to Nirosha Mahendraratnam Lederer, a managing associate at the Duke-Margolis Center for Health Policy and lead staff for its RWE Collaborative. Up to now, FDA has used RWD primarily in its evaluation of safety. 

The new Real-World Evidence Program, as envisioned by the FDA, will evaluate the potential use of RWE to back changes to product labeling about drug product effectiveness, including adding or modifying an indication or adding a new population or comparative effectiveness or safety information. 

One of the most significant contributions of the initial framework document was “clearly stated definitions” for the words RWD and RWE, which had been an ongoing point of confusion, says Lederer. But misunderstanding about other basic terminology remain potential roadblocks to progress. 

The term “real world,” for instance, denotes a healthcare setting where data is routinely collected and not a study design with a pre-specified protocol about how and when it gets collected, Lederer says. RWD isn’t limited to secondary data sources, as is often assumed. It can be used in support of a study’s primary outcome measure.  

RWD and RWE can also support clinical effectiveness research in two distinct ways, continues Lederer. One is to improve trial efficiency in areas such as patient recruitment, patient stratification, and the use of novel study designs that involve the use of Bayesian statistical methods. The other is to analyze data about the real-world use of products to create RWE with its own unique value as a “complementary resource” to, rather than replacement for, randomized clinical trials (RCTs). 

RWE Collaborative 

Established in 2018, the Duke-Margolis Center for Health Policy RWE Collaborative spent 2019 exploring ways to inform development of guidance, policies, and procedures around the use of RWD and RWE in effectiveness decision-making, Lederer says. 

Assessing RWD fitness-for-use will require evaluation standards because data vendors are currently using a hodgepodge of processes that “aren’t necessarily gold standard best practices for how to curate the data,” says Lederer. There is still no consensus on what those curation practices should be, she adds, although they probably do not make sense for rapidly evolving technologies, such as natural language processing of free-text narratives in electronic health records (EHRs).  

“If you can’t trust the data, whether or not it’s relevant doesn’t even matter,” Lederer notes. “[The] FDA has a public health-oriented mission related to the safety and efficacy of drugs, so we need to be able to trust the data.” 

The RWE Collaborative published several white papers on the RWD/RWE topic last fall. One looks at evaluating whether RWD are fit for use by using verification checks to assess reliability. A second focuses on noninterventional studies using secondary data and their potential application in support of effectiveness decision-making, and a third explores how incorporating RWE studies into an evidence package can be used to support an effectiveness labeling change through a totality of evidence approach. 

One suggestion in the December 2019 paper is to conduct pilot projects using a novel outcome measurement tool, possibly as an exploratory endpoint within a traditional clinical trial. Primary endpoints are the ones that tend to make or break a trial “from a purely statistical point of view,” Lederer says. But other types of information can help physicians and regulatory decision-makers alike better understand the product—including data collected by wearables and sensors that may not yet have been validated in a trial setting.     

Early in 2020, Duke-Margolis will post a new white paper on considerations for identifying endpoints, including incorporating novel data sources, to answer regulatory research questions in real-world healthcare settings, Lederer says. Among the other topics to be explored are the use of RWD as an external control or comparator arm and harmonizing the evidence generation strategies for regulatory submissions and payer reimbursement to create time and cost efficiencies. 

Several pilot projects related to data fitness-for-use and valid study methodologies are also underway to help determine how to make valid causal inferences with real-world study designs as well as explore issues related to regulatory conduct, including patient privacy and how to collect and store data, says Lederer. 

This being a new arm of regulatory science, the chief challenge for the RWE Collaborative has been informing FDA guidance and policy development amid uncertainty. “It’s a little bit of a chicken-and-egg situation,” says Lederer, “but what’s exciting is that everyone has the same goal… and there is a concerted effort by stakeholders across the healthcare spectrum, including [the] FDA, to work together to advance things forward.”  

Lederer points to Pfizer’s Ibrance as an example of how RWD/RWE can be used to advance clinical development. Last April, the company secured expanded approval for the breast cancer drug to men by relying on data from EHRs and post-marketing reports about real-world use of the therapy. 

Other examples are listed in the appendix of the December 2019 white paper. Many are cancer drugs using RWD as a comparator for a single-arm study because the condition was rare, or the unmet need was so high that randomization would be unethical. Effective RWE use in regulatory approvals have included data on historical response rates drawn from chart reviews, expanded access, and other practice settings, according to the FDA’s framework document. 

Linked Datasets 

Uniting different big healthcare databases to get a more complete picture of patients is critical to advancing the use of RWE in regulatory decision-making and fulfilling the promise of precision medicine development. In oncology, the pioneering partnership between Foundation Medicine and Flatiron Health has linked vast quantities of clinical outcomes data from EHRs with genomic data on people who have their genome sequenced by the molecular insights company. 

The resulting clinico-genomic database (CGDB) represents all patients that have both received Foundation Medicine testing and been treated in the Flatiron network, says Alan Braly, director of the newly assembled real-world evidence team at Foundation Medicine. “While we filter for quality and completeness, we do not have any filters on patient demographic or disease characteristics.” 

The CGDB is now being used to validate the potential of RWD for understanding and optimizing personalized cancer care. In an April 2019 study published in JAMA (doi: 10.1001/jama.2019.3241), researchers demonstrated CGDB could be used to recapitulate previously described associations between clinical and genomic characteristics, driver mutations, and response to targeted therapy, and tumor mutation burden (TMB) and response to immunotherapy. 

Patients in the study all had non-small cell lung cancer, but similar replication studies can be expected moving forward to demonstrate the clinical utility of the database for other types of cancer, says Braly. “We’ll also try to assess over time how generalizable the [CGDB] database is relative to the broader population and to understand where there may be limitations we should recognize in our analyses.” 

The stakeholders that stand to benefit from the CGDB include clinicians trying to make better treatment decisions and biopharmaceutical companies seeking to improve the speed and efficiency of drug development, including clinical trial planning and execution as well as drug regulatory approval, says Braly. “We also look to CGDB to help us advance our own biomarker and diagnostic assay research and development efforts.” 

“There are multiple places where we might be able to make an impact with our biopharma partners,” Braly says. These include identifying populations who are super-responders or non-responders to therapy, and understanding the genetic basis of that response, as well as supporting potential regulatory submissions and the search for new indications for existing drugs. 

The use of RWD for the comparator arm of single-arm studies is “an area of ongoing and specific interest” to Foundation Medicine and partner Flatiron Health, Braly says. “It’s a compelling opportunity, if only to reduce the impact on patients of control arms in oncology drug development.”  

In addition to providing RWD for the control arm in clinical studies, the CGDB might also help support regulatory submissions by demonstrating evidence of an unmet need or characterizing a population in whom a drug’s effectiveness is being evaluated, Braly continues.  

Tokenized Data 

Since 2016, Flatiron Health has been collaborating with the FDA to help the agency better understand how RWE, derived from de-identified patient datasets curated from EHRs, can support regulatory decision-making. It has also been assisting biopharma companies in their efforts to incorporate RWE into regulatory filings for post-marketing studies and label expansions. 

“Foundation Medicine has a long, productive history of working with the FDA and we expect that to continue as we start to build out our capabilities in this space,” says Braly. “Transparency and guidance from the FDA will go a long way toward helping stakeholders—including Foundation Medicine, Flatiron Health and our biopharma partners—understand the ways we can best collaborate on real-world evidence for regulatory submission purposes.” 

Foundation Medicine’s FoundationCORE database holds more than 350,000 patient records and Flatiron Health’s oncology-specific EHR platform represents more than two million cancer patients being treated at 275 cancer clinics nationwide, Braly says. At present, the CGDB includes linked, de-identified data on more than 50,000 patients—demographic, medication history, laboratory testing, outcomes (including survival), genomic findings, variant annotations, and computational biomarkers such as TMB. 

The patients represented in CGDB all have advanced cancer, Braly adds, because that is typically what qualifies them to receive genetic testing and have the cost covered by payers. Foundation Medicine’s FDA-approved test is covered for eligible Medicare beneficiaries and at varying levels by private insurers. 

Tokenization is used whenever patient information is shared, to safeguard data security and privacy, says Braly. The process turns patient information into de-identified tokens—hashed patient identifiers derived from, but not including, protected health information. 

The tokens get “deterministically generated from each patient’s demographic data and overlapped by a third party, Management Science Associates (MSA),” Braly explains. “Deidentified clinical and genomic data are submitted independently to MSA for linkage using the tokens without accessing personal health information. The linked, deidentified database are transferred for analysis with new deidentified tokens replacing the original tokens, preventing relinking to internal identified data sets.”