By Deborah Borfitz

January 18, 2019 | For the last handful of years, the U.S. Food and Drug Administration (FDA) has issued a succession of guidance documents clarifying its views on Software as a Medical Device (SaMD). Collectively, they speak to the agency’s interest in deregulating low-risk SaMD products so as not to unnecessarily hinder innovation.

Congress moved in the same direction with a provision in the 21^st Century Cures Act exempting five categories of software from the definition of medical devices, says Bethany Hills, a healthcare and regulatory reimbursement attorney who chairs the FDA practice at law firm Mintz. These include electronic medical records, administrative software, products capturing only information about wellness, Medical Device Data Systems, and clinical decision support (CDS) software that allows providers to independently review the basis for recommendations. The FDA then updated guidance documents to align with the law.

In 2013, the International Medical Device Regulators Forum (IMDRF), a voluntary group of medical device regulators from around the world seeking global regulatory harmonization, convened a Software as a Medical Device Working Group chaired by the FDA. The group defined SaMD as "software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device."

The FDA has been “pretty consistently regulating standalone software [SaMD] for decades,” says Hills. In fact, the agency has stayed “well ahead of the curve” as SaMD moved beyond the radiological realm of image analysis to include smart phone apps evaluating patient symptoms and wearable sensors measuring sleep duration. It has issued about a dozen software-specific guidance documents on everything from mobile medical applications and application accessories to wellness apps and, most recently, devices posing cybersecurity risks that all spell out what gets regulated and with how much rigor.

A trio of new documents released January 7 suggests the FDA is trying to create a new oversight paradigm better aligned to the realities of software development without updating existing regulations or issuing new ones. The FDA has the statutory mandate to evaluate products prior to their marketing but has a lot of freedom in how to do that, notes Aaron Josephson, senior director of ML Strategies, the consulting and government relations arm of Mintz, and a former senior policy advisor at FDA’s Center for Devices and Radiological Health.

Rules created in the 1970s for hardware-based medical devices such as pacemakers and MRI machines don’t work for software that gets updated more or less continuously, Josephson says. The traditional 510(k) device review process is viewed as unnecessarily burdensome and out of step with how the software industry works, plus the agency has neither the expertise nor “boots-on the-ground” capacity to evaluate every single piece of software. It’s also unclear how reviewing a bunch of software code would “shine a light” on a product’s safety and effectiveness, he adds.

All Eyes On ‘Pre-Cert’ Program

The process of changing the rules picked up steam in 2018 when the agency released beta versions of a working model of a software precertification program and actively solicited and responded to public comments, Josephson says.

This month, the FDA formally launched its Software Precertification (Pre-Cert) Pilot Program, a new regulatory approach for SaMD meant to model “how the FDA has reimagined its way of regulating digital health products.” Nine companies involved in beta testing—including Apple, Fitbit, and Roche—will also participate in the pilot.

FDA’s working model describes the structure of the Pre-Cert program.

1. Excellence appraisal and two Pre-Cert levels determine how the FDA evaluates companies. Level 1 Pre-Cert applies to organizations determined by the FDA to be low risk with “limited or no experience in delivering software products.” Level 2 is for companies deemed to be low- to moderate-risk with a “proven record” and “extensive experience in delivering software products.” The agency proposes to evaluate an organization based on five Excellence Principles (product quality, patient safety, clinical responsibility, cybersecurity responsibility, and proactive culture). Questions remain about how this will be accomplished, says Josephson. For example, is it meaningful to evaluate product quality based on minimal bugs in the software or users not being constantly bombarded with error messages? Patient safety should be easier to gauge by looking at whether a developer addresses user safety concerns in subsequent iterations of the software and whether there are accountable parties for any software changes with potentially big safety impacts.
2. Pre-Cert level and device risk determines review pathway. No review will be required of initial “Type I” products (devices that drive clinical management for non-serious conditions and those that inform clinical management of serious or non-serious conditions) and “Type II” products (devices that treat or diagnose non-serious conditions, drive clinical management of serious conditions, or inform clinical management of critical conditions) that are manufactured by Level 2 Pre-Cert organizations, as well as many major changes subsequently made to those products. Review is also not required for any minor changes made to products by any Pre-Cert organization. The risk category framework for applications was developed by IMDRF.

The model leaves open plenty of questions. Pre-Cert relies more on a company’s “culture of quality and operational excellence” for approval than the efficacy and safety of the individual products it produces. Hills likens the strategy to allowing standup citizens to go through TSA PreCheck on behalf of their entire family.

Implementing the model will also require inspectors with expertise in organizational excellence. “What is the FDA going to do, hire the entire graduating class of Wharton Business School?” Hills wonders.

Documentation demands could be challenging for organizations unfamiliar with the medical device space, adds Scott Thiel, director of life sciences at Navigant Consulting. For its intended excellence appraisals, the FDA will require companies to generate and share an evidentiary dossier that could be an onerous task for those who have never compiled those types of records.

The actual review process remains a gray area, Josephson says. Among the current proposals are review of software code and test logs, hands-on product demos, and evidence of safety mitigation strategies where needed.

But if an organization proceeds through the process, the Pre-Cert program would apply to all of an applicant’s SaMD products, a “very different approach” than the FDA’s current process of evaluating one product at a time, Hills notes. One concern—as several prominent senators have already pointed out—is the legal authority FDA will use to fully implement the program, and how the agency otherwise intends to expedite some SaMD approvals.

“Our prediction is that there’s a low likelihood Pre-Cert will go anywhere fast,” says Hills. “There’s a new Congress and Democrats don’t really like this.”

Thiel, on the other hand, is “cautiously optimistic” about the Pre-Cert program and believes it’s a reasonable approach to accommodating the iterative, agile nature of software development while ensuring safe and effective products in the marketplace. “The devil is in the details,” including when the FDA should use the program for software apps and how to do so “within its available arsenal of regulations and responsibilities.” Software manufacturers can also expect to face execution challenges, he adds, since existing quality management systems were designed for more tangible types of medical devices with longer development timelines.

Getting Started With the De Novo Pathway

The FDA is clearly committed to the Pre-Cert program, based on the Working Model and two other new companion documents issued on January 7.

While “still looking” at whether it will ultimately need new authority, Josephson says, the FDA describes in the regulatory framework for the pilot program its position that the Pre-Cert program fits under the existing De Novo pathway for novel technologies.

Notably, FDA will establish a device classification known as a Pre-Cert 510(k) that would enable “excellence-appraised” sponsors to make 510(k) submissions relying on materials submitted during the Excellence Appraisal process and an optional Pre-Submission step. Thiel says the agency’s approach of using two available mechanisms—the De Novo pathway and Pre-Submission program—appears to be a response to criticism that the agency was overstepping its legal authority.

Testing The System

The agency will compare the streamlined De Novo pathway to its traditional submission pathway, an “engineering approach” similar to how software algorithms get confirmed, Thiel says. Information will be pulled from previous software approvals in a series of mock submissions under the new regulatory framework to determine if the information requirements are sufficient to make a proper decision.

Excellence appraisal will be the subject of much scrutiny, checking to see if the current criteria adequately map to the five Excellence Principles and suit different organizational structures. The FDA will also be collecting real-world information on the effectiveness and ease of the appraisal, as well as exploring how to prioritize the patient safety and clinical responsibility elements and assess organizations that produce software using artificial intelligence (AI) or machine learning (ML) algorithms.

The bottom line with the excellence appraisal, according to Thiel, is that companies need a well-established set of principles and controls and to “keep patient safety at the forefront” of all product-related decisions.

The FDA will also create a decision tree that organizations can use to determine if their SaMD products are regulated and, if so, in what risk categorization, as well as a framework for how major and minor software changes will be reviewed. FDA may end up requiring companies to summarize all the accumulated minor changes made to a device when a streamlined review is required for a subsequent major change, says Thiel. It is unclear how many minor changes might equate to a major change in the eyes of the agency.

Additionally, the FDA will conduct “after-action” reviews on any rejected submissions by pilot participants to identify gaps in supporting evidence and develop a process for reassessing Pre-Cert status. It will delineate agency and company responsibilities during the streamlined review process, an efficient method for conducting product demos, and a “robust training program” for FDA reviewers.

According to the working model, guidance will be developed for Pre-Cert organizations about appropriate types of analytics for verification of their ongoing commitment to excellence. Pilot participants will also provide the FDA with access to their real-world performance analytics for product monitoring to refine the agency’s framework and assess how well it would detect post-market signals.

Next Steps for SaMD Companies

The FDA is seeking feedback on the working model of its Pre-Cert program by March 8, 2019. But the comment period on the three documents will remain open as the agency continues to build and refine the program, says Thiel.

Software and start-up companies are largely taking a wait-and-see approach to the Pre-Cert program, Thiel says, though some wrongly believe it is a “free ticket to the market.” His best advice for organizations interested in the new product pathway is to implement a quality management system to document the good engineering and business practices they are likely already doing. They just need to start “capturing it in a way that is demonstrable and recoverable.”

Real-world performance needs to be integrated into the product development process, Josephson advises. This will require developers to continually collect and analyze data about how products are functioning in the market and the experience of end users, including physicians, nurses, and patients, he says. Companies need to have a process for applying user feedback to ongoing product development and ensure the team responsible for collecting real-world data is talking to the team writing software code.

Josephson further advises SaMD companies to “be part of the conversation” and to look at what the FDA is proposing. “[The FDA] wants the oversight model to succeed and work well for the software developers that will be subject to it, which requires [their] input.”

The FDA is aware that its own post-marketing surveillance system is immature and needs to move beyond passive reporting of adverse events and other information to also capture the voice of the “perfectly satisfied” who may not be inclined to submit a report, he says. Third-party evaluators may be needed, as could additional support for the evaluation system FDA has been championing for the past several years.

Defining Risk for CDS Software

Companies whose software is powered by AI or ML will have to wait for the agency’s yet-to-be-finalized clinical decision support guidance to learn how it draws the line between moderate- and high-risk software, Josephson says. His guess is that the former will be defined as products that make recommendations but require the signoff of a healthcare provider, while the latter will be software that makes decisions and automatically administers a prescribed therapy or treatment. The FDA has said Pre-Cert is for low- and moderate-risk software, he adds, so anything that is high-risk will likely be subject to the existing regulatory model.

The draft guidance has been widely challenged for its silence on the benefit/risk oversight approach it would take with CDS software. Of particular concern to Hills is that the FDA would regulate all CDS software powered by machine learning regardless of risk.

Based on existing guidance, Thiel says he suspects the complexity of ML and how it’s applied will determine the SaMD risk classification. A software app that does simple sorting and prioritization might well be exempt from regulation, for example.

The FDA has yet to be presented with any product that truly incorporates AI, says Hills, meaning software that’s “making its own characterizations, classifying things itself and changing its programming as it evolves.” The first company to do so will by default be highly regulated via the premarket approval or De Novo pathway because there would be no predicate device on which to base a 510(k) approval.