Studies Supporting PDTs Often Short on Rigor, Inclusivity, Research Finds

The evidence supporting the safety and efficacy of 20 FDA-authorized prescription digital therapeutics (PDTs) is spotty, with 40% having no clinical studies corresponding with their FDA authorization and only 2 were supported by studies that met the criteria for quality research, according to a study published in this month’s issue of the

Corresponding author Sanket S. Dhruva, M.D., M.H.S, said in an interview with Managed Healthcare Executive that the study’s finding was not that the evidence is uniformly poor. Dhruva and his colleagues found, for example, that 90% of the PDTs were supported by one or more randomized controlled trial and all of them by studies with a clinical primary end point.The shortcoming, he said, was that most studies of the studies supporting PDTs were missing elements that would qualify them as producing rigorous evidence,

“It’s important to conduct a randomized trial, in multiple centers, that has, ideally, double blinding, as well as demographic factors that includes diverse patient populations,” he said. “We found that all of those didn’t package together as often as we’d like to see in order to be able to consider the evidence as reliable, as sturdy as possible, to have causal inference— that is, does the prescription digital therapeutic do what it is intended to do.”

Dhruva said individual PDTs would need to be judged on a case-by-case basis but that most of the PDTs in this review lacked the “rigorous evidence base that gives us confidence that if we see a positive result on a meaningful clinical end point, we can say, we believe, this really works.”

Dhruva, an assistant professor at the University of California, San Francisco (UCSF) School of Medicine and UCSF Medical Center, and his colleagues, also found that almost half (46.9%, or 46 of 98) of the nonpediatric studies conducted of PDTs had an upper age limit; the median of those limits was age 75. They noted in the discussion section of their Health Affairs article that with Congress considering a prescription digital benefit category for Medicare and Medicaid beneficiaries “prescription digital therapeutic studies could be required to demonstrate improved outcomes in populations representative of these beneficiaries.”

The researchers found that more than half of the studies did not have data on the study volunteers’ race and more than 80% did not report on their ethnicity.

Dhruva complimented FDA regulators working under the current rules that regulate PDTs as “software as a medical device,” but he said a separate FDA process for PDTs would help centralize information about the PDTs and increase the understanding of their safety and efficacy.

Dhruva conducted their study by identifying 20 PDTs that had been FDA authorized as of Nov. 29, 2022. They used a variety of source to identify them, including Express Scripts and CVS Caremark formularies, the Digital Therapeutics Alliance product library and an FDA product codes. None of the 20 had been authorized through the agency’s premarket approval, which has the highest requirements for evidence and is reserved for high-risk devices. Thirteen were authorized through the 501(k) pathway, which is used for low- to moderate-risk devices that the agency deems similar to previously approved devices. According to Dhruva and his colleagues, manufacturers are not required to provide clinical data but the FDA may ask for it. The other seven PDTs in the study were approved in the De Novo pathway, which is for low- to moderate-risk devices that don’t have FDA-approved predecessors. The most common indication was diabetes; 25% of the PDTs had a diabetes indication.

Their dragnet for studies related to the PDTs included searches of the FDA decision summaries related to the agency’s authorizations, ClinicalTrials.gov and Pubmed. They also searched the websites of the manufacturers for mentions of the studies. The end result was a cache of 117 studies of the 20 PDTs that could be analyzed for their strengths and weakness.

As Dhruva noted, the results were mixed. For example, three-quarters (59 of 79) of the studies that had been completed or terminated had results published in a peer-reviewed journal. Of the 179 primary end points studied, 83.2% (149) were clinical outcomes.

The shortcomings of the studies that Dhruva and his colleagues identified were lack of blinding: just 30% of the clinical studies of the PDT used blinding. They noted that although 40% of the PDTs were studied in randomized controlled trial, more than 60% did not report results on ClinicalTrials.gov.

When they organized the studies on per PDT basis, they found some positives: 18 of the 20 PDTs had been tested in at least one randomized controlled trial and 16 were evaluated in at least completed study were all the primary end points. But they also found that eight PDTs had not been evaluated in a blinded study and only two had a study that hit all the mark of a high -quality study: randomized, blinded, multicenter study, no language proficiency requirement, clinical primary end point and all the primary endpoints were met.