Authored by Ruby Gadelrab and Greg Nagy

For years, we’ve emphasized the urgent need for standardized assessments in digital health. Currently, there’s no central resource to gauge the clinical outcomes and ROI of digital health interventions. As the digital health sector holds immense potential to revolutionize healthcare, it’s imperative to establish clear standards and assessments to validate efficacy.  This is particularly important in a time where larger healthcare stakeholders such as payors, employers and health systems are experiencing “vendor fatigue” and frustrations with point solutions and as a result, are being more discerning about which innovations they are willing to pilot.   

The recent assessment conducted by the Peterson Health Technology Institute (PHTI) on digital diabetes management tools exemplifies this need. The study evaluated various digital health solutions catering to individuals with Type 2 diabetes, including offerings from prominent companies like DarioHealth, Glooko, Omada, Perry Health, Teladoc (Livongo), Verily (Onduo), Vida Health, and Virta Health.

PHTI’s analysis aimed to determine whether these digital tools deliver significant clinical benefits and contribute to cost savings within the healthcare system. Led by a team of health technology assessment experts and informed by clinical advisors, the evaluation underscored several key findings:

1. Limited Clinical Benefits: The study revealed that digital diabetes management tools often fall short in delivering substantial clinical benefits and may even lead to increased healthcare spending. It highlighted limitations in improving glycemic control and questioned the tools’ impact on patient health outcomes, including cardiovascular risks.

2. Economic Concerns: Additionally, concerns were raised about the economic impact of these tools, suggesting that they may not generate cost savings for payers as initially anticipated. Despite significant investments in digital diabetes care, the study suggests a need for a shift towards innovative solutions that can deliver measurable improvements in patient care while remaining cost-effective.

While PHTI’s approach is important, their findings have also prompted some critical analysis from industry insiders:

  • Clinical Voices and Frontline Experience: The absence of direct input from endocrinologists and other clinicians in the study’s methodology is one of the most significant points of criticism. Given their frontline experience in diabetes care, incorporating clinicians’ insights is essential for a holistic understanding of how digital tools impact patient care and outcomes. Their exclusion calls into question the study’s conclusions and highlights the need for a more inclusive evaluative framework that values and integrates clinical perspectives.  We at MDisrupt believe that clinicians (particularly those treating a condition and utilizing the tools) need to be a key part of any assessment or evaluation.   
  • Company Involvement and Collaboration: A notable critique centers on the study’s methodology, particularly regarding the engagement of digital diabetes companies. Transparent collaboration between researchers and these companies is crucial for a fair and comprehensive assessment. Such collaboration would ensure that the evaluation accurately includes the latest data and studies which may not have yet been published.  Respondents from the companies mentioned have highlighted that recent data showing positive outcomes from their digital diabetes management solutions were not considered by the study. This points to the need for active engagement and collaboration between the companies and the assessors. 
  • Beyond A1C Levels: Criticisms also target the study’s focus on A1C levels as the primary metric for evaluating digital diabetes tools. This narrow focus overlooks the multifaceted nature of diabetes management, which extends beyond mere A1C reduction to include broader health impacts such as a reduction in hyperglycemic and hypoglycemic events, weight management, hypertension and other comorbidities.  Effective diabetes management involves addressing the diverse challenges and goals inherent in chronic disease management. This critique underscores the necessity of adopting a more comprehensive view in evaluating digital diabetes solutions.
  • User Engagement Consideration: Digital health tools are only effective if they are actively used by patients. Critics have observed that the study methodology seems to disregard the crucial element of heightened user engagement within a comprehensive care model incorporating digital health interventions. Failing to account for this factor could result in an incomplete or inaccurate assessment of cost-effectiveness.
  • Assessment of Quality Metrics: Effective diabetes management encompasses more than just glycemic control; it involves assessing various quality metrics like patient satisfaction, overall well-being, quality of life, and the mitigation of diabetes-related complications. Failing to assess these broader outcomes could lead the study to overlook crucial aspects of the benefits offered by digital health tools. Furthermore, as highlighted by industry critics, healthcare resource utilization data (HCRU) are deemed superior to predictive models in determining actual cost savings.

We believe the inclusion of clinician voices is crucial in the assessment of any digital health innovation.  Clinicians can provide a real world perspective,  they are closest to the patients and they are the ones who can advise on how these products should be holistically evaluated.  Furthermore they can evaluate how an innovation fits into a clinical workflow, the types of patients that see success, which can be a key factor to an innovation being adopted broadly. 

Incorporating these industry critiques, it’s evident that while the PHTI study offers valuable insights, the conversation is far from complete.  Collaborative research methodologies, the inclusion of clinical voices, the cooperation of the digital health companies and  a broader evaluative lens, are crucial if we are to be able to assess digital health innovations in an objective and clinically meaningful way.

At MDisrupt we believe that the most impactful health products should make it market quickly. We do this by uniting digital health companies with experts from the healthcare industry to help them accelerate their time to market responsibly.

Our expert consultants span the healthcare continuum and can assist with all stages of health product development: This includes regulatory, clinical studies and evidence generation, payor strategies, commercialization, and channel strategies. If you are building a health product, talk to us.