Skip to content
    Resources

    Methodology brief

    Evidence generation
    methodology in practice

    How we structure evidence programs for digital therapeutics, wearable biosensors, and clinical AI.

    Published January 2025 · HM Healthcare Partners

    PrefaceA methodology brief

    About this document

    Engagement specifics, including sponsor names, populations, sample sizes, and outcome figures, are shared under NDA in conversation, not in published material. This brief covers the methodology we apply, the frameworks we use, and the literature we draw on. Contact us to discuss projects relevant to your program.

    Evidence programs rarely run to plan. Regulatory, clinical, and commercial pressure shift the methods as you go, often faster than the team can keep up.[1] We work in digital therapeutics, wearable biosensors, and clinical AI. The regulatory pathways differ; the methods overlap.

    The chapters that follow describe the frameworks we apply across these domains, and what you would receive in an engagement.

    Chapter 1Digital therapeutics: the prescription evidence stack

    For digital therapeutics (DTx) targeting FDA classification (510(k), De Novo, or PMA) and payer coverage, we design phased evidence programs. A single-arm feasibility study tests whether the intervention and study procedures are workable and may identify preliminary signals for further evaluation. Those signals do not establish efficacy. Findings can inform the protocol and comparator for a subsequent randomized controlled trial.[10] Comparator options include a sham application, treatment as usual, or an active intervention; the choice depends on the research question.[2] We plan clinical endpoints, digital engagement metrics and implementation outcomes together at the start.[3]

    Why phased design matters

    We capture health economic endpoints (utilization, medication costs, productivity) inside the same RCT instead of a separate economic study. Payers then receive the economic evidence in the dataset that supports the regulatory submission, which can shorten the gap between approval and reimbursement.

    Chapter 2Wearable biosensors: validating digital endpoints

    For wearable biosensors used as endpoints in clinical trials, we apply the V3 framework (npj Digital Medicine, 2020).[5] Phase 1 addresses sensor verification and analytical validation through controlled studies comparing the wearable's measurements against gold-standard reference methods. Phase 2 covers clinical validation through prospective studies examining the correlation between wearable-derived metrics and established clinical outcomes.[4]

    The wearable validation trap

    V3 provides regulatory clarity but needs adaptation for any specific wearable. Motion artifacts, skin-tone variability, environmental conditions, battery dynamics, and the software-update lifecycle are rarely addressed in traditional validation protocols. Building those factors into the validation design upfront reduces the risk of a regulatory request for supplemental data.

    Chapter 3Clinical AI: evidence for decision support

    For AI-driven clinical decision support deployed in operational care settings, retrospective validation of the algorithm is not enough. Health systems require prospective evidence of clinical impact: clinician adoption, workflow integration, and patient outcomes.[6] We design stepped-wedge cluster randomized trials with sequential rollout, measure clinical outcomes alongside implementation outcomes using RE-AIM, and run CFIR-based pre-implementation assessments at each site to identify barriers and tailor deployment.[7]

    CFIR before deployment

    We run a pre-implementation site assessment before each clinical AI deployment. It scores each site on CFIR Inner Setting variables, and sites that score low receive targeted implementation support before go-live. The evaluation then reports site context separately from the performance of the technology.

    Chapter 4Clinical impact / medical education: audit and feedback as practice change

    For industry-sponsored medical education paired with clinical impact, the deliverable is no longer activity-level reports. ACCME and sponsor compliance teams want defensible evidence that the activity changed clinician behavior (Moore Level 5).[8] We design pre/post chart-abstraction clinical impact studies across participating sites. At each site we secure IRB approvals, execute BAAs and DUAs, and recruit a respected specialist as the local physician champion. Baseline chart abstraction against a standardized set of guideline-directed care variables anchors the audit and feedback intervention, delivered as a facilitated grand rounds session. Six months later we return for re-audit.[9]

    The local champion presents the data

    We ask each site to name a local champion who presents the baseline data. An outside facilitator supports the session and does not replace the champion. We decline engagements where a site cannot commit to supporting that champion through both the baseline and follow-up sessions. We analyze results by site and report the differences between sites.

    Chapter 5Cross-cutting principles

    Five principles apply across these four areas:

    Five principles

    1. Design for multiple audiences in the same study. Every study should generate evidence for at least two stakeholder groups (regulatory + payer, clinical + implementation). The marginal cost of adding endpoints is far less than the cost of a separate study.
    2. Measure implementation from the start. Implementation outcomes cannot be retrofitted. Embed RE-AIM or Proctor outcomes into the initial study design.
    3. Apply V3 rigorously for digital endpoints. Analytical shortcuts in early validation compound. Sensor verification and algorithm validation must precede any clinical claims.
    4. Use CFIR assessments before deployment. Pre-implementation site assessments identify barriers before they become failures.
    5. Build the economic case concurrently. Collect health economic evidence inside the clinical study so payers do not wait for a second study.

    References

    1. Digital health: a path to validation. NPJ Digit Med. 2019;2:38. doi:10.1038/s41746-019-0111-3
    2. Effect of a web-based cognitive behavior therapy for insomnia intervention with 1-year follow-up. JAMA Psychiatry. 2017;74(1):68–75. doi:10.1001/jamapsychiatry.2016.3249
    3. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. doi:10.2105/AJPH.89.9.1322
    4. Developing and adopting safe and effective digital biomarkers. NPJ Digit Med. 2019;2:14. doi:10.1038/s41746-019-0090-4
    5. Verification, analytical validation, and clinical validation (V3). NPJ Digit Med. 2020;3:55. doi:10.1038/s41746-020-0260-4
    6. Big data and machine learning in health care. JAMA. 2018;319(13):1317–1318. doi:10.1001/jama.2017.18391
    7. Fostering implementation of health services research findings into practice. Implement Sci. 2009;4:50. doi:10.1186/1748-5908-4-50
    8. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1–15. doi:10.1002/chp.20001
    9. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259. doi:10.1002/14651858.CD000259.pub3
    10. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239. doi:10.1136/bmj.i5239
    Contact details

    Provide contact details only. Do not enter patient information or health details. We use your details to handle this request. Privacy policy.

    Discuss your evidence program

    Book a call Take the readiness assessment