Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations

Appendix E Example Model Card

Sponsors must provide comprehensive documentation on device user interface, labeling, AI architecture, development and validation data, model training methods, risk assessment, and postmarket monitoring to FDA marketing submissions for AI-enabled devices. Documentation must address the degree of automation, model inputs and outputs, training and test data sources and representativeness, performance validation with subgroup analyses, cybersecurity risks including data poisoning and model theft, and plans to monitor performance changes post-launch, with information detailed enough that competent AI practitioners could reproduce equivalent models and sufficient for FDA to determine safety and effectiveness. Device labeling must explain AI involvement and how users should interpret outputs at appropriate reading levels, validation must demonstrate users can comprehend and correctly use device information in real-world conditions, and submissions should identify populations at risk of underperformance, justify data characteristics and management differences between development and validation phases, and include baseline study population distributions, statistical analysis plans with pre-specified success criteria and subgroup analyses by sex, age, race, ethnicity, and disease markers, with powered subgroup analysis required when specific performance claims are made for subgroups.

Appendix B Transparency Design Considerations

Sponsors are recommended to integrate transparency considerations throughout the product lifecycle starting in the design phase by identifying what information users need, when they need it, and what risks arise from missing or misunderstood information, while recognizing that transparency about significant clinical risks alone may not be adequate control. When designing communication, sponsors should consider user characteristics including functional capabilities, experience and education levels, interaction frequency, expected training, and familiarity with similar information, and select communication timing, mode, and format based on reading level, information location, and design elements such as hierarchy and visualizations. Sponsors should develop and validate explainability tools or visualizations through testing to ensure intended users understand key information and avoid misleading users.

Appendix C Performance Validation Considerations

Appendix D Usability Evaluation Considerations

Sponsors must perform human factors evaluations during design controls for all devices needing premarket submission; if critical tasks exist, formal validation is required, but sponsors without critical tasks may use alternative methods. For AI-enabled devices, sponsors should demonstrate that intended users can find, understand, and appropriately apply device information including AI outputs in clinical decision-making, considering that explainability varies from decision trees to deep neural networks; task analysis techniques and risk documentation of misuse and misinterpretation are recommended, with Appendix B transparency design recommendations available to identify user tasks and develop controls. Model cards are a recommended but not required format covering device identification, intended users and use, clinical benefits, inputs and outputs, performance metrics (sensitivity, specificity with confidence intervals), known biases and data gaps, risk management, deployment and update procedures, and software quality, with sponsors documenting development data sources (such as clinical trials or databases), data format categories, number of observations, statistical magnitude, quality aspects, comparison standard, population variation, and suitability to intended use.

Appendix F Example 510(k) Summary with Model Card

The Disease X Screening Model version 1.0.1, a convolutional neural network for patients above age 22 using 12-lead ECG from four compatible devices, was validated through a retrospective study of 25,000 patient records (eligible patients aged over 22 with 10-second ECGs and echocardiograms within 30 days, excluding those under 22, with missing data, using different ECG models, with non-10-second ECGs, or with pacemakers) and training on 30,000 patients (20% with Disease X, 80% without), achieving sensitivity of 84%, specificity of 83%, positive predictive value of 55.3%, and negative predictive value of 95.4% in detecting Disease X with subgroup analyses by demographics, ECG device models (A, B, C, D), and hospital sites (A through E). The model is not a stand-alone diagnostic tool; positive results should prompt further clinical evaluation such as echocardiography, and negative results should not exclude additional cardiac evaluation if high-risk factors exist. Deployment requires monitoring through site-specific validation testing, automated performance monitoring every 6 months, transparent reporting of successes and failures across sites, change management procedures, and user reporting of issues within 48 hours; publicly available summaries must follow applicable marketing submission requirements and may include technological characteristics, indications for use, and device description, with the FDA able to request additional information beyond examples provided.