This course explores testing strategies and study design considerations for diagnostic AI models requiring premarket clearance by regulatory bodies like the USA FDA, EU MDR, and those adhering to ISO 13485 standards. Focusing on detection AI, the training covers bench and clinical testing, emphasizing their roles in bias mitigation, clinical effectiveness, and regulatory compliance.

Key topics include the purpose of bench testing for performance assessment and design changes, and clinical testing (reader studies) for clinical effectiveness. The course also covers test scope, MRMC study design, acceptance criteria, statistical techniques such as AFROC, sensitivity, specificity, sample size rationales, and bias controls. Additionally, it provides an overview of regulatory guidance, including AAMI 34971 and FDA Good Machine Learning Practices, stressing the importance of early test protocol development in the design and submission process.

Over the course of four (4) hours, the attendee/participant will be able to:

• Develop a diagnostic AI V&V test protocol and begin the outline of their own test protocol from their jobs from the case study portion of the training.
• Apply regulatory requirements and considerations in their test protocol.
• Understand the study design and statistical techniques used in these protocols.