(1082) A Novel Evaluation Method for LLM Contextual Prompting in a Virtual Maternity Setting

Objective:
The safety and efficacy of generative AI in clinical settings are critically dependent on the context provided to the model. Our goal was to develop and validate an evaluation framework to de-risk and improve the quality of an AI clinical co-pilot by simulating real-world conditions and comparing the impact of different contextual "prompting" strategies on the quality of AI-generated patient communications.

Study Design:
We selected 69 historical patient conversations on preeclampsia risk, labor, and pediatric respiratory concerns. Each conversation was truncated at a key clinical turn. An LLM then generated seven responses using varied strategies, from a base model with no context to prompts using published guidelines (e.g., ACOG), conversation-derived protocols, or retrieval-augmented generation (RAG) with institutional protocols combined with published guidelines. Three blinded, board-certified clinicians reviewed all 483 responses, rating each as "Best," "Acceptable," or "Unacceptable" (defined as clinically unsafe).

Results:
Analysis of 490 responses revealed context was a critical determinant of safety (Table 1). Strategies grounded in curated, institutional protocols performed best, yielding the lowest ‘Unacceptable’ rate (16%) and highest ‘Acceptable’ rate (49%). In contrast, the base model produced the most 'Unacceptable' responses (28%). Notably, AI-derived protocols from historical conversations alone also performed poorly (‘Unacceptable’ rate of 23%). Qualitative clinician feedback captured that unacceptable responses often failed to recognize clinical nuance or escalation needs, such as for symptoms of preeclampsia.

Conclusion:
Clinical AI safety and efficacy are a product of rigorous context engineering, not the model alone. This framework effectively validates safe prompting strategies before live deployment. Grounding generative AI in curated, institution-specific protocols is essential to minimize risk and is significantly safer than relying on unstructured historical conversations alone, ensuring high-quality patient communications.