(820) Assessing the Accuracy and Safety of ChatGPT Responses to Common Questions About Preeclampsia in Pregnancy

To evaluate the accuracy, completeness, and safety of ChatGPT-generated responses to common patient and provider questions about preeclampsia (preE) in pregnancy, assessing its potential role as an educational and triage-support tool in obstetric care.

Study Design:

Clinically relevant questions were developed from ACOG, SMFM, and WHO guidelines, as well as frequently discussed topics in patient forums and prenatal education resources. Responses were generated using GPT‑4 and independently evaluated by five maternal‑fetal medicine (MFM) specialists. Reviewers rated each response on a 5‑point Likert scale for accuracy, completeness, and safety. Two standardized prompts simulated patient- and provider-directed queries. Questions were grouped into four categories: General Knowledge, Public Health/Prevention, Management & Treatment, and Diagnostic Interpretation.

Results:

Across all domains, ChatGPT responses demonstrated high performance. Mean ratings (1–5 scale) were 4.60 for accuracy, 4.29 for completeness, and 4.43 for safety. The proportion of evaluations rated ≥4 was 92% for accuracy, 86% for completeness, and 85% for safety. Public Health/Prevention questions achieved the highest ratings across domains. Management & Treatment responses showed good alignment with guideline recommendations. Diagnostic Interpretation responses were generally accurate and safe but had slightly lower completeness, reflecting less detail in complex clinical decision-making. No unsafe or potentially harmful content was identified.

Conclusion:

ChatGPT generated safe, accurate, and generally complete responses to preE-related questions. Its high accuracy and safety, particularly for public health and patient education topics, suggest potential for integration into prenatal counseling and triage support. Opportunities remain to enhance completeness in complex diagnostic contexts while maintaining its strong safety profile.