TL;DR
Get health and wellness essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
At an ASRM meeting, researchers reported that physicians rated ChatGPT responses to IVF questions lower than ASRM educational materials on accuracy, completeness, safety and overall quality. The chatbot’s responses were rated similarly clear, but half included at least one unsupported claim and 4.2% contained misinformation in this small study.
A small comparison presented at the American Society for Reproductive Medicine (ASRM) annual meeting found that physicians rated ChatGPT’s answers to common IVF questions lower than ASRM patient-education materials for accuracy, completeness, safety and quality. The chatbot’s answers were rated nearly as clear as the society’s materials, but researchers also found unsupported claims in half of its responses and misinformation in 4.2%.
Nicole Fischer, MD, MPH, a reproductive endocrinology and infertility fellow at the University of Pittsburgh, presented the analysis. Four physicians rated the responses on a five-point scale. ChatGPT scored 3.98 for accuracy, 3.40 for completeness, 2.92 for safety and 3.38 for quality. ASRM materials scored 5.00, 4.88, 4.88 and 5.00, respectively. For clarity, the ratings were similar: 4.60 for ChatGPT and 4.63 for ASRM.
The team compared ChatGPT’s answers with two ASRM documents and checked whether responses covered key concepts in the society’s ReproductiveFacts.org materials. ASRM materials met all checklist items on average; ChatGPT responses averaged 94.1% checklist adherence. Half of the chatbot answers included an unsupported claim, while 4.2% contained misinformation; reviewers found none in the ASRM materials.
The researchers also asked ChatGPT to estimate IVF success for each age from 20 to 45. The answers generally followed CDC and Society for Assisted Reproductive Technology (SART) data in showing lower birth rates with increasing age, but the chatbot overestimated live birth rates at age 35 and underestimated them at ages 37 and 40. The report does not provide the size of those differences.
Clear Answers Can Still Miss Key Facts
The findings describe differences between chatbot responses and ASRM educational materials on questions about IVF, its risks and the likelihood of success. In this analysis, clarity ratings were similar, while ChatGPT scored lower on safety and completeness and some responses included unsupported statements. A clear response may still omit qualifications or contain claims that reviewers did not support.
The study did not examine how patients interpret chatbot responses or whether they affect treatment decisions. Fischer recommended that clinicians ask patients whether they have used AI for fertility information, discuss ways to check claims against reliable sources, and direct them to trusted educational materials. She also recommended teaching patients to ask AI tools to cite sources.
Products Worth Considering
Withings Body Smart Wi-Fi and Bluetooth Body Composition Scale
Levoit Core 300S-P Smart Wi-Fi Air Purifier
Oura Ring 4 Smart Ring – Silver – Size 8
As an affiliate, we earn on qualifying purchases.
How Researchers Tested IVF Answers
The team used the free GPT-5.6 Luna version of ChatGPT. For its comparison of common questions about the IVF process and associated risks, researchers ran each prompt three times, producing 12 responses. Four physicians reviewed those responses alongside two ASRM documents, scoring accuracy, completeness, clarity, safety and quality and examining checklist coverage, red flags and reading level.
Prompt wording affected readability. ChatGPT answers to conversational prompts were at about an eighth-grade reading level, while answers to generic prompts were closer to a 10th-grade level; the ASRM documents were around a 12th-grade level. Fischer said this suggests that how a patient asks a question can affect how readable the response is. The report was presented October 30, 2026, at ASRM’s annual meeting.
“Fluent, confident wording can coexist with omissions, outdated information, and errors.”
— Nicole Fischer, MD, MPH, reproductive endocrinology and infertility fellow at the University of Pittsburgh
Limits of This Small Comparison
The analysis had a small sample and tested only one version of one large language model. Its reviewers also used some subjective and descriptive measures, and the study did not examine patient outcomes or whether chatbot use affected medical decisions. The results cannot establish how every ChatGPT version, other AI systems or specialized health tools would perform.
The report says that 4.2% of ChatGPT answers contained misinformation but does not specify in the supplied material how many answers that percentage represents. It also does not give the exact CDC or SART figures used for each age or quantify the deviations at ages 35, 37 and 40. Those details limit what can be inferred from the success-rate comparison.
Further Testing and Patient Discussions
Fischer said newer and specialized systems, including ChatGPT Health, warrant testing. The supplied report does not identify a date or planned follow-up study. For now, she encouraged clinicians to ask patients about their use of chatbots, help them request cited sources and point them toward trusted fertility information such as ASRM materials.
Further research would need to test more models and versions, use larger sets of questions and examine whether answers are accurate and useful for patients in practice. Until then, the findings apply to the system and prompts tested—not to every AI response about IVF.
Key Questions
Did ChatGPT match ASRM materials on IVF questions?
No. Physicians rated ChatGPT lower for accuracy, completeness, safety and quality. Its clarity rating was similar to ASRM’s.
How often did reviewers find unsupported claims or misinformation?
Half of the ChatGPT answers contained at least one unsupported claim, and 4.2% contained misinformation. Reviewers found no misinformation in the ASRM materials.
Were ChatGPT’s IVF success estimates accurate?
For most ages from 20 to 45, the answers aligned with the general pattern in CDC and SART data, with birth rates declining as age increased. The chatbot overestimated live birth rates at age 35 and underestimated them at ages 37 and 40.
Can these findings be applied to every AI health chatbot?
No. The study tested one version of ChatGPT with a small set of prompts. The researchers said other versions and specialized models need further evaluation.
What did the study’s presenter recommend to clinicians?
Fischer encouraged clinicians to ask patients whether they use chatbots, show them how to request sources and direct them to trusted educational resources.
Source: medpagetoday.com
