OpenAI's ChatGPT is no closer to replacing your family physicians, as the increasingly advanced chatbot failed to accurately diagnose the vast majority of hypothetical pediatric cases.
The findings were part of a new study published in JAMA Pediatrics on Jan. 2, conducted by researchers from Cohen Children's Medical Center in New York. The researchers analyzed the bot's responses to requests for medical diagnosis of child illnesses and found that the bot had an 83 percent error rate across tests.
The study used what are known as pediatric case challenges, or medical cases originally posted to groups of physicians as learning opportunities (or diagnostic challenges) involving unusual or limited information. Researchers sampled 100 challenges published on JAMA Pediatrics and NEJM between the years 2013 and 2023.
ChatGPT provided incorrect diagnoses for 72 out of 100 of the experimental cases provided, and generated 11 answers that were deemed "clinically related" to the correct diagnosis but considered too broad to be correct.
The researchers attribute part of this failure to the generative AI's inability to recognize relationships between certain conditions and external or preexisting circumstances, often used to help diagnose patients in a clinical setting. For example, ChatGPT did not connect "neuropsychiatric conditions" (such as autism) to commonly seen cases of vitamin deficiency and other restrictive-diet-based conditions.
The study concludes that ChatGPT needs continued training and involvement of medical professionals that feeds the AI not with an internet-generated well of information, which can often cycle in misinformation, but on vetted medical literature and expertise.
AI-based chatbots relying on Large Language Models (LLMs) have been previously studied for their efficacy in diagnosing medical cases and in accomplishing the daily tasks of physicians. Last year, researchers tested generative AI's ability to pass the three-part United States Medical Licensing Exam It passed.
But while it's still highly criticized for its training limits and potential to exacerbate medical bias, many medical groups, including the American Medical Association, don't view the advancement of AI in the field just as a threat of replacement. Instead, better trained AI's are considered ripe for their administrative and communicative potential, like generating patient-side text, explaining diagnoses in common terms, or in generating instructions. Clinical uses, like diagnostics, remain a controversial, and hard to research, topic.
To that extent, the new report represents the first analysis of a chatbot's diagnostic potential in a purely pediatric setting acknowledging the specialized medical training undertaken by medical professionals. Its current limitations show that even the most advanced chatbot on the public market can't yet compete with the full range of human expertise.
Originally posted here:
ChatGPT is terrible at diagnosing child medical cases, according to new study - Mashable
- Announcing Lau as interim chair of the Department of Pediatrics - Heersink School of Medicine News | UAB - University of Alabama at Birmingham - November 12th, 2024 [November 12th, 2024]
- Implementing artificial intelligence and the importance of transparency for patients - Contemporary Pediatrics - October 26th, 2024 [October 26th, 2024]
- Pediatrics with collaborative care explored in CDA Journal collection - CDA (California Dental Association) - October 13th, 2024 [October 13th, 2024]
- PAK Pediatrics offering drive-thru flu clinics - Yahoo Life - October 13th, 2024 [October 13th, 2024]
- Nonoperative treatment emerging for pediatric appendicitis - Contemporary Pediatrics - October 13th, 2024 [October 13th, 2024]
- Rep. Paul Lee honored as 'Legislative Hero' by American Academy of Pediatrics - WTVY - October 13th, 2024 [October 13th, 2024]
- Pediatrics Leveraging Yosi Health for Mental Health and Other Screenings to Ensure Healthy Children and Parents - Healthcare IT Today - October 2nd, 2024 [October 2nd, 2024]
- Chroming trend among teens linked to TikTok videos - Contemporary Pediatrics - October 2nd, 2024 [October 2nd, 2024]
- Water-absorbing beads pose growing threat to children, new study highlights - Contemporary Pediatrics - October 2nd, 2024 [October 2nd, 2024]
- Steven Selbst, MD: 'We have to be ready' to respond, help in mass shooting situations - Contemporary Pediatrics - October 2nd, 2024 [October 2nd, 2024]
- Pediatrics academy accused of fearmongering over GMO ingredients in kids diets - Science - September 23rd, 2024 [September 23rd, 2024]
- XpertDox Partners with Blue Fish Pediatrics to Enhance Medical Coding Efficiency with AI-Powered Healthcare Coding Software - 24-7 Press Release - September 23rd, 2024 [September 23rd, 2024]
- Exploring associations of inpatient food insecurity and pediatric hospitalizations - Contemporary Pediatrics - September 23rd, 2024 [September 23rd, 2024]
- Infants' tongue-tie may be overdiagnosed and needlessly treated, pediatrics group says - Tulsa World - August 5th, 2024 [August 5th, 2024]
- Infants' tongue-tie may be overdiagnosed and needlessly treated, American Academy of Pediatrics says - The Associated Press - August 5th, 2024 [August 5th, 2024]
- Infants tongue-tie may be overdiagnosed and needlessly treated, American Academy of Pediatrics says - Drgnews - August 5th, 2024 [August 5th, 2024]
- Pediatric patients with uveitis at elevated risk of cataracts, study finds - Contemporary Pediatrics - July 26th, 2024 [July 26th, 2024]
- Finding the Signal in the Noise on Pediatric Gender-Affirming Care - The Hastings Center - June 27th, 2024 [June 27th, 2024]
- Texas abortion law was followed by a rise in infant deaths: study | STAT - STAT - June 27th, 2024 [June 27th, 2024]
- Generation Next Dr. Lisa Costello, 37, Assistant Professor of Pediatrics, WVU - WV News - June 27th, 2024 [June 27th, 2024]
- Local pediatricians tips on keeping kids active and eating healthy this summer - WTOP - June 27th, 2024 [June 27th, 2024]
- Discussing health care sustainability, climate change, and WHO's One Health goal - Contemporary Pediatrics - June 27th, 2024 [June 27th, 2024]
- Pediatric chiropractic boosta physical, emotional well-being - The Times of Northwest Indiana - June 27th, 2024 [June 27th, 2024]
- Ask the Pediatrician: How firearm safety begins at home - Union Democrat - June 27th, 2024 [June 27th, 2024]
- Expert consensus and guidance for integrating discussions of racism in pediatric care - News-Medical.Net - June 27th, 2024 [June 27th, 2024]
- JAMA Pediatrics Published Study Rise in Infant Deaths in Texas to Abortion Ban - ASEAN NOW - June 27th, 2024 [June 27th, 2024]
- Supreme Court Will Take Up State Bans on Gender Care for Minors: Here's What to Know - Medpage Today - June 27th, 2024 [June 27th, 2024]
- Weekly review: Drowning prevention, coding updates, and more - Contemporary Pediatrics - June 9th, 2024 [June 9th, 2024]
- A call to action for pediatrics: Caring for students with learning disabilities in custody and community settings | Pediatric ... - Nature.com - June 9th, 2024 [June 9th, 2024]
- Milestones Pediatrics begins summer camp with sports at Rec Center - The Post-Searchlight - Post Searchlight - June 9th, 2024 [June 9th, 2024]
- Pediatric Associates to Deploy Innovaccer's AI Platform to Enhance Quality of Care for 1.5 Million Patients - HIT Consultant - June 9th, 2024 [June 9th, 2024]
- With drownings on the rise, prevention conversations with caregivers are crucial - Contemporary Pediatrics - June 9th, 2024 [June 9th, 2024]
- Chief Nursing Officer Honored with Miracle Maker Award - URMC - June 9th, 2024 [June 9th, 2024]
- EEG and ECG are overused in children with breath-holding spells - Contemporary Pediatrics - June 9th, 2024 [June 9th, 2024]
- New Close To Home Center to Open in Grove City - Newswise - June 9th, 2024 [June 9th, 2024]
- Dr. Robert Wilmott, pediatrics chair who gave parents advice as 'Dr. Bob,' dies at 75 - St. Louis Post-Dispatch - May 24th, 2024 [May 24th, 2024]
- American Academy of Pediatrics launches 'Rx4DC' initiative to address gun violence - DC News Now | Washington, DC - May 24th, 2024 [May 24th, 2024]
- Utah Valley Pediatrics expands to Sanpete County by opening Ephraim location - Daily Herald - May 24th, 2024 [May 24th, 2024]
- Pruitt named director of academic pediatrics division - The Source - Washington University in St. Louis - May 24th, 2024 [May 24th, 2024]
- Integrating behavioral health within primary care settings - Contemporary Pediatrics - May 24th, 2024 [May 24th, 2024]
- Phones and kids: new pediatric guidelines, expert advice and info on new school rules - Kidsburgh - May 24th, 2024 [May 24th, 2024]
- Suicide: Blueprint for Youth Suicide Prevention - American Academy of Pediatrics - May 24th, 2024 [May 24th, 2024]
- Dr Julie Linton Addresses Disparities in Pediatric Care for Immigrant Children in the US - AJMC.com Managed Markets Network - May 24th, 2024 [May 24th, 2024]
- CAR T-Cell Therapies Move Ahead in Pediatric and Adult ALL - OncLive - May 24th, 2024 [May 24th, 2024]
- American Academy of Pediatrics reverses long-standing ban on breastfeeding for mothers with HIV - WION - May 24th, 2024 [May 24th, 2024]
- In a Shift, Pediatricians' Group Says Breastfeeding Safe When HIV-Positive Mom Is Properly Treated - jacksonprogress-argus - May 24th, 2024 [May 24th, 2024]
- US Pediatricians Group Reverses Decades-Old Ban On Breastfeeding For Those With HIV - 1340 WJOL - 1340 WJOL - May 24th, 2024 [May 24th, 2024]
- Sang Pediatrics: When to see the doctor for a fever - YourCentralValley.com - May 12th, 2024 [May 12th, 2024]
- Should You Put Sunscreen on Infants? Not Usually - FDA.gov - May 12th, 2024 [May 12th, 2024]
- Thousands of bike helmets to be distributed to Ohio children | 10tv.com - 10TV - May 12th, 2024 [May 12th, 2024]
- How to talk to your teenaged patient about pregnancy and contraception - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Adult RSV Shots Given to More Than 30 Babies by Mistake - Medpage Today - May 12th, 2024 [May 12th, 2024]
- Pediatric Health Care Teams Can Help Prevent Youth Suicide - The Pew Charitable Trusts - May 12th, 2024 [May 12th, 2024]
- Small patients, big discoveries | Penn Today - Penn Today - May 12th, 2024 [May 12th, 2024]
- HCPs can play an important role in addressing the US infant formula shortage - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- East Wenatchee Doctor Awarded For Boosting Child Vaccinations - KPQ - May 12th, 2024 [May 12th, 2024]
- HPV vaccination coverage of US-children aged 9-17 years in 2022 - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Weekly review: Profound genetic deafness gene therapy, measles increases, and more - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Pediatric Telehealth Platform Market is expected to Expand at a Massive CAGR of 26.6% through 2030 - openPR - May 12th, 2024 [May 12th, 2024]
- DB-OTO improved hearing to normal in child with profound genetic deafness - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- The IT strategy behind a groundbreaking new $2B pediatric hospital - Healthcare IT News - May 12th, 2024 [May 12th, 2024]
- Modest Changes in Kids' Developmental Scores Seen From Pandemic - Medpage Today - April 23rd, 2024 [April 23rd, 2024]
- This pediatrics subspecialty has the best work-life balance - MD Linx - April 23rd, 2024 [April 23rd, 2024]
- PM Pediatrics Cobble Hill closing Pediatric urgent care options in Brooklyn and lower Manhattan - Brooklyn Bridge Parents - April 23rd, 2024 [April 23rd, 2024]
- Bethesda Pediatrics awarded $18,000 grant by GIVE - Yahoo! Voices - April 23rd, 2024 [April 23rd, 2024]
- Nemours Children's Health, UCF Partner to Elevate the Quality of Healthcare for Florida's Children - UCF - April 23rd, 2024 [April 23rd, 2024]
- Pediatricians told to talk to patients and parents about climate. - Grist - April 23rd, 2024 [April 23rd, 2024]
- Getting a good night's rest is vital for neurodiverse children pediatric sleep experts explain why - The Conversation - April 23rd, 2024 [April 23rd, 2024]
- Tina Cheng Honored as Exceptional CMO - Research Horizons - Research Horizons - April 23rd, 2024 [April 23rd, 2024]
- Let's talk about childhood injury prevention | Hub - The Hub at Johns Hopkins - April 23rd, 2024 [April 23rd, 2024]
- More young people with depression, anxiety in Virginia | Headlines - InsideNoVa - April 23rd, 2024 [April 23rd, 2024]
- Pediatrics fundraising event at Altru helps children with critical needs - Grand Forks Herald - April 15th, 2024 [April 15th, 2024]
- Public perceptions of families affected by pediatric cancer and educational work in pediatric oncology | Pediatric ... - Nature.com - April 15th, 2024 [April 15th, 2024]
- The pediatrician, the resident, the strawberry monster, and the zebra - VUMC News - VUMC Reporter - April 15th, 2024 [April 15th, 2024]
- Receptor targeting to identify medications for pediatric mental health concerns - Contemporary Pediatrics - April 15th, 2024 [April 15th, 2024]
- Hypoglossal nerve stimulation offers hope for pediatric Down syndrome patients with OSA - News-Medical.Net - April 15th, 2024 [April 15th, 2024]
- Efforts to reduce wait times for developmental evaluations - Baylor College of Medicine | BCM - April 15th, 2024 [April 15th, 2024]
- Nasal oxidative stress mediating the effects of colder temperature exposure on pediatric asthma symptoms | Pediatric ... - Nature.com - April 15th, 2024 [April 15th, 2024]
- Week in review: A new dermatology case study, and a new treatment for severe asthma - Contemporary Pediatrics - April 15th, 2024 [April 15th, 2024]
- Chordae Tendinea Rupture and Pulmonary Valve Vegetation in Pediatric Endocarditis: A Rare Condition - Cureus - April 7th, 2024 [April 7th, 2024]