In recent months we have always talked about how theArtificial intelligence can be used in medicine.
Despite some comforting results from this point of view, great (and justified) doubts persist in this regard. Apparently, in fact, ChatGPT it’s a real disaster when used as pediatrician. According to a study published by the magazine JAMA Pediatricsil chatbot OpenAI turns out to be incompetent when it comes to treating young patients.
Some tests, it seems, have revealed that ChatGPT offers a paltry accuracy rate of 17% in pediatric medical cases. For the study, researchers compared the chatbot to 100 pediatric case challenges published in JAMA Pediatrics and NEJM between 2013 and 2023. All with results, as just stated, that are anything but flattering.
However, experts have underlined how, if used as support tool, the chatbot proves to be able to cover an essential role. In this sense, AI is seen as integrating and useful in any laboratory, albeit under the constant supervision of a professional.
ChatGPT pediatrician: effective in only 17% of cases, but everything could change in the future
But how did the test happen, in simple words?
The researchers reported the relevant text of the cases into the ChatGPT prompt, then two trained clinician-researchers rated the AI-generated responses as correct, incorrect, or not understood. In the latter case, ChatGPT identified a clinically related condition that was too broad or nonspecific to be considered correct.
As mentioned, the chatbot provided the right answer only in 17 cases out of 100. In 72 occasions, the output provided was considered incorrect by doctors while in 11 cases the answer was considered too vague. Despite that, there is no talk of a definitive rejection.
For researchers, in fact, with an adequate selective training, ChatGPT could become an efficient pediatrician. To achieve this goal, however, according to experts it would be necessary to feed the AI with notions that go beyond what is present online, focusing on scientific texts and the like.