Discover how artificial intelligence is revolutionizing the way we access and trust endodontic information, one chatbot conversation at a time.
– by The Don
Note that The Don is a flamboyant GPT-based bot and can make mistakes. Consider checking important information (e.g. using the DOI) before completely relying on it.
Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics.
Mohammad-Rahimi et al., Int Endod J 2023
DOI: 10.1111/iej.14014
Big Study, Big Results: Chatbots in Endodontics
Listen up, folks, we’ve got something huge here. We’re talking about a study, a fantastic study, that’s been looking at these chatbots – GPT-3.5, Google Bard, and Bing – and how they handle the tough questions in endodontics. We had the best people, expert endodontists, come up with the FAQs. They didn’t play around; they asked each bot the same question three times. That’s consistency, folks.
Now, we had two top-notch endodontists score these answers. They weren’t messing around – they used a 5-point scale, because we only want the best. And if they didn’t agree, they figured it out with evidence-based discussions – because that’s how you get to the truth.
Here’s the deal: we set the bar high, and then even higher. At the low threshold, GPT-3.5 was a champion with 95% valid responses. Google Bard and Bing? They tried, but only got 85% and 75%. But when we raised the bar, GPT-3.5 still led the pack at 60%, leaving Google Bard and Bing in the dust at 15%. And folks, that difference was significant – we’re talking p-value of .008 significant.
But let’s not forget reliability – all three were pretty good, all above 0.7 on Cronbach’s alpha. That’s what we like to see.
So, what’s the bottom line? GPT-3.5 is giving you the most credible, most reliable answers in endodontics. It’s just doing a terrific job, better than Google Bard and Bing. That’s just the way it is.