Evaluating the accuracy and reproducibility of ChatGPT responses in the context of cochlear implantation
ACTA OTO-LARYNGOLOGICA, vol.1, no.1, pp.1-6, 2025 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Volume: 1 Issue: 1
- Publication Date: 2025
- Doi Number: 10.1080/00016489.2025.2577834
- Journal Name: ACTA OTO-LARYNGOLOGICA
- Journal Indexes: Scopus, Science Citation Index Expanded (SCI-EXPANDED), Academic Search Premier, International Bibliography of Social Sciences, Biotechnology Research Abstracts, CINAHL, EMBASE, CAB Abstracts, Linguistics & Language Behavior Abstracts, Veterinary Science Database
- Page Numbers: pp.1-6
- Hacettepe University Affiliated: Yes
Abstract
Aim/Objectives
To assess the accuracy and reproducibility of ChatGPT (GPT-4o) responses to questions about cochlear implantation, evaluating its potential role in patient education.
Material and Methods
A total of 104 questions across five categories (basic, preoperative, surgical, postoperative care, postoperative expectations) were selected from reliable online sources. Each was posed twice to ChatGPT-4o in separate sessions. Responses were graded by two reviewers for accuracy (comprehensive/correct, partially correct, misleading, or incorrect/irrelevant). Reproducibility was assessed based on consistency across sessions. Discrepancies were resolved by a third expert reviewer.
Results
Of 104 responses, 84.6% were correct, 5.8% partially correct, 6.7% misleading, and 2.9% incorrect. Reproducibility was 88.4% overall and 100% in the surgery category. No significant differences were found between question categories for accuracy (p = 0.829) or reproducibility (p = 0.348).
Conclusion and Significance
ChatGPT provided highly accurate and reproducible responses to cochlear implant-related questions, supporting its use as an educational tool. Nonetheless, expert review remains essential for complex or critical topics.