Evaluating the accuracy and reproducibility of ChatGPT responses in the context of cochlear implantation
ACTA OTO-LARYNGOLOGICA, cilt.1, sa.1, ss.1-6, 2025 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 1 Sayı: 1
- Basım Tarihi: 2025
- Doi Numarası: 10.1080/00016489.2025.2577834
- Dergi Adı: ACTA OTO-LARYNGOLOGICA
- Derginin Tarandığı İndeksler: Scopus, Science Citation Index Expanded (SCI-EXPANDED), Academic Search Premier, International Bibliography of Social Sciences, Biotechnology Research Abstracts, CINAHL, EMBASE, CAB Abstracts, Linguistics & Language Behavior Abstracts, Veterinary Science Database
- Sayfa Sayıları: ss.1-6
- Hacettepe Üniversitesi Adresli: Evet
Özet
Aim/Objectives
To assess the accuracy and reproducibility of ChatGPT (GPT-4o) responses to questions about cochlear implantation, evaluating its potential role in patient education.
Material and Methods
A total of 104 questions across five categories (basic, preoperative, surgical, postoperative care, postoperative expectations) were selected from reliable online sources. Each was posed twice to ChatGPT-4o in separate sessions. Responses were graded by two reviewers for accuracy (comprehensive/correct, partially correct, misleading, or incorrect/irrelevant). Reproducibility was assessed based on consistency across sessions. Discrepancies were resolved by a third expert reviewer.
Results
Of 104 responses, 84.6% were correct, 5.8% partially correct, 6.7% misleading, and 2.9% incorrect. Reproducibility was 88.4% overall and 100% in the surgery category. No significant differences were found between question categories for accuracy (p = 0.829) or reproducibility (p = 0.348).
Conclusion and Significance
ChatGPT provided highly accurate and reproducible responses to cochlear implant-related questions, supporting its use as an educational tool. Nonetheless, expert review remains essential for complex or critical topics.