Evaluating the accuracy and reproducibility of ChatGPT responses in the context of cochlear implantation

Eroğlu E., Pamuk A. E., Bajin M. D., Sennaroğlu L.

ACTA OTO-LARYNGOLOGICA, cilt.1, sa.1, ss.1-6, 2025 (SCI-Expanded, Scopus) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 1 Sayı: 1
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1080/00016489.2025.2577834
  • Dergi Adı: ACTA OTO-LARYNGOLOGICA
  • Derginin Tarandığı İndeksler: Scopus, Science Citation Index Expanded (SCI-EXPANDED), Academic Search Premier, International Bibliography of Social Sciences, Biotechnology Research Abstracts, CINAHL, EMBASE, CAB Abstracts, Linguistics & Language Behavior Abstracts, Veterinary Science Database
  • Sayfa Sayıları: ss.1-6
  • Hacettepe Üniversitesi Adresli: Evet

Özet

Aim/Objectives 

To assess the accuracy and reproducibility of ChatGPT (GPT-4o) responses to questions about cochlear implantation, evaluating its potential role in patient education.

Material and Methods 

A total of 104 questions across five categories (basic, preoperative, surgical, postoperative care, postoperative expectations) were selected from reliable online sources. Each was posed twice to ChatGPT-4o in separate sessions. Responses were graded by two reviewers for accuracy (comprehensive/correct, partially correct, misleading, or incorrect/irrelevant). Reproducibility was assessed based on consistency across sessions. Discrepancies were resolved by a third expert reviewer.

Results 

Of 104 responses, 84.6% were correct, 5.8% partially correct, 6.7% misleading, and 2.9% incorrect. Reproducibility was 88.4% overall and 100% in the surgery category. No significant differences were found between question categories for accuracy (p = 0.829) or reproducibility (p = 0.348).

Conclusion and Significance 

ChatGPT provided highly accurate and reproducible responses to cochlear implant-related questions, supporting its use as an educational tool. Nonetheless, expert review remains essential for complex or critical topics.