Artificial Intelligence in Patient Education: A Comparative Evaluation of Chatbot Reliability After Total Hip Arthroplasty
American Journal of Physical Medicine and Rehabilitation, vol.Publish Ahead of Print, 2026 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Volume: Publish Ahead of Print
- Publication Date: 2026
- Doi Number: 10.1097/phm.0000000000003061
- Journal Name: American Journal of Physical Medicine and Rehabilitation
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, CINAHL, EMBASE, MEDLINE, SportDiscus, Academic Search Ultimate (EBSCO)
- Keywords: Arthroplasty, artificial intelligence, hip, patient education, postoperative care, rehabilitation
- Hacettepe University Affiliated: Yes
Abstract
Objective: – This study aimed to evaluate and compare the responses of different artificial intelligence–based large language models (AI LLMs) to frequently asked patient questions regarding post-operative care following total hip arthroplasty (THA) across four domains: reliability, quality, accuracy, and readability. Design: – Twenty-seven commonly asked questions were identified through Google Trends and expert consensus, covering exercises, activities of daily living, and dislocation precautions. Responses generated by the three AI models (May 2–5, 2025) were assessed using the modified DISCERN scale, Global Quality Scale (GQS), Accuracy Scale, and Flesch Reading Ease Score (FRES). Results: – Inter-rater reliability was highest for Gemini (ICC = 0.71), while ChatGPT models demonstrated moderate-to-good reliability (ICC = 0.67–0.71). Gemini scored significantly higher in reliability (P < 0.001), whereas ChatGPT 3.5 achieved the greatest readability (median FRES = 55). Significant differences were observed among models across reliability, quality, and readability metrics. Conclusion: – AI LLMs can serve as supplementary tools for patient education following THA; however, their use requires clinician oversight to ensure accuracy and safety. Future studies should explore patient-centered evaluations and hybrid approaches combining AI guidance with professional supervision in orthopedic rehabilitation.