Safety and Accuracy of AI-Generated Return-to-Sport Recommendations After ACL Reconstruction: A Content Analysis Against the Bern Consensus
Keywords:
Sports Physical Therapy, Clinical Decision-making, Orthopedic Rehabilitation, Large Language Models, Psychological ReadinessAbstract
Large language models (LLMs) are increasingly consulted by athletes for return-to-sport (RTS) guidance after anterior cruciate ligament (ACL) reconstruction, yet their alignment with clinical consensus remains unknown. This study evaluated the accuracy, completeness, and safety of LLM-generated RTS recommendations against the 2016 Bern Consensus criteria. Fifteen clinical vignettes (five standard, five borderline, five high-risk) were each queried three times across three LLMs (DeepSeek-V3, Grok-4-Fast, GPT-5-nano), yielding 135 responses scored on a Bern Consensus–derived six-dimension framework, supplemented by a 3-level Likert sensitivity analysis for psychological readiness and sport-specific skills. All models achieved perfect scores for time factor, risk–benefit discussion, and safety warnings. Sport-specific skills were nearly absent (Likert scores below 0.07), and psychological readiness varied by model (DeepSeek 0.42, GPT 0.23, Grok 0.00). In high-risk cases, data hallucination and comorbidity omission reduced mean total scores from 4.36 to 3.69, and mixed-effects modelling confirmed significant case-type effects (p=0.001). Current LLMs demonstrate adequate baseline ACL knowledge but fail in sport-specific precision; AI-generated RTS advice should therefore serve only as a preliminary draft requiring mandatory individualized clinical validation.
References
Sanders, T. L., Maradit Kremers, H., Bryan, A. J., Larson, D. R., Dahm, D. L., Levy, B. A., Stuart, M. J., & Krych, A. J. (2016). Incidence of anterior cruciate ligament tears and reconstruction: A 21-year population-based study. The American Journal of Sports Medicine, 44(6), 1502–1507. https://doi.org/10.1177/0363546516629944
Martinez-Calderon, J., Infante-Cano, M., Matias-Soto, J., Perez-Cabezas, V., Galan-Mercant, A., & Garcia-Muñoz, C. (2025). The incidence of sport-related anterior cruciate ligament injuries: An overview of systematic reviews including 51 meta-analyses. Journal of Functional Morphology and Kinesiology, 10(2), 174. https://doi.org/10.3390/jfmk10020174
Ardern, C. L., Glasgow, P., Schneiders, A., Witvrouw, E., Clarsen, B., Cools, A., Gojanovic, B., Griffin, S., Khan, K. M., Moksnes, H., Mutch, S. A., Phillips, N., Reurink, G., Sadler, R., Grävare Silbernagel, K., Thorborg, K., Wangensteen, A., Wilk, K. E., & Bizzini, M. (2016). 2016 consensus statement on return to sport from the First World Congress in Sports Physical Therapy, Bern. British Journal of Sports Medicine, 50(14), 853–864. https://doi.org/10.1136/bjsports-2016-096278
Ardern, C. L., Webster, K. E., Taylor, N. F., & Feller, J. A. (2011). Return to sport following anterior cruciate ligament reconstruction surgery: A systematic review and meta-analysis of the state of play. British Journal of Sports Medicine, 45(7), 596–606. https://doi.org/10.1136/bjsm.2010.076364
Meredith, S. J., Rauer, T., Chmielewski, T. L., Fink, C., Diermeier, T., Rothrauff, B. B., Svantesson, E., Hamrin Senorski, E., Hewett, T. E., Sherman, S. L., Lesniak, B. P., , Bizzini, M., Chen, S., Cohen, M., Villa, S. D., Engebretsen, L., Feng, H., Ferretti, M., ... Wilk, K. (2020). Return to sport after anterior cruciate ligament injury: Panther Symposium ACL Injury Return to Sport Consensus Group. Orthopaedic Journal of Sports Medicine, 8(6), 2325967120930829. https://doi.org/10.1177/2325967120930829
Webster, K. E., Feller, J. A., & Lambros, C. (2008). Development and preliminary validation of a scale to measure the psychological impact of returning to sport following anterior cruciate ligament reconstruction surgery. Physical Therapy in Sport, 9(1), 9–15. https://doi.org/10.1016/j.ptsp.2007.09.003
McBain, R. K., Bozick, R., Diliberti, M., Zhang, L. A., Zhang, F., Burnett, A., Kofner, A., Rader, B., Breslau, J., Stein, B. D., Mehrotra, A., Pines, L. U., Cantor, J., & Yu, H. (2025). Use of generative AI for mental health advice among US adolescents and young adults. JAMA Network Open, 8(11), e2542281. https://doi.org/10.1001/jamanetworkopen.2025.42281
Wiggins, A. J., Grandhi, R. K., Schneider, D. K., Stanfield, D., Webster, K. E., & Myer, G. D. (2016). Risk of secondary injury in younger athletes after anterior cruciate ligament reconstruction: A systematic review and meta-analysis. The American Journal of Sports Medicine, 44(7), 1861–1876. https://doi.org/10.1177/0363546515621554
Beischer, S., Gustavsson, L., Senorski, E. H., Karlsson, J., Thomeé, C., Samuelsson, K., & Thomeé, R. (2020). Young athletes who return to sport before 9 months after anterior cruciate ligament reconstruction have a rate of new injury 7 times that of those who delay return. Journal of Orthopaedic & Sports Physical Therapy, 50(2), 83–90. https://doi.org/10.2519/jospt.2020.9071
Grindem, H., Snyder-Mackler, L., Moksnes, H., Engebretsen, L., & Risberg, M. A. (2016). Simple decision rules can reduce reinjury risk by 84% after ACL reconstruction: The Delaware-Oslo ACL cohort study. British Journal of Sports Medicine, 50(13), 804–808. https://doi.org/10.1136/bjsports-2016-096031
Ardern, C. L., Taylor, N. F., Feller, J. A., & Webster, K. E. (2013). A systematic review of the psychological factors associated with returning to sport following injury. British Journal of Sports Medicine, 47(17), 1120–1126. https://doi.org/10.1136/bjsports-2012-091203
Webster, K. E., & Feller, J. A. (2020). Who passes return-to-sport tests, and which tests are most strongly associated with return to play after anterior cruciate ligament reconstruction? Orthopaedic Journal of Sports Medicine, 8(12), 2325967120969425. https://doi.org/10.1177/2325967120969425
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Babiker, A., Schärli, N., Chowdhery, A., Mansfield, P., Demner-Fushman, D., ... Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180. https://doi.org/10.1038/s41586-023-06291-2
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., & Smith, D. M. (2023). Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine, 183(6), 589–596. https://doi.org/10.1001/jamainternmed.2023.1838
Kaarre, J., Feldt, R., Keeling, L. E., Dadoo, S., Zsidai, B., Hughes, J. D., Samuelsson, K., & Musahl, V. (2023). Exploring the potential of ChatGPT as a supplementary tool for providing orthopaedic information. Knee Surgery, Sports Traumatology, Arthroscopy, 31(11), 5190–5198. https://doi.org/10.1007/s00167-023-07529-2
Kodra, J. D., Saroyan, A., Darby, F., Surucu, S., Fong, S., Gillinov, S., Girardi, K., Vasudevan, R., Ansah‐Twum, J. K., Atadja, L., Moran, J., & Jimenez, A. E. (2025). ChatGPT-generated responses across orthopaedic sports medicine surgery vary in accuracy, quality, and readability: A systematic review. Arthroscopy, Sports Medicine, and Rehabilitation, 7(4), 101210. https://doi.org/10.1016/j.asmr.2025.101210
Li, L. T., Sinkler, M. A., Adelstein, J. M., Voos, J. E., & Calcei, J. G. (2024). ChatGPT responses to common questions about anterior cruciate ligament reconstruction are frequently satisfactory. Arthroscopy, 40(7), 2058–2066. https://doi.org/10.1016/j.arthro.2023.12.009
Johns, W. L., Martinazzi, B. J., Miltenberg, B., Nam, H. H., & Hammoud, S. (2024). ChatGPT provides unsatisfactory responses to frequently asked questions regarding anterior cruciate ligament reconstruction. Arthroscopy, 40(7), 2067–2079. https://doi.org/10.1016/j.arthro.2024.01.017
Gaudiani, M. A., Castle, J. P., Abbas, M. J., Pratt, B. A., Myles, M. D., Moutzouros, V., & Lynch, T. S. (2024). ChatGPT-4 generates more accurate and complete responses to common patient questions about anterior cruciate ligament reconstruction than Google's search engine. Arthroscopy, Sports Medicine, and Rehabilitation, 6(3), 100939. https://doi.org/10.1016/j.asmr.2024.100939
Quinn, M., Milner, J. D., Schmitt, P., Morrissey, P., Lemme, N., Marcaccio, S., DeFroda, S., Tabaddor, R., & Owens, B. D. (2025). Artificial intelligence large language models address anterior cruciate ligament reconstruction: Superior clarity and completeness by Gemini compared with ChatGPT-4 in response to American Academy of Orthopaedic Surgeons clinical practice guidelines. Arthroscopy, 41(6), 2002–2008. https://doi.org/10.1016/j.arthro.2024.09.020
Gültekin, O., Inoue, J., Yilmaz, B., Cerci, M. H., Kilinc, B. E., Yilmaz, H., Prill, R., & Kayaalp, M. E. (2025). Evaluating DeepResearch and DeepThink in anterior cruciate ligament surgery patient education: ChatGPT-4o excels in comprehensiveness, DeepSeek R1 leads in readability. Knee Surgery, Sports Traumatology, Arthroscopy, 33(8), 3025–3031. https://doi.org/10.1002/ksa.12711
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38. https://doi.org/10.1145/3571730
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023). Med-HALT: Medical domain hallucination test for large language models. Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), , 314–334. https://doi.org/10.18653/v1/2023.conll-1.21
Wang, L., Chen, X., Deng, X., Wen, H., You, M., Liu, W., Li, Q., & Li, J. (2024). Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Medicine, 7(1), 41. https://doi.org/10.1038/s41746-024-01029-4
Tang, L., Sun, Z., Idnay, B., Nestor, J. G., Soroush, A., Elias, P. A., Xu, Z., Ding, Y., Durrett, G., Rousseau, J. F., Weng, C., & Peng, Y. (2023). Evaluating large language models on medical evidence summarization. npj Digital Medicine, 6(1), 158. https://doi.org/10.1038/s41746-023-00896-7
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
van Grinsven, S., van Cingel, R. E. H., Holla, C. J. M., & van Loon, C. J. M. (2010). Evidence-based rehabilitation following anterior cruciate ligament reconstruction. Knee Surgery, Sports Traumatology, Arthroscopy, 18(8), 1128–1144. https://doi.org/10.1007/s00167-009-1027-2
McPherson, A. L., Feller, J. A., Hewett, T. E., & Webster, K. E. (2019). Smaller change in psychological readiness to return to sport is associated with second anterior cruciate ligament injury among younger patients. The American Journal of Sports Medicine, 47(5), 1209–1215. https://doi.org/10.1177/0363546519825499
