Temporal trends in large language model (LLM) accuracy: A meta-analysis of multiple-choice question performance in dentistry and dental education

Authors

• Alison F. Doubleday, University of Illinois Chicago, Chicago, IL
• Colleen M. Cheverko, Rush University, Chicago, IL
• Olena Bolgova, Alfaisal University, Riyadh, Saudi Arabia
• Volodymyr Mavrych, Alfaisal University, Riyadh, Saudi Arabia
• Fathima Raahima Mohamed, Alfaisal University, Riyadh, Saudi Arabia
• Jennifer Westrick, Rush University, Chicago, IL
• Lorena Juarez, Rush University, Chicago, IL
• Emily Rush, Rush University, Chicago, IL
• Kathryn A. Solka, Rush University, Chicago, IL
• Jessica N. Byram, Indiana University School of Medicine, Indianapolis, IN
• Robert Beacker, Indiana University School of Medicine, Indianapolis, IN
• Victoria Gomez, Indiana University School of Medicine, Indianapolis, IN
• Brenda K. Ganeng, Indiana University School of Medicine, Indianapolis, IN
• Leslie A. Hoffman, Indiana University School of Medicine, Fort Wayne, IN
• Victoria A. Roach, University of Washington, Seattle, WA
• Kirsten M. Brown, The George Washington University School of Medicine and Health Sciences, Washington, DC
• Nicole DeVaul, The George Washington University School of Medicine and Health Sciences, Washington, DC
• Colleen N. Garnett, University of Alabama at Birmingham Heersink School of Medicine, Birmingham, AL
• Hannah L. Herriott, Regis University, Denver, CO
• Rebecca S. Lufler, Tufts University School of Medicine, Boston, MA
• Jason C. Mussell, LSU Health Sciences Center - New OrleansFollow
• Joy Y. Balta, Point Loma Nazarene University, San Diego, CA
• Michael A. Pascoe, University of Colorado Anschutz Medical Campus, Aurora, CO
• Jason W. Middleton, LSU Health Sciences Center - New OrleansFollow
• Sharon Duffy, LSU Health Sciences Center - New OrleansFollow
• Georgina C. Stephens, Monash University, Melbourne, Victoria, Australia
• Adam B. Wilson, Rush University, Chicago, IL

Document Type

Article

Publication Date

4-29-2026

Publication Title

Journal of Dentistry

Abstract

OBJECTIVES: Studies have investigated the potential applications of Large Language Models (LLMs) in dental education. As LLMs evolve, it is necessary to determine the extent to which their accuracy improves before they can be integrated into dental curricula and clinical practice. This meta-analysis investigates the performance of LLM versions on text-based single-best-answer multiple-choice questions (MCQs) in dentistry over time and assesses the impact of question type and source on LLM accuracy. MATERIALS AND METHODS: Typical guidelines for reporting systematic reviews and meta-analyses were followed. Study eligibility followed the PICO (Population, Intervention, Comparison, Outcomes) model. Included records were published between January 2022 and August 2025. Screenings and data extractions were performed by two reviewers, with disagreements resolved by a third reviewer. Proportional meta-analysis was used to determine the pooled accuracy of LLM versions across studies. Meta-regression explored sources of heterogeneity. RESULTS: Twenty-seven articles met the inclusion criteria. Over time, LLM accuracy in answering dentistry MCQs has significantly improved (p < 0.001). Heterogeneity across all LLMs was high ( ≥ 90%). OpenAI's models (ChatGPT-3.5, 4, and 4o or newer) significantly outperformed competitors (p < 0.001). Question type (stem-only vs. clinical vignette + stem) and question source (locally developed, for-profit commercial product, or free resources) did not significantly affect accuracy. CONCLUSIONS: As LLMs evolve, their accuracy on MCQs is improving, increasing their potential as supplemental tools in dental education.

First Page

106724

PubMed ID

42066887

Volume

171

Rights

© 2026 Elsevier Ltd.

Recommended Citation

Doubleday, Alison F.; Cheverko, Colleen M.; Bolgova, Olena; Mavrych, Volodymyr; Mohamed, Fathima Raahima; Westrick, Jennifer; Juarez, Lorena; Rush, Emily; Solka, Kathryn A.; Byram, Jessica N.; Beacker, Robert; Gomez, Victoria; Ganeng, Brenda K.; Hoffman, Leslie A.; Roach, Victoria A.; Brown, Kirsten M.; DeVaul, Nicole; Garnett, Colleen N.; Herriott, Hannah L.; Lufler, Rebecca S.; Mussell, Jason C.; Balta, Joy Y.; Pascoe, Michael A.; Middleton, Jason W.; Duffy, Sharon; Stephens, Georgina C.; and Wilson, Adam B., "Temporal trends in large language model (LLM) accuracy: A meta-analysis of multiple-choice question performance in dentistry and dental education" (2026). School of Graduate Studies Faculty Publications. 594.
https://digitalscholar.lsuhsc.edu/sogs_facpubs/594
10.1016/j.jdent.2026.106724