An evaluation of Gemini, ChatGPT, and DeepSeek large language model responses to ophthalmology case-based questions
Author:
Corresponding Author:

Vishaal Bhambhwani. 1620 Mountain Road, Thunder Bay, Ontario P7J 1C8, Canada. vishaalb@ymail.com

Affiliation:

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    AIM: To evaluate the accuracy and appropriateness of responses of different large language models (LLMs) to ophthalmology case-based questions. METHODS: Prospective, observational, cross-sectional study conducted in 2025 in Ontario, Canada. Three LLMs (Gemini Advanced 2.5 Pro, ChatGPT 4.5, and DeepSeek DeepThink R1) were asked to respond to all “Clinical Challenge” cases published in The Journal of the American Medical Association Ophthalmology from 2021 to 2024 (135 multiple-choice questions). They were evaluated for accuracy and response appropriateness. Chi-square and Kruskal-Wallis tests were used to compare their performances. RESULTS: The average accuracy across all LLMs was 303/405 (74.8%), with Gemini leading at 106/135 (78.5%), followed by ChatGPT (99/135; 73.3%) and DeepSeek (98/135; 72.6%). The pairwise differences among the LLMs were not statistically significant. The average appropriateness score of all LLMs together was 4.44 out of 6. Gemini, ChatGPT, and DeepSeek had average appropriateness scores of 4.6, 4.2, and 4.5, respectively (P=0.007). CONCLUSION: These LLMs reveal a moderate-to-high level of accuracy and appropriateness of reasoning when answering complex ophthalmology case-based questions. While ChatGPT and DeepSeek differ by one correct answer, Gemini outperforms them by over 5%. When using LLMs for clinical decision-making, the model’s ability to provide quality explanations for the selected course of action is an asset. Although Gemini is the overall leader in appropriateness of reasoning, the fact that DeepSeek outperforms ChatGPT in appropriateness, combined with its open-source, free-to-use, and offline-capable features, makes it an exciting prospect for future use in ophthalmology.

    Reference
    Related
    Cited by
Get Citation

Amir-Ali Golrokhian-Sani, Jennifer L. Patnaik, Vishaal Bhambhwani. An evaluation of Gemini, ChatGPT, and DeepSeek large language model responses to ophthalmology case-based questions. Int J Ophthalmol, 2026,19(12):2321-2326

Copy
Article Metrics
  • Abstract:
  • PDF:
Publication History
  • Received:March 13,2026
  • Revised:September 18,2026
  • Adopted:
  • Online: November 16,2026
  • Published: