Watch The Presentation
Presentation Description
Institution: Waikato Hospital - Waikato, Aotearoa New Zealand
Background: The Generic Surgical Sciences Examination (GSSE) is the post-graduate examination required for surgical training across Australia and New Zealand. With the ability to comprehend information provided by the user, language learning models, such as Chat-GPT, are increasingly being used as learning tools. This study investigated Chat-GPT’s ability to produce revision materials for the GSSE, and how these materials stand up to practice questions in terms of accuracy and breadth of information.
Methods: Commonly recommended textbooks were provided to Chat-GPT 4o as a source material. The prompt “provide revision material from the source material for (topic) here” was used, and topics were based on a commonly used practice question bank resource. If the material provided was insufficient to answer the question with no prior knowledge, a second prompt “provide more details from the source material for this topic” was used. 100 questions were answered in anatomy, physiology and pathology sections respectively.
Results: 69% of questions were answered correctly only using information from the initial prompt, raising to 72% after the second. 1% of questions answered were incorrect. Physiology was the highest scoring section (78%) followed by anatomy (71%) and pathology (58%). Based on the pass mark for October 2022, answering questions only using these responses would be sufficient to pass all sections.
Conclusions: Chat-GPT 4o provides rapid responses to prompts for revision material in an easy-to-read format. Despite some visual limitations, it appears to provide succinct and accurate revision notes from a reputable source and would be a valuable revision tool used alongside classic revision materials.
Presenters
Authors
Authors
Dr Toby Ball - , Dr Thomas Hockey -