ABSTRACT INTRODUCTION The General Surgery Fellowship Examination is the final hurdle examination prior to being eligible for fellowship of the Royal Australasian College of Surgeons. The purpose of this study was to assess the consistency and reliability of this examination, specifically whether there was evidence of systemic gender bias. METHODS This is a retrospective audit of marks awarded in the examination between 2018 and 2022. Examination components were marked by separately by two examiners, who assigned a consensus mark after discussion. Candidates’ marks were analysed according to their demographics. Inter- observer agreement between examining pairs was analysed, as were marks awarded by individual examiners. RESULTS Overall, 811 candidates (548 (67%) men and 263 (33%) women) sat the examination, of whom 520/811 (64%) passed. The pass rate was consistent over the study period. A total of 5672 pairs of examiners’ marks were analysed to determine inter- observer agreement. Inter- observer correlation was 0.89 (95% C.I. 0.88 - 0.89). There were 110 examiners (92 men and 18 women). No examiner’s marks were outside the 95% control limit for strictness or leniency compared with their peers. Pass rates were similar for male and female candidates (343/548 (63%) male vs. 177/263 (67%), P=0.20). There were 102 marking pairs with one female and one male examiner. Inter- rater agreement between these examiners was 0.86 (95% C.I. 0.82-0.91) when examining men, and 0.76 (95% C.I. 0.76-0.85) for women. CONCLUSION We found the examination to be consistent, fair and reliable, with no evidence of systemic gender bias.