Purpose – Artificial intelligence (AI) chatbots like ChatGPT-4 have made significant strides to generate human-like responses. Trained on an extensive medical literature corpus, ChatGPT-4 holds potential in augmenting patient education. These chatbots may be beneficial to populations considering colorectal cancer (CRC). However, the accuracy and quality of patient education materials are crucial for informed decision-making. Given workforce demands impacting holistic care, AI chatbots can bridge gaps in CRC information, reaching rural demographics and crossing language barriers. However, rigorous evaluation is essential to ensure accuracy, quality, and readability. Methodology – To evaluate ChatGPT-4, 8 CRC-related questions were derived using peer-reviewed literature and Google Trends. Responses were evaluated by 8 colorectal surgeons for accuracy, safety, appropriateness, actionability, and effectiveness. Quality metrics included Patient Education Materials Assessment Tool (PEMAT-AI), modified DISCERN (DISCERN-AI), and Global Quality Score (GQS). Readability was assessed using Flesch Reading Ease (FRE) and Gunning Fog Index (GFI). Results – ChatGPT-4 responses were generally accurate (median 4.00), safe (4.25), appropriate (4.00), actionable (4.00), and effective (4.00). Quality ratings showed PEMAT-AI as “very good” (71.43), DISCERN-AI as “fair” (12.00), and GQS as “high” (4.00). Readability scores revealed challenges (FRE 47.00, GFI 12.40), requiring higher educational levels. Conclusion – ChatGPT-4 provides safe but non-specific CRC information, highlighting its potential as a patient education tool. Improvements in readability through contextual prompting and fine-tuning are necessary before clinical implementation.