Browsing by Author "Isa, Salman"
Now showing 1 - 2 of 2
- Results Per Page
- Sort Options
Item Assessing the Reliability of Current AI Platforms in Delivering Health Information Related to Crohn Disease, Ulcerative Colitis, and Colorectal Cancer(2024-03-21) Isa, Salman; Elchehabi, Sahar; Jafri, Faraz; Hoang, Long; Sharma, Mukesh; Hapuarachchi, Menalee; Gonzales, Gabriel; Lewis, Trina; Richardson, Justin; Nguyen, Elizabeth; Hyman, CharlesPurpose In recent years, the advancements in artificial intelligence (AI) have revolutionized the way we seek and access health information. With more people turning towards AI for answers to their problems, it is important to question how safe it is to rely on AI for answers to health-related issues. We explored the accuracy of ChatGPT—a language model developed by OpenAI—and Gemini—Google’s AI platform—in providing health information related to Crohn disease, ulcerative colitis, and colorectal cancer. Methods We generated 10 questions relating to Crohn disease, ulcerative colitis, and colorectal cancer in relation to the social, psychological, economic, and physical aspects that patients with these diseases may face. Each query was remastered for each disease, resulting in 30 total questions which were posed to the two separate AI models. We then regenerated the responses for a total of three times ending up with 90 generated responses per AI model. We also measured the Flesch-Kincaid Readability scores for each response and analyzed the sentiment of the text using natural language processing and computational linguistics. The Centers of Disease Control and Prevention (CDC) recommend that medical information for the public be written at no higher than an eighth-grade reading level. Generated AI responses were evaluated by six gastroenterologist attendings and fellows for accuracy within the context of a patient seeking information. Sets were deemed inappropriate if any of the three responses contained inaccurate or misleading information, based on clinical judgment. Evaluators were blinded to model names and prices. Interrater agreement (94%) and reliability (κ score, 0.87) were ideal. The study was performed in July 2023. Results Of the 60 questions posed to the two different AI language models, 45% (n = 27) of the responses were found to be inaccurate. When the two AI models were compared, 43.33% (n = 13) of ChatGPT’s responses were accurate while 46.7% (n = 14) of Gemini’s responses were deemed accurate. ChatGPT also had a 13.20 average Flesch Kincaid Reading grade level and a 31.06 average Flesch Kincaid Readability score. Gemini’s responses received an average Flesch Kincaid Reading grade level of 8.34 and an average Flesch Kincaid Readability score of 56.92. ChatGPT’s average sentiment score was a 1.23 while Gemini’s average score was a 0.92. Conclusion While OpenAI’s ChatGPT and Google's Gemini platform can serve as valuable resources for information retrieval, they possess certain limitations when it comes to health-related information for Crohn disease, ulcerative colitis, and colorectal cancer. Importantly, both AI models in the study provided inappropriate responses to common patient questions regarding these conditions. Medical professionals should be aware of these limitations as they may lead to the spread of misinformation in populations with limited access to health care.Item Optimizing Medical Education: Integrating Palpation and Radiologic Imaging in Simulation Labs(2024-03-21) Sankhavaram, Mira; Selby, Samuel; Mize, Will; Wieters, Matthew; Isa, Salman; Smith, Spencer; Papa, FrankPurpose: UNTHSC’s Regional Simulation Lab has created many opportunities for faculty to move beyond traditional, classroom-based approaches to an advanced, simulation-based learning environment. Unfortunately, there is minimal literature regarding the effective integration of distinct skills given that these sophisticated technologies can portray case scenarios that require the successful application of several skill sets. The investigators will utilize approximately 10 – 15 minutes of the two hours of time scheduled in the year two GI course, to support students in integrating the two primary skill sets introduced 'separately' earlier. This instructional activity will provide students an opportunity to simultaneously see CT and Ultrasound scans representing an abnormal abdominal finding while simultaneously palpating (on a high-fidelity abdominal simulator) the associated abnormal physical examination findings. Methods: UNTHSC utilizes a post-course evaluation tool collecting student feedback for continuous quality assurance of its curriculum. This year's post-course evaluation questions contain both a standardized set of questions and an additional set of questions designed to gather student opinions about how the course might be improved. Included in the survey were two sets of questions about the skills developed in this two-hour GI training activity. The first set of questions is about the adequacy of the time spent in: 1) Ultrasound training in isolation, 2) abdominal palpation training in isolation, 3) the simultaneous training of both Ultrasound and CT images, and palpation training. The second set of questions represents student opinion regarding the percentage of the two hours of instructional activity that should be spent with 4) Ultrasound training in isolation, 5) abdominal palpation training in isolation, and 6) simultaneous training in both Ultrasound and CT images, and palpation. Results: Our findings, based upon a survey of 99 students, reveal the value of integrating palpation with radiologic imaging more than performing either skill in isolation. The survey suggests that students would prefer spending about three times longer in the lab integrating these two skill sets compared to the original 15 minutes allocated. Conclusion: While traditionally topics in medical education are delivered to students in a piecemeal fashion, with the focus on mastering each skill in isolation before moving on to the next, this survey demonstrates that students value more opportunities to integrate various skills as they are first being introduced. This provides students with a more realistic understanding of how to use the resources they will have in the clinical setting to arrive at a diagnosis. This notion is further supported by the open-ended feedback solicited from the participants. In their comments, there is a strong desire for a more case-based approach to learning, in which students have the autonomy to work to solve a patient problem and dictate their diagnostic process in a more realistic, linear fashion rather than learning by simple demonstration and repetition. While more research is needed, these results demonstrate the value students place in integrating emerging simulation technology with pre-clinical medical education to help prepare students for their role in the clinical environment.