AbstractObjectivesIn the context of competency-based education emphasized in Korean Medicine, this study aimed to develop a pilot version of a CPX (Clinical Performance Examination) Practicing Chatbot utilizing large language models with prompt engineering.
MethodsA standardized patient scenario was acquired from the National Institute of Korean Medicine and transformed into text format. Prompt engineering was then conducted using role prompting and few-shot prompting techniques. The GPT-4 API was employed, and a web application was created using the gradio package. An internal evaluation criterion was established for the quantitative assessment of the chatbot’s performance.
참고문헌1. Gwak D, Kim M, Kwon J, & Yin C. Trends in Korean Medical education research from 2003 to 2022. The Journal of Korean Medical History;(2023). 3689-98.
2. Jeong S-H, Kim J-P, Kang Y-J, Jeong HI, & Kim KH. A Survey of Recognitions and Satisfaction with Education in Traditional Korean Medicine. Journal of Society of Preventive Korean Medicine;(2020). 2449-56.
3. Cho E, Kim Y, Ha J, Park J, & Jung H-J. Developing a manual for clinical practice on real patients in Korean Medicine. J Kor Med Edu;(2023). 115-22.
4. Han SY, Lee SH, & Chae H. Developing a best practice framework for clinical competency education in the traditional East-Asian medicine curriculum. BMC Med Educ;(2022). 22
5. Lim C, Han H, Hong J, & Kang Y. 2016 Competency Modeling for Doctor of Korean Medicine & Application Plans. Journal of Korean Medicine;(2016). 37101-13.
6. Han C-Y, Kang D-W, Park J-G, Kim B-H, Kim K-S, & Kim Y-B, et al. An Analysis of Clerkship Satisfaction in College of Korean Medicine : Focusing on Doctor-patient Role-play and mock CPX. J Korean Med Ophthalmol Otolaryngol Dermatol;(2020). 3312-24.
7. Jo H, & Min S. The current status and future operations of Clinical Performance Evaluation (CPX) in the nationwide colleges (graduate schools) of Traditional Korean Medicine. The Journal of Korean Medical History;(2020). 339-21.
8. Jo H-J, Roh J-D, Sung HK, & Park J-S. A Survey on Students’ Perception of Clinical Performance Examination (CPX) in College of Korean Medicine Using Student Standardized Patients. Journal of Society of Preventive Korean Medicine;(2020). 241-13.
9. Park S, & Kim C. Enhancing Korean Medicine Education with Large Language Models: Focusing on the Development of Educational Artificial Intelligence. Journal of Physiology & Pathology in Korean Medicine;(2023). 37134-8.
10. Chen B, Zhang Z, Langrené N, & Zhu S. Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review;(2023.
11. Renze M, & Guven E. The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models;(2024.
12. Institute of Medical Education and Training. Revised Clinical Diagnosis. 2nd edition. Seoul. Seoul National University Press;(2015.
13. Bae H, Lee S, Lee CY, & Kim CE. A Novel Framework for Understanding the Pattern Identification of Traditional Asian Medicine From the Machine Learning Perspective. Front Med (Lausanne);(2022). 8763533
14. Jang D, Yun T-R, Lee C-Y, Kwon Y-K, & Kim C-E. GPT-4 can pass the Korean National Licensing Examination for Korean Medicine Doctors. PLOS Digital Health;(2023). 2e0000416
15. Banerjee D, Singh P, Avadhanam A, & Srivastava S. Benchmarking LLM powered Chatbots: Methods and Metrics;(2023.
|
|