The goal of this thesis is to evaluate the effectiveness of IRIS, a GPT-based chatbot for higher education. The chatbot is integrated into the Artemis learning platform and is designed to provide assistance to students when they encounter difficulties. In addition, the chatbot is capable of answering questions related to the course content. Instructors can benefit from IRIS through assistance in exercise generation.
In this thesis, the effectiveness of IRIS will be evaluated through a combination of quantitative and qualitative methods. The quantitative evaluation will be conducted through a comprehensive assessment of the chatbot’s performance both in real course and experimental settings. The qualitative evaluation will be conducted through a survey of students and instructors and expert interviews. The results of the evaluation will be used to identify the strengths and weaknesses of the chatbot and to provide recommendations for future improvements.
Artemis is open source and available on https://github.com/ls1intum/Artemis