2024: Evaluation of a GPT-based Chatbot for Higher Education

Master's theses

Student
Anna Lottner

Supervisor(s)Advisor(s)

Abstract

The goal of this thesis is to evaluate the effectiveness of IRIS, a GPT-based chatbot for higher education. The chatbot is integrated into the Artemis learning platform and is designed to provide assistance to students when they encounter difficulties. In addition, the chatbot is capable of answering questions related to the course content. Instructors can benefit from IRIS through assistance in exercise generation.

In this thesis, the effectiveness of IRIS will be evaluated through a combination of quantitative and qualitative methods. The quantitative evaluation will be conducted through a comprehensive assessment of the chatbot’s performance both in real course and experimental settings. The qualitative evaluation will be conducted through a survey of students and instructors and expert interviews. The results of the evaluation will be used to identify the strengths and weaknesses of the chatbot and to provide recommendations for future improvements.

Artemis is open source and available on https://github.com/ls1intum/Artemis

Head of Research Group



Prof. Dr. Stephan Krusche

Administrative office


Sophia Adelmeier
adelmeie@cit.tum.de

Office: 01.07.052
Phone: +49 (89) 289-18206
Fax: +49 (89) 289-18207 Boltzmannstraße 3
85748 Garching b. München