Article: A framework for effectively utilising human grading input in automated short answer grading Journal: International Journal of Mobile Learning and Organisation (IJMLO) 2022 Vol.16 No.3 pp.266 - 286 Abstract: Short answer questions are effective for recall knowledge assessment. Grading a large amount of short answers is costly and time consuming. To apply short answer questions on MOOCs platforms, the issues of scalability and responsiveness must be addressed. Automated grading uses a computing process and a machine learning grading model to classify answers into correct, wrong, and other levels of correctness. The divide-and-grade approach is proven effective in reducing the annotation effort needed for the learning the grading model. This paper presents an improvement on the divide-and-grade approach that is designed to increase the utility of human actions. A novel short answer grading framework is proposed that addresses the selection of impactful answers for grading, the injection of the ground-truth grades for steering towards purer final clusters, and the final grade assignments. Experiment results indicate the grading quality can be improved with the same level of human actions. Inderscience Publishers - linking academia, business and industry through research

Title: A framework for effectively utilising human grading input in automated short answer grading

Authors: Andrew Kwok-Fai Lui; Sin-Chun Ng; Stella Wing-Nga Cheung

Addresses: School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China ' School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China ' School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China

Abstract: Short answer questions are effective for recall knowledge assessment. Grading a large amount of short answers is costly and time consuming. To apply short answer questions on MOOCs platforms, the issues of scalability and responsiveness must be addressed. Automated grading uses a computing process and a machine learning grading model to classify answers into correct, wrong, and other levels of correctness. The divide-and-grade approach is proven effective in reducing the annotation effort needed for the learning the grading model. This paper presents an improvement on the divide-and-grade approach that is designed to increase the utility of human actions. A novel short answer grading framework is proposed that addresses the selection of impactful answers for grading, the injection of the ground-truth grades for steering towards purer final clusters, and the final grade assignments. Experiment results indicate the grading quality can be improved with the same level of human actions.

Keywords: automated short answer grading; clustering; semi-supervised clustering; MOOCs; automated grading.

DOI: 10.1504/IJMLO.2022.124160

International Journal of Mobile Learning and Organisation, 2022 Vol.16 No.3, pp.266 - 286

Received: 04 Nov 2020
Accepted: 08 Mar 2021
Published online: 15 Jul 2022 *

Title: A framework for effectively utilising human grading input in automated short answer grading

Keep up-to-date