Title: Applying machine learning to analyse teachers' instructional questions

Authors: Anwar Ali Yahya; Mohammad Said El Bashir

Addresses: Faculty of Computer Science and Information Systems, Najran University, Najran – 61441, Saudi Arabia ' Faculty of Information Technology, Al al-Bayt University, Mafraq – 25113, Jordan

Abstract: This paper introduces machine learning approaches to a new application in the field of education. More specifically, it explores the effectiveness of three machine learning approaches, namely, k-nearest neighbours, naïve Bayes, and support vector machines with term frequency as term selection approach, on the task of evaluating teaching effectiveness by classifying teachers' classroom questions into different cognitive levels identified in Bloom's taxonomy. In doing so, a dataset of questions has been collected and annotated manually with Bloom's cognitive levels. Several steps of pre-processing have been applied to convert these questions into a representation suitable for machine learning approaches. Using the dataset, the performance of the machine learning approaches and the traditional rule-based approach have been evaluated. The obtained results lead to several conclusions: First, machine learning approaches have a superior performance over rule-based approach. Second, the term frequency as a term selection approach plays a crucial rule in the performance of machine learning approaches. Third, SVM shows a superior performance over k-nearest neighbour and naïve Bayes which shows a comparable performance in term of F-measure and accuracy. Finally, machine learning approaches show different levels of sensitivity to the number of terms used in question representation.

Keywords: machine learning; Bloom's taxonomy; questions analysis; text classification; higher education; k-nearest neighbour; kNN; naive Bayes; support vector machines; SVM; term frequency; teaching effectiveness; classroom questions; teacher effectiveness; computer science; computing education.

DOI: 10.1504/IJAIP.2014.066985

International Journal of Advanced Intelligence Paradigms, 2014 Vol.6 No.4, pp.312 - 327

Received: 12 Mar 2014
Accepted: 13 Sep 2014
Published online: 24 Jan 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article