Title: Learning automata-based approach to learn dialogue policies in large state space

Authors: G. Kumaravelan; R. Sivakumar

Addresses: Department of Computer Science, Pondicherry University, Karaikal Centre, Karaikal – 609605, India. ' Department of Computer Science, AVVM Sri Pushpam College, Thanjavur – 613503, Tamilnadu, India

Abstract: This paper addresses the problem of scalable optimisation of dialogue policies in speech-based conversational systems using reinforcement learning. More specifically, for large state spaces several difficulties like large tables, an account of prior knowledge and data sparsity are faced. Hence, we present an online policy learning algorithm based on hierarchical structure learning automata using eligibility trace method to find optimal dialogue strategies that cover large state spaces. The proposed algorithm is capable of deriving an optimal policy that prescribes what action should be taken in various states of conversation so as to maximise the expected total reward to attain the goal and incorporates good exploration and exploitation in its updates to improve the naturalness of human-computer interaction. The proposed model is tested using the most sophisticated evaluation framework PARADISE for accessing the travel information system.

Keywords: spoken dialogue system; SDS; reinforcement learning; learning automata; Markov decision process; MDP; human-computer interaction; HCI; speech; conversation; prior knowledge; data sparsity; dialogue strategies; travel information systems.

DOI: 10.1504/IJIIDS.2012.045844

International Journal of Intelligent Information and Database Systems, 2012 Vol.6 No.2, pp.180 - 199

Received: 08 Sep 2010
Accepted: 11 Jan 2011
Published online: 16 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article