Article: Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model Journal: International Journal of Modelling, Identification and Control (IJMIC) 2006 Vol.1 No.4 pp.245 - 256 Abstract: A sparse regression modelling technique is developed using a generalised kernel model in which each kernel regressor has its individually tuned position (centre) vector and diagonal covariance matrix. An orthogonal least squares forward selection procedure is employed to append the regressors one by one. After the determination of the model structure, namely the selection of an appropriate number of regressors, the model weight parameters are calculated from the Lagrange dual problem of the original least squares problem. Different from the least squares support vector regression, this regression modelling procedure involves neither reproducing kernel Hilbert space nor Mercer decomposition concepts. As the regressors used are not restricted to be positioned at training input points and each regressor has its own diagonal covariance matrix, a very sparse representation can be obtained with excellent generalisation capability. Experimental results involving two real data sets demonstrate the effectiveness of the proposed regression modelling approach. Inderscience Publishers - linking academia, business and industry through research

Title: Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model

Authors: Xunxian Wang, David Lowe, Sheng Chen, Chris J. Harris

Addresses: Neural Computing Research Group, Aston University, Birmingham B4 7ET, UK. ' Neural Computing Research Group, Aston University, Birmingham B4 7ET, UK. ' School of Electronics and Computer Science, University of Southampton, Southampton SO17 1BJ, UK. ' School of Electronics and Computer Science, University of Southampton, Southampton SO17 1BJ, UK

Abstract: A sparse regression modelling technique is developed using a generalised kernel model in which each kernel regressor has its individually tuned position (centre) vector and diagonal covariance matrix. An orthogonal least squares forward selection procedure is employed to append the regressors one by one. After the determination of the model structure, namely the selection of an appropriate number of regressors, the model weight parameters are calculated from the Lagrange dual problem of the original least squares problem. Different from the least squares support vector regression, this regression modelling procedure involves neither reproducing kernel Hilbert space nor Mercer decomposition concepts. As the regressors used are not restricted to be positioned at training input points and each regressor has its own diagonal covariance matrix, a very sparse representation can be obtained with excellent generalisation capability. Experimental results involving two real data sets demonstrate the effectiveness of the proposed regression modelling approach.

Keywords: generalised kernel model; least squares support vector machine; SVM; orthogonal least squares; forward selection; regression; sparse modelling; learning machines.

DOI: 10.1504/IJMIC.2006.012612

International Journal of Modelling, Identification and Control, 2006 Vol.1 No.4, pp.245 - 256

Published online: 27 Feb 2007 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model

Keep up-to-date