Tree-structured Partial Least Squares with an Application to Orthodontics Data

Abstract

Partial least squares (PLS) regression is an alternative to classical regression for handling multicollinearity by combining the merits of principal component analysis and multiple linear regression. The aim of PLS is to predict a set of responses from a set of predictors by finding latent variables that capture the variability in both responses and predictors. Latent variables used in PLS are defined as the linear combinations of original variables. Among predictors, some variables can come from an extremely different nature or some can be indirectly related to responses. In that case, taking all predictors into account for extracting latent variables is not suitable from the model selection point of view. More latent variables can be selected for model fitting. If the variable from other natures has a different role rather then a predictor in PLS, it can be possible to improve the prediction performance by using the less number of latent variables. A disadvantage of PLS would be the deficiency in data visualization and model interpretation among others, which may be as important as building an optimal predictive model. A recursive partitioning algorithm emerges as one of the solutions capable of achieving these purposes. We here propose new piecewise regression by combining the model-based recursive partitioning and PLS in order to improve prediction performance, and provide a visually interpretable model. Recursive partitioning, also known as tree-structured modeling, has been widely used because they allow us to give easy data visualization and interpretation. The objective of our proposed method is to select the most relevant predictors recursively, and provide more accurate prediction performance. Instead of fitting a global PLS to the whole data, one might fit a collection of local PLS to subsets of the data so that a better fit and higher predictive accuracy are obtained. We alleviate variable selection bias using the merits of the residual analysis approach of [1] and conditional inference of [2]. Our developed software program can be obtained from the authors upon request.