AModifiedLeastAngleRegressionAlgorithmforHierarchicalInteraction

Abstract

Variable selection is important in high dimensional regression. Traditional variable selection methods such as stepwise selection are unstable which means that the set of the selected variables is sensitive according to the change of data sets. As an alternative to those methods, a series of sparse penalized methods are used for estimation and variable selection simultaneously. The full set of LASSO solutions can be calculated by a minor modification of the LARS algorithm. In many important practical problems, the main effect alone may not be enough to capture the relationship between the response and predictors, and high-order interactions are often of interest to scientific researchers. In considering two-way interaction models with a large number of covariates, we often would like to determine a smaller subset that exhibits strong effects on the response variable have been suggested. Considering all possible interactions, however, is almost impossible due to computational burden when the number of covariate is large. To resolve this problem, the heredity structure between the main and interaction effect can be considered, algorithms for LASSO with heredity structure. However these algorithms cannot be executed if the number of main effects is large since still computational burden is large. To resolve this issue, we suggest a hierarchical LARS algorithm which can be parallelized easily with MPI. The proposed hierarchical LARS is a modified version of LARS, but it is more faster than LARS and has comparable prediction accuracy. It can be scaled up since it is possible to be executed in parallel process. MPI is a well known parallel model and we suggest a MPI-version of hierarchical LARS.