Comparison of Ridge Penalized Logistic Model and Ridge Penalized Simple Linear Model in Dichotomous Response

Abstract

Notwithstanding the situation where the value of response is a continuous, there are many other situations where the value of response is a discrete or categorical value, representing that a subject belongs to.

In this regard, logistic regression is one of the most widely used model for classication problem. Generally people believe that the linear regression analysis is not appropriate when we want to build a model to predict what category a new subject should belong to. Nonetheless, simple linear regression can also model dichotomous predictors using linear probability models.

Both models have its own pros and cons. It is important to choose appropriate model depends on the situation. Therefore, in this paper, we investigate that two methods, especially ridge penalized logistic regression and ridge penalized linear regression models, can produce similar results with the same data set in case of dichotomous classication problem. The comparison between the methods is based on the area under the ROC curve value. The goal of this paper is not to discourage the current practice but rather to present an idea that there's no huge difference between two methods.