A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions

• Published in: Communications in statistics. Simulation and computation Vol. 39; no. 8; pp. 1639 - 1670
• Main Authors: Månsson, Kristofer, Shukur, Ghazi, Golam Kibria, B. M.
• Format: Journal Article
• Language: English
• Published: Colchester Taylor & Francis Group 01-09-2010; Taylor & Francis; Taylor & Francis Ltd
• Subjects: Computation; Computer simulation; Estimation; Estimators; Exact sciences and technology; Inference from stochastic processes; time series analysis; Linear inference, regression; LSE; Mathematics; Mean square errors; MSE; Multicollinearity; Numerical analysis; Numerical analysis. Scientific computation; Numerical methods in probability and statistics; PRESS; Primary 62J07; Probability and statistics; Probability theory and stochastic processes; Regression; Ridge regression; Ridges; SAMHÄLLSVETENSKAP; Sciences and techniques of general use; Secondary 62F10; Simulation; SOCIAL SCIENCES; Statistics; Statistics, computer and systems science; Statistics/Econometrics; Statistik; Statistik, data- och systemvetenskap; Stochastic processes; Studies; Ridge regression; Freedom degree; Correlation; Parameter estimation; Error estimation; Sample size; Primary 62J07; Prediction theory; Multivariate analysis; Stochastic process; Statistical simulation; Mean square error; F distribution; Distribution function; Student distribution; Sample survey; MSE; LSE; Secondary 62F10; Estimation; Prediction; Statistical association; Statistical estimation; Independent variable; PRESS; Statistical method; Statistical regression; Sampling theory; Simulation; Correlation analysis; Regression model; Multicollinearity; Numerical simulation; Filtering theory
• ISSN: 0361-0918; 1532-4141; 1532-4141
• DOI: 10.1080/03610918.2010.508862

Based on the work of Khalaf and Shukur (
2005
), Alkhamisi et al. (
2006
), and Muniz et al. (
2010
), this article considers several estimators for estimating the ridge parameter k. This article differs from aforementioned articles in three ways: (1) Data are generated from Normal, Student's t, and F distributions with appropriate degrees of freedom; (2) The number of regressors considered are from 4-12 instead of 2-4, which are the usual practice; (3) Both mean square error (MSE) and prediction sum of square (PRESS) are considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has negative effect on the MSE and PRESS. However, increasing the number of regressors has positive effect on MSE and PRESS. When the sample size increases the MSE decreases even when the correlation between the independent variables is large. It is interesting to note that the dominance pictures of the estimators are remained the same under both the MSE and PRESS criterion. However, the performance of the estimators depends on the choice of the assumption of the error distribution of the regression model.