CALCULATING THE SAMPLE SIZE FOR ORDINARY LEAST SQUARE ESTIMATION IN PRESENCE OF MULTICOLLINEARITY

Abstract

The relationship between a variable (the response variable) and the scores of several other variables (the independent variables) may be described using multiple linear regression analysis. This study compares the L2 (ridge) shrinkage method and least squares shrinkage method when multicollinearity is present in a dataset across various sample sizes. For different sample sizes $(n = 25, n = 50, n = 200,$ and $n = 1000)$, this process was repeated. The relationship between larger sample sizes and covariance was not linear in the simulated data. The results demonstrated that L2 regression is best and generates parsimonious models in the presence of multicollinearity; the higher the degree of multicollinearity, the smaller the shrinkage parameter. The L2 regularization technique also helps to reduce standard errors of regression coefficients and the prediction error of the generated model. This implies that for every change in the dataset values, there is always an optimal value of the shrinkage parameter  that minimizes multicollinearity and produces more stable and reliable regression models. In moderation studies where we would like to keep all of the predictor variables, L2 regularization would be the best alternative. Increasing sample size gives stable results after estimation as it helps to reduce the standard errors of the regression coefficients of the predictor variables. It is also the best method to use for greatly inflated standard errors of OLS regression coefficients. OLS works best for independent samples, but correlated covariates should be handled with modern regression methods (L2).

Received: February 24, 2023
Revised: May 10, 2023
Accepted: June 7, 2023