Posted by Abstract: Multicollinearity – Don’t Throw The Baby (Independent Variable) Out With The Bath Water
Lessons Learned Track
Downloadable Files:
Abstract:
One of the cardinal assumptions with respect to Ordinary Least Squares Regression is the assumption that the Independent Variables are independent of each other. That is to say they are not correlated with each other. Collinear independent variables do not lead to biased estimates, however, they do produce estimates with relatively large variances. This is another way of saying that the estimate may be very far from its true value.
Besides leading to larger variances in the estimates, multicollinearity may cause some of the independent variables to appear to have little or no predictive impact. Worse yet, it sometimes makes them appear to have the opposite effect from what would be expected based on other studies and the underlying theory. This often results in the “offending” independent variable being “thrown out” of the analysis.
Experiments conducted in highly controlled environments usually do not result in multicollinearity because the experiments are designed to avoid it. In the world outside of laboratories, we don’t have the luxury of building multiple identical businesses and factories to produce products with controlled variations so that we can isolate the impact of each independent variable. We have to work with the data that fate presents us.
This presentation will explore and discuss from a pragmatic standpoint, alternative solutions to this problem. It will offer up variable substitution, analogous relationships, pooled regression and ridge regression as potentially viable alternatives to simply dropping an independent variable out of the analysis.
Author:
Jim Byrd
Mr. Byrd is currently the Manager of Cost Estimating and Risk Analysis for the Integrated Systems Sector of Northrop Grumman Corporation (NGC). He has been with NGC for the last thirty years, working in various estimating and pricing positions on a wide variety of aircraft programs.
Before joining NGC he was an Assistant Professor for six years, teaching Economics, Statistics and Management Science/Operations Research. He has also been an adjunct lecturer at various colleges and universities.
He earned a B.A. in Economics from Cal-State University, Fullerton in 1971; and an M.A. in Economics from the University of Southern California in 1972. In addition, extensive doctoral coursework was completed in Economics at the University of Southern California as well as graduate work in Finance and Management Science/Operations Research.