Regression Analysis

Simple linear and Multiple Regressions

The simple linear regression denotes the relationship between the immensity of one variable in relation to the other. For instance, while variable A increase, variable B could also be increasing, or while variable A increases, B could be decreasing. While correlation measures the magnitude of the relationship between the two variables, regression quantifies the type of relationship. Multiple regression is the second type linear regression, with one dependent variable and more than one independent variables. Multiple regression's assumption is that the responding variable follows a linear function of the model's parameters, with the model containing more than one independent variables.

Ordinary least squares regression

Ordinary least regression (OLS) is one of the common statistical techniques in social sciences. OLS facilitates the prediction of values of a continuous response variable through explanatory variables while determining the strengths of the relationship between the involved variables. OLS assumption is that all variables included in the analysis have an aspect of continuity with the regression processes elevating the importance of the actual values.

Logistic and multinomial regressions

Despite the similarity between the logistic regression with the multiple linear regression, the former is used for a dichotomous outcome variable. The conventional coding for logistic regression outcome variables is 0-1. 0 shows the absence of a characteristic and 1 shows availability of the same. Multinomial regression applies in situations of logistic regressions with the outcome variables having different categories.

Ridge and lasso regressions

Ridge regression perfectly fits situations characterized by numerous predictors, with all having a non-zero coefficient and emanating from different normal distributions. Specifically, this regression works well with many predictors bearing minimal effect and curtailing linear regression models with many predictors from a poor determination and displaying high variance. Lasso regressions apply in situation containing domains with huge data sets. Computing such data necessitates the use of fast and efficient algorithms. Lasso does not display vigor in high correlations involving predictors and will pick on predictor leaving the other and disintegrates when all predictors are identical.

Multi-collinearity

Multi-collinearity is the inclusion of extra variables, raising the collinearity of the entire data set of the explanatory variables, reaching a dangerous level. The attainment of the harmful level occurs if the additional variables have a close correlation with one or more linear combination of variables included in the model.

Coefficient of determination

The coefficient of determination, (R² ), is commonly used as a measure to select models and variables. R² does not outline the quality of predicted values and how the observed values could potentially influence the predicted values.

Testing of significance

Test of significance is one of the statistical inferences to evaluate the data provided by evidence about some data claims on the population. Testing of significance is one of the statistical processes to comparing observed data against the claim, which is the hypothesis. The expression of results of the significance test are in form of a probability.

Regression Equation Modeling

Confidence and prediction intervals

Prediction intervals refer to the range whose determination is through the observed data to demote the expected fall for future data range. The prediction interval incorporates a random variable's distribution, using a confidence coefficient to describe observed data for the future. Reporting and highlighting confidence data provide a better understanding of the data, enabling comparisons of different studies on the same topic.