Linear Regression Calculator: Understanding Results – Simply Explained

- Hamid: What's that?
- Rambo: It's blue light.
- Hamid: What does it do?
- Rambo: It turns blue.

The linear regression analysis has been calculated, the model is ready, and now we see something like this:


What does this result tell us? This section explains how to interpret the regression table and what conclusions we can draw from the data. While we see the table from the linear regression, simply calculating and presenting it is, of course, not enough. It would be too easy if the analysis ended here. This article focuses on the individual metrics that emerge from the data analysis and how they should be understood. Some statistical background knowledge is assumed, but the theoretical foundations and background can also be reviewed here.

We know that the linear regression model looks like this:

• $\hat{y}$ is the estimated value for our target – the variable we are interested in. In the example above, this is the price of a property.
• ${\hat{\beta }}_0$ is the estimated Y-intercept.
• $X_i$ represents the explanatory / independent variables (also called regressors), such as the area or condition of a property.
• ${\hat{\beta }}_i$ represents the estimated changes of the regression line (called the regression coefficient or beta coefficient) with respect to $X_i.$

Using the linear regression equation, we examine a linear relationship within a defined research question.
The following happens when calculating this linear regression equation: The best line that most closely matches the data points X and Y is sought. Visually, this corresponds to a line that best fits through a scatter plot of X and Y. More precisely, the squared deviations between $\hat{y}$, the linear regression line, and y, the actual values of Y, are minimized. This method is called OLS (Ordinary Least Squares) or the method of least squares. A more detailed explanation and more information can be found in the introduction to linear regression..
In our example, the linear regression equation looks like this: With this knowledge, we can now understand the results of our linear regression and interpret them in the context of the respective question.

Beta coefficients of a Regression Analysis

After using the linear regression calculator, you will find the variables of the data analysis in the first column.

• $\mathrm{const}$ is the constant of the linear regression equation (the Y-intercept).
• $\mathrm{Area\; (sq\; ft)}$ is the variable whose influence on the target we want to measure (regressor/predictor).
• $\mathrm{Condition}$ is a control variable in this linear regression. There can be any number of these control variables.

The 2nd column 'coef' contains the values of the beta coefficients ${\hat{\beta }}_0,\hspace{0.17em}{\hat{\beta }}_1,...,\hspace{0.17em}{\hat{\beta }}_N$ that belong to the corresponding variables. The coefficients are defined as: and
The latter refers to the regression coefficients of the variables included in the linear regression: ${\hat{\beta }}_1$ is the coefficient of $X_1$, ${\hat{\beta }}_2$ is the coefficient of $X_2$ and so on. It is a convention to refer to the explanatory variable (also called the predictor or exogenous variable) as $X_1$, with ${\hat{\beta }}_1$ as its corresponding coefficient. All subsequent variables $X_2\mathrm{,...,}{X}_N$ are control variables. In this example, this is also reflected in the results table of the online linear regression.
First, 'const' appears as the constant of the linear model with ${\hat{\beta }}_0$, then 'Area' appears as the explanatory variable $X_1$ with its coefficient ${\hat{\beta }}_1=\mathrm{3268.2735}$. Then comes the control variable 'Condition' as $X_2$ with its coefficient ${\hat{\beta }}_2=\mathrm{124200}$.

If ${\hat{\beta }}_1$ is positive, i.e. ${\hat{\beta }}_1>0$, this indicates a positive relationship between $X_1$ and Y. If $X_1$ increases by one unit, Y increases by ${\hat{\beta }}_1$. If ${\hat{\beta }}_1$ is negative, i.e. ${\hat{\beta }}_1<0$, this indicates a negative relationship between the variable $X_1$and the target Y. The larger $X_1$ becomes, the smaller Y becomes. The same applies to ${\hat{\beta }}_2$, ${\hat{\beta }}_3$, etc.

${\hat{\beta }}_0$ is the constant in the linear regression model. The target variable Y is equal to ${\hat{\beta }}_0$ if all other $X_1\mathrm{,...,}{X}_N=0$. This can be clearly seen from the regression equation. Visually ${\hat{\beta }}_0$ corresponds to the Y-intercept in the regression plot.

The Standard Error of Linear Regression

The column "std err" stands for standard error (SE) and shows the standard errors for our regression coefficients ${\hat{\beta }}_0,...,\hspace{0.17em}{\hat{\beta }}_N$. The standard error generally refers to the standard deviation, i.e. dispersion, of a parameter. The standard deviation itself, however, is the dispersion of the raw data points. In the case of linear regression, the standard error is therefore the standard deviation of the beta coefficients ${\hat{\beta }}_0,...,\hspace{0.17em}{\hat{\beta }}_N$. It indicates how much each ${\hat{\beta }}_i$ in the sample differs on average from the true ${\stackrel{}{\beta }}_i$ in the population. The standard error is defined by :

with

• $n$ as sample size,
• $\mathrm{Var}\left(X_i\right)$ as variance of the predictor $X_i$ and
• $\mathrm{Var}\left(\epsilon \right)$ as the variance of the residuals ε from the regression.
• $\sigma$ is the corresponding square root of the variance – that is, the respective standard deviation.

The standard errors increase (i.e. the estimated regression coefficients are less precise) when the variance of the residuals $\epsilon$ is large. A high variance of the residuals $\epsilon$ indicates that the linear regression line does not fit the data particularly well. Unsurprisingly, the standard error decreases with increasing sample size n, as the estimate for the beta coefficients becomes closer to the population parameters. Similarly, a large spread (variance) of the regressor $X_i$ is beneficial: As ${\sigma }_{X_i}$ increases, the estimation of the coefficients becomes more precise. More varied and "diverse" data points help to determine the linear regression line more accurately.

In addition to the "normal" standard error, there are also so-called "robust" standard errors. These robust standard errors have a key advantage in that they take into account heteroskedastic data. Heteroskedastic data is data that does not have a constant variance. This means that the dispersion of the data (along one or more dimensions) increases or decreases. When this happens, the standard error is biased. This leads to incorrect conclusions about beta coefficients, confidence intervals, etc.

However, robust standard errors are generally larger than normal standard errors. As a result, the average deviation from the true ${\stackrel{}{\beta }}_i$ increases, making it more difficult to estimate the true parameter ${\stackrel{}{\beta }}_i$. Consequently, it becomes more difficult to obtain statistically significant results because the test statistic T becomes smaller (SE is in the denominator) and the confidence interval around the null hypothesis becomes wider. However, if the calculated ${\hat{\beta }}_i$ coefficients fall outside a confidence interval with robust standard errors, one can be even more confident that the observed effect of ${\hat{\beta }}_i$ on Y is real. There are several types of robust standard errors; for those interested in more detail, see HC1, HC2, HC3, HC4 and Eicker, White, Huber, MacKinnon. A possible definition of robust standard errors (RSE) is as follows:


In the robust standard error equation, the unequal variance is accounted for by an interaction term $\left(X_i-E\left(X_i\right)\right)$. If $X_i$ deviates significantly from the mean $E\left(X_i\right)$, the multiplication with the residual e "penalizes" the RSE more. A wider distribution of Y at the edges (with respect to X) of the data contributes more to the RSE. Thus, the RSE reflects the increased spread of Y at large (or small, depending on the case) X values. The normal standard error does not take this uneven variance into account. However, it can be seen that if the data does indeed have constant variance, i.e. is homoscedastic, then RSE ≈ SE.

The Test Statistic T of the Linear Regression Model

Under the column "t" you will find the test statistics for the respective beta coefficients ${\hat{\beta }}_0,...,\hspace{0.17em}{\hat{\beta }}_N$. The test statistic T is an aggregate value calculated from the available data sample. T is used to decide whether the effect of $X_i$ on Y is also statistically significant. Statistical significance refers to the situation where the effect of $X_i$ actually exists in principle and is not just a result of chance (see null hypothesis). This concept is discussed in more detail in the article on hypothesis testing. The test statistic T measures the extent to which our data reflect the null hypothesis $H_0$, and whether or not we can reject $H_0$. T is defined differently for each statistical parameter; in the case of linear regression and beta coefficients, T is defined as follows: $$T\hspace{0.17em}=\hspace{0.17em}\frac{{\hat{\beta }}_{\mathrm{i}}-{{\stackrel{}{\beta }}_i}^{H_0}}{\mathrm{SE}\left({\hat{\beta }}_{\mathrm{i}}\right)}$$ Because T is calculated from a sample of data, it is also subject to random variation, which is represented by the t-distribution.
For example, if T is greater than +1.96 or less than -1.96, the regression coefficient is considered statistically significant with a 5% probability of error (also known as the significance level Alpha). This means that the influence of $X_i$ on Y is real and more than just chance. "Mere" chance is represented by the null hypothesis $H_0:{{\stackrel{}{\beta }}_i}^{H_0}=0$.
The values for the test statistic T vary depending on the significance level being considered or the desired probability of error. The values for T correspond to quantiles of the t-distribution and can be found in corresponding tables. The most relevant t-values and their associated metrics are:

Significance level Alpha	Critical value t	Confidence interval
5%	|1.96|	95%
1%	|2.576|	99%
0.01%	|3.291|	99.99%

The p-value of Regression Analysis

The p-value indicates the probability of obtaining a T' that is even more extreme than the calculated T from the available data sample. Thus, p-values represent probabilities under which the null hypothesis would be (falsely) rejected.
For example, if $\mathrm{p-value}\le \mathrm{0.05}$ (or 5%), the beta coefficient is considered statistically significant and the influence of X on Y is real and not due to chance. This statement can then be made at the 5% significance level, which means there is a 5% chance of error. Additional p-values, along with their corresponding significance levels and T-values, are as follows:

Significance level Alpha / p-value	Critical value t	Confidence interval
5%	|1.96|	95%
1%	|2.576|	99%
0.01%	|3.291|	99.99%

It is common to see decimal values in a regression table like the one shown above with an e, such as $7.945e-11$. This is scientific notation and means that the number $\mathrm{7.945}$ is multiplied by ${\mathrm{10}}^{\mathrm{-11}}$. The exponent -11 indicates that the decimal point is moved 11 places to the left. Therefore, the number $7.945\times {10}^{-11}=0.00000000007945$.

Confidence Intervals of Regression Coefficients

The last two columns show the borders of the 95% confidence intervals around $H_0:{{\stackrel{}{\beta }}_i}^{H_0}=0$. Since the beta coefficients also vary and can take different values depending on the sample, their distribution can be described by intervals. The second to last column on the left shows the lower bound, i.e. the 2.5% quantile of this distribution. The last column, the right one, represents the 97.5% quantile. If a calculated ${\hat{\beta }}_i$ falls outside the 95% confidence interval, it can be said with a 5% probability of error that ${\hat{\beta }}_i$ is statistically significant, and the null hypothesis $H_0$ can be rejected. In this case, it is said that the corresponding variable $X_i$has a real effect on the target Y.
There are several types of confidence intervals; in statistical regression analysis, confidence intervals that are closed at both the upper and lower ends are usually considered. The formula for their calculation is: The formula for their calculation is:

In general, ${{\stackrel{}{\beta }}_i}^{H_0}=0$ .

Conclusion: Lineare Regression Online

- Gollum: My Precious!

We now understand each column of the online linear regression table and are ready to interpret the results of the data analysis.
We'll start with the second row, since it contains the results for the variable we're interested in: $X_1$ (area). It lists all the metrics relevant to the statistical significance of the variable $X_1$, which in this example is the area of a property. The results indicate that area does indeed have a real impact on the price of a property (who would have guessed?!) and that the observed ${\hat{\beta }}_1$ does not just differ from 0 by chance. Of course, it is more interesting to describe the strength of this effect. In this case, the price increases by $3268.2735 per additional square meter, since ${\hat{\beta }}_1=3268.2735$.

The following rows in the regression table, namely $X_2,...,X_2$, represent the results for ${\hat{\beta }}_2,...,{\hat{\beta }}_N$ of the control variables, such as the condition of a property in this case. Recall that it is a convention to denote the explanatory variable (also called predictor or exogenous variable) as $X_1$ with ${\hat{\beta }}_1$ as the corresponding coefficient. All subsequent variables $X_2\mathrm{,...,}{X}_N$ are control variables. Again, we can see which ones are statistically significant.
The top row contains the evaluation for ${\hat{\beta }}_0$, the constant in the linear regression model. The constant represents the value of Y, when all $X_1,...,X_N=0$. It is the base value from which the linear regression line starts. Visually, ${\hat{\beta }}_0$ corresponds to the y-intercept. This initial value, the constant in the linear regression model, should always be considered in the context of the problem. Here, it makes little sense to assume a property with an area of 0 square feet.

Bonus: Linear Regression Calculator

All of the above analyses and metrics are calculated using the online linear regression calculator. Once the data analysis is complete, all of these results can be downloaded as a CSV or Excel file. This allows the results to be used for further analysis, such as inclusion in a report or scientific study. A plot of the linear regression line is also provided, which can be customized with a variety of graphical options.

The Linear Regression Online Calculator also provides additional valuable information about the data being analyzed. Further details about the data can be found below the regression results, which can provide deeper insight into the target variable under investigation. As a convenience, the interpretation of the results is provided directly. If certain analyses are no longer needed, they can be easily discarded to maintain clarity without affecting other experiments. To restart the calculator, simply reload the page.