11 Simple Linear Regression Models Essay

Submitted By Mark-Elakawi
Words: 1362
Pages: 6
Open Document
Performance Evaluation:

Simple Linear Regression Models
Hongwei Zhang http://www.cs.wayne.edu/~hzhang Statistics is the art of lying by means of figures.
--- Dr. Wilhelm Stekhel

Acknowledgement: this lecture is partially based on the slides of Dr. Raj Jain.

Simple linear regression models
Response Variable: Estimated variable
Predictor Variables: Variables used to predict the response
Also called predictors or factors

Regression Model: Predict a response for a given set of predictor variables
Linear Regression Models: Response is a linear function of predictors Simple Linear Regression Models: Only one predictor

Outline
Definition of a Good Model
Estimation of Model parameters
Allocation of Variation
Standard deviation of Errors
Confidence Intervals for Regression Parameters
Confidence Intervals for Predictions
Visual Tests for verifying Regression Assumption

Outline
Definition of a Good Model
Estimation of Model parameters
Allocation of Variation
Standard deviation of Errors
Confidence Intervals for Regression Parameters
Confidence Intervals for Predictions
Visual Tests for verifying Regression Assumption

Definition of a good model?

Good models (contd.)
Regression models attempt to minimize the distance measured vertically between the observation point and the model line (or curve) The length of the line segment is called residual, modeling error, or simply error

The negative and positive errors should cancel out => Zero overall error Many lines will satisfy this criterion

Choose the line that minimizes the sum of squares of the errors

Good models (contd.)
Formally,
where,

is the predicted response when the predictor variable is

x. The parameter b0 and b1 are fixed regression parameters to be determined from the data.
Given n observation pairs {(x1, y1), …, (xn, yn)}, the estimated response for the i-th observation is:

The error is:

Good models (contd.)
The best linear model minimizes the sum of squared errors
(SSE):

subject to the constraint that the overall mean error is zero:

This is equivalent to the unconstrained minimization of the variance of errors (Exercise 14.1)

Outline
Definition of a Good Model
Estimation of Model parameters
Allocation of Variation
Standard deviation of Errors
Confidence Intervals for Regression Parameters
Confidence Intervals for Predictions
Visual Tests for verifying Regression Assumption

Estimation of model parameters
Regression parameters that give minimum error variance are:

where,

Example 14.1

Example (contd.)

Example (contd.)

Derivation of regression parameters?

Derivation (contd.)

Derivation (contd.)

Least Squares Regression vs. Least Absolute
Deviations Regression?
Least Squares Regression

Least Absolute Deviations
Regression

Not very robust to outliers

Robust to outliers

Simple analytical solution

No analytical solving method
(have to use iterative computation-intensive method)

Stable solution

Unstable solution

Always one unique solution

Possibly multiple solutions

The unstable property of the method of least absolute deviations means that, for any small horizontal adjustment of a data point, the regression line may jump a large amount. In contrast, the least squares solutions is stable in that, for any small horizontal adjustment of a data point, the regression line will always move only slightly, or continuously.

Outline
Definition of a Good Model
Estimation of Model parameters
Allocation of Variation
Standard deviation of Errors
Confidence Intervals for Regression Parameters
Confidence Intervals for Predictions
Visual Tests for verifying Regression Assumption

Allocation of variation

Allocation of variation (contd.)
The sum of squared errors without regression would be:

This is called total sum of squares or (SST). It is a measure of

y's variability and is called variation of y. SST can be computed as follows:

Where, SSY is the sum of squares of y (or Σy2). SS0 is the sum of squares of

and is equal to

Allocation of
Show More

Related Documents: 11 Simple Linear Regression Models Essay

Simple Linear Regression

Chapter 11 – Simple linear regression Types of Regression Models (Sec. 11-1) Linear Regression: • - Outcome of Dependent Variable (response) for ith experimental/sampling unit • - Level of the Independent (predictor) variable for ith experimental/sampling unit • - Linear (systematic) relation between Yi and Xi (aka conditional mean) • - Mean of Y when X=0 (Y-intercept) • - Change in mean of Y when X increases by 1 (slope) • - Random error term Note that…

Words 447 - Pages 2

BANA 5368 Final Exam Essay

Final Exam 1. (a) A regression of “hrly_wage” on “female” was computed using Gretl. The coefficient on “female” in context of “hrly_wage” is -2.51. Interpretation in the context of the problem is the difference in a person’s estimated mean amount of hourly wages (in dollars) between male and female is approximately -$2.51. This shows that there is a negative relationship with “female” and “hourly wage.” This implies a negative relationship between “hourly wage” and “gender.” Thus, hourly wages…

Words 1241 - Pages 5

Essay on Applying Time

RES/342 Instructor: Leo M September 11, 2011 Applying Time Series Methodologies To forecast variables such as sales and demand, models such as simple linear regression, simple moving averages, and weighted moving averages. To be able to arrive at a reasonably accurate forecast, variation in the data provided needs to be analyzed. Variations can be cyclical, seasonal, secular, or irregular. This simulation illustrates the use of regression and time series methodologies to forecast…

Words 347 - Pages 2

Project Last Essay

CONTENTS PART I. SIMPLE REGRESSION ANALYSIS 4 (a) Statement of estimated equation 4 (b) Interpretation of coefficient 4 (c) Comment on the significance of the model 4 (d) Test of regression assumptions 5 PART II. MULTIPLE REGRESSION ANALYSIS 6 (a) Explanation of selected variables for the model 6 (b) Statement of estimated equation 6 (c) Interpretation of coefficients 7 (d) Comment on the significance of the model 7 (e) Test for regression assumptions 8 APPENDIX LIST…

Words 1808 - Pages 8

Analysis On Forecasting

forecasting models – Compare moving averages, exponential smoothing, and trend time-series models – Seasonally adjust data. – Understand Delphi and other qualitative decision-making approaches 1 Learning Objectives (2) Students will be able to: – Identify independent and dependent variables and use them in a linear regression model. – Compute a variety of error measures. 2 Topics (1) • Importance of Forecasting • Steps of Forecasting • Types of Forecasting • Scatter Diagrams • Time-Series Models • Moving…

Words 5226 - Pages 21

Assignment 4 Essay

Assignment-4 (Chs. 10, 12 and 13 : these chapters are marked different in the 7th ed. Chs 12 and 13 of the 6th ed are marked as Chs 13 and 14 in the 7th ed) Due by Midnight of Sunday, June 29th, 2014 (Dropbox 4): Total 125 points True/False (two points each) Chapter10 1. In an experiment involving matched pairs, a sample of 15 pairs of observations is collected. The degree of freedom for the t statistic is 14. true 2. In testing the difference between two means from two independent populations,…

Words 2826 - Pages 12

Electric: Statistical Hypothesis Testing and Significance Approach Essay example

Zhaoxia Xu Agenda  Linear regressions  Simple linear model  Ordinary Least Squares  Estimations  Inferences  Applications 1 Financial Econometrics  Application of statistical techniques to problems in finance  Estimating and testing relationships between variables as suggested by theories in finance, e.g. CAPM or APT  Modeling and forecasting asset returns  Evaluating the performance of portfolio management  Examining determinants of firms’ financial decisions…

Words 2887 - Pages 12

Written Assignment 2 Essay

the simple linear regression equation Y = β0 + β1X1 + ǫ, and superimpose your regression line on the scatterplot in part (a). β1= Sx1y/Sx1x1= 18762573.43/17526467.18 = 1.07 β0 = ˉy- β1ˉx1 = 6290.714 – 1.0705x6353.214 = -510.58 ≈ -511 Thus, the regression equation is: Y = -511+ 1.07X1 + ǫ (d) Enter the data from the table above into two separate columns of a Minitab worksheet. Now use Minitab to perform a simple linear regression of Y…

Words 865 - Pages 4

Essay on Statistics Curriculum

Chapter 6: The Standard Deviation as a Ruler and the Normal Model (Pages 83­97) How does adding, subtracting, multiplying or dividing by a constant change the center and/or spread of a variable? When can standardization be used to compare values? Does standardization use standard deviation as a ruler? When is the normal model appropriate? How do we calculate the z­score of an observation? How do we compare the values of two different variables y using their z­scores When using the normal model what percentage of observations should be within one…

Words 4395 - Pages 18

Case Study of Store24 (a): Managing Employee Retention Essay

can explain more of the variation in profit. According to the result of Model 3 regression analysis, we come to the model equation as follows: Expected profit= 60731.36 + 842.27(Manager Tenure) + 1003.62(Crew Tenure) + 3.48(Population) - 26708.90(Competitors) + 73146.90(Pedestrian 2) + 101085.25(Pedestrian 3) + 117757.20(Pedestrian 4) + 168799.40(Pedestrian 5) + 88856.49(Resident) + 59846.77*(hours24) We run a simple calculation that increasing 1 percent of average manager tenure increases…

Words 1647 - Pages 7