Honors Statistics Unit 12 Review: Linear Regression and Correlation

Key Concepts

• Linear regression models the relationship between a dependent variable and one or more independent variables
• Correlation measures the strength and direction of the linear relationship between two variables
• Scatter plots visualize the relationship between two quantitative variables (height and weight)
• The line of best fit minimizes the sum of the squared vertical distances between the data points and the line
• The correlation coefficient (r) quantifies the strength and direction of the linear relationship between two variables
  • Ranges from -1 to 1, with 0 indicating no linear relationship
  • Positive values indicate a positive linear relationship, while negative values indicate a negative linear relationship
• Simple linear regression involves one independent variable, while multiple linear regression involves two or more independent variables
• Residuals represent the differences between the observed values and the predicted values from the regression line

Linear Relationships

• A linear relationship between two variables means that as one variable changes, the other variable changes at a constant rate
• The slope of the line represents the rate of change in the dependent variable for a one-unit change in the independent variable
• The y-intercept represents the value of the dependent variable when the independent variable is zero
• Positive linear relationships have a positive slope, indicating that as one variable increases, the other variable also increases (temperature and ice cream sales)
• Negative linear relationships have a negative slope, indicating that as one variable increases, the other variable decreases (age and reaction time)
• Perfect linear relationships have all data points falling exactly on the line of best fit
  • Rarely occur in real-world data due to measurement error and other factors

Correlation Coefficient

• The correlation coefficient (r) measures the strength and direction of the linear relationship between two variables
• Ranges from -1 to 1, with 0 indicating no linear relationship
  • Values closer to -1 or 1 indicate a stronger linear relationship
  • Values closer to 0 indicate a weaker linear relationship
• Positive correlation coefficients indicate a positive linear relationship, where both variables increase or decrease together (height and weight)
• Negative correlation coefficients indicate a negative linear relationship, where one variable increases as the other decreases (age and physical fitness)
• The square of the correlation coefficient (r²) represents the proportion of variance in the dependent variable explained by the independent variable
• Correlation does not imply causation, as other factors may influence the relationship between the variables

Scatter Plots and Line of Best Fit

• Scatter plots display the relationship between two quantitative variables, with each data point represented by a dot
• The independent variable is plotted on the x-axis, while the dependent variable is plotted on the y-axis
• The line of best fit is a straight line that best represents the trend in the data
  • Minimizes the sum of the squared vertical distances between the data points and the line
  • Can be used to make predictions for values of the dependent variable based on the independent variable
• Outliers are data points that deviate significantly from the overall pattern and can influence the line of best fit
• The strength of the linear relationship can be visually assessed by the proximity of the data points to the line of best fit
  • Data points clustered closely around the line indicate a strong linear relationship
  • Data points scattered far from the line indicate a weak or no linear relationship

Simple Linear Regression

• Simple linear regression models the relationship between a dependent variable and a single independent variable
• The regression equation is written as $y = b_0 + b_1x$, where $y$ is the dependent variable, $x$ is the independent variable, $b_0$ is the y-intercept, and $b_1$ is the slope
• The least squares method is used to estimate the values of the y-intercept and slope that minimize the sum of the squared residuals
• The coefficient of determination (R²) measures the proportion of variance in the dependent variable explained by the independent variable
  • Ranges from 0 to 1, with higher values indicating a better fit of the model to the data
• Confidence intervals and prediction intervals can be constructed around the regression line to quantify the uncertainty in the estimates and predictions

Interpreting Regression Results

• The slope ($b_1$) represents the change in the dependent variable for a one-unit change in the independent variable, holding all other variables constant
• The y-intercept ($b_0$) represents the value of the dependent variable when the independent variable is zero
• The p-value associated with the slope tests the null hypothesis that the slope is equal to zero (no linear relationship)
  • A small p-value (typically < 0.05) indicates strong evidence against the null hypothesis, suggesting a significant linear relationship
• The standard error of the slope measures the variability of the estimated slope across different samples
• The residual standard error measures the average deviation of the observed values from the predicted values
• Residual plots can be used to assess the assumptions of linearity, homoscedasticity, and normality of the residuals

Assumptions and Limitations

• Linear regression assumes a linear relationship between the dependent and independent variables
  • Nonlinear relationships may require transformations or alternative models
• Homoscedasticity assumes that the variability of the residuals is constant across all levels of the independent variable
  • Heteroscedasticity (non-constant variance) can affect the validity of the model and the accuracy of the standard errors
• Independence of observations assumes that the residuals are not correlated with each other
  • Autocorrelation can occur in time series data or when observations are clustered
• Normality of residuals assumes that the residuals follow a normal distribution
  • Non-normality can affect the validity of confidence intervals and hypothesis tests
• Outliers and influential points can have a significant impact on the regression results and should be carefully examined
• Extrapolation beyond the range of the observed data can lead to unreliable predictions

Real-World Applications

• Linear regression is widely used in various fields to model and predict relationships between variables
• In finance, linear regression can be used to predict stock prices based on market indicators (interest rates, GDP growth)
• In healthcare, linear regression can be used to model the relationship between patient characteristics and health outcomes (age and blood pressure)
• In marketing, linear regression can be used to analyze the impact of advertising expenditure on sales
• In social sciences, linear regression can be used to study the relationship between socioeconomic factors and educational attainment
• In environmental studies, linear regression can be used to model the relationship between pollution levels and health effects
• In sports analytics, linear regression can be used to predict player performance based on various statistics (points scored and minutes played)
• Linear regression can also be used for quality control, demand forecasting, and resource allocation in various industries