🫁Intro to Biostatistics Unit 6 – Regression Analysis

What's Regression Analysis?

• Statistical method used to examine the relationship between a dependent variable and one or more independent variables
• Helps predict the value of the dependent variable based on the values of the independent variables
• Estimates the strength and direction of the relationship between variables
• Useful for understanding how changes in independent variables affect the dependent variable
• Can be used for prediction, forecasting, and inferring causal relationships (with caution)
• Widely applied in various fields, including biostatistics, economics, and social sciences
• Provides a quantitative measure of the impact of each independent variable on the dependent variable

Types of Regression Models

• Linear regression
  • Assumes a linear relationship between the dependent and independent variables
  • Simple linear regression involves one independent variable
  • Multiple linear regression involves two or more independent variables
• Logistic regression
  • Used when the dependent variable is binary or categorical (e.g., presence or absence of a disease)
  • Estimates the probability of an event occurring based on the independent variables
• Polynomial regression
  • Models non-linear relationships between the dependent and independent variables
  • Includes higher-order terms (squared, cubed, etc.) of the independent variables
• Stepwise regression
  • Iterative process of adding or removing independent variables based on their statistical significance
  • Helps identify the most relevant variables for the model
• Ridge regression and Lasso regression
  • Used to handle multicollinearity (high correlation) among independent variables
  • Apply regularization techniques to shrink the coefficients of less important variables

Key Concepts and Terminology

• Dependent variable (response variable)
  • The variable being predicted or explained by the model
  • Usually denoted as Y
• Independent variables (predictor variables, explanatory variables)
  • The variables used to predict or explain the dependent variable
  • Usually denoted as X1, X2, etc.
• Coefficients (parameters)
  • Numerical values that represent the change in the dependent variable for a one-unit change in the corresponding independent variable, holding other variables constant
  • Denoted as β0 (intercept), β1, β2, etc.
• Residuals
  • The differences between the observed values of the dependent variable and the predicted values from the regression model
• R-squared (coefficient of determination)
  • Measures the proportion of variance in the dependent variable that is explained by the independent variables
  • Ranges from 0 to 1, with higher values indicating a better fit of the model to the data
• P-value
  • Indicates the statistical significance of the relationship between an independent variable and the dependent variable
  • A small p-value (typically < 0.05) suggests that the relationship is unlikely to have occurred by chance

Building a Regression Model

• Define the research question and identify the dependent and independent variables
• Collect and preprocess the data
  • Clean the data by handling missing values, outliers, and inconsistencies
  • Transform variables if necessary (e.g., log transformation for skewed data)
• Explore the data using descriptive statistics and visualizations
  • Examine the distribution of variables and their relationships
  • Check for potential multicollinearity among independent variables
• Select the appropriate regression model based on the nature of the dependent variable and the relationships observed in the data
• Estimate the model coefficients using a fitting method (e.g., least squares, maximum likelihood)
• Assess the model's goodness of fit and performance
  • Evaluate R-squared, adjusted R-squared, and other fit statistics
  • Check the significance of the coefficients using p-values and confidence intervals
• Validate the model using techniques such as cross-validation or holdout samples
• Refine the model if necessary by adding or removing variables, transforming variables, or considering interaction terms

Interpreting Regression Results

• Coefficient estimates
  • Represent the change in the dependent variable for a one-unit change in the corresponding independent variable, holding other variables constant
  • The sign of the coefficient indicates the direction of the relationship (positive or negative)
• Standard errors
  • Measure the precision of the coefficient estimates
  • Smaller standard errors indicate more precise estimates
• P-values and confidence intervals
  • Assess the statistical significance of the coefficients
  • A small p-value (typically < 0.05) and a confidence interval not containing zero suggest a significant relationship
• Residual analysis
  • Examine the distribution of residuals to check for model assumptions (e.g., normality, homoscedasticity)
  • Identify potential outliers or influential observations
• Practical significance
  • Consider the practical implications of the coefficient estimates
  • Assess whether the magnitude of the effects is meaningful in the context of the problem

Assumptions and Diagnostics

• Linearity
  • The relationship between the dependent variable and independent variables should be linear
  • Can be assessed using residual plots or by adding non-linear terms to the model
• Independence
  • The observations should be independent of each other
  • Violations can occur with time series data or clustered data
• Normality
  • The residuals should be normally distributed
  • Can be assessed using histograms, Q-Q plots, or statistical tests (e.g., Shapiro-Wilk test)
• Homoscedasticity
  • The variance of the residuals should be constant across all levels of the independent variables
  • Can be assessed using residual plots or statistical tests (e.g., Breusch-Pagan test)
• No multicollinearity
  • The independent variables should not be highly correlated with each other
  • Can be assessed using correlation matrices or variance inflation factors (VIF)
• Influential observations and outliers
  • Identify observations that have a disproportionate impact on the model
  • Can be assessed using leverage values, Cook's distance, or residual plots

Applications in Biostatistics

• Epidemiology
  • Identifying risk factors for diseases
  • Estimating the strength of associations between exposures and health outcomes
• Clinical trials
  • Evaluating the effectiveness of treatments or interventions
  • Adjusting for confounding variables to isolate the treatment effect
• Genetics and genomics
  • Associating genetic variants with phenotypic traits or diseases
  • Predicting disease risk based on genetic profiles
• Environmental health
  • Assessing the impact of environmental exposures on health outcomes
  • Identifying environmental risk factors for diseases
• Health services research
  • Analyzing factors associated with healthcare utilization and costs
  • Predicting patient outcomes based on demographic and clinical characteristics

Common Pitfalls and How to Avoid Them

• Overfitting
  • Occurs when the model is too complex and fits the noise in the data rather than the underlying patterns
  • Can be avoided by using model selection techniques (e.g., stepwise regression, regularization) and validating the model on independent data
• Underfitting
  • Occurs when the model is too simple and fails to capture important relationships in the data
  • Can be avoided by considering a wider range of variables and non-linear relationships
• Extrapolation
  • Applying the model to predict outcomes outside the range of the observed data
  • Can lead to unreliable predictions and should be done with caution
• Confounding
  • Occurs when an unmeasured variable influences both the dependent and independent variables, leading to spurious associations
  • Can be addressed by carefully selecting variables, using randomization in experiments, or applying statistical techniques (e.g., propensity score matching)
• Misinterpretation of coefficients
  • Interpreting coefficients without considering the scale and units of the variables
  • Can be avoided by carefully examining the units and scale of the variables and interpreting the coefficients in the appropriate context
• Ignoring model assumptions
  • Failing to check and address violations of model assumptions
  • Can lead to biased and unreliable results
  • Should be addressed by assessing assumptions using diagnostic tools and applying appropriate remedial measures (e.g., transformations, robust standard errors)