9.2 Logistic regression for binary outcomes

Logistic regression predicts binary outcomes, extending linear regression for categorical variables. It uses a logistic function to model probabilities between 0 and 1, with applications in churn prediction, credit risk assessment, and fraud detection.

The model is fitted using maximum likelihood estimation, with coefficients interpreted as log-odds. Performance is assessed using likelihood ratio tests, ROC curves, and confusion matrices, while applications in business include customer retention and credit risk evaluation.

Logistic Regression Fundamentals

Concept of logistic regression

• Statistical method predicts binary outcomes extends linear regression for categorical dependent variables
• Uses logistic function models probability outputs range between 0 and 1
• Applications include customer churn prediction, credit risk assessment, marketing campaign response prediction, fraud detection

Fitting and interpreting models

• Maximum likelihood estimation fits model using iterative algorithms (Newton-Raphson method)
• Coefficients interpreted as log-odds indicate direction and magnitude of effect
• Odds ratios calculated as $e^{\beta}$ show change in odds for one-unit increase in predictor
• Probability estimates derived from logistic function $P(Y=1) = \frac{1}{1 + e^{-(β_0 + β_1X_1 + ... + β_nX_n)}}$

Assessing model performance

• Likelihood ratio test compares nested models using chi-square distribution
• Pseudo R-squared measures (McFadden's, Cox and Snell, Nagelkerke) assess goodness-of-fit
• ROC curves plot true positive rate vs false positive rate AUC serves as performance metric
• Confusion matrix displays true positives, true negatives, false positives, false negatives calculates accuracy, precision, recall, F1 score

Applications in business problems

• Customer churn prediction identifies factors influencing retention develops targeted strategies
• Credit risk assessment evaluates loan applicant creditworthiness sets credit limits and interest rates
• Model development involves data preprocessing, feature selection, handling imbalanced datasets, cross-validation
• Results interpretation identifies key predictors quantifies variable impact on probabilities
• Ethical considerations address bias in training data ensure fairness in predictions comply with regulations (GDPR, FCRA)