📊Principles of Data Science Unit 5 – Statistical Inference & Hypothesis Testing

Key Concepts and Definitions

• Statistical inference draws conclusions about a population based on a sample of data
• Populations refer to the entire group of individuals, objects, or events of interest
• Samples are subsets of the population used to make inferences about the whole population
• Parameters are numerical summaries that describe characteristics of a population (mean, standard deviation)
• Statistics are numerical summaries calculated from sample data to estimate population parameters
• Probability distributions describe the likelihood of different outcomes in a random process
  • Discrete probability distributions have a finite or countable number of possible outcomes (binomial, Poisson)
  • Continuous probability distributions have an infinite number of possible outcomes within a range (normal, exponential)
• Hypothesis testing evaluates claims or assumptions about a population using sample data
• Null hypothesis ($H_0$) represents the default or status quo assumption about a population parameter
• Alternative hypothesis ($H_a$ or $H_1$) represents the claim or assertion being tested against the null hypothesis

Types of Statistical Inference

• Estimation involves using sample statistics to estimate unknown population parameters
  • Point estimation provides a single value estimate of a population parameter (sample mean, sample proportion)
  • Interval estimation provides a range of plausible values for a population parameter (confidence intervals)
• Hypothesis testing uses sample data to assess the plausibility of a claim or assumption about a population
  • Tests whether the observed data provides sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis
• Prediction uses patterns or relationships in sample data to forecast future outcomes or behaviors
  • Regression analysis models the relationship between predictor variables and a response variable to make predictions
• Classification assigns observations into predefined categories or classes based on their characteristics
  • Discriminant analysis and logistic regression are common classification techniques in statistics
• Clustering identifies natural groupings or structures within a dataset based on similarity or distance measures
  • K-means and hierarchical clustering are popular unsupervised learning methods for grouping observations

Probability Distributions

• Normal distribution is a symmetric, bell-shaped curve characterized by its mean ($\mu$) and standard deviation ($\sigma$)
  • Approximately 68%, 95%, and 99.7% of observations fall within 1, 2, and 3 standard deviations of the mean, respectively
• Standard normal distribution is a normal distribution with a mean of 0 and a standard deviation of 1
  • Z-scores standardize observations by measuring the number of standard deviations they are from the mean
• t-distribution is similar to the normal distribution but has heavier tails and is used for small sample sizes ($n < 30$)
  • Degrees of freedom ($df$) determine the shape of the t-distribution and are based on the sample size ($df = n - 1$)
• Binomial distribution models the number of successes in a fixed number of independent trials with a constant probability of success
  • Characterized by the number of trials ($n$) and the probability of success ($p$)
• Poisson distribution models the number of rare events occurring in a fixed interval of time or space
  • Characterized by the average rate of occurrence ($\lambda$) per unit of time or space

Sampling Methods and Sample Statistics

• Simple random sampling selects individuals from a population such that each individual has an equal chance of being chosen
• Stratified sampling divides a population into subgroups (strata) based on a characteristic and randomly samples from each stratum
  • Ensures representation of important subgroups and can increase precision of estimates
• Cluster sampling divides a population into clusters, randomly selects a subset of clusters, and samples all individuals within chosen clusters
  • Useful when a complete list of individuals in the population is not available or when clusters are geographically dispersed
• Systematic sampling selects individuals from a population at regular intervals after a random starting point
  • Easy to implement but can introduce bias if the sampling interval is related to a periodic pattern in the population
• Sample mean ($\bar{x}$) is the arithmetic average of all observations in a sample
  • Calculated as the sum of observations divided by the sample size ($\bar{x} = \frac{\sum x_i}{n}$)
• Sample variance ($s^2$) measures the average squared deviation of observations from the sample mean
  • Calculated as $s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1}$
• Sample standard deviation ($s$) is the square root of the sample variance and measures the average distance of observations from the mean
• Sample proportion ($\hat{p}$) is the fraction or percentage of observations in a sample that possess a particular characteristic of interest

Hypothesis Testing Basics

• Null hypothesis ($H_0$) represents the default or status quo assumption about a population parameter
  • Often states that there is no difference or relationship between variables
• Alternative hypothesis ($H_a$ or $H_1$) represents the claim or assertion being tested against the null hypothesis
  • Can be one-sided (greater than or less than) or two-sided (not equal to)
• Test statistic measures the discrepancy between the observed data and what is expected under the null hypothesis
  • Calculated from sample data and follows a known probability distribution under the null hypothesis (Z, t, F, chi-square)
• P-value is the probability of observing a test statistic as extreme or more extreme than the one calculated, assuming the null hypothesis is true
  • Smaller p-values provide stronger evidence against the null hypothesis
• Significance level ($\alpha$) is the threshold used to determine whether to reject the null hypothesis
  • Commonly set at 0.05, meaning there is a 5% chance of rejecting the null hypothesis when it is actually true (Type I error)
• Critical value is the value of the test statistic that corresponds to the significance level and separates the rejection and non-rejection regions
• Rejection region is the range of test statistic values that lead to rejecting the null hypothesis
  • Determined by the significance level and the direction of the alternative hypothesis (one-tailed or two-tailed)

Common Statistical Tests

• Z-test compares a sample mean to a known population mean when the population standard deviation is known and the sample size is large ($n \geq 30$)
• One-sample t-test compares a sample mean to a known population mean when the population standard deviation is unknown and the sample size is small ($n < 30$)
• Two-sample t-test compares the means of two independent samples to determine if they are significantly different from each other
  • Assumes equal variances and normal distributions for both populations
• Paired t-test compares the means of two related or dependent samples to determine if they are significantly different from each other
  • Used when observations are paired or measured on the same individuals before and after a treatment
• Analysis of Variance (ANOVA) tests for differences among the means of three or more independent groups
  • One-way ANOVA examines the effect of one categorical factor on a continuous response variable
  • Two-way ANOVA examines the effects of two categorical factors and their interaction on a continuous response variable
• Chi-square test for independence assesses whether two categorical variables are associated or independent in a population
  • Compares observed frequencies in a contingency table to expected frequencies under the assumption of independence
• Chi-square goodness-of-fit test determines whether an observed frequency distribution differs significantly from a theoretical or expected distribution

Interpreting Results and P-values

• P-value measures the strength of evidence against the null hypothesis provided by the sample data
  • Smaller p-values indicate stronger evidence against the null hypothesis
• If the p-value is less than the chosen significance level ($\alpha$), reject the null hypothesis in favor of the alternative hypothesis
  • Concludes that the observed data is statistically significant and unlikely to have occurred by chance alone
• If the p-value is greater than the chosen significance level ($\alpha$), fail to reject the null hypothesis
  • Concludes that there is insufficient evidence to support the alternative hypothesis based on the observed data
• Confidence intervals provide a range of plausible values for a population parameter with a specified level of confidence (usually 95%)
  • Narrower intervals indicate greater precision in the estimate, while wider intervals suggest more uncertainty
• Effect size measures the magnitude or practical significance of a difference or relationship
  • Cohen's d, Pearson's r, and eta-squared are common effect size measures for comparing means, correlations, and ANOVA, respectively
• Statistical significance does not necessarily imply practical or clinical significance
  • Large sample sizes can detect statistically significant differences that may not be meaningful in practice
• Interpret results in the context of the research question, study design, and domain knowledge
  • Consider potential confounding variables, limitations, and alternative explanations for the findings

Practical Applications in Data Science

• A/B testing compares two versions of a product, website, or app to determine which performs better on a key metric (click-through rate, conversion rate)
  • Randomly assigns users to either the control (A) or treatment (B) group and uses hypothesis testing to assess differences
• Sentiment analysis classifies text data (reviews, tweets, feedback) into positive, negative, or neutral categories
  • Uses natural language processing and machine learning techniques to train models on labeled data and predict sentiment for new data
• Fraud detection identifies unusual patterns or anomalies in financial transactions that may indicate fraudulent activity
  • Applies statistical methods (Benford's law, outlier detection) and machine learning algorithms (logistic regression, decision trees) to flag suspicious cases
• Customer segmentation divides a customer base into distinct groups based on demographic, behavioral, or psychographic characteristics
  • Uses clustering algorithms (k-means, hierarchical) to identify segments and tailor marketing strategies and product recommendations
• Predictive maintenance forecasts when equipment or machinery is likely to fail based on sensor data and historical maintenance records
  • Employs time series analysis, regression models, and survival analysis to optimize maintenance schedules and minimize downtime
• Recommender systems suggest relevant products, content, or services to users based on their preferences and behavior
  • Utilizes collaborative filtering (matrix factorization) and content-based filtering (item similarity) to generate personalized recommendations
• Churn prediction identifies customers who are likely to stop using a product or service based on their characteristics and usage patterns
  • Applies classification algorithms (logistic regression, random forests) to predict churn probability and inform retention strategies