7.2 Inferential statistics

Inferential statistics is the backbone of data-driven decision-making in reproducible research. It allows scientists to draw conclusions about populations from sample data, providing a framework for quantifying uncertainty and making probabilistic statements about hypotheses.

This topic covers key concepts like sampling distributions, point and interval estimation, hypothesis testing, and various statistical tests. It also explores effect size, power analysis, Bayesian inference, and resampling methods, emphasizing their importance in conducting robust and reproducible statistical analyses.

Foundations of inferential statistics

• Inferential statistics forms the cornerstone of data-driven decision-making in reproducible and collaborative statistical data science
• Enables researchers to draw conclusions about populations based on sample data, crucial for making generalizable insights
• Provides a framework for quantifying uncertainty and making probabilistic statements about hypotheses

Population vs sample

Top images from around the web for Population vs sample

• 

  Why It Matters: Linking Probability to Statistical Inference | Concepts in Statistics View original

  Is this image relevant?

• 

  Stratified sampling - Wikipedia View original

  Is this image relevant?

• 

  Cluster sampling - Wikipedia View original

  Is this image relevant?

• 

  Why It Matters: Linking Probability to Statistical Inference | Concepts in Statistics View original

  Is this image relevant?

• 

  Stratified sampling - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Population vs sample

• 

  Why It Matters: Linking Probability to Statistical Inference | Concepts in Statistics View original

  Is this image relevant?

• 

  Stratified sampling - Wikipedia View original

  Is this image relevant?

• 

  Cluster sampling - Wikipedia View original

  Is this image relevant?

• 

  Why It Matters: Linking Probability to Statistical Inference | Concepts in Statistics View original

  Is this image relevant?

• 

  Stratified sampling - Wikipedia View original

  Is this image relevant?

1 of 3

• Population encompasses all individuals or items of interest in a study
• Sample represents a subset of the population selected for analysis
• Random sampling techniques ensure representativeness and minimize bias
• Stratified sampling divides the population into subgroups before sampling
• Cluster sampling selects groups rather than individuals

Statistical inference process

• Starts with defining the research question and identifying relevant variables
• Involves collecting data through appropriate sampling methods
• Analyzes sample data using statistical techniques (descriptive and inferential)
• Draws conclusions about the population based on sample results
• Assesses the reliability and validity of inferences through statistical measures

Types of statistical inference

• Estimation infers population parameters from sample statistics
• Hypothesis testing evaluates claims about population characteristics
• Prediction forecasts future outcomes based on observed patterns
• Causal inference determines cause-and-effect relationships between variables
• Model selection chooses the best statistical model to explain observed data

Sampling distributions

• Sampling distributions play a crucial role in understanding variability in statistical estimates
• Form the basis for many inferential techniques used in reproducible data science
• Enable researchers to quantify uncertainty and make probabilistic statements about population parameters

Central limit theorem

• States that the distribution of sample means approaches a normal distribution as sample size increases
• Applies regardless of the underlying population distribution (with some exceptions)
• Sample size of 30 or more often considered sufficient for normality assumption
• Enables the use of z-scores and t-scores in statistical inference
• Facilitates the construction of confidence intervals and hypothesis tests

Standard error

• Measures the variability of a sample statistic across multiple samples
• Calculated as the standard deviation of the sampling distribution
• Decreases as sample size increases, improving estimate precision
• Used in calculating confidence intervals and test statistics
• Formula for standard error of the mean: $SE = \frac{s}{\sqrt{n}}$

Sampling variability

• Refers to the variation in sample statistics from sample to sample
• Influenced by factors such as sample size, population variability, and sampling method
• Larger samples generally lead to less sampling variability
• Quantified through measures like standard error and confidence intervals
• Understanding sampling variability crucial for assessing the reliability of statistical inferences

Point estimation

• Point estimation provides single best-guess values for population parameters
• Crucial in reproducible data science for summarizing and interpreting data
• Forms the basis for more complex inferential techniques and model building

Maximum likelihood estimation

• Estimates parameters by maximizing the likelihood function
• Widely used in statistical modeling and machine learning algorithms
• Assumes a probability distribution for the data (Gaussian, Poisson)
• Iterative process often requires numerical optimization techniques
• Produces asymptotically unbiased and efficient estimators under certain conditions

Method of moments

• Equates sample moments to population moments to estimate parameters
• Simple and computationally efficient compared to maximum likelihood estimation
• Can be used when maximum likelihood estimation is difficult or intractable
• May produce biased estimators, especially for small sample sizes
• Useful for obtaining initial parameter estimates for more complex methods

Properties of estimators

• Unbiasedness measures the average closeness of estimates to true parameter values
• Consistency ensures estimates converge to true values as sample size increases
• Efficiency compares the variance of estimators, with lower variance being more efficient
• Sufficiency indicates whether an estimator uses all relevant information from the data
• Robustness evaluates an estimator's performance under departures from assumptions

Interval estimation

• Interval estimation provides a range of plausible values for population parameters
• Essential for quantifying uncertainty in reproducible and collaborative data science
• Allows researchers to communicate the precision of their estimates effectively

Confidence intervals

• Provide a range of values likely to contain the true population parameter
• Calculated using point estimate, standard error, and desired confidence level
• Wider intervals indicate less precise estimates, narrower intervals more precise
• Formula for a 95% confidence interval: $\text{CI} = \text{Point Estimate} \pm 1.96 \times \text{Standard Error}$
• Assumptions include random sampling and approximately normal sampling distribution

Margin of error

• Represents the maximum expected difference between the sample estimate and population parameter
• Calculated as the product of the critical value and standard error
• Decreases as sample size increases, improving estimate precision
• Often reported in survey results and opinion polls
• Formula for margin of error: $\text{MOE} = \text{Critical Value} \times \text{Standard Error}$

Interpreting confidence levels

• Confidence level represents the long-run frequency of intervals containing the true parameter
• 95% confidence level means 95% of similarly constructed intervals would contain the true value
• Higher confidence levels result in wider intervals, lower levels in narrower intervals
• Misinterpreted as the probability of the parameter falling within the specific interval
• Correct interpretation focuses on the method's long-run performance, not individual intervals

Hypothesis testing

• Hypothesis testing forms the foundation for making decisions based on data in statistical research
• Crucial for reproducible science by providing a systematic framework for evaluating claims
• Enables researchers to quantify evidence against null hypotheses and support alternative explanations

Null vs alternative hypotheses

• Null hypothesis (H0) represents the status quo or no effect
• Alternative hypothesis (Ha) proposes a specific effect or difference
• Hypotheses must be mutually exclusive and exhaustive
• Directional hypotheses specify the direction of the effect (one-tailed tests)
• Non-directional hypotheses only claim a difference exists (two-tailed tests)

Type I and Type II errors

• Type I error occurs when rejecting a true null hypothesis (false positive)
• Type II error involves failing to reject a false null hypothesis (false negative)
• Probability of Type I error equals the significance level (α)
• Probability of Type II error denoted as β, with power equal to 1 - β
• Trade-off exists between Type I and Type II errors, influenced by sample size and effect size

P-values and significance levels

• P-value represents the probability of obtaining results as extreme as observed, assuming H0 is true
• Significance level (α) sets the threshold for rejecting the null hypothesis
• Common significance levels include 0.05 and 0.01
• Reject H0 if p-value < α, fail to reject if p-value ≥ α
• Interpretation focuses on strength of evidence against H0, not proof of Ha

Parametric tests

• Parametric tests assume specific probability distributions for the data
• Widely used in reproducible data science due to their power and efficiency
• Require certain assumptions to be met for valid inferences

Z-test

• Used when population standard deviation is known and sample size is large
• Tests hypotheses about population means or proportions
• Assumes normally distributed data or large sample sizes
• Z-score calculated as: $Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}}$
• Appropriate for comparing sample mean to known population mean

T-test

• Employed when population standard deviation is unknown
• Various types include one-sample, independent samples, and paired samples t-tests
• Assumes normally distributed data or sufficiently large sample sizes
• T-statistic calculated as: $t = \frac{\bar{X} - \mu}{s / \sqrt{n}}$
• Degrees of freedom influence the shape of the t-distribution

ANOVA

• Analysis of Variance compares means across multiple groups
• One-way ANOVA examines the effect of one factor on a continuous outcome
• Two-way ANOVA investigates the effects of two factors and their interaction
• Assumes normality, homogeneity of variances, and independence of observations
• F-statistic compares between-group variance to within-group variance

Chi-square test

• Used for categorical data analysis and goodness-of-fit testing
• Independence test examines relationships between categorical variables
• Goodness-of-fit test compares observed frequencies to expected frequencies
• Assumes large expected frequencies in each cell (typically > 5)
• Chi-square statistic calculated as: $\chi^2 = \sum \frac{(O - E)^2}{E}$

Non-parametric tests

• Non-parametric tests make fewer assumptions about the underlying data distribution
• Crucial in reproducible data science when dealing with non-normal or ordinal data
• Generally less powerful than parametric tests but more robust to assumption violations

Mann-Whitney U test

• Non-parametric alternative to the independent samples t-test
• Compares two independent groups based on rank-ordered data
• Does not assume normality but requires similar shaped distributions
• Null hypothesis states the two populations have the same distribution
• Test statistic U based on the sum of ranks for each group

Wilcoxon signed-rank test

• Non-parametric alternative to the paired samples t-test
• Used for comparing two related samples or repeated measurements
• Assumes symmetry of differences around the median
• Ranks the absolute differences and sums the ranks of positive differences
• Null hypothesis states the median difference between pairs is zero

Kruskal-Wallis test

• Non-parametric alternative to one-way ANOVA
• Compares three or more independent groups based on rank-ordered data
• Does not assume normality but requires similar shaped distributions
• Tests whether samples come from the same distribution
• Test statistic H approximates a chi-square distribution under the null hypothesis

Effect size and power

• Effect size and power analysis are crucial for reproducible and meaningful statistical research
• Help researchers design studies with adequate sample sizes and interpret practical significance
• Essential for meta-analyses and comparing results across different studies

Cohen's d

• Standardized measure of effect size for comparing two group means
• Calculated as the difference between means divided by pooled standard deviation
• Interpretations: small (0.2), medium (0.5), and large (0.8) effects
• Useful for meta-analyses and comparing effects across different scales
• Formula: $d = \frac{\bar{X}_1 - \bar{X}_2}{s_\text{pooled}}$

Statistical power

• Probability of correctly rejecting a false null hypothesis (1 - β)
• Influenced by effect size, sample size, significance level, and test directionality
• Conventional target power is 0.80 (80% chance of detecting a true effect)
• Increases with larger sample sizes and effect sizes
• Power analysis crucial for determining appropriate sample sizes in study design

Sample size determination

• Calculates the required sample size to achieve desired statistical power
• Considers effect size, desired power, significance level, and test type
• Larger sample sizes needed for smaller effect sizes and higher power
• Trade-off between cost/feasibility and statistical power
• Various software tools available for sample size calculations (G*Power, R packages)

Bayesian inference

• Bayesian inference provides an alternative framework to frequentist statistics
• Increasingly important in reproducible data science for handling uncertainty
• Allows incorporation of prior knowledge and updating beliefs based on new data

Bayes' theorem

• Fundamental to Bayesian inference, relates conditional and marginal probabilities
• Formula: $P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$
• P(A|B) represents the posterior probability of A given B
• P(B|A) is the likelihood of B given A
• P(A) represents the prior probability of A

Prior vs posterior distributions

• Prior distribution represents initial beliefs about parameters before observing data
• Can be informative (based on previous knowledge) or non-informative (vague)
• Likelihood function represents the probability of observing the data given the parameters
• Posterior distribution combines prior and likelihood to update beliefs about parameters
• Posterior serves as the basis for Bayesian inference and decision-making

Credible intervals

• Bayesian alternative to frequentist confidence intervals
• Provide a range of values containing the true parameter with a specified probability
• Interpretation more straightforward than confidence intervals
• Can be asymmetric and directly reflect the shape of the posterior distribution
• Highest Density Interval (HDI) represents the most probable parameter values

Resampling methods

• Resampling techniques provide powerful tools for inference without relying on parametric assumptions
• Essential in reproducible data science for robust estimation and hypothesis testing
• Particularly useful when dealing with complex data structures or small sample sizes

Bootstrap

• Involves repeatedly sampling with replacement from the original dataset
• Estimates the sampling distribution of a statistic empirically
• Used for constructing confidence intervals and hypothesis testing
• Non-parametric bootstrap makes no assumptions about the underlying distribution
• Parametric bootstrap assumes a specific distribution and resamples from it

Jackknife

• Systematically leaves out one observation at a time and recalculates the statistic
• Used for estimating bias and standard error of a statistic
• Particularly useful for complex estimators without known sampling distributions
• Can be extended to delete-d jackknife for leaving out d observations at a time
• Less computationally intensive than bootstrap but may be less accurate for some applications

Permutation tests

• Randomly shuffles the observed data to generate the null distribution
• Tests the null hypothesis of no association between variables
• Makes no assumptions about the underlying distribution of the test statistic
• Particularly useful for small sample sizes or when parametric assumptions are violated
• Can be applied to a wide range of test statistics and experimental designs

Reproducibility in inference

• Reproducibility in statistical inference is crucial for advancing scientific knowledge
• Ensures transparency, reliability, and credibility of research findings
• Fundamental to collaborative data science and building upon previous work

Reporting statistical results

• Clearly state research questions, hypotheses, and analysis plans
• Report effect sizes and confidence intervals alongside p-values
• Provide descriptive statistics and visualizations of data distributions
• Discuss assumptions, limitations, and potential sources of bias
• Use standardized reporting guidelines (CONSORT, STROBE) when applicable

Replication studies

• Attempt to recreate previous research findings using similar methods
• Direct replication uses identical procedures, conceptual replication tests the same hypothesis with different methods
• Important for validating and generalizing research findings
• Can reveal potential issues with original studies or strengthen confidence in results
• Challenges include publication bias and difficulty publishing null results

Open science practices

• Share data, code, and materials to enable others to verify and build upon research
• Use version control systems (Git) to track changes and collaborate effectively
• Preregister study designs and analysis plans to reduce researcher degrees of freedom
• Utilize open-access platforms for disseminating research findings
• Participate in collaborative research efforts and multi-lab replication projects