🧰Engineering Applications of Statistics Unit 5 – Hypothesis Testing

What's Hypothesis Testing?

• Hypothesis testing is a statistical method used to make decisions or draw conclusions about a population based on sample data
• Involves formulating a null hypothesis ($H_0$) and an alternative hypothesis ($H_a$) about a population parameter
• The null hypothesis assumes no significant difference or effect, while the alternative hypothesis suggests a significant difference or effect exists
• Collects sample data and uses statistical tests to determine whether there is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis
• The decision to reject or fail to reject the null hypothesis is based on the calculated test statistic and the chosen significance level ($\alpha$)
• Helps engineers and researchers make data-driven decisions and draw meaningful conclusions from experimental or observational data
• Enables the assessment of the effectiveness of new designs, processes, or interventions compared to existing ones

Key Concepts and Terms

• Null hypothesis ($H_0$): The default assumption that there is no significant difference or effect in the population
• Alternative hypothesis ($H_a$): The claim that contradicts the null hypothesis, suggesting a significant difference or effect exists
• Significance level ($\alpha$): The probability of rejecting the null hypothesis when it is actually true (Type I error)
  • Commonly used significance levels are 0.05 (5%) and 0.01 (1%)
• Test statistic: A value calculated from the sample data used to determine whether to reject the null hypothesis
  • Examples include z-score, t-score, and F-score
• p-value: The probability of obtaining a test statistic as extreme as or more extreme than the observed value, assuming the null hypothesis is true
• Critical value: The threshold value of the test statistic that separates the rejection and non-rejection regions of the null hypothesis
• Type I error (false positive): Rejecting the null hypothesis when it is actually true
• Type II error (false negative): Failing to reject the null hypothesis when it is actually false

Types of Hypothesis Tests

• One-sample tests: Compare a sample mean or proportion to a known population parameter
  • One-sample z-test: Used when the population standard deviation is known and the sample size is large (n ≥ 30) or the population is normally distributed
  • One-sample t-test: Used when the population standard deviation is unknown and the sample size is small (n < 30)
• Two-sample tests: Compare the means or proportions of two independent samples
  • Independent two-sample t-test: Used when comparing the means of two independent samples with unknown population standard deviations
  • Paired t-test: Used when comparing the means of two related or paired samples
• ANOVA (Analysis of Variance): Compares the means of three or more groups simultaneously
  • One-way ANOVA: Used when there is one categorical independent variable and one continuous dependent variable
  • Two-way ANOVA: Used when there are two categorical independent variables and one continuous dependent variable
• Chi-square tests: Used for categorical data to test the independence of two variables or the goodness of fit of a distribution
  • Chi-square test of independence: Tests whether two categorical variables are independent or associated
  • Chi-square goodness of fit test: Tests whether an observed distribution fits an expected distribution

Steps in Hypothesis Testing

1. State the null and alternative hypotheses: Clearly define $H_0$ and $H_a$ based on the research question or problem statement
2. Choose the appropriate test: Select the suitable hypothesis test based on the type of data, sample size, and research question
3. Set the significance level ($\alpha$): Determine the acceptable probability of making a Type I error (usually 0.05 or 0.01)
4. Collect and summarize data: Gather relevant sample data and calculate descriptive statistics (e.g., mean, standard deviation)
5. Calculate the test statistic: Use the appropriate formula to compute the test statistic based on the chosen hypothesis test
6. Determine the p-value or critical value: Find the p-value associated with the test statistic or calculate the critical value using the significance level and degrees of freedom
7. Make a decision: Compare the p-value to the significance level or the test statistic to the critical value to decide whether to reject or fail to reject the null hypothesis
8. Interpret the results: Draw meaningful conclusions based on the decision and relate them to the original research question or problem

Statistical Significance and p-values

• Statistical significance indicates the likelihood that the observed differences or effects in the sample data are not due to chance alone
• The p-value is a measure of the strength of evidence against the null hypothesis
  • A smaller p-value suggests stronger evidence against the null hypothesis
• If the p-value is less than or equal to the chosen significance level ($\alpha$), the result is considered statistically significant, and the null hypothesis is rejected
• If the p-value is greater than the significance level, the result is not statistically significant, and there is insufficient evidence to reject the null hypothesis
• Statistical significance does not necessarily imply practical or clinical significance, as small differences can be statistically significant with large sample sizes
• It is essential to consider the context, effect size, and practical implications when interpreting statistically significant results

Common Test Statistics

• Z-score: Used in one-sample and two-sample tests when the population standard deviation is known or the sample size is large
  • Calculated as $z = \frac{\bar{x} - \mu}{\sigma / \sqrt{n}}$ for one-sample tests and $z = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}}$ for two-sample tests
• T-score: Used in one-sample and two-sample tests when the population standard deviation is unknown and the sample size is small
  • Calculated as $t = \frac{\bar{x} - \mu}{s / \sqrt{n}}$ for one-sample tests and $t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$ for two-sample tests
• F-score: Used in ANOVA tests to compare the variance between groups to the variance within groups
  • Calculated as $F = \frac{MS_{between}}{MS_{within}}$, where $MS$ stands for mean square
• Chi-square statistic: Used in chi-square tests for categorical data
  • Calculated as $\chi^2 = \sum \frac{(O - E)^2}{E}$, where $O$ is the observed frequency and $E$ is the expected frequency

Interpreting Test Results

• If the null hypothesis is rejected, conclude that there is sufficient evidence to support the alternative hypothesis
  • For example, if the null hypothesis of no difference between two population means is rejected, conclude that there is a significant difference between the means
• If the null hypothesis is not rejected, conclude that there is insufficient evidence to support the alternative hypothesis
  • This does not necessarily mean that the null hypothesis is true, but rather that there is not enough evidence to reject it based on the sample data
• Consider the practical significance of the results in addition to statistical significance
  • A statistically significant result may not always be practically meaningful, depending on the context and the magnitude of the effect
• Be cautious when interpreting non-significant results, as they may be due to insufficient sample size or low statistical power
• Always interpret the results in the context of the research question, study design, and limitations of the data

Real-World Engineering Applications

• Quality control: Hypothesis testing is used to monitor and improve product quality by comparing sample means or proportions to specified target values
  • For example, testing whether the mean strength of a material meets the required specifications
• Process optimization: Hypothesis tests can help determine the optimal settings for process parameters by comparing the performance of different configurations
  • For instance, comparing the yield of a chemical process at different temperature and pressure settings
• Design of experiments (DOE): Hypothesis testing is a crucial component of DOE, which involves systematically varying input factors to assess their impact on a response variable
  • ANOVA is commonly used in DOE to determine the significance of main effects and interactions between factors
• Reliability engineering: Hypothesis tests can be employed to assess the reliability of components or systems by comparing failure rates or mean time between failures (MTBF) to industry standards or target values
• Simulation validation: Hypothesis testing can be used to validate simulation models by comparing the model outputs to real-world data
  • For example, using a t-test to compare the simulated and observed performance of a manufacturing system
• A/B testing: Hypothesis tests are used in online experiments to compare the effectiveness of different designs, layouts, or features on user engagement or conversion rates
  • For instance, using a two-sample proportion test to compare the click-through rates of two website variants