Key Concepts in Hypothesis Testing

Hypothesis testing is a key concept in Engineering Probability and Mathematical Probability Theory. It helps determine if there's enough evidence to reject a default assumption, known as the null hypothesis, in favor of an alternative hypothesis based on sample data.

1. Null and alternative hypotheses

   • The null hypothesis (H0) represents a statement of no effect or no difference, serving as the default assumption.
   • The alternative hypothesis (H1 or Ha) is what you aim to support, indicating the presence of an effect or difference.
   • Hypothesis testing involves determining whether to reject H0 in favor of Ha based on sample data.

2. Type I and Type II errors

   • Type I error (α) occurs when the null hypothesis is incorrectly rejected when it is true.
   • Type II error (β) occurs when the null hypothesis is not rejected when it is false.
   • Balancing these errors is crucial; reducing one often increases the other.

3. Significance level (α)

   • The significance level (α) is the threshold for deciding whether to reject the null hypothesis, commonly set at 0.05.
   • It represents the probability of making a Type I error.
   • A lower α reduces the likelihood of Type I errors but increases the risk of Type II errors.

4. Power of a test

   • The power of a test is the probability of correctly rejecting the null hypothesis when it is false (1 - β).
   • Higher power increases the likelihood of detecting a true effect.
   • Factors affecting power include sample size, effect size, and significance level.

5. P-value

   • The P-value measures the strength of evidence against the null hypothesis.
   • A low P-value (typically < 0.05) indicates strong evidence to reject H0.
   • It does not measure the probability that H0 is true or false.

6. Test statistic

   • A test statistic is a standardized value calculated from sample data used to determine whether to reject the null hypothesis.
   • It compares the observed data to what is expected under the null hypothesis.
   • Common test statistics include Z, T, Chi-square, and F.

7. Critical region

   • The critical region is the set of values for the test statistic that leads to the rejection of the null hypothesis.
   • It is determined by the significance level (α) and the distribution of the test statistic.
   • If the test statistic falls within this region, H0 is rejected.

8. One-tailed and two-tailed tests

   • A one-tailed test assesses the direction of an effect (greater than or less than).
   • A two-tailed test evaluates whether there is any difference, regardless of direction.
   • The choice between them affects the critical region and P-value interpretation.

9. Z-test

   • The Z-test is used when the population variance is known or the sample size is large (n > 30).
   • It compares the sample mean to the population mean using the standard normal distribution.
   • Commonly used for hypothesis testing regarding population means.

10. T-test

    • The T-test is used when the population variance is unknown and the sample size is small (n ≤ 30).
    • It compares the sample mean to the population mean using the t-distribution.
    • Variants include independent, paired, and one-sample T-tests.

11. Chi-square test

    • The Chi-square test assesses the association between categorical variables.
    • It compares observed frequencies to expected frequencies under the null hypothesis.
    • Commonly used in contingency tables and goodness-of-fit tests.

12. F-test

    • The F-test compares the variances of two populations to determine if they are significantly different.
    • It is often used in the context of ANOVA (Analysis of Variance).
    • The test statistic follows an F-distribution.

13. Confidence intervals

    • Confidence intervals provide a range of values within which the true population parameter is expected to lie.
    • They are constructed using sample data and a specified confidence level (e.g., 95%).
    • Wider intervals indicate more uncertainty, while narrower intervals suggest more precision.

14. Sample size determination

    • Sample size determination is crucial for ensuring adequate power to detect an effect.
    • It depends on the expected effect size, significance level (α), and desired power (1 - β).
    • Larger sample sizes reduce variability and increase the reliability of results.

15. Hypothesis testing for population mean

    • Involves testing whether the sample mean significantly differs from a known population mean.
    • Can be conducted using Z-tests or T-tests depending on sample size and variance knowledge.
    • Assumes normality of the data or large sample sizes for the Central Limit Theorem to apply.

16. Hypothesis testing for population proportion

    • Tests whether the sample proportion significantly differs from a hypothesized population proportion.
    • Typically uses the Z-test for proportions when sample sizes are large enough.
    • Assumes a binomial distribution of successes and failures.

17. Hypothesis testing for population variance

    • Involves testing whether the sample variance significantly differs from a hypothesized population variance.
    • The Chi-square test is commonly used for this purpose.
    • Assumes normality of the data for valid results.

18. Likelihood ratio test

    • The likelihood ratio test compares the goodness of fit of two competing hypotheses.
    • It calculates the ratio of the maximum likelihoods under the null and alternative hypotheses.
    • Often used in complex models and nested hypotheses.

19. Multiple hypothesis testing

    • Involves testing multiple hypotheses simultaneously, increasing the risk of Type I errors.
    • Techniques like the Bonferroni correction are used to adjust significance levels.
    • Important in fields like genomics and clinical trials where many tests are conducted.

20. Non-parametric tests

    • Non-parametric tests do not assume a specific distribution for the data.
    • They are useful when data do not meet the assumptions of parametric tests (e.g., normality).
    • Common examples include the Mann-Whitney U test and Kruskal-Wallis test.