15.3 Non-parametric tests
4 min read•august 9, 2024
Non-parametric tests are statistical methods that don't assume data follows a specific distribution. They're crucial when dealing with , small samples, or when parametric assumptions are violated. These tests offer flexibility and robustness in various research scenarios.
This section covers rank-based tests like Mann-Whitney U and Wilcoxon signed-rank, as well as distribution-free methods such as chi-square and . Understanding these alternatives to parametric tests is essential for conducting accurate statistical analyses across diverse datasets.
Rank-Based Tests
Mann-Whitney U and Wilcoxon Signed-Rank Tests
Top images from around the web for Mann-Whitney U and Wilcoxon Signed-Rank Tests
scipy - Am I using the Mann-Whitney U test correctly? - Cross Validated View original
Is this image relevant?
Wilcoxon Signed Rank Test reports "U" instead of "Z" in SPSS - Cross Validated View original
Is this image relevant?
Frontiers | Impact on children of a parent with ALS: a case-control study | Psychology View original
Is this image relevant?
scipy - Am I using the Mann-Whitney U test correctly? - Cross Validated View original
Is this image relevant?
Wilcoxon Signed Rank Test reports "U" instead of "Z" in SPSS - Cross Validated View original
Is this image relevant?
1 of 3
Top images from around the web for Mann-Whitney U and Wilcoxon Signed-Rank Tests
scipy - Am I using the Mann-Whitney U test correctly? - Cross Validated View original
Is this image relevant?
Wilcoxon Signed Rank Test reports "U" instead of "Z" in SPSS - Cross Validated View original
Is this image relevant?
Frontiers | Impact on children of a parent with ALS: a case-control study | Psychology View original
Is this image relevant?
scipy - Am I using the Mann-Whitney U test correctly? - Cross Validated View original
Is this image relevant?
Wilcoxon Signed Rank Test reports "U" instead of "Z" in SPSS - Cross Validated View original
Is this image relevant?
1 of 3
- compares two independent groups
- Used when data is ordinal or continuous but not normally distributed
- Ranks data from both groups together, then compares rank sums
- assumes no difference in distribution between groups
- Test statistic U calculated by counting number of times a score from group 1 precedes a score from group 2
- analyzes paired data
- Suitable for before-and-after studies or matched pairs
- Calculates differences between pairs, ranks absolute differences, and assigns signs
- Compares sum of positive ranks to sum of negative ranks
- Null hypothesis assumes median difference between pairs is zero
- Both tests provide alternatives to t-tests when normality assumptions are violated
Kruskal-Wallis and Friedman Tests
- extends Mann-Whitney U to three or more independent groups
- Non-parametric alternative to one-way ANOVA
- Ranks all data together, then compares mean ranks across groups
- Test statistic H follows chi-square distribution with k-1 degrees of freedom (k = number of groups)
- Null hypothesis assumes all groups have the same distribution
- analyzes repeated measures for three or more conditions
- Non-parametric alternative to repeated measures ANOVA
- Ranks data within each subject, then compares mean ranks across conditions
- Test statistic Q follows chi-square distribution with k-1 degrees of freedom (k = number of conditions)
- Null hypothesis assumes no difference in distributions across conditions
- Both tests useful for comparing multiple groups or conditions without assuming normality
Correlation and Other Rank-Based Methods
- measures monotonic relationships between variables
- Converts raw scores to ranks, then calculates Pearson correlation on ranks
- Ranges from -1 to 1, indicating strength and direction of relationship
- More robust to outliers and non-linear relationships than Pearson correlation
- compares medians of two or more groups
- Dichotomizes data above and below overall median
- Uses to compare observed and expected frequencies
- Less powerful than Kruskal-Wallis but simpler to understand and calculate
- evaluates consistency of differences between paired observations
- Considers only direction of difference, not magnitude
- Useful for ordinal data or when exact differences cannot be measured
- Test statistic follows binomial distribution under null hypothesis
- offer robust alternatives to parametric tests
- Less affected by outliers and extreme values
- Do not require assumptions about underlying distributions
- May have lower power than parametric tests when assumptions are met
Distribution-Free Tests
Chi-Square and Kolmogorov-Smirnov Tests
- Chi-square test analyzes
- Compares observed frequencies to expected frequencies under null hypothesis
- Used for goodness-of-fit, independence, and homogeneity tests
- Test statistic follows chi-square distribution with degrees of freedom based on number of categories
- Assumes large sample sizes and expected frequencies greater than 5 in each cell
- compares distributions
- One-sample version compares observed distribution to theoretical distribution
- Two-sample version compares two observed distributions
- Based on maximum difference between cumulative distribution functions
- Sensitive to differences in both location and shape of distributions
- More powerful than chi-square for continuous data but less flexible for discrete data
Permutation and Bootstrap Methods
- Permutation tests assess significance by resampling without replacement
- Randomly reassign observed data to groups many times (10,000+ permutations)
- Calculate test statistic for each permutation
- Compare observed test statistic to distribution of permuted statistics
- is proportion of permuted statistics as extreme as observed
- Exact for small samples, approximates randomization test for larger samples
- estimate sampling distributions by resampling with replacement
- Create multiple resamples of original data, each same size as original
- Calculate statistic of interest for each resample
- Use distribution of resampled statistics to estimate confidence intervals or conduct hypothesis tests
- Particularly useful for complex statistics without known sampling distributions
- Can be applied to wide range of problems, including regression and time series analysis
- Both methods provide flexible, distribution-free alternatives to parametric tests
- Do not require assumptions about underlying population distributions
- Can be computationally intensive but increasingly feasible with modern computing power
- Offer robust solutions for analyzing complex datasets and non-standard statistical problems
Key Terms to Review (24)
Bootstrap methods: Bootstrap methods are statistical techniques that involve resampling data with replacement to estimate the distribution of a statistic. These methods are particularly useful when the underlying distribution is unknown or when traditional parametric assumptions cannot be met. By creating many resampled datasets, bootstrap methods allow for the estimation of confidence intervals and the testing of hypotheses without relying on normality assumptions.
Categorical data: Categorical data refers to variables that can be divided into distinct groups or categories, where each category represents a specific characteristic or attribute. This type of data is often qualitative and is used to classify items based on non-numeric traits, such as color, gender, or brand. Understanding categorical data is crucial for analyzing trends and relationships in non-parametric tests, which often rely on frequency counts rather than mean values.
Chi-square test: A chi-square test is a statistical method used to determine whether there is a significant association between categorical variables. It assesses how observed frequencies in a contingency table compare to expected frequencies under the assumption of independence. This test is particularly useful in non-parametric statistics because it does not require the data to follow a normal distribution, making it ideal for analyzing nominal or ordinal data.
Conover Test: The Conover Test is a non-parametric statistical method used to compare multiple groups when the assumptions of normality and homogeneity of variances are violated. It is often applied as a post-hoc test following the Kruskal-Wallis test, allowing for the evaluation of pairwise differences between groups. This test helps to determine if there are significant differences among the ranks of several independent samples, making it valuable in various fields such as psychology and medical research.
Distribution-free tests: Distribution-free tests, also known as non-parametric tests, are statistical methods that do not assume a specific distribution for the data being analyzed. These tests are particularly useful when the data does not meet the assumptions required for parametric tests, such as normality or homogeneity of variance. By focusing on ranks or signs rather than specific numerical values, distribution-free tests offer greater flexibility and robustness in various statistical analyses.
Effect Size: Effect size is a quantitative measure of the magnitude of a phenomenon, often used to indicate the strength or importance of a relationship or difference between groups. It helps in understanding how substantial a result is beyond just statistical significance, providing context to the findings in studies. This measure can inform researchers about the practical implications of their results, especially when using non-parametric tests where traditional measures may not apply.
Friedman Test: The Friedman test is a non-parametric statistical test used to detect differences in treatments across multiple test attempts. It’s particularly useful when the assumptions of the repeated measures ANOVA are not met, allowing for analysis of data that may not follow a normal distribution. This test evaluates if there are statistically significant differences among groups when the same subjects are exposed to different conditions.
Kolmogorov-Smirnov Test: The Kolmogorov-Smirnov test is a non-parametric statistical test used to compare two probability distributions or to compare a sample distribution with a reference probability distribution. It evaluates the goodness-of-fit between empirical data and theoretical distributions without making strong assumptions about the underlying data, making it a versatile tool for assessing distributional differences.
Kruskal-Wallis Test: The Kruskal-Wallis test is a non-parametric statistical method used to determine if there are statistically significant differences between the medians of three or more independent groups. It is an alternative to the one-way ANOVA when the data does not meet the assumptions of normality and homogeneity of variance, making it a valuable tool for analyzing ordinal data or non-normally distributed interval data.
Kruskal.test(): The `kruskal.test()` function in R is used to perform the Kruskal-Wallis rank sum test, a non-parametric method for comparing two or more independent groups. This test is particularly useful when the assumptions of normality or homogeneity of variance are not met, making it a robust alternative to ANOVA for analyzing differences among groups based on ranked data.
Mann-Whitney U Test: The Mann-Whitney U Test is a non-parametric statistical test used to determine whether there are differences between two independent groups on a continuous or ordinal outcome. This test is particularly useful when the assumptions of normality and homogeneity of variances for parametric tests, like the t-test, are not met, making it a reliable alternative in many research scenarios.
Median Test: The median test is a non-parametric statistical method used to determine whether two or more groups have different medians. This test is particularly useful when the data does not meet the assumptions required for parametric tests, such as normality. By comparing the medians across groups, this test provides insights into the central tendency of the data without assuming a specific distribution.
Null hypothesis: The null hypothesis is a statement that assumes there is no effect or no difference in a given population or dataset. It serves as a starting point for statistical testing, allowing researchers to determine if observed data significantly deviates from this baseline assumption. If evidence suggests otherwise, the null hypothesis can be rejected in favor of an alternative hypothesis.
Ordinal data: Ordinal data refers to a type of categorical data where the values can be ordered or ranked, but the differences between the values are not meaningful or uniform. This kind of data helps in understanding the relative position of observations, such as in surveys with rating scales, where responses can indicate levels of satisfaction or agreement without specifying how much more one level is than another.
P-value: A p-value is a statistical measure that helps to determine the significance of results obtained from hypothesis testing. It quantifies the probability of observing results at least as extreme as the ones obtained, assuming that the null hypothesis is true. A smaller p-value indicates stronger evidence against the null hypothesis, leading researchers to consider alternative explanations.
Permutation tests: Permutation tests are a type of non-parametric statistical test used to determine the significance of observed differences between groups by comparing them to a distribution of differences generated through random rearrangement of the data. This approach allows researchers to assess the null hypothesis without relying on traditional assumptions about the data's distribution, making it particularly useful in situations where those assumptions might not hold.
Rank-based methods: Rank-based methods are statistical techniques that use the ranks of data rather than their actual values to conduct analyses, particularly when the assumptions of parametric tests cannot be met. These methods are especially useful in non-parametric tests where the data may not follow a normal distribution or may be ordinal in nature. By focusing on the relative order of data points, rank-based methods provide a way to analyze and interpret data without requiring strict adherence to parametric assumptions.
Siegel and Castellan: Siegel and Castellan are non-parametric statistical tests used to determine if there are significant differences between two or more groups. These tests are particularly useful when data do not meet the assumptions required for parametric tests, such as normality and homogeneity of variance, making them essential tools in statistical analysis for handling non-normally distributed data.
Sign Test: The sign test is a non-parametric statistical method used to assess whether the median of a population differs from a specified value. This test is particularly useful when the assumptions required for parametric tests, like normality, cannot be met. It analyzes paired data by focusing on the direction of differences between pairs rather than their specific values, making it suitable for ordinal data or non-normally distributed interval data.
Small sample sizes: Small sample sizes refer to a limited number of observations or data points collected for analysis, which can affect the reliability and validity of statistical results. In research, small sample sizes may lead to increased variability and uncertainty in estimates, making it challenging to draw general conclusions. This is particularly important in the context of statistical tests, where smaller samples may not meet the assumptions required for parametric tests, thus leading researchers to consider non-parametric alternatives.
Spearman's rank correlation: Spearman's rank correlation is a non-parametric measure of the strength and direction of association between two ranked variables. This method evaluates how well the relationship between the two variables can be described using a monotonic function, meaning it looks at whether one variable tends to increase or decrease as the other does, without making assumptions about the specific distribution of the data. It's particularly useful when the data is ordinal or not normally distributed, highlighting its role in descriptive statistics, correlation analysis, and non-parametric testing.
Statistical Significance: Statistical significance is a determination that the relationship or effect observed in data is unlikely to have occurred by chance alone, given a predetermined threshold for probability. It is commonly evaluated using a p-value, which indicates the probability of observing the data if the null hypothesis were true. When statistical significance is established, it suggests that there is strong evidence against the null hypothesis, leading to the potential for further investigation into the nature of the relationship or effect.
Wilcox.test(): The `wilcox.test()` function in R is used to perform the Wilcoxon rank sum test or the Wilcoxon signed-rank test, which are non-parametric tests for comparing two groups. These tests are particularly useful when the assumptions of normality for parametric tests are not met, allowing for the analysis of ordinal data or non-normally distributed continuous data.
Wilcoxon signed-rank test: The Wilcoxon signed-rank test is a non-parametric statistical method used to compare two related samples, matched samples, or repeated measurements on a single sample to assess whether their population mean ranks differ. This test is particularly useful when the data does not meet the assumptions of normality required for parametric tests, making it an essential tool in the realm of non-parametric statistics.