Power Analysis of Mann Whitney, Two Sample Kolmogorov Smirnov and Wald-Wolfowitz Runs Tests: Comparison with Monte Carlo Simulation in the Case of Variance Heterogeneity

EasyChair Preprint 10501

Abstract

When assumptions like normality and homogeneity of variance are violated, it is advisable to utilize nonparametric statistical tests in the realm of social sciences. When researchers need to test data from two independent samples, they often come across several different nonparametric tests. The two-sample Kolmogorov-Smirnov test, the Mann-Whitney test, and the Wald-Wolfowitz Runs test are the most frequently used tests for this purpose. The purpose of this study was to compare the statistical power of three commonly used tests - Mann Whitney, Two Sample Kolmogorov Smirnov, and Wald-Wolfowitz Runs Test - for testing independent samples data. As these tests may yield varying outcomes based on specific circumstances, we conducted a power analysis of each test when faced with heterogeneity of variance, taking into account varying sample sizes. In the study, standard deviation ratios were set as 2, 3, 4, 1/2, 1/3 and 1/4 and power comparisons were made for small and large sample sizes. For equal sample sizes, small sample sizes of 5, 8, 10, 12, 16 and 20 and large sample sizes of 25, 50, 75 and 100 were used. For different sample sizes, small sample size combinations of (4, 16), (8, 16), (10, 20), (16, 4), (16, 8) and (20, 10) and large sample size combinations of (10, 30), (30, 10), (50, 75), (50, 100), (75, 50), (75, 100), (100, 50) and (100, 75) were examined. The Monte Carlo simulation technique was used to run simulations 20,000 times for every combination.

Keyphrases: Mann-Whitney test, Monte Carlo study, Two Sample Kolmogorov Smirnov Test, Wald-Wolfowitz Runs test, power

@booklet{EasyChair:10501,
  author    = {Sahib Ramazanov and Rehile Askerbeyli},
  title     = {Power Analysis of Mann Whitney, Two Sample Kolmogorov Smirnov and Wald-Wolfowitz Runs Tests: Comparison with Monte Carlo Simulation in  the Case of Variance Heterogeneity},
  howpublished = {EasyChair Preprint 10501},
  year      = {EasyChair, 2023}}