I had a conversation (ok–an electronic conversation) with colleagues a few weeks ago. The conversation was about the use of inappropriate analyses for manuscripts being submitted. The specific question raised was about the t-test and went something like this:
Should a t-test be used on a sample that has not been RANDOMLY drawn from a population. If the sample is not randomly drawn, or if the entire population is used (as is often the case with Extension evaluations), then a t-test is NOT appropriate.
What exactly is the appropriate test? First, one has to identify if the underlying assumptions have been met; if not, a parametric, i.e., t–test, is not appropriate.
So what exactly are the assumptions underlying the use of a t-test?
Glass and Stanley (1970, pg. 297) [a classic statistics text in the field of education and psychology and there may be a more recent edition than the one I have on my shelf] say that for dependent samples (the kind most often used by Extension professionals ), the sample:
- is normally distributed;
- has homogeneous variances (i.e., same spread); and
- is RANDOMLY drawn from a population.
Some would add that the scale of measurement needs to result in interval or ratio data (not nominal–ordinal is a questionable case) and have a sample size of 30 or over.
This presents a quandary, to say the least. Extension professionals know that journals want a probability value (if the study is quantitative) and how do you get a probability value with out a t-test?
Answer: Use a nonparametric equivalent test.
The nonparametric equivalent for a test of dependent means is a Wilcoxon matched pair test.
Marascuilo and McSweeney (1977, pg 5) say that that the researcher needs to select “…a test for which the power of rejection is maximized when the hypothesis is tested false.” They go on to say, “If the data adhere to the assumptions required for a classical, normally based t or F test, a researcher would be foolish…not (to) use them, since they are optimum, when justified.” Justification is the key here. And the justification is: Do the data meet the assumptions for the test?
Their bottom line is that a researcher should never think that a nonparametric test is exclusively a substitute for a parametric test. It is not. Use the right test for the hypothesis being tested. It may be (probably is) a nonparametric test.
Citations
Glass, G. V., & Stanley, J. C. (1970). Statistical Methods in Education and Psychology. Englewood Cliffs, NJ: Prentice-Hall, Inc.
Marascuilo, L. A. & McSweeney, M. (1977). Nonparametric and Distribution-free Methods for the Social Sciences. Monterey, CA: Brooks/Cole Publishing.
