Statistical significance and replicability - Why the former does not presage the latter

Journal/Book: Theor Psychol. 1998; 8: 2455 Teller Rd, Thousand Oaks, CA 91320. Sage Publications Inc. 291-311.

Abstract: In spite of arguments to the contrary, psychologists, it is shown here, believe statistical significance (SS) signifies that a finding will replicate. The most visible argument that SS is not an index of replicability, one that is based in notions of Bayesian statistical inference, is considered and shown to be flawed. Two different arguments are presented that demonstrate the irrelevance of SS to replicability: (a) SS may not be taken as a sign of the truth of the research hypothesis; and (b) statistical significance tests do not generate verifiable predictions of replication attempts. Direct tests of replicability and effect-size measures of replicability are shown to have comparable problems. A solution to the replicability problem is proposed for atheoretical research that replaces 'once-and-for-all' tests of replicability with the requirement that the treatment effect be demonstrable (a) in the individual (b) on a continuing basis, and (c) in a way that is clearly discernible. The question of the role of significance testing in research is answered by denying a role to such testing.

Note: Article Sohn D, Univ N Carolina, Dept Psychol, Charlotte,NC 28223 USA

Keyword(s): null hypothesis testing; replicability and replication; significance testing; statistical significance; SIGNIFICANCE TESTS; NULL HYPOTHESIS; P-VALUES; PROBABILITY; REPLICATION

Zurück | Weiter