A practitioner’s guide to developing critical appraisal skills What is the difference between clinical and statistical significance?

Abstract

Background. It is common to find published studies in which the authors claim to have found significant results. However, many times these results are only statistically significant with no meaningful impact in clinical settings. Methods. The authors aim to clarify and differentiate the concepts of statistical and clinical significance, as well as to provide guidance on how to interpret research results to determine whether an observed difference is meaningful. Results. Study results are considered to be statistically significant if statistical tests that examine the null hypothesis of no difference yield P values that are smaller than the significance level prespecified by the authors. In this way, researchers can use hypothesis testing to assess the possibility that observed results could have arisen by chance. However, hypothesis testing cannot establish the clinical implications of these results. Rather, clinical significance can be established once the magnitude of results is larger than the minimal clinically important difference. Clinical significance then would encompass not only statistical significance, but also the importance of the outcomes to patients, clinicians and policymakers. Conclusion. The values for statistical significance alone cannot convey the complete picture of the effectiveness of an intervention or of a difference between two groups. Both clinical and statistical significance are important measures for interpretation of clinical research results and should complement each other. Practical Implications. Any benefit in terms of improved health outcomes must be both clinically and statistically significant. If there is no benefit at the threshold of both clinical and statistical improvement, then the intervention should not be used for that purpose.