A common problem faced by scientists and clinicians in the analysis of data is in the interpretation of how impactful a measured difference is, which is typically represented by the “significance” of a result. Very broadly, statistical significance reflects the fact that it’s likely that something is happening, whereas clinical significance reflects to what extent that phenomenon is happening, and how impactful it is on clinical practice. Unfortunately, in clinical research, study results which are statistically significant are sometimes assumed to thus be clinically important without further proof. However, it is crucial for medical researchers and providers to understand the difference between clinically significant and statistically significant results 1.
In statistical analyses, the P value is often what conclusions are based on. Stated simply, the P value represents the probability that the observed results would have occurred if nothing was actually happening (e.g. there was no association between two variables, or there was no effect from the intervention). A P value below a set threshold indicates statistical significance. However, a statistically significant P value is sometimes erroneously interpreted as an indicator that results are clinically significant 2.
In clinical practice, the “clinical significance” of a result relates to how it can drive treatment decisions. Some clinicians explicitly suggest that clinical significance should reflect “the extent of change, whether the change makes a real difference to subject lives, how long the effects last, consumer acceptability, cost-effectiveness, and ease of implementation” 3. For example, a drug candidate may improve numbers in a blood test without having a measurable impact on quality of life.
While there are established thresholds for statistical significance testing, these are still lacking in the assessment of clinical significance 4. More often than not, it is the clinician’s (and/or patient’s) judgment which decides whether a result is clinically significant or not.
As a result of the lack of clarity, academic journals have taken initiatives to clarify the interpretation of results. Most journals now endorse the use of a CONSORT statement for the reporting of parallel-group randomized trials. This statement emphasizes the need to report the estimated effect size and its precision (such as a 95% confidence interval) for all primary and secondary outcomes 5.
Readers should keep in mind, though, that the interpretation of study results must consider clinical significance by holistically looking at the big picture: P values and statistical significance as well as the actual treatment effect (with confidence intervals).
It is essential for clinicians, and valuable for patients, to learn the parallels, distinctions, and applications of clinically significant results and statistically significant results 6. Although clinically vs. statistically significant continue to be frequently confused concepts 7clearly delineating the two can reduce inefficiencies and waste in healthcare.
References
1. Ranganathan, P., Pramesh, C. & Buyse, M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect. Clin. Res. (2015). doi:10.4103/2229-3485.159943
2. Sharma, H. Statistical significance or clinical significance? A researcher’s dilemma for appropriate interpretation of research results. Saudi Journal of Anaesthesia (2021). doi:10.4103/sja.sja_158_21
3. LeFort, S. M. The Statistical versus Clinical Significance Debate. Image J. Nurs. Scholarsh. (1993). doi:10.1111/j.1547-5069.1993.tb00754.x
4. Fethney, J. Statistical and clinical significance, and how to use confidence intervals to help interpret both. Aust. Crit. Care (2010). doi:10.1016/j.aucc.2010.03.001
5. Schulz, K. F., Altman, D. G. & Moher, D. CONSORT 2010 Statement. Obstet. Gynecol. (2010). doi:10.1097/aog.0b013e3181d9d421
6. Clinical Significance vs. Statistical Significance – Side-by-Side Comparison. Available at: https://www.mhaonline.com/faq/clinical-vs-statistical-significance. (Accessed: 2nd September 2023)
7. Sedgwick, P. Clinical significance versus statistical significance. BMJ 348, (2014). doi: https://doi.org/10.1136/bmj.g2130
