Ferri Nicola [Department of Biomedical and Neuromotor Sciences (DIBINEM), Alma Mater Studiorum University of Bologna, Bologna, Italy]
Ravizzotti Elisa [Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics and Maternal Child Health (DINOGMI), University of Genoa, Genoa, Italy]
Bracci Alessandro [Department for Life Quality Studies (QUVI), University of Bologna, Rimini, Italy]
Carreras Giulia [Oncologic Network, Prevention and Research Institute (ISPRO), Florence, Italy]
Pillastrini Paolo [Department of Biomedical and Neuromotor Sciences (DIBINEM), Alma Mater Studiorum University of Bologna, Bologna, Italy]
Di Bari Mauro [Department of Experimental and Clinical Medicine, University of Florence, Florence, Italy]
Clinicians implement the best scientific evidence to support decision-making, according to the Evidence-Based Practice paradigm. Systematic reviews with meta-analysis of RCTs (SRs) represent the best evidence for questions of intervention effectiveness. These studies have been growing exponentially for decades now, with the paradox that in recent years, the secondary literature has overcome the primary literature in relative growth. This resulted in the emergence of overlapping SRs on the same question, with sometimes conflicting results or conclusions. A recent survey reported that most decision-makers find it difficult to choose the best evidence out of many. This study aims to investigate musculoskeletal physiotherapy literature assessing the methodological quality of SRs over the past 10 years and to analyze possible quality predictors.
This is an observational study in the meta-research field. We searched four scientific databases (MEDLINE, CDSR, CINAHL, PEDro) using both free text and MeSH terms. Two independent assessors selected only the SRs concerning treatment effectiveness on musculoskeletal conditions, published between 2012 and 2022. A random list of included full texts was generated, and the first 100 SRs were included. The study outcomes included the SRs type (Cochrane or non-Cochrane), the Journal Impact Factor (JIF), the Journal Quartile, the h-index of the first and last author, the publication policy, the total number of studies, and the direction of results and conclusions. Each SR was then qualitatively assessed using the AMSTAR-2 tool by two independent assessors, leading to a four-level score (critically low, low, moderate, and high). Descriptive analyses were performed, followed by multivariable ordinal logistic regression. The STROBE checklist for conference abstracts was followed.
A total of 2078 records were retrieved, and 395 full texts were included, from which the random sample of 100 SRs was obtained. Ninety SRs (90%) are of critically low quality, 4 of low quality (4%), 2 of moderate quality (2%) and 4 of high quality (4%). The overall methodological quality has not increased over the past 10 years, and the high-quality SRs are all Cochrane SRs. Quality seemed to differ according to the JIF; however, after performing a sensitivity analysis, Cochrane SRs proved to be a confounding factor. Thus, in our sample, JIF is not associated with SR quality. The h-index of the first and last authors are significantly different across SR quality, and the last author’s h-index is the only weak predictor (OR 1.04; 95% CI: 1.01, 1.06) among all the variables considered.
This study confirms previous research on different topics and highlights how the quality of current literature is extremely low while there is a staggering increase in publications. The physiotherapy profession has ethical and legal responsibilities through an obligation of means. But what if the best evidence is of low methodological quality? How should it be used to support clinical practice? In the first instance, the recommendation for the clinician is to always assess the quality of SRs carefully and not to trust the prestige of the journal, the most recent publication, or the one that included the more primary studies. The study design alone (i.e., SRs) is not enough to support a clinical choice. We suggest that researchers strictly follow the existing methodological recommendations for conducting a SR and that editors implement a thorough quality assessment during peer-review processes; this could improve the quality of the scientific literature and, finally, patient care.
Ferri N, Ravizzotti E, Bracci A, Carreras G, Pillastrini P, Di Bari M. The confidence in the results of physiotherapy systematic reviews in the musculoskeletal field is not increasing over time: a meta-epidemiological study using AMSTAR 2 tool. J Clin Epidemiol. 2024 Feb 24;169:111303. doi: 10.1016/j.jclinepi.2024.111303. Epub ahead of print. PMID: 38402999.
