Sarah Khan edited untitled.tex  almost 9 years ago

Commit id: 77070012a9a6fd2ea216e53c7b25fac7d6d77e1e

deletions | additions      

       

meta-analysis;oncology;quality; risk of bias;systematic review  \section{Introduction}  The use of systematic reviews and meta-analyses has become increasingly important in evidence-based medicine as clinicians seek out consistent and reliable information on treatments and care guidelines in medicine \cite{14764293}. In part, the reasoning for reliance on systematic reviews and meta-analyses is that both are considered to be helpful due to pooling of results of multiple studies to provide a broader view of information of interest, and by broadening the pool of data, the idea is that bias would become less of an issue in studies \cite{7500513}. Quality assessment is a crucial component of any study, and consequences of inadequate quality reporting or evaluation within research studies, can lead to exaggerated treatment effects when bias of participants is not taken into account in study design \cite{gurusamy2009assessment}. Assessing benefits and harms of interventional procedures is crucial in clinical application of trials and it is of great value to determine whether studies are conducted after assessing bias or low quality of studies.   Many scales have been designed to address concerns about high risk of bias or low quality in earlier studies, however, recent evidence indicates scales may not be the best means of assessing quality measures, and rather certain design features should be viewed to give a clearer picture of bias in trials \cite{lohr1999assessing}.The Cochrane Handbook for Systematic Review of Interventions has been updated continually to address changes in assessment of quality \cite{higgins2011cochrane}. Similarly the Jadad grading scale has also been designed as a means of unifying quality measures amongst clinical trials \cite{jadad1996assessing}. The first major guideline for reporting in systematic reviews and meta-analyses came out in 1996 and was referred to as the Quality of Reporting of Meta-Analyses (QUORUM), and has been followed thereafter by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement \cite{shamseer2015preferred}. The \textit{Cochrane Handbook for Systematic Review of Interventions} have been update continually to address changes in assessment of quality \cite{Higgins_2011}. Similarly the \textit{Jadad} grading scale has also been designed as a means of unifying quality measures amongst clinical trials \cite{jadad1996assessing}.   The first major guideline for reporting in systematic reviews and meta-analyses came out in 1996 and was referred to as the Quality of Reporting of Meta-Analyses (QUORUM), and has been followed thereafter by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement \cite{Moher_2011}. After publication of the QUORUM statement there was an improvement in how items from the checklist were reported and quality of reporting improved in critical care literature\cite{Delaney_2007}. Despite a clear move toward progress in reporting of quality measures, there still remains significant differences in designs and gradings between each study\cite{higgins2008cochrane}. According to The Cochrane Collaboration, criteria for evaluation of bias in studies should be domain-based and in fact that further analysis would be needed if high risk of bias or low quality studies are included \cite{higgins2008cochrane}. Reporting of quality measures and methodological quality assessment is crucial for clinicians to have the best information for patient care.   The aim of our study was to assess how quality and risk of bias are evaluated in a sample of oncology systematic reviews and meta-analyses, how quality and risk of bias are graded, whether high risk of bias or low quality studies tend to be included, and if so, what are the methods of analyzing that are used to deal with high risk of bias and low quality. In addition we assessed how this information is reported within studies analyzed.   \section{Methods} 

Our overall dataset included 337 studies during just the identification process of articles (Figure 1). From that initial dataset 79 articles were excluded during the screening process due to the fact that they were neither meta-analyses nor systematic reviews. The remaining set of articles that were assessed for meeting our eligibility criteria was 258. An additional 76 studies were removed after individual and group consensus was reached about reasons to remove those articles from our dataset. The studies removed were genetic studies, individual patient data meta-analyses, genomic studies, histological studies, and a letter to an editor. Our final dataset contained 182 articles.   Within this data set, quality or risk of bias assessment was conducted in 91 articles (50\%) \ref{fig:FIGURE_3}. Most common tools used were those adapted from other sources (24\%, n=25/91) such as other authors \ref{fig:FIGURE_3}. The second highest used tools were those in which the author independently assessed (21\%, n=19/91) and those that were unspecified (13\%, n=12/91) \ref{fig:FIGURE_3}. QUORUM was the fourth highest used tool in Oncology Journals and was used 12\%, n=11/91\ref{fig:FIGURE_3}.  Quality or High Risk of Bias studies were isolated \ref{fig:FIGURE_4}.There were 35 studies in which low quality or high risk of bias were found and included with (76\%, n=35/46) \ref{fig:FIGURE_4}.From included studies, subgroup analysis was conducted in 17\%, n=16/91) \ref{fig:FIGURE_4}. Meta regression was used to address bias and quality problems in 9\% of the 46 articles that assessed quality \ref{fig:FIGURE_4}. Sensitivity analysis was used to address bias and quality reporting issues in 18\% of studies analyzed \ref{fig:FIGURE_4}.   In assessing risk of bias, high/medium/low scale was used most commonly (19\%, n=11/58) followed by high/medium/unclear (14\%, n=8/58), and quality was assessed through author created scales (29\%, n=17/58) and the Jadad scale (16\%, n=9/58) \ref{fig:FIGURE_5}. Low Quality or High risk of bias studies were found in 46 studies out of the 91 studies that assessed quality \ref{fig:FIGURE_5}. There were 37 studies in which it could not be determined whether Low  Quality measures were articulated largely in narrative format (47\%, n=43/91) or not at all (40\%, n=36/91) \ref{fig:FIGURE_6}. Additional forms of presentation included combinations of figures and narratives (4\%, n=4/91) \ref{fig:FIGURE_7}. The combination of table and narrative was also used more than single formats of presentation (3\%, n=3/91) \ref{fig:FIGURE_7}.  \section{Discussion/Conclusion}  This study provides a comprehensive and recent assessment of methodological quality and risk of bias assessment in journals associated with oncology.  Our main findings were that comprehensive reporting of quality measures in systematic reviews and meta-analyses in major oncology journals was moderate to low, with actual assessment of methodological being present in 91 of the 182 articles (50\%), and inclusion of studies with high risk of bias or low quality present in 35 of the 46 or 76\% of studies. This is surprisingly high in comparison to similar studies assessing risk of bias evaluations such as in Hopewell et al., where 20\% of non-Cochrane reviews neglected to report methods used.  In addition the inclusion of studies with high risk of bias or low quality was also an issue, with 76\% of studies with high risk of bias or low quality studies being included. This is comparable to the proportion of trials with high risk of bias included in previous studies where 75\% of trials contained one or more trials with high risk of bias \cite{hopewell2013incorporation}. Of course Hopewell et al. used the Cochrane Database of Systematic Reviews, which is known for its stringent criteria for methodological quality. However inclusion of high risk of bias or low quality studies was  not the only problem in these studies, but rather than the problem was that  out of the included articles, articles  further analysis was not conducted to address additional bias. bias or low quality.  The completeness of quality measure reporting in oncology journals appears to be lower compared to reports in related fields such as orthodontics in which PRISMA results were 64\% compared to the quality assessment found in oncology journals of 50\% \cite{tunis2013association}. The use of high risk of bias or low quality appears to not be the focus in the assessment of quality in oncology journals. The variety of quality assessment scales also indicates a problem in reporting consistently and makes comparison amongst similar studies problematic \cite{Balk_2002}.   Another point of interest was that despite presence of high quality bias or low quality studies being included in the data set, most oncology journals did not conduct further analysis to address increased bias, and it is possible that due to varied criteria of assessing quality, most studies lack a clear awareness of which types of tools to use \cite{chalmers1983bias}. Grading of scales proves to be a problem due to lack of consistent types of scales within papers \cite{J_ni_1999}.  Our study faced certain limitations, but also maintained strengths in evaluating quality of reporting. In addition, the articles pooled from our search were not distributed equally in number, which would indicate that the results refer to one particular journal rather than many. Our coding procedure also assessed over several years, so that the trend of reporting was not of primarily one year, but that of several years.