Sarah Khan edited untitled.tex  almost 9 years ago

Commit id: 55f1f2b5ad6bfdaf33d9aafce2cb6fc96f1a90b7

deletions | additions      

       

\subsection{\textbf{Methods:}}  We performed a search of systematic reviews from high impact factor journals in oncology from 2007 to 2015 through PubMed. Covidence was used to screen articles based on the title and abstract. The methodological quality and reporting of risk of bias were evaluated by three rounds of coding from two independent reviewers using the same checklist. Differences in assessment were resolved through group consensus.  \subsection{\textbf{Results:}}  Quality assessment was studied on 182 articles after exclusion. Quality or risk of bias assessment was assessed in 48\% of articles. More common were tools adapted from other author's custom  sources (29\%), author independently assessed (22\%) (23\%), other (14\%)  and unspecified (14\%). Cochrane Risk of Bias Tool (13\%).  Low Quality or High risk of bias studies were found in 45 40  studies. From included studies, subgroup analysis was conducted in 13\%, 14\%,  meta-regression in 9\%, 10\%,  and sensitivity analysis in 18\%. 21\%.  Quality and risk of bias was not reported in 34 32  studies. Quality measures were articulated in narrative format (47\%) (44\%)  or not at all (40\%). (44\%).  \subsection{\textbf{Conclusions:}}  Quality and risk of bias were assessed in only half of systematic reviews, and even when addressed, methods of assessment are more commonly determined by authors rather than following recommended guidelines. This analysis provides further evidence for inconsistent quality measure reporting for clinical findings in oncology manuscripts. Differences between bias assessment and quality reporting could misdirect intervention results in oncology journals.  \subsection{\textbf{Keywords:}} 

We performed a descriptive analysis of the frequency and percentage use of quality assessment tools. We tabulated the frequency of quality assessment tools used, type of tools, types of scales used, how the quality information was presented, types of methods used to deal with risk of bias or low quality. In assessing the types of tools used to measure quality, we had some additional categories such as author custom measure, adapted criteria, and unspecified. For studies to fit into the author's custom measure category an author had to describe their own method of evaluating study quality. In a situation where the author of the study used a quality assessment method adapted from another study that was similar to their own we listed it as adapted criteria. There were studies where it was indicated either in the abstract or in the methods section that quality was assessed, but there was no specification of how quality was assessed, and so those articles were listed under the 'unspecified' category. We also looked at frequency of high risk of bias or low quality studies being included in data set of articles, and if studies were included, were they dealt with using subgroup analysis, meta-regression, or sensitivity analysis \ref{fig:FIGURE_2}. Statistical analyses were performed with STATA version 13.1 software (State Corporation, College Station, Texas, USA).     \section{Results}  Our overall dataset included 337 studies during just the identification process of articles (Figure 1). From that initial dataset 79 articles were excluded during the screening process due to the fact that they were neither meta-analyses nor systematic reviews. The remaining set of articles that were assessed for meeting our eligibility criteria was 258. An additional 76 studies were removed after individual and group consensus was reached about reasons to remove those articles from our dataset. The studies removed were genetic studies, individual patient data meta-analyses, genomic studies, histological studies, and a letter to an editor. Our final dataset contained 182 articles. Within this data set, quality or risk of bias assessment was conducted in 91 77  articles (48\%) (42\%)  \ref{fig:FIGURE_3}. Most common tools used were those adapted from other sources (29\%, n=25/87) designed by authors (23\%, n=18/77)  such as other authors \ref{fig:FIGURE_3}. The second highest used tools were those in which the author independently assessed (22\%, n=19/87) other (14\%, n=11/77)  and those that were unspecified (14\%, n=12/87) under the Cochrane Risk of Bias tool (13\%, n=10/77)  \ref{fig:FIGURE_3}. Jadad and Newcastle-Ottawa Scale tied for Adapted measures were  the fourth highest used tools in Oncology Journals and was used 9\%, n=8/87 12\%, n=9/77  \ref{fig:FIGURE_3}. Quality or High Risk of Bias studies were isolated \ref{fig:FIGURE_4}.There were 35 studies in which low quality or high risk of bias were found and included with (78\%, n=35/45) \ref{fig:FIGURE_4}.From included studies, subgroup analysis was conducted in 13\%, n=11/87) \ref{fig:FIGURE_4}. Meta regression was used to address bias and quality problems in 9\% of the 45 articles that assessed quality \ref{fig:FIGURE_4}. Sensitivity analysis was used to address bias and quality reporting issues in 18\% of studies analyzed \ref{fig:FIGURE_4}.   In assessing risk of bias, high/medium/low scale was used most commonly (20\%, n=11/56) followed by high/medium/unclear (14\%, n=8/56), and quality was assessed through author created scales (30\%, n=17/56) and the Jadad scale (14\%, n=8/56) \ref{fig:FIGURE_5}. Low Quality or High risk of bias studies were found in 45 studies out of the 87 studies that assessed quality \ref{fig:FIGURE_6}. There were 34 studies in which it could not be determined whether low quality or high risk of bias studies were isolated.  Quality measures were articulated largely in narrative format (49\%, n=43/87) (44\%, n=34/77)  or not at all (41\%, n=36/87) \ref{fig:FIGURE_7}. (44\%, n=34/77) \ref{fig:FIGURE_6}.  Additional forms of presentation included combinations of figures and narratives (5\%, n=4/87) \ref{fig:FIGURE_7}. n=4/77) \ref{fig:FIGURE_6}.  The combination of table and narrative was also used more than single formats of presentation (3\%, n=3/87) \ref{fig:FIGURE_7}. n=2/77) \ref{fig:FIGURE_6}.  \section{Discussion/Conclusion}  This study provides a comprehensive and recent assessment of methodological quality and risk of bias assessment in journals associated with oncology. Our main findings were that comprehensive reporting of quality measures in systematic reviews and meta-analyses in major oncology journals was moderate to low, with actual assessment of methodological being present in 87 of the 182 articles (48\%), and inclusion of studies with high risk of bias or low quality present in 35 of the 45 or 76\% of studies. This is surprisingly high in comparison to similar studies assessing risk of bias evaluations such as in Hopewell et al., where 20\% of non-Cochrane reviews neglected to report methods used. In addition the inclusion of studies with high risk of bias or low quality was also an issue, with 76\% of studies with high risk of bias or low quality studies being included. This is comparable to the proportion of trials with high risk of bias included in previous studies where 75\% of trials contained one or more trials with high risk of bias \cite{hopewell2013incorporation}. Of course Hopewell et al. used the Cochrane Database of Systematic Reviews, which is known for its stringent criteria for methodological quality. Our study also focused only on oncology journals while the Hopewell et al. study focused on Cochrane and non-cochrane reviews across many fields.