The broad influence that systematic reviews of randomized controlled trials have on clinical decision-making proves how crucial these types of studies are for patient treatment. Health care providers use the information gained systematic reviews to attempt to give their patients the best treatment possible. Nonetheless, when the original studies that constitute these systematic reviews have foundational errors in study design they can be put at high risk of bias, which, as a result, decreases the overall benefit gained from systematic reviews \cite{25481532}. In our examination of ophthalmology journals, only 47.80% (87/182) of reviews reported the assessment of MQ/ROB.

We found that the Cochrane Risk of Bias Tool was used most commonly for evaluation of MQ/ROB at 20.69% (18/87) followed closely by the Jadad scale (19.54%; 17/87). There is extensive disagreement over which MQ/ROB tool gives the most accurate portrayal of a study’s quality \cite{Zeng_2015} \cite{10493204}. To assess the effectiveness of each tool, we examined their component parts to evaluate which provides the most of extensive evaluation of MQ/ROB. The Downs and Black Checklist was the most extensive, assessing 62.79% (27/43) of listed measures. The CASP Checklist for RTCs and the Newcastle-Ottawa scale followed, both assessing 34.88% (15/43) of measures. QUADAS-2 (32.56%; 14/43) was slightly more thorough than that QUADAS (30.23%; 13/43) at assessing MQ/ROB. We found that Jadad (27.91%; 12/43), the Delphi List (18.60%; 8/43), and the Cochrane Risk of Bias Tool (16.28%; 7/43) were the three least extensive of the tools used. However, the Cochrane Handbook for Systematic Reviews of Interventions states that, “It is preferable to use simple approaches for assessing validity that can be fully reported (i.e. how each trial was rated on each criterion)” \cite{2008}. Furthermore, the Cochrane Handbook discourages the next most commonly used tool in our study, the Jadad scale. The handbook states that not only are there generic problems with the use of this scale, but Jadad also strongly emphasizes reporting over conduct and does not implement allocation concealment, one of the most important biases, into the scale.

Many authors in our study used custom measures to evaluate MQ/ROB. These custom measures were either a combination of previously published tools and the author’s own custom quality criteria or solely the author’s choice of quality criteria. A subgroup of these custom measures did not evaluate important aspects of study quality. Allocation concealment, which is highly agreed upon as a very important component for MQ/RO \cite{18426565}, was only present in 50.0% (7/14) of custom measures. Blinding, specifically double blinding of both the patient and the assessor, was assessed in only 42.86% (6/14) reviews using a custom created tool. Response rate/withdrawal (64.29%; 9/14), blinding of the assessor alone (57.14%; 8/14), valid and objective outcome measures (57.14%; 8/14), and inclusion/exclusion criteria (57.14%; 8/14) were the only components evaluated in >50.0% of the studies. In addition, there were seven included studies (Tanna 2010, Diener-West 1992, Markowitz 1992, Ding 2008, Rogers 2010, McGimpsey 2009, and Dueker 2007) that used clinical information in addition to MQ/ROB measures that could falsely inflate the quality score of the review.

Out of 87 reviews reporting assessment of MQ/ROB, only 12.64% (11/87) explicitly stated that low MQ/high ROB were excluded from their review while 57.47% (50/87) included articles with low MQ/high ROB. More than twice as many studies were unclear about inclusion of low MQ/high ROB (29.89%; 26/87) as those that were excluded (12.64%; 11/87). Of the 50 articles that included articles with low MQ/high, only 42.0% (21/50) performed a subgroup analysis, 32.0% (16/67) conducted a meta-regression analysis, and 62.0% (31/50) performed a sensitivity analysis. The quintessential systematic review would exclude all primary studies that contained low MQ/high ROB. Yet, in the least, a review should perform a subgroup, meta-regression, or sensitivity analysis to account for the possible affect that inclusion may have on summary effect size \cite{25481532}.

In conclusion, our study has suggested that most authors in ophthalmology rarely assess MQ/ROB in their systematic reviews. Moreover, when MQ/ROB is evaluated, the majority of authors use a tool that does not provide an accurate assessment of study validity. The implications of these conclusions provide that many therapeutic interventions in ophthalmology are founded on systematic reviews that may be composed of unsound evidence, thus resulting in unwarranted treatment. We recommend that systematic reviewers adopt the use of the Cochrane Risk of Bias Tool for the evaluation of MQ/ROB due to the well-founded indication that other scales, custom measures, and checklists provide an invalid assessment of quality and risk of bias. Future research could examine what determines the authors’ choice of MQ/ROB tools in systematic reviews. A thorough review of studies that contain custom measures may be warranted, as these custom measures may have no basis in the evalutation of quality or risk of bias, and, alternatively, may only inflate MQ/ROB scores.