4.4 Strengths and limitations
To the best of our knowledge, this is the first study to investigate the characteristics of OG COSs and assess whether the OG COSs meet the minimum design standards and minimal COS study reporting requirements set by COS-STAD checklist. In addition, we explore the potential factors that may impact the methodological and reporting standards of COSs.
This work had the following limitations. First, some studies only conducted the scoping process or consensus process which led to the low reporting rate of some criteria. However, because we focused on investigating the current issues of COSs and explored how to improve the COS standards instead of criticizing the quality of the existing COSs, the impact of this limitation was limited. Second, some of the included COSs were published before the COMET initiative was set up, hence were not registered. Therefore, our definition of COSs developed with the protocol was not limited to this website, but also other registration platforms, and protocols published in the peer-reviewed journal that met our set criteria. Third, we applied the same COS-STAD criteria to compare the standards between OG COSs with cancer COSs based on previous research. Moreover, variation in standards for item compliance may also exist in the assessment progress between different reviewers. Given that COS-STAD and COS-STAR did not provide a standard to score the overall quality of COSs, our analysis was limited to the compliance rate. Further studies are required to explore the appropriate statistical methods for estimating an average value of the quality scores of multiple COSs.
Conclusion
The compliance of COS-STAD and COS-STAR criteria in OG COSs are not optimistic. Therefore, the reporting and methodological standards of OG COSs should be further improved, especially, the design of important stakeholders involved and consensus process; and reporting standards in methods, results, and other information specifications. OG COS developed with the prior protocol is beneficial to improve the reporting standards.
Disclosure of interests: We declare that we have no competing interests