4.4 Strengths and limitations
To the best of our knowledge, this is the first study to investigate the
characteristics of OG COSs and assess whether the OG COSs meet the
minimum design standards and minimal COS study reporting requirements
set by COS-STAD checklist. In addition, we explore the potential factors
that may impact the methodological and reporting standards of COSs.
This work had the following limitations. First, some studies only
conducted the scoping process or consensus process which led to the low
reporting rate of some criteria. However, because we focused on
investigating the current issues of COSs and explored how to improve the
COS standards instead of criticizing the quality of the existing COSs,
the impact of this limitation was limited. Second, some of the included
COSs were published before the COMET initiative was set up, hence were
not registered. Therefore, our definition of COSs developed with the
protocol was not limited to this website, but also other registration
platforms, and protocols published in the peer-reviewed journal that met
our set criteria. Third, we applied the same COS-STAD criteria to
compare the standards between OG COSs with cancer COSs based on previous
research. Moreover, variation in standards for item compliance may also
exist in the assessment progress between different reviewers. Given that
COS-STAD and COS-STAR did not provide a standard to score the overall
quality of COSs, our analysis was limited to the compliance rate.
Further studies are required to explore the appropriate statistical
methods for estimating an average value of the quality scores of
multiple COSs.
Conclusion
The compliance of COS-STAD and COS-STAR criteria in OG COSs are not
optimistic. Therefore, the reporting and methodological standards of OG
COSs should be further improved, especially, the design of important
stakeholders involved and consensus process; and reporting standards in
methods, results, and other information specifications. OG COS developed
with the prior protocol is beneficial to improve the reporting
standards.
Disclosure of interests: We declare that we have no competing
interests