Quality assessment of included studies
The agreement between the reviewers regarding the report and methodological quality was 82.35% and 94.12%, respectively. All discrepancies were removed by two experts. Regarding the methodological quality, most of the studies (15) had a low risk of bias (Da Silva et al., 2019; Guo et al., 2016; He et al., 2019; M.-J. Kim et al., 2019; Y. S. Kim et al., 2016; Laksmitawati et al., 2016; H. A. Lee et al., 2017; S. B. Lee et al., 2017; Lim, Kim, et al., 2015; Lim, Lee, et al., 2015; Meram & Wu, 2017; Mohr et al., 2019; Pang et al., 2019; Sun et al., 2017; Zhang et al., 2015) and two had a moderate risk of bias (Ghate et al., 2018; Karatoprak et al., 2019) (Supplementary Material S8 - Table 4B, and S9). No study had a high risk of bias. Regarding the quality of the reports, eight of the 17 studies had a moderate risk, while 9 had a low risk of bias.
In all the SCIRAP evaluation criteria, there was an item that was not covered by almost any of the studies, but one that we consider crucial for the quality of the articles. For example, in the test compound and controls session, only one author (H. A. Lee et al., 2017) described the vehicle used to dilute the compounds. Similarly, in the test system session, only two authors reported the number of cell passages used to perform the experiments (M.-J. Kim et al., 2019; Mohr et al., 2019), and no author provided information on the measures taken to track or avoid contaminations by mycoplasma, bacteria, fungi or virus. Furthermore, only two authors (Guo et al., 2016; H. A. Lee et al., 2017) provided complete information regarding the toxicity of the test compound as well as the phlogistic agent (LPS) (Supplementary Material S8 - Table 4A, and S9).
In the methodological quality, the lack of information about the vehicle used, as well as the lack of tests to verify the interference of possible cytotoxicity of the compounds in the results, were the negative aspects that were present in practically all the studies. In addition, 9 studies did not provide information on replicates or repetitions of the experiments to verify the reproducibility/reliability of the results (Da Silva et al., 2019; Ghate et al., 2018; Karatoprak et al., 2019; M.-J. Kim et al., 2019; Y. S. Kim et al., 2016; S. B. Lee et al., 2017; Lim, Kim, et al., 2015; Lim, Lee, et al., 2015; Sun et al., 2017). It is important to note that all the studies provided complete information and were extremely reliable in regard to the administration of the test compound. This was to be expected, given the strictness of the pre-established eligibility criteria (Supplementary Material S8 - Table 4B, and S9).
In the relation of the phlogistic agent (LPS), we considered the LPS effects with indirect relevance for all the studies that did not provide information on the manufacturers or sources of LPS strains used (Da Silva et al., 2019; Ghate et al., 2018; Karatoprak et al., 2019; M.-J. Kim et al., 2019; Laksmitawati et al., 2016) (Supplementary Material S8 - Table 4C, and S9). The other items (test systems, endpoint and concentrations) were considered of direct relevance.  For the tested system, as expected, all the studies were of direct relevance, because the experimental model in RAW cells induced by LPS was the most used to mimic the inflammatory process in vitro , and was one of the eligibility criteria used for the inclusion of studies. Likewise, in terms of the endpoint and concentration, all the articles were also of direct relevance, as the studies had to meet specific eligibility criteria at the time of collection/treatment, and a particular concentration of LPS. All the studies were considered reliable, with restrictions (Supplementary Material S8 - Fig 9), as none completely met with the guidelines in regard to the reporting and methodological assessment. In summary, the studies presented some restrictions as to the origin of the cell, a number of cell passages (15), and microbiological control (All studies). Also, some studies did not report toxicity (15), while others did not report experimental and biological replicates (9).