Public Articles
Methodological Quality and Risk of Bias Inconsistently Evaluated and Addressed in Meta-Analyses and Systematic Reviews in Major Oncology Journals
and 4 collaborators
This study aimed to the reporting and utilization of methodological quality measures in addressing low quality and risk of bias in major oncology journals.
We performed a search of systematic reviews from high impact factor journals in oncology from 2007 to 2015 through PubMed. Covidence was used to screen articles based on the title and abstract. The methodological quality and reporting of risk of bias were evaluated by three rounds of coding from two independent reviewers using the same checklist. Differences in assessment were resolved through group consensus.
Quality assessment was examined in 182 articles after exclusion. Quality or risk of bias assessment was assessed in 48% of articles. More common were tools adapted from authors’custom sources (23%), others (14%), and the Cochrane Risk of Bias Tool (13%). Low quality or high risk of bias studies was detected in 40 studies. Subgroup analysis was conducted in 14%, meta-regression in 10%, and sensitivity analysis in 21%. Low quality or risk of bias were not reported in 32 studies. Quality measures were articulated in narrative format (44%), not at all (44%), or in a combination of tables and figures (12%) .
Quality and risk of bias were assessed in only half of systematic reviews; moreover, when addressed, the methods of assessment were more commonly determined by the authors rather than following recommended guidelines. This analysis provides further evidence for inconsistent quality measure reporting for clinical findings in oncology manuscripts. Differences between bias assessment and quality reporting could misrepresent intervention results in oncology journals.
meta-analysis;oncology;quality; risk of bias;systematic review
The use of systematic reviews and meta-analyses has become increasingly important in evidence-based medicine as clinicians seek reliable information on treatments and care guidelines in their medical practice \cite{14764293}. Since systematic reviews synthesize evidence from multiple studies, clinicians are able to better understand the individual trials comprising the review as well as the efficacy of the therapy summarized across all available, relevant evidence. One essential feature that lends confidence to the findings of a review is an appraisal of the methodology of studies comprising the review. In cases where systematic reviewers have concluded that primary studies are of high methodological quality or have low potential for biased outcomes, clinicians can have more confidence in the study findings. For example, Yang et al. evaluated the toxicity and efficacy of chemotherapy plus cetuximab in relation to chemotherapy alone in patients with advanced non-small cell lung cancer. The systematic review comprised of four trials. A risk of bias assessment of these trials was conducted, and the authors concluded that risk of bias was low for overall survival and one-year survival rates but high for all other outcomes due to a lack of blinding. Hence, the reviewers concluded that chemotherapy plus cetuximab was better than chemotherapy alone for improving overall survival; the risk of bias assessment played an important role in the interpretation of the summary effect.
Many scales are designed in response to concerns regarding methodological quality among primary studies; however, recent evidence indicates scales may not be the best way to appraise studies \cite{juni2001assessing}\cite{juni1999hazards}. Rather, certain design features should be reviewed to provide a clearer picture of bias in trials \cite{lohr1999assessing}.The Cochrane Handbook for Systematic Review of Interventions is continually updated to improve the assessment of methodological quality in clinical studies and advocates for appraising the risk of bias of all primary studies included for review \cite{higgins2011cochrane}. Major reporting guidelines for systematic reviews have been published and suggest some form of quality appraisal. The first guideline, published in 1996, was referred to as the Quality of Reporting of Meta-Analyses (QUORUM) and advocated use of a methodological quality measures tool for appraisal. More recently, however, QUORUM’s predecessor, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), called for evaluating the risk of bias of primary studies. This recommendation, consistent with the Cochrane Collaboration, accounts for criticisms of the quality scales, including that certain components of these tools often have no known role in contributing to the validity of findings, such as whether investigators reported oversight by an institutional review board. Inclusion of such items can artificially inflate the overall quality score of a particular study.
Despite a clear move toward progress in this area, there are still significant differences in quality assessment practices between systematic reviews \cite{higgins2008cochrane}. In fact, little is known about the application of methodological quality or risk of bias measures in clinical specialties like oncology. To address this issue, we conducted a study of the oncology literature to assess how often quality and risk of bias assessments were used in oncology systematic reviews, determine the prevalence of approaches reported by the authors, and examine the ways that such evaluations are incorporated into the reviews.
Using the h5-Index from Google Scholar Metrics, we selected the six oncology journals with the highest index scores. We searched PubMed search using the following search string: ((((((“Journal of clinical oncology : official journal of the American Society of Clinical Oncology”[Journal] OR “Nature reviews. Cancer”[Journal]) OR “Cancer research”[Journal]) OR “The Lancet. Oncology”[Journal]) OR “Clinical cancer research : an official journal of the American Association for Cancer Research”[Journal]) OR “Cancer cell”[Journal]) AND (“2007/01/01”[PDAT] : “2015/12/31”[PDAT]) AND “humans”[MeSH Terms]) AND (((meta-analysis[Title/Abstract] OR meta-analysis[Publication Type]) OR systematic review[Title/Abstract]) AND (“2007/01/01”[PDAT] : “2015/12/31”[PDAT]) AND “humans”[MeSH Terms]) AND ((“2007/01/01”[PDAT] : “2015/12/31”[PDAT]) AND “humans”[MeSH Terms]). This search strategy was adapted from a previously established method that is sensitive to identifying systematic reviews and meta-analyses (Montori 2005). Searches were conducted on May 18 and May 26, 2015.
Screening and data extraction We used Covidence (covidence.org) to initially screen articles based on title and abstract. To qualify as a systematic review, articles had to summarize evidence across multiple studies and provide information on the search strategy, such as search terms, databases, or inclusion/exclusion criteria \cite{babineau2014product}. Meta-analyses were classified as quantitative syntheses of results across multiple studies (Onishi 2014). Two screeners independently reviewed the titles and abstracts of each citation and made a decision regarding its suitability for inclusion based on the definitions previously described. Next, the screeners held a meeting to revisit the citations in conflict and arrive at a final consensus. Following the screening process, full-text versions of included articles were obtained via EndNote.
To standardize the coding process, an abstraction manual was developed and pilot tested. After completing this process, a training session was conducted to familiarize coders with abstracting the data elements. A subset of studies was jointly coded. After the training exercise, each coder was provided with three new articles to code independently. Each coder was next assigned an equal subset of articles for data abstraction. We coded the following elements: a) whether methodological quality or risk of bias was assessed, and if so the tool used; b) whether authors developed a customized measure; c) whether methodological quality was scored, and if so, what scale was used; d) whether authors identified high risk of bias or low-quality studies; e) whether high risk of bias or low-quality studies were included in the estimation of summary effects; f) how risk of bias or quality appraisal information was presented in the article; and g) whether follow-up analyses were conducted to explore the effects of bias on study outcomes (such as subgroup analysis, sensitivity analysis, or meta-regression).
We performed a descriptive analysis of the frequency and percent use of quality assessment tools used, types of tools, types of scales used, how the quality information was presented, and types of methods used to deal with risk of bias or low quality. In assessing the types of tools used to measure quality, we created some additional categories to account for the variations in approaches. We coded an appraisal as “author’s custom measure” if authors described their own approach to evaluating study quality. In situations where the author used a quality assessment method adapted from another study, we coded this as “adapted criteria.” Some studies indicated (either in the abstract or from the methods section) that methodological quality was assessed, but there was no specific detail beyond this generic statement. These were coded as “unspecified.” Statistical analyses were performed with STATA version 13.1 software (State Corporation, College Station, Texas, USA).
The PubMed search resulted in 337 articles from four journals. After screening titles and abstracts, 79 were excluded because they were not systematic reviews or meta-analyses. An additional 76 articles were excluded after full text screening. Two articles could not be retrieved after multiple attempts. A total of 182 articles were included in this study (Figure 1).
Methodological quality or risk of bias assessment was conducted in 42% (77/182) of systematic reviews. Of the 77 articles where assessment of methodological quality or risk of bias was identified, 51.95% (40/77) found either low methodological quality or high risk of bias in primary studies comprising systematic reviews. Studies with an unclear risk of bias or unknown methodological quality were reported in 41.56% (32/77) of reviews; Five cases (6.49%) reported no issues with study quality or risk of bias.
The most common approaches to evaluating risk of bias or methodological quality were those designed by authors (23.4%, 18/77). The Cochrane Risk of Bias Tool was the most commonly reported standardized measure used by systematic reviewers (14.3%, 11/77), followed by the Newcastle–Ottawa scale (10.4%, 8/77), the Jadad scale (10.4%, 8/77), QUADAS-2 (5.19%, 4/77), and QUADAS (3.9%, 3/77). Measures adapted from previous work were reported by 13% (10/77). Other measures used only once are reported in Table 1 and represented 10.4% (8/77) of the approaches used. There were 25 studies with low quality or high risk of bias that were included with (78%, 35/45).From included studies, subgroup analysis was conducted in 13%, 11/77). Meta regression was used to address bias and quality problems in 9% of the 45 articles that assessed quality. Sensitivity analysis was used to address bias and quality reporting issues in 18% of studies analyzed.
We examined the scales by which reviewers scored or categorized studies. This information was reported in 56 systematic reviews. For risk of bias assessments, the high/medium/low format was used most commonly (20%, 11/56) followed by high/low/unclear (14%, 8/56). Methodological quality was most commonly assessed using a 0-5 point scale (16.07%, 9/56) followed by Good/Fair/Poor (7.14%, 4/56) and 1-9 point scale (5.36%, 3/56).
Methodological quality information was articulated largely in narrative format (44%, 34/77) or not at all (44%, 34/77). Additional forms of presentation included combinations of figures and narratives (5%, 4/77) . The combination of table and narrative was also used more than single formats of presentation (3%, 2/77). Single formats of presentation either as a table or figure were used more than the combination of all three forms of presentation (3%, 2/77). The combination of tables, figures, and narrative was used in 1% of assessed articles.
This study provides a comprehensive and recent assessment of methodological quality and risk of bias assessment in oncology journals. Our main findings indicate that reporting of quality assessment in systematic reviews and meta-analyses in major oncology journals is moderate to low, with actual assessment of methodological quality being present in only 48% of studies. This is low in comparison with similar studies assessing frequency of risk of bias evaluations. Hopewell et al., for example, found that 80% of non-Cochrane reviews reported methods for evaluating methodological quality or risk of bias.
The inclusion of studies with high risk of bias or low quality in deriving summary effects was also an issue, with 76% of studies in our sample including such studies in calculating results; however, this is comparable with the proportion of trials with high risk of bias included in previous studies. Hopewell et al. reported that 75% of trials in their study contained one or more trials with a high risk of bias \cite{hopewell2013incorporation}. It should be noted that Hopewell et al. used the Cochrane Database of Systematic Reviews, which is known for its stringent adherence to Cochrane guidelines, of which risk of bias evaluations are a routine part. This may contribute to differences between our findings and theirs.
Despite the presence of high risk of bias or low quality studies, most review authors did not conduct a further analysis to explore the influence of bias on study outcomes. Perhaps more interesting was the number of systematic reviews reporting an assessment of study quality, but leave the reader to wonder what had become of those evaluations. A significant number included such studies without further mention of the quality of evidence. Narrative styles of presenting information for quality assessment were the most common means of presenting this information; however, the use of a table format would provide readers with easier access to quality or risk of bias information. We advocate for a more structured approach, such as tables published in the review, to display such information.
Future research should continue to investigate these evaluation practices in systematic reviews in other clinical specialties. While the majority of research has been confined largely to Cochrane systematic reviews, as well as those published in high profile medical journals, there is a need to understand whether systematic reviewers in clinical specialties conduct these assessments and, if so, what assessments they use.
Chemotherapy with cetuximab versus chemotherapy alone for chemotherapy-naive advanced non-small cell lung cancer.Yang ZY1, Liu L, Mao C, Wu XY, Huang YF, Hu XF, Tang JL.
Reverse engineering an open-source MOOC: optional open notebook from PPE 101
and 3 collaborators
Welcome! This is Cobi's open notebook for tutoring work within the Melbourne School of Population and Global Health. You can email me on [email protected] or check the course guide for more information.
A microbial survey of the most extreme built environment, the International Space Station (ISS)
\newpage
A microbial survey of the most extreme built environment: the International Space Station (ISS)
#Abstract
Modern advances in sequencing technology have enabled the census of microbial members of many natural ecosystems. Recently, attention is increasingly being paid to the microbial residents of human-made, built ecosystems, both private (homes) and very public (subways, office buildings, and hospitals). Here, we report results of the characterization of the microbial ecology of a singular built environment, the International Space Station (ISS). This ISS sampling involved the collection and microbial analysis (via 16S rDNA PCR) of 15 samples swabbed from surfaces onboard the ISS. This sampling is a component of Project MERCCURI - a collaborative effort of the "microbiology of the Built Environment network" (microBEnet) project, Science Cheerleaders, NanoRacks, Space Florida, and Scistarter.com. Learning more about the microbial inhabitants of the "buildings" in which we travel through space will take on increasing importance, as plans for human exploration and colonization of the solar system come to fruition.
Sterile swabs were used to sample 15 surfaces onboard the ISS. The sites sampled were designed to be analogous to samples collected for 1) the Wildlife of Our Homes project and 2) a study of cell phones and shoes that were concurrently being collected for another component of Project MERCCURI. Sequencing of the 16S rRNA genes amplified from DNA extracted from each swab was used to produce a "census" of the microbes present on each surface sampled. We compared the microbes found on the ISS swabs to those from both the Earth homes and the Human Microbiome Project.
While significantly different from homes on Earth and the Human Microbiome Project samples analyzed here, the microbial community composition on the ISS was more similar to home surfaces than to the human microbiome samples. The ISS surfaces are species-rich with 1036-4294 operational taxonomic units (OTUs per sample). There was no discernible biogeography of microbes on the 15 ISS surfaces, although this may be a reflection of the small sample size we were able to obtain.
EEMIAN DISTRIBUTION OF NEANDERTHALS
and 1 collaborator
Aim: We hypothesize that the northern and southern edges of Neanderthals distribution during the Last Interglacial were respectively limited by low winter temperatures, and high temperatures and lack of rainfall during the summer, while high topographic diversity combined with moderate slopes favored presence at the local scale.
Location: Europe (20ºN to 70ºN, 10ºW to 70ºE).
Methods: We used Neanderthal presence, palaeoclimatic, and topographic data to calibrate a species distribution model. We analyzed variable importance at the continental scale with Randon Forest, and at the local scale with local regression and recursive partition trees.
Results: Highest habitat suitability was observed along the Mediterranean coast. Main mountain ranges and continental plains showed low habitat suitability values. Extreme seasonal temperatures and annual rainfall were the most important predictors at the continental scale, while topography defined habitat suitability at the local scale.
Main conclusions: Our results challenge the notion of Neanderthals as a central European species. Therefore, many current interpretations of Neanderthal livelihood during the Last Interglacial may not accurately represent this species preferred habitat.
Keywords: palaeo-species distribution modeling, hominins, habitat suitability, ecological niche
Preparation of manuscripts for the American Journal of Physics using LaTeX
and 2 collaborators
LaTeX is typesetting software that is widely used by mathematicians and physicists because it is so good at typesetting equations. It is also completely programmable, so it can be configured to produce documents with almost any desired formatting, and to automatically number equations, figures, endnotes, and so on. \cite{25165807}
To prepare manuscripts for the American Journal of Physics (AJP), you should use the REVTeX 4.1 format for Physical Review B preprints, as indicated in the documentclass
line at the top of this article’s source file. (If you’re already familiar with LaTeX and have used other LaTeX formats, please resist the temptation to use them, or to otherwise override REVTeX’s formatting conventions, in manuscripts that you prepare for AJP.)
This sample article is intended as a tutorial, template, and reference for AJP authors, illustrating most of the LaTeX and REVTeX features that authors will need. For a more comprehensive introduction to LaTeX, numerous books and online references are available. Documentation for the REVTeX package can be found on the APS web site.\cite{revtex}
LaTeX is free software, available for Unix/Linux, Mac OS X, and Windows operating systems. For downloading and installation instructions, follow the links from the LaTeX web site.\cite{latexsite} It is most convenient\cite{cloudLaTeX} to install a “complete TeX distribution,” which will include LaTeX, the underlying TeX engine, macro packages such as REVTeX, a large collection of fonts, and GUI tools for editing and viewing your documents. To test your installation, try to process this sample article.
MADX Demo: Input and Output
\label{chap:tfs}
TFS
\cite{TFS} is a an acronym for the “Table File System”. TFS
files have been used in the LEP control system. The program knows only coded TFS
files. The TFS
format has been chosen for all table output of . TFS
formatted tables can be read back into , and may then be further processed.
PHYS321 Derivation
In pursuit of an empirically determined value for the electron charge-to-mass e/m ratio we must guide electrons in a variable circular path using a homogeneous magnetic field. The Helmholtz configuration of two current-carrying coils with radius R separated by a distance d as shown in Fig. [fig1] provides the necessary field. To determine the magnetic field $\vec{\textit{B}}$ we will first find the magnetic field due to a single current-carrying loop using the Biot-Savart Law.
Summary of research for: “Local Radiation Hydrodynamic Simulations of Massive Star Envelopes at The Iron Opacity Peak”
This is a layman summary of “Local Radiation Hydrodynamic Simulations of Massive Star Envelopes at The Iron Opacity Peak” from Yan-Fei Jiang (姜燕飞), Matteo Cantiello , Lars Bildsten, Eliot Quataert and Omer Blaes. The full article can be downloaded from the arXiv.
This layman summary is part of the Public Friendly Open Science initiative.
A Grateful Dead Analysis: The Relationship Between Concert and Listening Behavior
Coupled Parametric Oscilators Proof of Concept in Radio Frequency
Optical Paramteric Oscillators (OPO) oscillate at a specific frequency by means of a non-linear interaction. A pump input laser with frequency ωp is converted through the non-linear interaction into a signal and idler frequencies, such that ωp = ωi + ωs. Both frequencies can resonate in the cavity (doubly-resonant) or just one (singly-resonant).
Diffusion-Limited Aggregation
Diffusion-Limited Aggregation (DLA) is a process whereby particles undergoing Brownian motion aggregate to form clusters of particles. “Diffusion” because the particles diffuse randomly before attaching themselves (“Aggregating”) to the structure. “Diffusion-limited” because the particles are considered to be in low concentrations so the structure grows one particle at a time.
DLA can be observed in many natural phenomenon, such as the formation of snowflakes and the formation of electrically conducting regions in a dielectric breakdown. These clusters are an example of a fractal, i.e. a pattern that replicates itself in any scale.
Analysis and visualization of emerging zoonoses through temporal networks
I present case tree plots and checkerboard plots for visualizing contagions. The visualizations are best suited for diseases like SARS, MERS-CoV and H7N9 for which there are a limited (less than 200) number of cases, with data available on human to human transmission. They a) allow for easy estimation of epidemiological parameters like basic reproduction number b) indicate the frequency of introductory events, e.g. spillovers in the case of zoonoses c) represent patterns of case attributes like patient sex both by generation and over time.
The Environmental Footprints Explorer - a database for global sustainable accounting
and 4 collaborators
Environmentally Extended Multi Regional Input Output tables and analysis (EE MRIOs) have emerged as one of the main tools to analyze resource use and environmental impacts across international supply chains. They provide insights into the life cycle impacts of the production and consumption of commodities world wide, taking into account the global supply chain of purchased commodities. Currently half a dozen EE MRIO databases are available which differ in their environmental and economic focus as well as in the level of detail. As these databases become more sophisticated, it has become increasingly difficult for the non-expert to access the most important attributes and results of basic calculations. Here we present an integrated web-platform, the Environmental Footprints Explorer (http://www.environmentalfootprints.com), designed to access indicator results calculated based on these databases. The main functionality of the web-platform include (1) exploring environmental accounts based on a single database (2) comparison between databases using a common classification system and (3) exporting analysis results visualization. The presented web-platform removes the obstacle for policy-makers and the public alike to access EE MRIO results.
Welcome to Authorea!
ProCS15: A DFT-based chemical shift predictor for backbone and C\(\beta\) atoms in proteins
and 3 collaborators
We present ProCS15: A program that computes the isotropic chemical shielding values of backbone and Cβ atoms given a protein structure in less than a second. ProCS15 is based on around 2.35 million OPBE/6-31G(d,p)//PM6 calculations on tripeptides and small structural models of hydrogen-bonding. The ProCS15-predicted chemical shielding values are compared to experimentally measured chemical shifts for Ubiquitin and the third IgG-binding domain of Protein G through linear regression and yield RMSD values below 2.2, 0.7, and 4.8 ppm for carbon, hydrogen, and nitrogen atoms respectively. These RMSD values are very similar to corresponding RMSD values computed using OPBE/6-31G(d,p) for the entire structure for each protein. The maximum RMSD values can be reduced by using NMR-derived structural ensembles of Ubiquitin. For example, for the largest ensemble the largest RMSD values are 1.7, 0.5, and 3.5 ppm for carbon, hydrogen, and nitrogen. The corresponding RMSD values predicted by several empirical chemical shift predictors range between 0.7 - 1.1, 0.2 - 0.4, and 1.8 - 2.8 ppm for carbon, hydrogen, and nitrogen atoms, respectively.
White paper
and 2 collaborators
Hey, welcome.
Double click anywhere on the text to start writing.
In addition to simple text you can also add text formatted in boldface, italic, and yes, math too: E = mc2!
Add images by drag’n’drop or click on the “Insert Figure” button.
Citing other papers is easy. Voilà: \cite{2012} or \cite{Holstein_2009}. Click on the cite
button in the toolbar to search articles and cite them.
Authorea also comes with a powerful commenting system. Highlight the text you want to discuss or click the comment button.
Find out more about using Authorea on our help page.
Giant anterior sacral meningocele repaired through a posterior approach.
and 3 collaborators
Anterior sacral meningoceles are rare forms of spinal dysraphism produced by herniation of the thecal sac through a bone defect in the anterior sacral wall. The patients may remain asymptomatic or present with nonspecific symptoms such as long-term constipation, urinary dysfunction, lower back pain, or perineal hypoalgesia These lesions are difficult to diagnose without a strong suspicion due to the multiple range of nonspecific symptoms such as low lumbar pain, obstetric problems, and bowel and bladder difficulties. Special care should be taken in order to avoid erroneous diagnoses that may expose the patient to unnecessary surgical procedures. Because these lesions usually do not regress spontaneously, surgical treatment is mandatory for symptomatic or growing masses. The dural defect can be repaired with a variety of anterior transabdominal or posterior transsacral approaches. We present the case of a 82-year-old female patient with a giant anterior sacral meningocele that was initially confused with various entities such as inguinal hernia and an ovarian cyst. Once the anterior sacral meningocele was conffirmed through a MRI-scan she was successfully treated using a posterior transsacral approach. We present a brief review of the current literature and discuss the surgical treatment options.
Keywords: Anterior sacral meningocele, posterior approach, abdominal pain.
Nara Desert, Pakistan
Hey, welcome. Double click anywhere on the text to start writing. In addition to simple text you can also add text formatted in boldface, italic, and yes, math too: E = mc2! Add images by drag’n’drop or click on the “Insert Figure” button.