Main Data History
Show Index Toggle 0 comments
  •  Quick Edit
  • Reproducible research and open science, the way to the future.

    Beyond the Impact Factor.

    Having been quite versed in the art of research 8 years post PhD, I have been very fortunate to witness a renaissance in publishing in two ways. First, I remember quite well during my PhD training (over 10 years ago), the process of preparing a manuscript for the highest ranked journal, submit, reject, reformat and submit to the next journal, reject, get the story. Published manuscripts were usually in print form. The second method utilises the Internet and open access publishing.

    During that time, impact factor was the key metric in which a publishing house was measured (The agony and ecstasy of the impact factor). This evolved quickly into a measure of a researcher’s performance. There was active push to get manuscripts into journals with a high impact factor as it reflects positively on the authors involved. Over the course of my career, merit in using the impact factor to judge a study was questioned (Porta 2006). These days, it is almost a profanity to even consider the Impact Factor as a measure of a manuscript’s value (Randy Schekman’s Piece on Nature, Cell Science and Occam’s Typewriter on Impact Factor). Believe it or not, I have heard a swear jar was enforced at a grant review panel, where whomever mentioned “Impact Factor” would contribute to the swear jar.

    Okay so Impact Factor is not relevant, so what is a good metric? What is a good measure of a published manuscript for the likes of major granting bodies and prospective employers to judge your performance by? Speaking to many in the field and to many of my mentors, I would say the jury is still out; there really isn’t a good metric to go by, but nonetheless there should be one. In preparing this little piece I have come across Eugene Garfield, it seems he is the pioneer of citation metrics and has written many commentaries on the history of the Impact Factor and on how it has been misused. There have been discussions on other metrics so-called altmetrics where page hits or PDF downloads for instance, are being considered as measures of research impact.

    Back to my inaugral, first author manuscript, I aimed my submissions to the major journals including Science, Nature, Cell, Nature Structural Biology, Developmental Cell each time sacrificing some impact factor points (see Figure \ref{fig:tour}). This was at my PhD supervisor’s guidance as we both thought of the body of research to be paradigm shifting at the time. I quickly won the boomerang award for the manuscript that came back the most times within the group (not exactly the prize worth singing a song about!). In the end, we settled on, what was at the time, a very new publishing house, the Public Library of Science (PLoS). “What is this PLoS my supervisor says with scorn, there is no impact factor and what is this open access business? No business if you ask me; how will they survive if you give everyone open access to their published papers?”

    I was sold on how PLoS came about with regard to it’s push for online access and publishing at the time. It took a little convincing (meeting the editor of PLoS at the time) for my superviosr to come around who settled on the analogy with the stockmarket, “well the impact factor can only go up with this one”. To make a long story short, my paper was published in PLoS Genetics (Wong 2006). It was an interesting finding to me and my supervisor, but wasn’t really to the editors of Nature and Science.

    Today, there are many open access publishers, such as open access pioneers BioMed Central (BMC), who publish over 260 journals spanning all topics of biology and medicine, and paved the way for other publishers, such as PLoS. As a profession, we all have come to realise the benefits of open access. Even the more unscrupulous entities that prey on unsuspecting scientists, have embraced open access (Beall’s List of Predatory Open Access Journals.). Even the likes of Nature and Science have embraced open access. It’s great and makes research more accessible much quicker than conventional print-based avenues.

    Measures of impact have evolved in the era of open access; however, they are only slowly being adopted by the important funding bodies. In fact it is a requirement by NIH in USA and more recently NHMRC and ARC in Australia, that government funded research should be published as open access.

    \label{fig:tour} The journey of my first first author paper. It was the world tour that finally ended at PLoS Genetics.