Alberto Pepe edited sectionDiscussion_The_ongoing_move.tex  about 11 years ago

Commit id: 6e5dedb3bdf1fc90e3d2726dd522faa919396ef7

deletions | additions      

       

\section{Discussion}  The ongoing move to online scholarly communication has introduced new possibilities for measuring scholarly impact. At the same time, it has become more difficult to determine which communities drive a particular form of online impact. For example, usage data, measured as volume of downloads, is generally assumed to reflect the interests and preferences of the general public, but what if the particular online service for which usage data was recorded is dedicated to serving scientists only? What if an online service for scientists increasingly becomes a tool for the general public to learn about scientific findings? The online user communities associated with particular services may in fact overlap to various degrees as the scholarly community progressively moves online and the online public moves toward scholarly information services. Naturally, scholarly impact metrics should acknowledge this new reality.  The research presented in this paper is based on data from two services which are arguably associated with and intended for two different audiences. ArXiv.org is focused on offering scientists an online platform to publish pre-prints. Twitter is designed to serve as a micro-blogging services for the public. In this study we did, however, not attempt to conceptualize arXiv downloads solely as scientific impact, and Twitter mentions solely as public chatter. Rather, we measured the correlation and temporal differences between these forms of responses, working under the assumption that these services naturally have overlapping and interacting user communities.  Our results, though preliminary, are highly suggestive of a strong tie between social media interest, article downloads, and even early citations. We find that Twitter mentions and arXiv downloads of scholarly articles follow two distinct temporal patterns of activity, with Twitter mentions having shorter delays and narrower time spans than arXiv downloads. We also find that volume of Twitter mention is statistically correlated with that of both downloads and ``early'' citations, i.e., citations in the scholarly record occurring less than 7 months after the publication of a preprint.  We can think of two possible explanations for these results. First, the manner in which Twitter mentions, arXiv downloads and article citations are correlated could indicate a causal relation. Scholars are increasingly exposed to social media such as Twitter, and therefore their scholarly download and citation behavior is unavoidably affected. A paper submitted to arXiv that happens to receive high levels of mentions in social media will, as a result, receive greater exposure among both the general public and scholars. As a consequence, it will receive greater levels of scholarly interest, and higher volumes of downloads and subsequent citations. Our results indeed indicate that early Twitter mentions of a paper seem to lead to more rapid and more intense download levels and subsequently higher citation levels. Second, an equally plausible, alternative explanation for our results lies in the intrinsic quality or popular appeal of different manuscripts. A manuscript of greater quality or appeal, either among the public or the scholarly community, will by virtue of this characteristic enjoy higher levels of mentions on Twitter, higher levels of downloads on arXiv, and higher levels of later citations. As a result these indicators will seem to be correlated, and even causative of each other.  We therefore acknowledge that these observations can be the result of a number of distinct or overlapping factors which our methodology confounds and fails to distinguish. Consequently, we caution against drawing the unwarranted conclusion that these results indicate that the scholarly impact of an article can be fully determined by its social media coverage, nor that one could increase the citation rate of an article by merely tweeting about it. The fact that some correlation --- no matter how small --- was observed between social media coverage, usage, and early citations may nevertheless indicate that the scientific communication process is increasingly affected by the growing societal importance of social media. In future research we will therefore continue to focus on unraveling the potential mechanisms that tie these various factors together. These efforts might shed light on whether and how social media is becoming a component of academic and scholarly life.