If a resource was not found via the portal, authors were given the option to submit the resource to obtain an identifier. For antibodies and software/databases, which are found in databases maintained within the NIF, submission was handled through the Resource Identification Portal. For model organisms, the author was referred to the authoritative model organism database for their organism (rat, mouse, zebrafish, worm, fly). All new submissions were curated by their respective databases and the data was pulled back into the RII portal weekly so that authors could see their newly registered antibodies or software tools in about a week.
To address the aims of the pilot project, we tracked the use of RRIDs in published papers and journals. We performed an in depth analysis of the first 100 papers found through Google Scholar that reported RRID’s. For each paper, we examined the methods section to determine the correct usage (i.e. if the RRID pointed to the correct resource), the synatatic correctness (i.e. if the author reported the RRID using the correct syntax), and the identifiability of the three resources. The total number of research resources reported in the first 100 papers reporting RRIDs was determined by manual inspection of each paper by two independent curators. A Google Scholar alert was used to track all new papers that contained the term RRID. Each paper was downloaded and examined for the snippets of text surrounding research resources (in the methods or data use sections).
Curation workflow to determine correct usage of reported RRIDs:
To determine if the RRIDs were reported correctly for the three resource types, the following criteria was applied.
A resource was considered correct if resource reported an RRID and that RRID pointed to the correct resource in the RII portal.
A resource was considered incorrect if the reported an RRID and that RRID pointed to the incorrect resource in the RII portal.
The percentage of resources requiring a minor correction for the RRID were calculated for each resource. A resource was marked as needing a minor correction if the resource contained an RRID, but the RRID did not use the correct syntax, i.e. if the RRID was not formatted correctly or had missing characters.
Curation workflow for identifiability of the three resource types:
To determine if the three resource types were identifiable in the journal articles that reported RRIDs (post-pilot), and in articles from the same journals before the pilot started. To select the pre-pilot articles, articles were selected by performing a PubMed search filtered for each journal and using the first five publications returned that contained the relevant resource types from approximately January–March 2013. The following criteria were applied: Resources (primary antibodies, organisms, and non-commercial tools) were considered identifiable if they contained a correct RRID or by using the same specific resource identification criteria as described in Vasilevsky et al., 2013 (PMID:24032093). For tools (non-commercial software and databases) (which were not previously analyzed), these resources were considered identifiable if they contained the correct RRID or reported the manufacturer and version number for that tool. Note, we distinguished commercially produced for-profit software from public or individually produced software (non-commercial).
Statistical analysis for identifiability of the three resources:
Since the data was binomial in that each resource was either identifiable or not, we used a binomial confidence interval strategy for calculating upper and lower 95% confidence intervals (CI) (http://www.danielsoper.com/statcalc3/calc.aspx?id=85, RRID:SCR_013827). Error bars for the corresponding 95% CI are displayed on the graphs. Statistical significance was determined by calculating the z-score.