Approved by CiSE AEiCs on 3 March 2020. Computing in Science and Engineering (CiSE) is a technical magazine of the IEEE Computer Society, in publication since 1999. CiSE publishes peer-reviewed research articles, and also runs departments spanning news and analyses, topical reviews, tutorials, case studies, and more. CiSE runs special issues on themes chosen by the editorial board, which are handled by Guest Editors. This document explains the editorial workflow of special issues, and the role and responsibilities of Guest Editors.Selection of Special Issue ThemesSpecial Issues are proposed by members of the editorial board, or other community members, for the consideration of the Editor-in-Chief (EiC), and Associate Editors-in-Chief (AEiCs). They are approved by the EiC, with concurrence of a majority of the AEICs. (CiSE currently has 5 AEiCs, so three of them should be supportive of the theme). A special issue needs to have at least two Guest Editors, and should be approved at least one year prior to the expected publication.Invited or Open SubmissionsMost special issues make an open call for submissions of papers relevant to the issue theme. Guest editors may propose an invited special issue, for which articles will be submitted via a closed CFP, and this may be approved by the EiC with support from AEiCs.  To avoid perceived conflict of interest, guest editors should refrain from inviting authors from their own institutions, or authors who were their graduate students or postdoctoral trainees. All invited articles undergo the same peer-review process as unsolicited ones.Invited Special Issues are generally limited to one per year.Special Issue ProposalsProposals for future CiSE special issues should be sent to the EiC directly, and should include the following materials:Description—An overview of topic and scope, and reasons why it is timely to dedicate a magazine issue to this theme.Guest editors—A short bio for each guest editor, addressing how they are well positioned to lead the community to submit and review manuscripts.Draft CFP—A preliminary drafting of the call for papers, subject to improvements after approval.Dissemination plans—How the guest editors plan to reach authors and ensure success of the special issue.Proposed issue date—Considering any related timelines (e.g., when articles stem from workshops, conferences, or world events), anticipated date of CFP and publication. Role and Responsibilities of Guest EditorsGuest editors assume the responsibility of an Associate Editor in charge of the manuscripts submitted to the special issue. They manage the peer review of submitted manuscripts, ensure publication quality, and adhere to the IEEE and Computer Society’s policies. \cite{manual2020}The IEEE Computer Society has published a Guest Editor Information webpage. CiSE complies with these guidelines, but is sometimes stricter.Follow these guidelines for CiSE Special Issues:Guest editors write a special-issue introduction, which should be 1000 to 2200 words in length (5 pages or less), discussing the state of the art and future directions in the theme topic. It should highlight how each article in the special issue contributes to the field, and it may introduce key concepts and terminology, to facilitate a smooth reading of the articles. The guest editors’ introduction should state whether the issue had an invited or open CFP. It should be sent directly to the EiC.Other than the introduction to the special issue, a guest editor may not be an author or co-author of another article in the special issue they are editing. If some special circumstances warrant it (e.g., a needed survey of the field), they should request special dispensation from the EiC, who will seek concurrence from two AEiCs. Guest editors should actively solicit submissions via outreach through their networks, and make every effort to ensure a healthy number of manuscripts are assessed via peer review to be part of the special issue. Guest editors invite reviewers who are experts in the topic of the special issue ahead of the submission deadline. The reviewers should confirm their willingness to serve, and be ready to review their assigned articles within three weeks. Computer Society policies require each manuscript to receive three independent reviews. Guest editors oversee the timeliness and quality of peer-review reports. They should not take the role of reviewers, themselves.Guest editors should not handle the peer review of manuscripts where a real or apparent conflict of interest (COI, see below) is present with any authors. These include: affiliation, previous co-authorship, having formally mentored the author (as PhD or postdoctoral supervisor), or having collaborated in a funded grant. In the case of a submission by an author where a potential COI may exist (real or apparent) with one or more of the guest editors, the guest editors should seek guidance from the EiC on how the COI might be mitigated. The submission may be allowed to go forward, in which case the guest editor will delegate the handling of peer review of this submission to another member of the editorial board, with guidance from the EiC. Guest Editors are also responsible for ensuring that articles conform to Computer Society policies, and the magazine style. Common issues that arise in this regard are the word limits, and limits on the number of references. According to CiSE guidelines for authors: Articles should be between 2,400 and 6,250 words, including all main body, abstract, keyword, bibliography, and biography text. Each table and figure counts for 250 words. They  should have no more than 12 references. Articles should be written for an interdisciplinary audience, be concise, and tend to be more readable than scholarly in style. Special issues typically consist of about 6 articles. Informed by the peer reviews, guest editors select the articles to appear in the issue, which may result in good-quality articles having to be rejected. In exceptional situations of a theme being very popular, guest editors may request approval from the EiC to run a Part 2 of the theme in a later issue. Transfer of special-issue submissions to the regular article queue is discouraged.Definition of Conflict of Interest“A conflict of interest is defined as any situation, transaction, or relationship in which a member’s, volunteer’s, or staff person’s decisions, actions, or votes could materially affect that individual’s professional, personal, financial, or business concerns.” [p.22 IEEE PSPB Operations Manual] An Associate Editor is regarded as having a COI with a manuscript if any author is employed at the same institution as the editor. It is also a COI if the editor has co-authored a paper or has closely collaborated in a research project with any of the manuscript's authors  in the previous five years.Editorial WorkflowWhen a special issue is approved, the editorial workflow is as follows:

SummaryThis is the editor’s review of manuscript CiSESI-2018-02-0016: “CoRR — The Cloud of Reproducible Records,” submitted to Computing in Science and Engineering Reproducible Research (RR) Track. We requested via email to the corresponding author that a preprint be made available on arXiv or an equivalent service, but did not receive a reply. It is policy of the RR Track to require a preprint; see our launch editorial \citep*{Barba_2017}.

AbstractA recent editorial, “Giving Software its Due,”1 described challenges related to the development of research software and highlighted, in particular, the challenge of software publication and citation. Here, we call attention to a system that we have developed enabling community-driven software review, publication, and citation. The Journal of Open Source Software (JOSS) is an open-source project and an open-access journal providing a lightweight publishing process for research software. Focused on and based in open platforms and on a community of contributors, JOSS satisfies a pressing need, evident in having already published more than 500 articles in approximately three years of existence.

Among the top challenges of reproducible computational science are: (1) creation, curation, usage and publication of research software; (2) acceptance, adoption and standardization of open-science practices; (3) misalignment with academic incentive structures and institutional processes for career progression. I will address here mainly the first two, proposing a praxis of reproducible computational science.

SummaryThis is the editor's review of manuscript CiSESI-2018-02-0013, submitted to Computing in Science & Engineering, Reproducible Research Track: "automan: a Python-based, automation framework for numerical computing"  \citep{p2018b}.We received two peer-review reports for this manuscript, and the reviewers were: Thomas Pasquier and Olivier Mesnard. Both reviewers recommended that the manuscript be accepted with a minor revision, and I agree. Reviewer 1 comments on software-engineering aspects: he asks if automan could be combined with continuous integration tool chains. He also asks to state plainly the license of automan—from the LICENSE.txt file in the GitHub repository, it looks like it’s under BSD3, but you could state this in the README and in the paper. Reviewer 2 acted as Reproducibility Reviewer. He inspected the code repository of automan, installed it, and tested it with a fabricated problem. He also provides several minor suggestions in his open report \citep{o2018}.Editor's reviewOne issue that the author needs to correct is the terminology. The CiSE RR Track adopted a set terminology that we request authors to consistently apply. See the Call for Papers \citep{g2018}.Citing from there:Reproducible research is defined as research in which authors provide all the necessary data and the computer codes to run the analysis again, re-creating the results. Replication, on the other hand, is defined as arriving at the same scientific findings as another study, collecting new data (possibly with different methods), and completing new analyses.Throughout the manuscript, the author is using “repeatability” and “repeating” in place of “reproducibility” and “reproducing.” Conflicting terminology is a source of confusion in the field, and the CiSE RR editors want consistent usage in the magazine.I also recommend reading: “Re-run, Repeat, Reproduce, Reuse, Replicate…” \citep{Benureau_2018}AbstractCiSE articles don’t normally carry a standard abstract. Instead, a short “blurb” appears under the title, usually about three sentences long, to pull focus. Please edit the abstract down, in line with this style. Here's a starting point for your editing (or simply re-use this, if you like it):Automating computational workflows leads to more reproducible research. The Python tool automan assembles a suite of long-running computations, deploys them to your computational resources, and produces final plots from output data, all with a single command.Introduction and DiscussionIn the Introduction, you say “What is perhaps more important is that we discovered that [reproducibility] can be very profitable to the researcher.” The paper then goes into the design details and explains the usage of automan in section 3, and you wrap up with a summary in the first paragraph of section 4 (Discussion). Towards the end, the Discussion explains how automan helped you manage a series of about 75 computational experiments with SPH schemes. But I miss here a stronger, more persuasive statement about how automan expedited and ensured quality of the research (it doesn’t help that the paragraph in question is plagued with passive phrases). I would like it if you re-wrote this passage to reveal more about your own experience, and how you benefitted from the tool you created. Convince your readers here that the initial overhead of setting up automan for their simulation-based workflow will pay off!Editorial / style requestsThe manuscript is well written, overall, but a few small improvements would polish it further. In CiSE, we aim for a more readable style than typical scholarly literature. In line with that, I want to encourage you to especially look at whether any passive constructions are strictly necessary, and look to remove syntactic-expletive phrases like “there are,” if you can, as well as superfluous adverbs and superlatives (e.g., clearly, very).Examples:First sentence! “It is well known that…” You can easily remove that, and combine the first two sentences: “Reproducibility is a cornerstone in all science fields, and is vitally important in computational science.”p. 1, l. 39: “Unfortunately, there are not many immediate or direct incentives for a researcher to invest time…” >> “Unfortunately, researchers have few immediate or direct incentives to invest time…”p. 1, l. 41: “there can be significant challenges involved in [reproducing] and replicating numerical experiments” >> “reproducing and replicating numerical experiments involves serious challenges” (or another adjective, but don’t use “significant” unless you mean statistically significant, to avoid misunderstanding).p. 1, l. 45: “we believe this is a very important step in facilitating reproducible research” >> This step is important for facilitating reproducible research” (the terminology here is consistent with CiSE usage).p. 1, l. 46: “The approach used in our automation framework is fairly general” >> “Our automation framework typifies a general approach”p. 1, .. 49: “[reproducibility] can be very profitable to the researcher” (remove “very”)p. 1, l. 51: “It is our hope that” >> “We hope that"— The fact that we can pick several examples from just the first two paragraphs shows that you could polish the manuscript by combing through the text looking for just these patterns. I expect about a 5–10% reduction in word count from this exercise!p. 10, l. 40: Bad sentence: “It is important to note again that the commands that are executed such that one may configure the directory in which they will generate output using a command line argument.” Typos:Title: you don’t need the comma.p. 1, l. 38: “some of [the] most important articles” (missing “the”)p. 3, l. 21: capitalize DockerSoftware AppendixAs indicated in the Call for Papers: “articles will include an appendix reporting on the software engineering and data management practices followed by the authors.” Please add a description to that effect, in the form of a brief Appendix.Extended AcknowledgementIn the short statement on "Peer review in the CiSE RR Track" \citep{Barba}, we describe our ambitions for opening up the peer-review process in this track. Your reviewers opted in to open identity and open reports. We ask that you recognize reviewers by name in an extended acknowledgements section, briefly stating their contribution to your paper. You can cite the open reports with the URL or DOI, in each case.ReferencesPlease provide DOIs wherever possible for all references.For Ref. [1], please add DOI of the PDF version of this blog post, published on Figshare (10.6084/m9.figshare.4879928.v1).For Ref. [8], please update the citation, as needed.

In our editorial launching the new Reproducible Research Track in CiSE \citep*{Barba_2017} , we promised to explore innovations to the peer-review process. Because we require articles submitted to this track to adhere to practices that safeguard reproducibility, we must review for these aspects deliberately. For each submission, a reproducibility reviewer will be charged with checking availability, quality and usability of digital artifacts (data, code, figures). This reviewer (sometimes one of the track editors) will be known to the authors, and may interact with the authors during the review—for example, opening issues on a code repository. For this service, we ask that the authors recognize the reviewer in the article's acknowledgements section.

Completing a full replication study of our previously published findings on bluff-body aerodynamics was harder than we thought. Despite the fact that we have good reproducible-research practices, sharing our code and data openly. Here's what we learned from three years, four CFD codes and hundreds of runs.

Hey, welcome. Double click anywhere on the text to start writing. In addition to simple text you can also add text formatted in BOLDFACE, _italic_, and yes, math too: E = mc²! Add images by drag’n’drop or click on the “Insert Figure” button. Trying _italics_.