\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage[round]{natbib}
\let\cite\citep




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}










\begin{document}

\title{A step by step guide to using Authorea, github, and Jupyter for
reproducible, dynamic, collaborative data science in public health}



\author[1]{Arindam Basu}%
\affil[1]{University of Canterbury}%


\vspace{-1em}



  
  \date{May 08, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup








\sloppy


\par\null

\subsection*{Preamble}

{\label{141528}}\par\null\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.28\columnwidth]{figures/move-fast-and-break-things-2x/move-fast-and-break-things-2x}
\caption{{Move fast and break things. This is an xkcd cartoon I found now. I can
upgrade or change it in one step with just an upload of another one.
{\label{382515}}%
}}
\end{center}
\end{figure}

\par\null

For those of us who are public health professionals, life is data and
evidence driven. At the same time, we are living in interesting times in
the sense we are drowned in a world of health data that are asking for
analyses and generation of insights. Evidence based data analysis,
particularly for public health is a necessity. Evidence \& causal
inference drive public policy and debates. The problem though is, more
often than not, the articles and knowledge-base that form the basis of
such evidence are rooted in an ancient opaque practice where the readers
are not allowed an access to the reproducibility of the process that
generated the information in the first place: at times the methods are
sparsely described, or the sources are behind paywalls, or not explicit
enough.~

\par\null

There is a growing demand of reproducible and replicable research
designs. What is replicable does not necessarily mean that is
reproducible. While the methods of data collection and analyses are made
explicit, one does not necessarily share the data and the methods used
to generate the insights, or the testing of hypotheses. Reproducible
research takes the additional steps of sharing the data and the steps
taken to conduct the analyses.~

\par\null

Yet this is a stepping stone into the larger more complex world where
research gets done not by one person but by a group. Research,
particularly in the context of public health, is team sport. The
theorist brings together the questions, the epidemiologist brings the
data and the methods, the analyst takes the data and crunches the
numbers. Together everyone writes. There is a need for transparency at
every end and we also owe it to ourselves to make it as easy as possible
for our readers to get a glimpse into that world and verify for
themselves what we have done.

\par\null

Authorea is a god-send. This is the only web based authoring system
where you can control a live web based writing system seamlessly online,
offline, store and forward, including opportunistic availability of
going online using a state of the art version control. This is
particularly valuable in computational public health and epidemiology
where we need to update as new data becomes available, sometimes in the
field, sometimes in the ivory towers, and indeed all sorts of
situations.~

\par\null

Here is a use case. Think about conducting a continuously updated meta
analysis that you can crowd source. Using a continuously updated crowd
sourced meta analysis where the codes are free and open to all will
avoid the need for re-analysis and revision of the meta analysis when
new studies are released. If my code can pick up the new studies soon as
they are made available in the data bases I searched, and fed into the
methods and streams I set up in the data analyses routine, I can have a
continuously updated data analysis system.~

\par\null

My intention here is to put a step by step guide on using Authorea and
Jupyter notebooks to do reproducible data science in public health that
brings in not just dynamically updating a document but doing so
seamlessly and brining in the power of a team to provide necessary
insights throughout the document~\cite{Peng_2011}. I will show in the
end that you can use this system to read data from WHO (and indeed other
repos) and run an analysis for yourself in the time it takes you to read
this paper.

\par\null

In brief,~ the steps are kind of as follows:

\par\null

Step 1: Start with Authorea and set up an article

Step 2: Link the article to github or a git repo, do a remote add origin
to the github to your local hard drive on a folder after installing git
locally

Step 3:~ Start a jupyter notebook in the root folder of the git repo and
use the python module (python 2 and python 3). You can use R as well;
I'd say if you want to use a notebook and want to use R, use an
Rnotebook in Rstudio; the effects will be same for the workflow that I
outline here. The reason I'd go the python way with an ipynb notebook is
that when you share the work and codes, most jupyter notebook servers
(such as colaboratory or mybinder.org) use Python 2 or Python 3. So if
your reader wants to verify your code and output, you provide him or her
the opportunity to do that right away. Else, use something like an
r-brain.io to work with julia or R codes to play with. I'd rather use an
Rstudio code set for R codes.

Step 4: Use git to push the ipynb into the Authorea repo; ditto for
Rnotebook markdown to push tables, figures, and text generated in the
Rnotebook.

\par\null

For this system to work, keep in mind these general principles:

\par\null

Write in markdown; for tables use csv files, and for figures use jpg
files

In Authorea, store the csv files in subfolders in the figures folder~

In Authorea, store the jpeg files in subfolders in the figures folder

Use the layout.md file to position the various components.

Learn the config.yml file for figures; there is magic in the config.yml
file worth mastering

Use bibtex for citation management

\par\null

Then, finish in Authorea itself. The html interface in Authorea is as
intuitive as it gets, and for plain tables that are not csv, use a
separate markdown chunk. The beauty of this system in Authorea is its
flexibility (this is as of 2018) where it allows you to blend markdown,
html, latex seamlessly. As a result, it does not matter what format you
use to write your text, you get a robust publishable text in the end
(you can channel this text to more static servers and more on these
things at the end of this article). So, here are the steps as to what
goes where for such a system to work.

\par\null

\subsection*{Step 1: Start with
authorea}

{\label{410256}}\par\null

So we want to build a dynamically updated publication system. By that I
mean that the document is \emph{so flexible that without repeated manual
upload of figures and tables, once you set a structure or a template,
you can practically just do your analyses and write wherever, whenever,
yet the ``production view'' in Authorea reflects that movable analyses
and get updated as you deal with new information and analyses}. This is
particularly important for open sourced reproducible research by a group
of diverse scholars where readers of the research become your partners.

\par\null

In order to dynamically update the paper, we need to establish that the
changing figures, tables and text happens only in some part of the paper
so that the structure does not get broken or be repetitive. In any
scholarly or information sharing context, as authors and also as
readers, we need at the least four elements:

\par\null

\begin{itemize}
\tightlist
\item
  Tables of information
\item
  Visualisation of elements
\item
  Attributable arguments or statements with sources (citations); no
  beliefs or opinions.
\item
  For reproducible research you need to provide source codes of your
  analyses (here we have used python but you can use R or Julia as
  well).
\end{itemize}

Mix Authorea with dynamically developed graphs and tables to work. You
can do all of these in the Authorea main interface by uploading from
your local hard drive but you may also want to work on the web on the
fly and here github is your friend. You will next need to do a couple of
things:

\par\null

\subsection*{Tables, figures, bibliography and
layout}

{\label{949138}}\par\null

For each of your figure and table you will need to create two files: a
config.yml file and a caption.html file. The caption file will contain
caption and the config.yml file will be different for figures and
tables. The table with csv data will contain the ipynb file and the
figure will contain the config.yml file and the caption.html file. The
caption.html file will only contain the title for the figure or the
table. The config.yml file is more involved and will contain the
following set of codes:

\par\null\par\null\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/survival-analysis/concept}
\caption{{This is a conceptual diagram of the dynamic process in which the paper
is written
{\label{442730}}%
}}
\end{center}
\end{figure}

\subsection*{The configuration file for a
figure}

{\label{986144}}

\href{http://colab.research.google.com/}{Colaboratory} for a hosted
python notebook is a great way to get things done in this scheme. My
only peeve with colaboratory is that I could not get it to work in
directly interfacing with~\href{https://github.com}{github}; otherwise
in terms of building real time collaborative data analysis in python,
hard to find a fault (particularly with Google power).
Try~\href{http://r-brain.io}{r-brain.io} if you want. Another option is
to use~\href{http://mybinder.org}{mybinder.org}. Any environment that
allows you some slack to use git along with analytical tools should be
fine.~

\par\null

While we are on this, some technical nitty gritty of setting up is in
order, and once you see through the process, you will realise there is
hardly any technicality involved other than just pushing a few files
here and there and it does the magic. To keep things simple, we will
focus on three different documents that can be updated and dynamically
change the configuration of this text depending on how the work is being
done and where the analyses are being conducted: the two config files
and the layout file.~

\par\null

Authorea is so good that if you can click on a linked
\href{https://jupyter.org}{jupyter} notebook, you can even verify the
analyses in Authorea's resulting jupyter notebook on their server in a
Python 2.7 version. The Python 3 version does not always work with all
modules, in particular the pandas module does not work. It also cannot
accommodate julia or R.~ A much better idea is to download the file and
process either at your end or use one of the web services.

\par\null

So let's start with the configuration file for the figure (it's called
config.yml) and it lives on the figure subfolder of the data folder.
This file is editable, best of all. If you use git and your plain text
editor, you can also get the job done that way. Anyway, it looks like as
follows:

\begin{verbatim}
---
version: 2
multicolumn: false
web-width: 500px
align: center
float: h!
grid:
  rows: 1
  columns: 1
  grid_elements:
  - path: figures/<subfolder_name>/<file_name.ext>
    type: static
    from:
    - 1
    - 1
    to:
    - 1
    - 1
    url: "/users/2191/articles/<article_id>/master/file/figures/<subfolder_name>/<filename.ext>"
    folder_url: "/users/2191/articles/<article_id>/master/folder/figures/<subfolder_name>"
\end{verbatim}

Most of what is written above in this code box is self-explanatory. For
beginners, if you change the path, the url, and optionally the folder
URL, you will be able to see the magic of dynamically updating the
figures in a document everytime you add a figure and want to change
things. For the tables, the configuration file will look different. Here
is a table and its associated configuration file.

\subsection*{The configuration file for a
table}

{\label{378128}}

Here is a table that we have loaded based on a larger data table\selectlanguage{english}
\begin{table}[h!]
\centering
\normalsize\begin{tabulary}{1.0\textwidth}{CCCCCC}
 & country & infectious & noncom & injuries & total \\
0 & Afghanistan & 71 & 230 & 49 & 26 \\
1 & Albania & 3 & 153 & 15 & 23 \\
2 & Algeria & 11 & 101 & 24 & 17 \\
3 & Andorra & 0 & 38 & 4 & 11 \\
4 & Angola & 188 & 157 & 62 & 26 \\
\end{tabulary}
\caption{{Data on deaths attributable to environment by country
{\label{675317}}%
}}
\end{table}The associated configuration file looks like this

\par\null

\begin{verbatim}
---
version: 2
multicolumn: false
web-width: 500px
align: center
float: h!
grid:
  rows: 1
  columns: 1
  grid_elements:
  - path: figures/<subfolder_name>/<name_of_the_file.csv>
    type: table
    from:
    - 1
    - 1
    to:
    - 1
    - 1
    url: "/users/2191/articles/<article_id>/master/file/figures/<subfolder_name>/<file_name.csv>"
    folder_url: "/users/2191/articles/<article_id>/master/folder/figures/<subfolder_name>"
\end{verbatim}

\textbf{The bibliography file}

The bibliography file is kept at bibliography/biblio.bib -- a bibtex
file. The bibtex file can be filled in from any source and if you work
with markdown, then use Latex citation style to cite the sources.
Authorea will adjust the citation and the references.~ Anyone can fill
in the bibtex file with entries and anyone who has access to this file
who can edit it can draw on the content of the bibtex file to add
citations and shape up the reference list as one wishes. There is
nothing to configure in the bibtex file beyond just adding citations. At
different point one can add and modify citations or a group of authors
can add/edit citations.

\par\null

\subsection*{The layout file}

{\label{702511}}\par\null

\begin{verbatim}
untitled.html
figures/survival_analysis
oa43tgiqatk.html
9vdgjid9kmk.md
kssnk71b08s.html
1h7kg7pk434.html
figures/who_env
8563fjuguvs.html
dplfdk8i5m8.md
1v7gd9bump.html
\end{verbatim}

As you can see, this file is nothing beyond a list of files and folders.
The linear order decides what you get to see, and you can edit the
layout.md file from the data folder to adjust the flow and logic of your
paper. You can see there are mix of html and markdown files. You do not
see individual images and tables but instead in the specific points
where they are located, you get to see their ``folder names'' are
organised. You can manipulate the configuration and the layout files to
change the appearance of the documents and add templating to the
Authorea document for your final data analysis results presentation. We
will discuss these now.

\par\null

\subsection*{Step 2: Link with github}

{\label{500687}}

Once you have set up the article (or say you use this as a template and
this one contains a jupyter file as well), you can just set up the
github links and you can work from github or git and jupyter to wwork
with the Authorea article. You can adjust the main sections once your
data analyses are complete but otherwise you can push your files from
within your computer or from the web using the github web interface. In
the rest of the article, I will guide you through it.

\par\null

\subsection*{Step 3: Use a jupyter notebook to conduct your data analysis
and link it
here}

{\label{369754}}\par\null

Start a jupyter notebook in the parent directory and start working. Here
are the principles:

\par\null

\begin{enumerate}
\tightlist
\item
  You can write in markdown and you can save the contents in markdown,
  upload and put it in the layout.md file
\item
  Save your outputs as jpg files or csv files and push them to the
  figures folder
\item
  Later edit the config.yml and layout md files
\item
  Leave the ipynb text in the form of markdown
\item
  git your changes
\item
  Do the final editing in Authorea.
\end{enumerate}

\par\null\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/death-hist/death-hist}
\caption{{Histogram on world wide deaths due to communicable and non-communicable diseases in 2012. see a bimodal distribution.%
}}
\end{center}
\end{figure}

As the above diagram suggests there is a bimodal distribution in the
age-standardised death rates from all causes in the world in 2012. We
can examine other causes of death and test them against each other.

\par\null

\subsection*{Case for a dynamic updating of this document with new
insights and new diagrams as they become
available}

{\label{745306}}\par\null

The power of this system is apparent as we dynamically change the
diagrams \& the analyses in different sections of the paper, and layout
the different parts of the document on the fly not directly visiting the
Authorea page but using the github and the other tools such as the
jupyter notebook that we can connect to the github and sharing our
analyses. More on these variations shortly. At any point, we can work
with the jupyter notebooks to change the analyses and change the
diagrams and the figures will dynamically change as well without any
further uploading and work on our part. As we continue to refine the
analyses and push the changes through github, the paper will continue to
evolve. The markdown with its own set of citations also get cooped and
dynamically updated as we continue to work. It take a one time set up to
configure the yaml files but beyond that the paper gets updated
dynamically. This helps in building an open yet collaborative paper or
piece of work. If we want to then share the work widely, we can either
use the jupyter notebook as a gist and share it, with a note here, or we
share the notebook directly here.

\par\null

As long as I settle on a set of diagrams and tables and continue to
dynamically revise and update the document, the document will grow
itself. All I have to do is to feed it with new analyses and
figures/tables to populate the different spaces and folders, add them to
the layout.md and continue to work with them. If I desire, I can change
the grid arrangement of the figure and the tables, as different tables
will take their place and continue to update each other. Once the system
is set up, its a matter of manually adding and pushing or updating a
document through github.

\par\null

The jupyter notebook and the git becomes the engine of dynamically
updating this document. We can of course change things as and when we
want here, but we retain the fine control of how the document is going
to look like. Watch this document as this will be updated dynamically.

\selectlanguage{english}
\FloatBarrier
\bibliographystyle{plainnat}
\bibliography{bibliography/converted_to_latex.bib%
}

\end{document}

