paper.Rmd

---
title: "Writing a reproducible paper with R Markdown and Pagedown^[Corresponding adress: mail@paulcbauer.de, camille.landesvatter@uni-mannheim.de]"
shorttitle: "Writing a reproducible paper with R Markdown and Pagedown"
author:
  - name: Paul C. Bauer
    affiliation: Mannheim Centre for European Social Research
    email: <mail@paulcbauer.de>
  - name: Camille Landesvatter
    affiliation: Mannheim Centre for European Social Research
    email: <camille.landesvatter@uni-mannheim.de>
date: 'First version: 20 June, 2021<br>This version: `r format(Sys.time(), "%d %B, %Y")`'
download-link: 'Download: https://osf.io/k8jhx'
abstract: 'The present paper provides a template to write a reproducible scientific paper with R Markdown and Pagedown.^[Based on an earlier R Markdown template that uses Latex and can be downloaded under https://osf.io/q395s [see @Bauer2018-dl].] Below we outline some of the "tricks"/code (e.g., referencing tables, sections etc.) we had to figure out to produce this document. The underlying files which produce this document can be [downloaded here](https://github.com/paulcbauer/Writing_a_reproducable_paper_in_pagedown) (click on Code -> Download ZIP). Importantly, we also provide different CSS and HTML files that can be used to achieve a pdf output with the look of a "working paper". We are convinced that in the future there will be many improvements and developments with regards to RStudio, R markdown and Pagedown. We intend to update this file when we discover more convenient code. You can follow any updates on the [github repository](https://github.com/paulcbauer/Writing_a_reproducable_paper_in_pagedown/).'
# at least one keyword must be supplied
keywords: [open science, transparency, replication, reproducible research, reproducibility, R, markdown, pagedown]
links-to-footnotes: false
paged-footnotes: true
bibliography: references.bib
output: 
  pagedown::jss_paged:
    template: wp_paged.html
    self_contained: true
    css: ['wp.css', 'wp-fonts.css', 'wp-page.css']
    csl: american-sociological-association.csl
# uncomment the next line if you only want to knit to HTML and not PDF
knit: pagedown::chrome_print
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(cache = FALSE)
# Use chache = TRUE if you want to speed up compilation
# A function to allow for showing some of the inline code
rinline <- function(code){
  html <- '<code  class="r">``` `r CODE` ```</code>'
  sub("CODE", code, html)
}
```


\newpage
# Why reproducible research (in R)?
Some arguments...

* **Access**: Research is normally funded by taxpayers (researchers are also taxpayers). Hence, it should be freely accessible to everyone without any barriers, e.g., without requiring commercial software. Importantly, researchers from developing countries are even more dependent on free access to knowledge [@Kirsop2005-ro].

* **Reproducability**: Even if you have written a study and analyzed the data yourself you will forget what you did after a few months. A fully reproducible setup will help you to trace back your own steps. Obviously, the same is true for other researchers who may want to understand your work and built on it. It may sound like a joke but why not aim for a document that can be used to reproduce your findings in 500 years. 

* **Errors**: Manual steps in data analysis (e.g., manually copy/pasting values into a table etc.) may introduce errors. R Markdown allows you to **automatize** such steps and/or avoid them.

* **Revisions**: Revising a paper takes much less time if you have all the code you need in one place, i.e., one `.rmd` file. For instance, if you decide to exclude a subset of your data you simply need to insert one line of your code at the beginning and everything is rebuilt/re-estimated automatically.



# Why Pagedown?

Formatting text as PDF is probably one of the most widespread standards in the scientific community, especially when it comes to submitting papers and similar documents. The traditional way to well-formatted and good-looking PDFs is often through LaTeX or Word. However, if you have spent hours and hours debugging latex code (or getting it to run) you may be on the lookout for something new.

The fairly new `pagedown` R package takes a completely new approach. While the main purpose of pagedown is to create high-quality PDFs, the idea is to take advantage of modern web technologies (HTML/JSS/Javascript) with which one can design web pages and eventually print those to PDF.

While web pages are usually single-page scrollable documents, pagedown uses the JavaScript library `Paged.js` which allows documents to be paginated with elements like headers, footers and everything a readable scientific paper will need. Additionally, pagedown documents are based on R Markdown. In our view, Pagedown and the underlying technology may replace Latex in the long run. In the near future it should also be possible to produce a PDF with static graph and an equivalent html with interactive graps (see dicsussion [here](https://github.com/rstudio/pagedown/pull/87)).


# Prerequesites

We assume that you are using R on a day-to-day basis and you may have even started to work in R Markdown. If you don't know what R Markdown is there are many great resources that you should use (e.g. watch this [short video](https://vimeo.com/178485416)). An older template [see @Bauer2018-dl; https://osf.io/q395s] on which this newer template is based, may provide a quick entry point to writing a reproducible with R Markdown and Latex.

Based on R Markdown, Pagedown allows you to create custom and well-formatted (paged) HTML Documents. For a comprehensive overview watch [this video](https://www.rstudio.com/resources/rstudioconf-2019/pagedown-creating-beautiful-pdfs-with-r-markdown-and-css/) which is a record of a talk introducing `pagedown` given by Yihui Xie (who in addition to Romain Lesur developed the `pagedown` package). If you are not in a video watching mood find the slides [here](https://slides.yihui.org/2019-rstudio-conf-pagedown.html#1).

Then...

* ...install [R](https://www.r-project.org/) and [Rstudio](https://www.rstudio.com/) (most recent versions) [@R2017; @Rstudio2015].
* ...install the "pagedown"-package from github using the code below [@Xie2021-bi; @Xie2021-ls].
```{r eval=FALSE, include=TRUE}
remotes::install_github('rstudio/pagedown')
```
* ...also install the packages below using the code below [@bookdown1; @bookdown2; @knitr1; @knitr2; @knitr3; @kableextra].
```{r eval=FALSE, include=TRUE}
install.packages(c("rmarkdown", "knitr", "kableExtra",
                   "stargazer", "modelsummary", "knitr", "gt"))
```
* ...download the 4 input files we created --- `paper.rmd`, `references.bib`, `data.csv` and `american-sociological-association.csl` --- from [Github](https://github.com/paulcbauer/Writing_a_reproducable_paper_in_pagedown) (click on Code -> Download ZIP). Ignore the other files.
* ...also download the 4 styling files we created: `wp_paged.html`, `wp.css`, `wp-fonts.css` and `wp-pages.css`.
* ...store all 8 files from above together in one folder (and use this folder as your working directory later on)
* ...learn R and read about the other underlying components namely [Markdown](https://en.wikipedia.org/wiki/Markdown), [R Markdown](https://rmarkdown.rstudio.com/lesson-1.html) and [Pagedown](https://pagedown.rbind.io/).
* ...pagedown comes with several Rmd-templates (presentations, poster, thesis, etc.) and via this review we provide another template for a working paper style. If however you want to modify single aspects or create your own template, you will need to at least gain some basic skills in [CSS](https://www.w3schools.com/css/) and [HTML](https://www.w3schools.com/html/).





# Basics: Input files, output files and the YAML header

All the files you need to produce the present PDF file are:

1. the input files:

* `paper.rmd` (the underlying R Markdown file).
* `references.bib` (the bibliography).
    + I use paperpile to manage my references and export the `.bib` file into the folder that contains my `.rmd` file.
* `data.csv` (some raw data).
* `american-sociological-association.csl` (defines the style of your bibliography).^[You can download various citation style files from this webpage: https://github.com/citation-style-language/styles.] 

2. the "styling" files:

Basically, these are files you will need to specify in the YAML of your rmd-file, so that R and ultimately pagedown recognizes the certain style you want to achieve for your document. With using our templates, you will create a document that has the "look" of a working paper (we based our files on the [jss_paged pagedown format](https://github.com/rstudio/pagedown#journal-of-statistical-software-article-pagedownjss_paged)). 

* `wp_paged.html` (based on `jss_paged.html`)
* `wp.css`
* `wp-fonts.css`
* `wp-pages.css`

Take `paper.rmd` (the underlying R Markdown file of this pdf) and have a look at the YAML (line \#18 - \#22) to see how to specifiy these files.
Basically, what happens here is that within the [jss_paged function](https://rdrr.io/cran/pagedown/man/jss_paged.html) we additionally specify that we want to use custom CSS and custom HTML.

[Download these files](https://github.com/paulcbauer/Writing_a_reproducable_paper_in_pagedown) (click on Code -> Download ZIP) and save them into a folder. Close R/Rstudio and directly open `paper_pagedown.rmd` with RStudio. Doing so assures that the working directory is set to the folder that contains `paper.rmd` and the other files.^[You can always check your working directory in R with `getwd()`.]

Once you run/compile the `paper.rmd` file in Rstudio it creates a output file called `paper_pagedown.html`.

By using pagedown's `chrome_print` function in the YAML (line \#25) your html based web page will be printed to `paper_pagedown.pdf` (the one you are reading right now).

Both outputs will be saved in your working directory.


# Referencing within your document

To see how referencing works simply see the different examples for figures, tables and sections below. For instance in Section \@ref(sec:tables) you can find different ways of referencing tables. The code of the underlying `paper.rmd` will show you how I referenced Section \@ref(sec:tables) right here namely with '`Section \@ref(sec:tables)`'.


# Software versioning

Software changes and gets updated, especially with an active developer community like that of R. Luckily you can always access [old versions of R](https://cran.r-project.org/bin/windows/base/old/) and old version of R packages in [the archive](https://cran.r-project.org/src/contrib/Archive/). In the archive you need to choose a particular package, e.g dplyr and search for the right version, e.g., `dplyr_0.2.tar.gz`. Then insert the path in the following function: `install.packages("https://....../dplyr_0.2.tar.gz", repos=NULL, type="source")`. Ideally, however, results will be simply reproducible in the most current R and package versions.

I would recommend to use the command below and simply add it to the appendix as I did here in Appendix \@ref(sec:rsessioninfo). This will make sure you always inform the reader about the package versions your relied on in your paper. For more advanced tools see [packrat](https://rstudio.github.io/packrat/).

```{r fig-versioning, echo=TRUE, eval=FALSE}
cat(paste("#", capture.output(sessionInfo()), "\n", collapse ="")) 
  # or use message() instead of cat()
```






# Data
## Import

Generally, code is evaluated by inserting regular `R Markdown` blocks. 

```{r}
x <- 1:10
x
```

Below we import an exemplary dataset (you can find `data.csv` in the folder with the other files).

```{r, echo=T, results="raw"}
data <- read.csv("data.csv")
head(data)
```

## Putting your entire data into the .rmd file
Applying the function `dput()` to an object gives you the code needed to reproduce that object. So you could paste that code into your `.rmd` file if you don't want to have extra data files. This makes sense were data files are small.

```{r}
dput(data[1:5,]) # here we only take a subset
```

You can then insert the dput output in your `.rmd` as below.

```{r eval=FALSE, include=TRUE}
data <- structure(list(Fertility = c(80.2, 83.1, 92.5, 85.8, 76.9), Agriculture = c(17, 
45.1, 39.7, 36.5, 43.5), Examination = c(15L, 6L, 5L, 12L, 17L
), Education = c(12L, 9L, 5L, 7L, 15L), Catholic = c(9.96, 84.84, 
93.4, 33.77, 5.16), Infant.Mortality = c(22.2, 22.2, 20.2, 20.3, 
20.6)), class = "data.frame", row.names = c(NA, -5L))
```













# Tables {#sec:tables}

Producing good tables and referencing these tables within a R Markdown document has been a hassle but got much better. Examples that you may use are shown below.



## Tables with kable() and kable_styling()

A great function is `kable()` (`knitr` package) in combination with `kableExtra`. Table \@ref(tab:table-2) provides an example. To reference the table produced by the chunk you need to add ´tab:´ to the chunk name, i.e., ´tab:table-2´ and would reference it by adding "`Table \@ref(tab:table-2)`" in your text.

```{r table-2, echo=TRUE, message=FALSE, warning=FALSE, results="asis"}
library(knitr)
library(kableExtra)

kable(swiss[1:10,], row.names = TRUE, 
      caption = 'Table with kable() and kablestyling()', 
      format = "html", booktabs = T) %>%
        kable_styling(full_width = T, 
                      latex_options = c("striped", 
                                        "scale_down",
                                        "HOLD_position"),
                      font_size = 10)


```





## Tables with modelsummary

The `modelsummary` package provides a variety of tables and plots to summarize statistical models and data in R. Modellsummary plots and tables are highly customizable and they can be saved to almost all formats, e.g., HTML, PDF and Markdown. This makes ist especially easy to embed them in dynamic documents. Please look at the package's extensive [documentation](https://vincentarelbundock.github.io/modelsummary/index.html) where they also show examples for almost any plot or table you might be looking for. In this template we demonstrate an example for modelsummary's `datasummary` function. `Datasummary` creates frequency tables, crosstab tables, correlation tables, balance tables and many **more**.

### Summarize numeric variables

Table \@ref(tab:table-3) shows a summary table for numeric variables. 

```{r table-3, echo=TRUE, message=FALSE, warning=FALSE}
library(modelsummary)
datasummary_skim(swiss, 
                 type="numeric", 
                 histogram=T, 
                 title = "Summary: Numeric variables")
```


### Summarize categorical variables

Table \@ref(tab:table-4) shows a summary table for categorical variables. 

```{r table-4, echo=TRUE, message=FALSE, warning=FALSE}
# Create categorical variables
swiss$Education_cat <- cut(swiss$Education, 
                   breaks=c(-Inf, 6, 12, Inf), 
                   labels=c("low","middle","high"))
swiss$Infant.Mortality_cat <- cut(swiss$Infant.Mortality, 
                   breaks=c(-Inf, 18.15, 21.70, Inf), 
                   labels=c("low","middle","high"))

library(flextable)
tab_cat <- datasummary_skim(swiss, 
                            type="categorical", 
                            title = "Summary: Categorical variables",
                            output = 'flextable')

# additionally we want to change the font, fontsize and spacing
library("gdtools")

library(dplyr)
tab_cat <- tab_cat %>%
  font(fontname="Times New Roman", part="header") %>%
  font(fontname="Times New Roman", j=1:4) %>% 
  fontsize(size=12, part="header") %>%
  fontsize(size=10, j=1:4) %>% 
  line_spacing(space = 0.3, part = "all")

tab_cat
```

## Regression table


Table \@ref(tab:table-5) shows the output for a regression table. Make sure you name all the models you estimate (even if its 50) and explicitly refer to model names (M1, M2 etc.) in the text.

```{r table-5, echo=TRUE, message=FALSE, warning=FALSE}
library(modelsummary)
M1 <- lm(Fertility ~ Education + Agriculture, data = swiss)
M2 <- lm(Fertility ~ Education + Catholic, data = swiss)
M3 <- lm(Fertility ~ Education + Infant.Mortality + Agriculture, data = swiss)
models <- list("M1" = M1, "M2" =  M2, "M3" = M3)


library(gt)
# additionally we want to change the font, font size and spacing
modelsummary(models,
             title = 'Linear regression',
             output = 'gt',
             notes = "Notes: some notes...") %>%
    tab_spanner(label = 'Dependent variable: Fertility', columns = 2:4) %>%
  tab_options(
    table.font.size = 10,
    data_row.padding = px(1),
    table.border.top.color = "white",
    heading.border.bottom.color = "black",
    row_group.border.top.color = "black",
    row_group.border.bottom.color = "white",
    table.border.bottom.color = "white",
    column_labels.border.top.color = "black",
    column_labels.border.bottom.color = "black",
    table_body.border.bottom.color = "black",
    table_body.hlines.color = "white"
  )


```





# Inline code & results

Reproduction reaches new heights when you work with inline code. For instance, you can automatize the display of certain coefficients within the text. An example is to include estimates, e.g., the coefficient of `dist` of the model we ran above. `r rinline("round(coef(M1)[2], 2)")` will insert the coefficient as follows: `r round(coef(M1)[2], 2)`. Or `r rinline("3 + 7")` will insert a `r 3 + 7` in the text.      
Inline code/results that depend on earlier objects in your document will automatically be updated once you change those objects. For instance, imagine a reviewer asks you to omit certain observations from your sample. You can simply do so in the beginning of your code and push play subsequently.. at time you might have to set `cache = FALSE` at the beginning so that all the code chunks are rerun.    
Researchers often avoid referring to results in-text etc. because you easily forget to change them when revising a manuscript. At the same it can make an article much more informative and easier to read, e.g., if you discuss a coefficient in the text you can directly show it in the section in which you discuss it. Inline code allows you to do just that. R Markdown allows you to that do so in a reproducible and automatized manner.










# Graphs

## R base graphs
Inserting figures can be slightly more complicated. Ideally, we would produce and insert them directly in the `.rmd` file. It's relatively simple to insert R base graphs as you can see in Figure \@ref(fig:fig-1).

```{r fig-1, fig.align="center", fig.cap="Scatterplot of Speed and Distance", fig.pos="H", message=FALSE, warning=FALSE, paged.print=FALSE}
plot(swiss$Catholic, swiss$Fertility)
``` 

But it turns out that it doesn't always work so well. 

## ggplot2 graphs
Same is true for ggplot2 as you can see in Figure \@ref(fig:fig-2).

```{r fig-2, fig.align="center", fig.cap="Miles per gallon according to the weight", fig.pos="H", fig.width=6, fig.height=3, message=FALSE, warning=FALSE, paged.print=FALSE}
library(ggplot2)
ggplot(swiss, aes(x=Catholic, y=Fertility, shape=Education_cat)) + geom_point() +
  labs(x="Agriculture", y = "Fertility", 
       shape="Education") + theme_classic()
``` 



# Compiling the document

To view your paper, pagedown requires a web server (since it is based on paged.js)^[open-source library to paginate content in the browser]. By compiling a document, R Studio will display your HTML page through a local web server, i.e., paged.js will work in RStudio Viewer.

There are several options, depending on your intention:

- click on the `Knit` button in R Studio which by default will provide a HTML document in the RStudio viewer pane (the HTML will be stored in your working directory)
- use pagedown's `chrome_print` function in the YAML (uncomment line \#24 of this `Rmd` file) if additionally you want your HTML based web page to be printed to be PDF (the PDF will be stored in your working directory)
- to "live"-preview your pages do not click on the `Knit` button but use the **xaringan** [@xaringan] RStudio add-in _Infinite Moon Reader_. You can simply call the function `xaringan::inf_mr()` (within your console). This will launch a local web server via the **servr** package (Xie 2021a) and display your pages in the RStudio viewer. Each time you save your document (_Ctrl+S_) xaringan updates your pages in the viewer.

- If you use the option `self_contained: false` (see line \#21 of this `Rmd` file) (change to true for a self-contained document, but it'll be a litte slower for Pandoc to render), don't click on the `Knit` button in RStudio. Use instead the **xaringan** [@xaringan] RStudio add-in _Infinite Moon Reader_.




# Good practices for reproducibility
Every researcher has his own optimized setup. Currently we would recommend the following:

* Keep all files of your project (that matter for producing the PDF) in one folder without subfolders. You can zip and directly upload that folder to the [Harvard dataverse](https://dataverse.harvard.edu/).^[Another good folder setup would be to store all files needed as input files for the R Markdown manuscript in a subfolder called "input" and all output files that are produced apart from paper.html and paper.pdf in a subfolder called "output".]
* Make sure that filenames have a logic to them.
    + Main file with text/code: "paper.rmd", "report.rmd" 
    + Data files: "data_xxxxxx.*"
    + Image files: "fig_xxxxxx.*"
    + Tables files: "table_xxxx.*"
    + etc.
    + Ideally, your filenames will correspond to the names in the paper. For instance, Figure 1 in the paper may have a corresponding file called `fig_1_xxxxx.pdf`.
* Use the document outline in R studio (Ctrl + Shift + O) when you work with R Markdown.
* Name rchunks according to what they do or produce:
    + "`fig-...`" for chunks producing figures
    + "`table-...`" for chunks producing tables
    + "`model-...`" for chunks producing model estimates
    + "`import-...`" for chunks importing data
    + "`recoding-...`" for chunks in which data is recoded
* Use "really" informative variable names:
    + Q: What do you think does the variable *trstep* measure? It actually measures trust in the European parliament. 
        + How could we call this variable instead? Yes, `trust.european.parliament` which is longer but will probably be understood by another researcher in 50 years. 
    + If your setup is truly reproducible you will probably re-use the variable names that you generate as variable names in the tables you produce. Hence, there is an incentive to use good names.
* Use unique identifiers in the final R Markdown document paper.rmd that you upload:
    + Think of someone who wants to produce Figure 1/Model 1/Table 1 in your paper but doesn't find it in your code...
        + Name the chunks "fig-1", "fig-2" as the are named in the published paper.
        + Name the chunks that produce tables "table-1", "table-2" etc. as they are named in the published paper.
        + Name your statistical models in your R code "M1", "M2" as they are named in the published paper.


# Additional tricks for publishing
* Make your script anonymous
    + Simply put a `<!-- ... -->` around any identifying information, e.g., author names, title footnote etc.
* Counting words
    + Use adobe acrobat (commerical software) to convert your file to a word file. Then open in word and delete all the parts that shouldn't go into the word count. The word count is displayed in the lower right.
    + Use an one of the online services to count your words (search for "pdf word count")
* Appendix: You can change the numbering format for the appendix in the rmd file
    + What is still not possible in this document is to automatically have separate reference sections for paper and appendix.
* Journals may require you to use their tex style: Sometimes you can simply use their template in your rmarkdown file. See [here](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/LDUMNY) for a PLOS one example.

# Citation styles
If your study needs to follow a particular citation style, you can set the corresponding style in the header of your `.rmd` document. To do so you have to download the corresponding `.csl` file.     

In the present document we use the style of the American Sociological Association and set it in the preamble with `csl: american-sociological-association.csl`. However, you also need to download the respective `.csl` file from the following github page: https://github.com/citation-style-language/styles and copy it into your working directory for it to work.     

The github directory contains a wide variety of citation style files depending on what discipline you work in.

\newpage

# References {-}

::: {#refs}

:::

<!--  to generate a citation entry for BibTeX, you can pass the returned object of citation() to toBibtex(), e.g. toBibtex(citation("xaringan")). Then  copy the output to a .bib file and add a unique citation key -->
<!-- https://bookdown.org/yihui/rmarkdown-cookbook/write-bib.html -->


\newpage

# Online appendix {-}

## Attach R session info in appendix {#sec:rsessioninfo}

Since R and R packages are constantly evolving you might want to add the R session info that contains information on the R version as well as the packages that are loaded. 

```{r echo=FALSE}
print(sessionInfo(), local = FALSE)
```

## All the code in the paper

To simply attach all the code you used in the PDF file in the appendix see the R chunk in the underlying `.rmd` file:

```{r, ref.label=knitr::all_labels(),echo=TRUE,eval=FALSE}
```