washr turns a cleaned dataset into a documented R data package that follows the FAIR principles, with a website, a citation file and machine readable metadata. This page walks through the workflow in the order you run it. Each step is one function, and every function is safe to run again after you have edited the files it wrote.
The FAIR Data Publishing guide covers the same steps with more explanation and the parts that happen outside R, such as creating the GitHub repository and the Zenodo release. The chapters of the guide follow the same order as the sections below.
Install
The stable release, from CRAN:
install.packages("washr")The current development build, from the openwashdata R-universe. It is rebuilt on every push and installs like any other package, with no compiler toolchain:
install.packages("washr", repos = c("https://openwashdata.r-universe.dev", getOption("repos")))The same development source from GitHub, built on your machine:
# install.packages("remotes")
remotes::install_github("openwashdata/washr")1. Create the package
Create an R package for the dataset and open it as a project. The package name is the name of the dataset, in lower case, without dots or underscores.
usethis::create_package("~/path/to/fslogisticskampala")Then put it under version control and on GitHub, for example with
usethis::use_git() and usethis::use_github().
The guide describes this step in detail.
setup_ci() adds the GitHub Actions workflow that runs
R CMD check on every push and pull request, on macOS,
Windows and three versions of R on Linux. The openwashdata review
standard requires it, so run it now and the package meets that part of
the standard from its first commit.
2. Bring in the raw data
Run setup_rawdata() from the package root. It creates
data-raw/ and a processing script,
data-raw/data_processing.R, with the steps the script
should follow.
Copy the raw files into data-raw/. In the processing
script, read them, clean them into one or more tidy data frames, and end
with usethis::use_data() so each data frame lands in
data/ as an .rda file. The script also exports
each data frame as CSV and XLSX into inst/extdata/, which
is where the website and the metadata find the downloadable files.
3. Write the dictionary
setup_dictionary() reads every data object in
data/ and writes data-raw/dictionary.csv with
one row per variable: the file it belongs to, its name, its type, and an
empty description.
Open the CSV and write a description for every variable. The dictionary is the one place where variables are described; the roxygen documentation and the metadata are generated from it.
When the data changes later, run update_dictionary(). It
adds a row for each new variable, removes the row of a variable that no
longer exists, refreshes the types, and keeps every description and
every column you added yourself, such as unit or
allowed_values. Then run setup_roxygen()
again.
4. Generate the roxygen documentation
setup_roxygen() writes one file per data object under
R/, with a title and description placeholder and the
variable table from the dictionary.
Open each file, replace the title and the description, then run
devtools::document() so the help pages exist. When you
change a description in the dictionary later, run
setup_roxygen() again: it regenerates only the variable
table and keeps the title, the description and anything else you wrote
in the file.
5. Complete DESCRIPTION
update_description() sets the fields the openwashdata
standard expects: the language, the date, the repository URL, the bug
report URL, and the CC BY 4.0 license when the package has none yet.
Existing values are kept and merged. It also writes a few
Config/washr/ fields, such as the funding sentence and the
analytics domain of the site. The later steps read them from
DESCRIPTION, and the last section of this page explains
them.
Then open DESCRIPTION and check the title, the
description and the authors. Authors are written with
person() and carry their ORCID in the comment field. Three
more facts have no other home and go into DESCRIPTION by
hand:
X-schema.org-keywords: sanitation, faecal sludge, Kampala
X-schema.org-spatialCoverage: Kampala, Uganda
X-schema.org-temporalCoverage: 2022-03-01/2022-09-30
The keywords are a comma separated list. The spatial coverage is a place name. The temporal coverage is a start and an end date separated by a slash.
6. Generate the metadata
update_metadata() derives a schema.org description of
the dataset from DESCRIPTION, the dictionary, the citation
file and the files in inst/extdata, and writes it as
JSON-LD into pkgdown/templates/in-header.html. The website
build embeds it in the head of every page, where dataset search engines
read it. The same JSON-LD goes to
pkgdown/assets/metadata.jsonld, which the build copies to
the root of the site, so the metadata also has its own address,
<site>/metadata.jsonld. The README links it in the
Citation section once the package names its site.
The function ends with the fields it could not fill and where to fill them. Nothing is edited by hand: change the source and run it again. This function is experimental; its output shape may still change.
7. Write the README
setup_readme() writes README.Rmd from the
openwashdata template, with the installation instructions, the download
table, and a section per data object. With
has_example = TRUE it adds an Example section with a
scaffold for a first plot.
setup_readme(has_example = TRUE)Write the text, add a section for each further data object, then render it:
devtools::build_readme()8. Build the website
setup_website() writes _pkgdown.yml from
the openwashdata template and builds the site into docs/.
The template sets the Pages URL, the analytics header, a link to
openwashdata.org in the navbar, the funding and the openwashdata blocks
of the sidebar, a footer that names the washr version and the publisher
(“Package built with washr 1.2.1.” and “Published by openwashdata, a
project of Global Health Engineering at ETH Zurich.”), and a reference
index with one entry per data object.
Running it again keeps your _pkgdown.yml and rebuilds
the site. docs/ is committed and served by GitHub Pages;
when the package deploys through a pkgdown GitHub Actions workflow
instead, the function leaves docs/ ignored.
update_website() brings an existing
_pkgdown.yml up to date after a washr release: it adds what
the template has gained, refreshes the washr version in the footer, and
keeps the reference index, your hand edits and the comments. A current
file is left untouched. update_description() refreshes the
version line on its own.
use_brand() installs the openwashdata brand, the fonts
and colors of the site, from the latest release of the central brand
repository and wires _pkgdown.yml to it. It installs the
small logo of the brand as the package logo, which the site shows beside
the title and the README shows on GitHub (with its white counterpart on
the dark theme of the site), and it sets the social preview of the brand
as the Open Graph image of the site, so a shared link shows the
openwashdata image. It records the release it installed in
DESCRIPTION as Config/washr/brand. Run it once
after setup_website(), and again after a new brand release;
a run with no new release changes nothing.
Branded PDF and Word reports about the package use the openwashdata
Quarto extension, quarto-owd, with
the formats owd-typst and owd-docx. Install it
in the package root with
quarto add openwashdata/quarto-owd; it reads the
_brand.yml that use_brand() installed, so no
second brand copy is needed. Add ^_extensions$ to
.Rbuildignore.
9. Cite and release
update_citation() writes CITATION.cff and
inst/CITATION from DESCRIPTION. Run it before
the first release so the files exist, then again with the DOI once the
release is deposited on Zenodo; the guide describes the Zenodo
steps.
Before you create the release, run
check_publication_readiness(). It reads the package and
reports, item by item, what is in place and what is missing, and for
every gap it names the step that closes it. It writes nothing, so run it
as often as you like.
After the Zenodo release, run update_citation() once
more with the DOI:
update_citation(doi = "10.5281/zenodo.11185699")With a DOI the function also adds the DOI badge to the README and
rebuilds it and the site. A later run without the DOI keeps the DOI on
file. Run update_metadata() once more after that, so the
metadata carries the DOI as well.
The same call writes .zenodo.json from
DESCRIPTION. Zenodo reads this file when it archives a
GitHub release and takes the resource type (Dataset), the community, the
creators and the license from it. Commit the file before you create the
release. update_zenodo_json() writes it on its own.
Once the DOI is on file, data_availability_statement()
drafts the text for the authors of an article that uses the data: the
reference list entry in APA style, the data availability statement and
the in-text citation. Name the data the article uses with
subject; when some data was left out of the package, write
one sentence that says what and why into
Config/washr/withheld in DESCRIPTION, and the
statement ends with it.
data_availability_statement(subject = "The household survey data")10. Versions after the release
The version and the DOI of a release identify the data someone cites. The version of a data package therefore changes only when the data, or the meaning of the data, change:
| Change | Version |
|---|---|
| Columns renamed or removed, the structure breaks | major |
| New data, rows or variables | minor |
| Corrected values, or a documentation fix that changes how the data are read (unit, variable description, dictionary) | patch |
| Website, articles, styling, CI, wording that does not change the meaning | none |
Work on the website or the documentation stays on the release version and can reach the main branch without a new release. The first commit that changes the data or their meaning sets the development version:
usethis::use_dev_version()
update_citation(build = FALSE)
update_metadata()check_publication_readiness() reports when
data/, inst/extdata/ or
data-raw/dictionary.csv differ from the tag of the release
while the version still names that release. A change of meaning outside
these files, for example in the description of a dataset in
R/, is not detected; the reviewer decides on it.
Running steps again
Every function above reads what is there, merges its changes, and
writes only what changed. The two exceptions stop instead of
overwriting: setup_dictionary() when the dictionary exists
(run update_dictionary() instead, which keeps what you
wrote), and setup_readme() when README.Rmd
exists (pass force = TRUE to replace it).
setup_website() keeps an existing
_pkgdown.yml; update_website() is the function
that changes it. A full second run of the other steps on an unchanged
package changes no file.
More than one dataset
A package can hold more than one dataset. Name them when you write the processing script:
setup_rawdata(datasets = c("trips", "trucks"))The export block of data-raw/data_processing.R then has
one usethis::use_data() line per dataset and exports each
of them to inst/extdata/ as CSV and XLSX in one loop over
the names. Without datasets, the script exports one dataset
named after the package, through the same loop.
setup_dictionary(), setup_roxygen(),
setup_website() and update_metadata() cover
every data object in data/ on their own, and so does
setup_readme(): it writes one data section per object, in
alphabetical order, or for the objects named in its own
datasets argument, in that order. It warns about an object
that has no rows in the dictionary yet, because the table of its
variables would stay empty.
Publishing outside openwashdata
washr ships with the openwashdata values as defaults, and every one
of them lives in DESCRIPTION, so another group publishes
with its own values by editing that file. Nothing else has to
change.
Start by naming your GitHub organisation, once:
update_description(github_user = "https://github.com/yourorg")From then on the organisation, the repository URL, the license and
the people are read from the standard DESCRIPTION fields
(URL, License, Authors@R), and
the README, the site, the citation files and the metadata follow them.
The values that have no standard field are the
Config/washr/ fields:
| Field | What it sets | Value for another group |
|---|---|---|
Config/washr/funding |
the funding sentence in the sidebar of the site | your funding sentence, Markdown allowed, or none
|
Config/washr/analytics-domain |
the Plausible analytics header of the site | your Plausible domain, or none
|
Config/washr/doi-provider |
the DOI badge of the README |
zenodo, or the name of another provider for a generic
DOI badge |
Config/washr/zenodo-community |
the community in .zenodo.json
|
your Zenodo community, or none
|
Config/washr/brand-source |
the repository use_brand() copies from |
the GitHub repository of your brand, such as
yourorg/brand, or none
|
Config/washr/publisher |
the publisher sentence in the footer of the site | your sentence, Markdown allowed, or none
|
Config/washr/home |
the website of your group: the navbar link, the line under the README badges, the publisher url in the metadata | the URL, or none
|
Config/washr/catalog |
the data catalogue named in the metadata | the URL of your catalogue, or none
|
Config/washr/community |
the text of the community block in the sidebar of the site | your text, Markdown allowed, or none
|
Config/washr/parent-organizations |
the organisations above the publisher in the metadata
(parentOrganization), nearest first; a
https://ror.org/ URL becomes the ROR identifier |
Name <url> entries separated by semicolons, such
as
Your Institute <https://ror.org/...>; Your University <https://ror.org/...>,
or none
|
Config/washr/pages-domain |
the domain of the site | only when the site is not served from
yourorg.github.io
|
For a package under another organisation,
update_description() writes the funding, analytics, Zenodo
community, brand, publisher, website, catalogue, community and parent
organisations fields as none and lists them, because the
openwashdata values would be wrong there. The footer then names washr
and nothing else. Edit them, then run the steps as above. A field that
is set is never overwritten. check_publication_readiness()
holds the package to these values.
A license other than CC BY 4.0 is set in License before
the first update_description(), for example with
usethis::use_cc0_license() or another
usethis::use_*_license() function.
update_description() leaves a license that is set as it
is.
