Skip to content

Commit

Permalink
install instructions and schedule draft
Browse files Browse the repository at this point in the history
  • Loading branch information
bast committed Sep 9, 2024
1 parent 110c6a5 commit 3e0feed
Show file tree
Hide file tree
Showing 3 changed files with 309 additions and 0 deletions.
140 changes: 140 additions & 0 deletions content/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Reproducible research software development using Python


## Big-picture goal

This is a hands-on course on research software engineering. Since most
participants use Python in their work, we will use Python as the example
language. We will work with an example project and go through all the steps of
a typical software project. Once we have seen the building blocks, we will try
to apply them to own projects. Workshop participants will receive and also
learn to give constructive code feedback.


## Prerequisites

:::{prereq} Preparation
1. Get a **GitHub account** following [these instructions](https://coderefinery.github.io/installation/github/).
1. You will need a **text editor**. If you don't have a favorite one, we recommend
[VS Code](https://coderefinery.github.io/installation/vscode/).
1. **If you prefer to work in the terminal** and not in VS Code, set up these two (skip this if you use VS Code):
- [Git in the terminal](https://coderefinery.github.io/installation/git-in-terminal/)
- [SSH or HTTPS connection to GitHub from terminal](https://coderefinery.github.io/installation/ssh/)
1. **One of these two software environments** (if you are not sure which one to
choose or have no preference, choose Conda):
- {ref}`conda`
- {ref}`venv` (Snakemake is not available in this environment)
1. **Optional** and only on Linux: [Apptainer](https://apptainer.org/) following
[these instructions](https://apptainer.org/docs/admin/1.3/installation.html#install-from-pre-built-packages).
:::


## Schedule

:::{note}
The schedule will very soon contain links to lesson material and exercises.
:::


### Day 1 (Sep 16)

- 13:00-13:30 (0.5h) - **Welcome and introduction**
- Motivation (reproducibility, robustness, distribution, improvement, trust, etc.)
- Practical information (tools, communication, breaks, etc.)
- What will learn and achieve from this course?
- Example Python project we will use and develop

- 13:30-14:45 (1.25h) - **Introduction to version control with Git and GitHub (1/2)**
- Creating a repository and porting your project to Git and GitHub
- Basic commands

- 15:00-16:30 (1.5h) - **Introduction to version control with Git and GitHub (2/2)**
- Branching and merging
- Recovering from typical mistakes

- 16:45-18:00 (1.25h) - **Documentation**
- In-code documentation including docstrings
- Writing good README files
- Markdown
- Sphinx
- Building documentation with GitHub Actions
- Jupyter Notebooks


### Day 2 (Sep 17)

- 09:00-10:30 (1.5h) - **Collaborative version control and code review (1/2)**
- Practice code review using issues and pull requests
- Forking workflow
- Contributing changes to projects of others

- 10:45-12:15 (1.5h) - **Collaborative version control and code review (2/2)**
- Organization strategies
- Merge vs. rebase
- Conflict resolution

- 16:45-18:00 (1.25h) - **Debriefing and Q&A**
- Participants work on their projects
- Together we study actual codes that participants wrote or work on
- Constructively we discuss possible improvements
- Give individual feedback on code projects


### Day 3 (Sep 18)

- 09:00-10:30 (1.5h) - **Automated testing**
- Unit tests
- End-to-end tests
- pytest
- GitHub Actions

- 10:45-12:15 (1.5h) - **How to make the project more reusable?**
- Tracking dependencies with requirements.txt and environment.yml
- Recording environments in containers

- 13:00-14:45 (1.75h) - **Code quality and good practices**
- Naming (and other) conventions, project organization, modularity
- Refactoring (explained through examples)
- Design patterns: functional design vs. object-oriented design
- How to design your code before writing it
- Structuring larger software projects in a modular way
- Command-line interfaces
- Workflows with Snakemake

- 15:00-16:30 (1.5h) - **How to release and publish your code**
- Packaging the code
- Licenses
- Sharing the code via PyPI
- Publishing the code via Zenodo

- 16:45-18:00 (1.25h) - **Debriefing and Q&A**
- Participants work on their projects
- Together we study actual codes that participants wrote or work on
- Constructively we discuss possible improvements
- Give individual feedback on code projects


### Extra material if we have time

- Profiling memory and CPU usage
- Strategies for parallelization


```{toctree}
:maxdepth: 1
:caption: Software environment
:hidden:
installation/conda
installation/virtual-environment
```

```{toctree}
:maxdepth: 1
:caption: Reference
:hidden:
All lessons <https://coderefinery.org/lessons/>
CodeRefinery <https://coderefinery.org/>
Reusing <https://coderefinery.org/lessons/reusing/>
```
87 changes: 87 additions & 0 deletions content/installation/conda.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
(conda)=

# Conda environment

A Conda environment is an isolated software environment that is used to manage dependencies for a project
and you decide where it is located.

You will need a `environment.yml` file that documents the dependencies:
```yaml
name: coderefinery
channels:
- conda-forge
- bioconda
dependencies:
- python >= 3.10
- black
- click
- flit
- ipywidgets
- isort
- jupyterlab
- jupyterlab_code_formatter
- jupyterlab-git
- matplotlib
- myst-parser
- nbdime
- numpy
- pandas
- pytest
- pytest-cov
- scalene
- seaborn
- snakemake-minimal
- sphinx
- sphinx-autoapi
- sphinx-autobuild
- sphinx_rtd_theme >= 2.0
- vulture
- scikit-image
```
## Before you create a virtual environment
1. Create a new directory for this course.
1. In this directory, create an `environment.yml` file and copy-paste the dependencies above into it.


## Choose the tool to manage the environment

If you are already using one of these tools, please continue using the tool that you like and know.
If you are new to this, **we recommend using Miniconda or Miniforge**.

- [Anaconda](https://docs.anaconda.com/anaconda/install/)
- Advantages: easy to install, easy to use, good for beginners
- Disadvantages: large download, installs more than we will need, license restrictions
- [Miniconda](https://docs.anaconda.com/miniconda/)
- Advantages: small size, installs only what you need
- Disadvantages: no graphical interface, license restrictions
- [Miniforge](https://github.com/conda-forge/miniforge)
- Advantages: small size, no license restrictions
- Disadvantages: no graphical interface
- [Micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)
- Advantages: fast, small size
- Disadvantages: no graphical interface
- [Pixi](https://pixi.sh/latest/)
- Advantages: fast and new
- Disadvantages: new and less tested and not documented here


## Creating the virtual environment

1. Open your terminal shell (e.g. Bash or Zsh).
2. Activate `conda` using `conda activate` or `source ~/miniconda3/bin/activate`.
3. Run the following command:
```console
$ conda env create --file environment.yml
```
4. Make sure that you see "coderefinery" in the output when you ask for a list of all available environments:
```console
$ conda env list
```


## How to verify that this worked

(this will be added)
82 changes: 82 additions & 0 deletions content/installation/virtual-environment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
(venv)=

# Virtual environment

A virtual environment is an isolated software environment that is used to manage dependencies for a project
and you decide where it is located.

You will need a `requirements.txt` file that documents the dependencies:
```
black
click
flit
ipywidgets
isort
jupyterlab
jupyterlab-code-formatter
jupyterlab-git
matplotlib
myst-parser
nbdime
numpy
pandas
pytest
pytest-cov
scalene
seaborn
sphinx
sphinx-autoapi
sphinx-autobuild
sphinx_rtd_theme >= 2.0
vulture
scikit-image
```


## Before you create a virtual environment

1. Create a new directory for this course.
1. In this directory, create a `requirements.txt` file and copy-paste the dependencies above into it.


## Creating the virtual environment

Now create a virtual environment in this directory either using [pip and
venv](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)
(more traditional and safer) or using [uv](https://docs.astral.sh/uv/) (more modern but also less tested):

:::::{tabs}

::::{group-tab} pip and venv
Create a new virtual environment and activate it:
```console
$ python3 -m venv coderefinery-environment
$ source coderefinery-environment/bin/activate
```

Install the dependencies into the environment:
```console
$ python3 -m pip install -r requirements.txt
```
::::


::::{group-tab} uv
Create a new virtual environment and activate it:
```console
$ uv venv coderefinery-environment
$ source coderefinery-environment/bin/activate
```

Install the dependencies into the environment:
```console
$ uv pip install -r requirements.txt
```
::::

:::::


## How to verify that this worked

(this will be added)

0 comments on commit 3e0feed

Please sign in to comment.