Scaling Up Experiments

This codebase contains a couple utilities to facilitate running a large number of experiments.

• scripts/launch.py: This script takes as input a yaml file containing all of the jobs you want to run and will manage distributing them accross the available GPUs to ensure that they are being fully utilized. It is intended to be used in place of the prior approach of bash scripts (which do not guarantee that work will be properly distributed across GPUs).
• jobs/: The jobs folder contains jinja2 templates used to create the aforementioned yaml files. To render them you can use scripts/render_template.py.

Example usage: