From 9b0c2384ee7367f0e7b6de98e44ea9475d2b1f0c Mon Sep 17 00:00:00 2001 From: Andy Teucher Date: Fri, 21 Jun 2024 13:06:14 -0700 Subject: [PATCH 1/4] Add section on testing image locally --- README.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/README.md b/README.md index e2563bd..2ae21f2 100644 --- a/README.md +++ b/README.md @@ -59,6 +59,43 @@ For other Jupyterhub deployments we can change the image using the hub configura > Note: Looks like 2i2c caches the user image so tags like `main` won't be updated even if they have changes. Using the actual commit hash is a better practice for now. +## Testing changes locally + +If you want to test your changes locally, you can do so using Docker: + +1. [Install Docker Desktop](https://docs.docker.com/desktop/) +2. Make sure Docker is running, then build the image with: + +``` +# make sure you are in the "ci" directory +cd ci +docker build -t openscapes/corn:test . --platform linux/amd64 +``` + +The `--platform linux/amd64` flag is only necessary if you are _not_ on a +machine with an x86-64 chip architecture (e.g., an M1 or M2 Mac, which have an +ARM-based architecture). + +Once the image has been built, you can run it with: + +``` +docker run -p 8888:8888 --platform linux/amd64 openscapes/corn:test jupyter lab --ip=0.0.0.0 +``` + +If a browser doesn't automatically open, you can open one of the links that +is generated in the output. It will look something like: + +``` +http://127.0.0.1:8888/lab?token=a74663dba15a5e5cab52ef4bd6a9346034fd1ab927f6a29b +``` + +Note that the home directory (`/home/jovyan`) will look different than you are +used to in the Hub. This is because in the local image the home directory still +contains artifacts from the image building process, while in the Hub a shared +AWS NFS drive is mounted to `home/jovyan`, giving you access to your persistent +home directory in the Hub. + + ## What's next? This is a effective but probably inefficient way of building environments, exploring staged partial builds in Docker or using [conda-store](https://github.com/Quansight/conda-store) to build each environment and then pulling them into a Docker image may be more efficient. From cb3a96b28e2b831861e1dc89314f31e2971d11c2 Mon Sep 17 00:00:00 2001 From: Andy Teucher Date: Fri, 21 Jun 2024 13:57:00 -0700 Subject: [PATCH 2/4] Update instructions for updating image --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 2ae21f2..4670072 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ amazing-env ## **That's it!** -> Note: if you have pip installable depencencies, they must be listed using a `requirements.txt` file. +> Note: if you have pip installable dependencies, they must be listed using a `requirements.txt` file. @@ -47,21 +47,22 @@ amazing-env To update the [quarto](https://quarto.org) installation you'll need to change the version number in two places in corn's [Dockerfile](https://github.com/NASA-Openscapes/corn/blob/main/ci/Dockerfile#L10). After committing changes, the GitHub Action will begin - see next. -## Using a Kernel +## Updating the image in the JupyterHub -After we commit our changes to this repo, create a tagged release with format `vYYYY.MM.DD`. This is important because the GitHub Action build only gets triggered with that tagged release format. Then, the Github Action will push the resulting Docker image to [dockerhub](https://hub.docker.com/repository/docker/openscapes/corn). This can take ~20 minutes. Then, we need to update the user image in our Jupyterhub configuration (admin > Services > configurator)(right now it's hard-coded to `openscapes/corn:$TAG`, previously was `betolink`.) For 2i2c deployments there is a GUI that allows administrators to do it. +After we commit our changes to the `main` branch of this repo, the GitHub Action build will be triggered. Then, the Github Action will push the resulting Docker image to [dockerhub](https://hub.docker.com/r/openscapes/python/tags), creating an image tagged with the commit hash. This can take ~20 minutes. -![configurator](https://user-images.githubusercontent.com/717735/139174138-f6eb011e-9cc5-4c15-af68-d77ae5d7ee00.png) +You can try this newly created image by using the "Bring your own image" functionality in the +JupyterHub. Specify the image with the name `openscapes/python:$TAG`, where `$TAG` is the tag of the Dockerhub image (which is the same as the commit hash). You can copy the name from the `docker pull` command shown in [Dockerhub](https://hub.docker.com/r/openscapes/python/tags). -Then, you'll go to > Stop My Server (or File > Log Out) to stop your server and restart it. Then the Docker image should be updated. +Once you've verified it is working the way you want, we need to update the python image in our Jupyterhub configuration. The quickest way to do this is to create a pull request [here](https://github.com/2i2c-org/infrastructure/blob/main/config/clusters/openscapes/common.values.yaml#L68), updating `openscapes/python:$TAG`, with the tag/commit hash. For 2i2c deployments there is a GUI that allows administrators to do it. -For other Jupyterhub deployments we can change the image using the hub configurator object or even in a Kubernetes chart. +Then, you'll go to > Stop My Server (or File > Log Out) to stop your server and restart it. Then the Docker image should be updated. > Note: Looks like 2i2c caches the user image so tags like `main` won't be updated even if they have changes. Using the actual commit hash is a better practice for now. -## Testing changes locally +## Testing changes to the image locally -If you want to test your changes locally, you can do so using Docker: +If you want to test your changes locally (i.e., without building in GitHub actions and pushing to Dockerhub), you can do so on your own computer using Docker: 1. [Install Docker Desktop](https://docs.docker.com/desktop/) 2. Make sure Docker is running, then build the image with: From fc592e7029375053a197044720d489365a03b686 Mon Sep 17 00:00:00 2001 From: Andy Teucher Date: Fri, 21 Jun 2024 14:16:52 -0700 Subject: [PATCH 3/4] Update Dockerfile --- ci/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/Dockerfile b/ci/Dockerfile index f15231f..e17bc86 100644 --- a/ci/Dockerfile +++ b/ci/Dockerfile @@ -6,7 +6,7 @@ COPY --chown=jovyan:jovyan . /home/jovyan/.kernels USER root -ENV QUARTO_CLI=https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.433/quarto-1.3.433-linux-amd64.deb +ENV QUARTO_CLI=https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.555/quarto-1.4.555-linux-amd64.deb RUN wget -O /tmp/quarto.deb ${QUARTO_CLI} && dpkg -i /tmp/quarto.deb && rm -rf /tmp/quarto.deb && apt-get clean From e17fe92d75e2bc4cec95f78a48b2c9b9f2eb5e72 Mon Sep 17 00:00:00 2001 From: Andy Teucher Date: Fri, 21 Jun 2024 14:29:51 -0700 Subject: [PATCH 4/4] update Quarto update instructions --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4670072..828351b 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ amazing-env ### Updating quarto -To update the [quarto](https://quarto.org) installation you'll need to change the version number in two places in corn's [Dockerfile](https://github.com/NASA-Openscapes/corn/blob/main/ci/Dockerfile#L10). After committing changes, the GitHub Action will begin - see next. +To update the [quarto](https://quarto.org) installation you'll need to change the version number in corn's [Dockerfile](https://github.com/NASA-Openscapes/corn/blob/main/ci/Dockerfile#L9). After committing changes, the GitHub Action will begin - see next. ## Updating the image in the JupyterHub