Install dependencies with uv

.autoupdate/preupdate

@@ -0,0 +1,17 @@
+
+#!/bin/bash
+
+# This script is called by our weekly dependency update job in Jenkins
+
+pip3 install uv > speech-to-text.txt &&
+    ~/.local/bin/uv lock --upgrade >> speech-to-text.txt
+
+retVal=$?
+
+git add uv.lock &&
+    git commit -m "Update Python dependencies"
+
+if [ $retVal -ne 0 ]; then
+    echo "ERROR UPDATING PYTHON (speech-to-text)"
+    cat speech-to-text.txt
+fi

.github/workflows/test.yml

@@ -2,11 +2,11 @@ name: Test
 on:
   - push
 jobs:
-  build:
+  test:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.8]
+        python-version: [3.11]
     steps:
 
       - name: checkout
@@ -31,15 +31,15 @@ jobs:
         run: |
           wget -O - https://raw.githubusercontent.com/jontybrook/ffmpeg-install-script/main/install-ffmpeg-static.sh | bash -s -- --stable --force
 
-      - name: Install Python dependencies
+      - name: Install uv
         run: |
-          pip install -r requirements.txt
+          pip install uv
 
       - name: Run type checking
-        run: mypy .
+        run: uv run mypy .
 
       - name: Run tests
-        run: pytest --cov-branch --cov-report=xml
+        run: uv run pytest --cov-branch --cov-report=xml
 
       - name: Upload coverage reports to Codecov
         uses: codecov/codecov-action@v5

Dockerfile

@@ -1,4 +1,4 @@
-FROM nvidia/cuda:12.1.0-devel-ubuntu20.04
+FROM nvidia/cuda:12.8.0-cudnn-devel-ubuntu22.04
 
 RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
     python3 \
@@ -10,14 +10,14 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
 WORKDIR /app
 
 ADD ./whisper_models whisper_models
-ADD ./requirements.txt requirements.txt
+ADD ./pyproject.toml pyproject.toml
 
 RUN python3 -m pip install --upgrade pip
-RUN python3 -m pip install -r requirements.txt
+RUN python3 -m pip install uv
 
 ADD ./error_reporting_wrapper.py error_reporting_wrapper.py
 
 ADD ./speech_to_text.py speech_to_text.py
-RUN python3 -m py_compile speech_to_text.py
+RUN uv run python3 -m py_compile speech_to_text.py
 
-ENTRYPOINT ["python3", "error_reporting_wrapper.py", "python3", "speech_to_text.py"]
+ENTRYPOINT ["uv", "run", "error_reporting_wrapper.py", "python3", "speech_to_text.py"]

README.md

@@ -305,6 +305,10 @@ When updating the base Docker image, in order to prevent random segmentation fau
 1. You are using an [nvidia/cuda](https://hub.docker.com/r/nvidia/cuda) base Docker image.
 2. The version of CUDA you are using in the Docker container aligns with the version of CUDA that is installed in the host operating system that is running Docker.
 
+## Benchmarking
+
+It may be useful to compare the output of speech-to-text with previous runs. See docs/README.md for more about that.
+
 ## Linting and Type Checking
 
 You may notice your changes fail in CI if they require reformatting or fail type checking. We use [ruff](https://docs.astral.sh/ruff/) for formatting Python code, and [mypy](https://mypy-lang.org/) for type checking. Both of those should be present in your virtual environment.

docs/README.md

@@ -0,0 +1,18 @@
+# Benchmark speech-to-text
+
+If you make changes to speech-to-text's software dependencies, especially the base nvidia/cuda Docker image or the Whisper software, it's a good idea to see what divergence there might be in the transcripts that are generated. To do this properly we should send the media through the complete speech-to-text workflow in the SDR, since it has certain options configured for Whisper as well as pre and post-processing.
+
+22 items have been deposited into the SDR development environment, and tagged so that they can be easily queued up for processing. See [the wiki](https://github.com/sul-dlss/speech-to-text/wiki/Load-testing-the-speech%E2%80%90to%E2%80%90text-workflow) for details on how to run them.
+
+The VTT file output for these transcripts as of 2025-01-31 has been stored in the `baseline` directory. When you are making updates to the speech-to-text software and want to examine the differences in output you can:
+
+1. Ensure your changes pass tests, either locally or in Github.
+2. Deploy your changes to the SDR QA environment by tagging a new version, e.g. `git tag rel-2025-01-29`, and pushing it to Github (`git push --tags`) so that the release Github Action runs. If this succeeds your changes will be live in the QA and Stage environments. Alternatively you can run deploy.sh directly (note that the Docker images generated come to around 20 GB, and may be slow to upload from a home connection).
+3. Use Argo's [bulk action](https://github.com/sul-dlss/speech-to-text/wiki/Load-testing-the-speech%E2%80%90to%E2%80%90text-workflow#running-text-extraction-as-a-bulk-action) to generate transcripts.
+4. Wait for the processing to be done. One easy way to do this is to look at them in the AWS Batch dashboard in the AWS Console. Or you can simply watch the last item in the batch to see when it completes.
+5. Run report.py: `python report.py`
+6. Commit your new report directory as a record.
+
+You should find a `index.md` Markdown file in a date stamped directory inside the `reports` directory.
+
+The `baseline` directory contains Cocina JSON and VTT files to use as baseline data. You may want to update these over time as understanding of what to use as a baseline changes. In an ideal world these would be more like ground truth data, or transcripts that have been vetted and corrected by people.