Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: trigger release process #961

Merged
merged 29 commits into from
Feb 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
43a710f
refactor: derive the _schema automatically
SzymonSzyszkowski Dec 3, 2024
c9e2901
feat: allow for skipping _schema if get_schema is defined
SzymonSzyszkowski Dec 17, 2024
ca14c10
Merge branch 'dev' into szsz-derive-schema-automatically
project-defiant Dec 17, 2024
faee7d8
Merge branch 'dev' into szsz-derive-schema-automatically
project-defiant Dec 17, 2024
897b1f0
chore: comments
Dec 18, 2024
cb64cc5
Merge pull request #941 from opentargets/szsz-derive-schema-automatic…
project-defiant Dec 18, 2024
5534908
test: improve fixture definition by constraining dataset IDs (#954)
ireneisdoomed Dec 19, 2024
e8724cf
fix: repair SusieFinemapperStep to work with new SL schema and fix lo…
Daniel-Considine Dec 19, 2024
0a958f8
chore: update poetry to version 2 (#969)
ireneisdoomed Jan 14, 2025
45ff1be
chore: clean documentation page about version engine (#960)
project-defiant Jan 14, 2025
1475f7e
fix: updating info in finngen study index ingestion (#972)
Daniel-Considine Jan 15, 2025
8e3c765
fix(`isProteinCoding`): get all overlapping genes from the variant in…
ireneisdoomed Jan 15, 2025
111d902
fix: skip leading 'v' in docker tag (#975)
project-defiant Jan 21, 2025
c847921
refactor: remove gene_index step (#946)
vivienho Jan 22, 2025
aa00959
feat(qtls): flagging trans QTL credible sets (#973)
DSuveges Jan 24, 2025
3d31edd
chore: multiple python version support with latest pyspark and hail (…
project-defiant Jan 28, 2025
49b62e7
chore: bump pycln version (#978)
project-defiant Jan 28, 2025
a99ca84
chore: pre-commit autoupdate (#976)
pre-commit-ci[bot] Jan 28, 2025
040bbf8
chore: revert pre-commit update (a99ca8490ae6feab2494f9283f49b329d488…
ireneisdoomed Jan 28, 2025
e7f7945
feat(l2gmodel): store features list metadata as instance attribute (#…
ireneisdoomed Jan 30, 2025
ce8afb9
chore: removing symbols from QTL study identifiers (#971)
DSuveges Jan 31, 2025
f3a2f9d
build(deps): update google-cloud-storage requirement from <2.15.0,>=2…
dependabot[bot] Feb 3, 2025
e2832a3
fix: tweak spark session builder to fix type hints (#982)
Daniel-Considine Feb 3, 2025
2c0ecd5
build(deps): update numpy requirement from <1.27.0,>=1.26.4 to >=1.26…
dependabot[bot] Feb 3, 2025
3f204dc
feat: labels for dataproc in dev mode (#987)
project-defiant Feb 3, 2025
bfddd2d
build(deps): update scipy requirement from <1.12.0,>=1.11.4 to >=1.11…
dependabot[bot] Feb 3, 2025
5e9638a
feat: FoldX predicted energies in variant annotation (#947)
DSuveges Feb 5, 2025
8622b5e
fix: foldX normalisation to cover full range (#989)
DSuveges Feb 5, 2025
8d018de
feat: implement posterior prob filter for COLOC at small overlaps N<1…
xyg123 Feb 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,26 @@ version: 1
labels:
- label: "size-XS"
size:
exclude-files: ["poetry.lock"]
exclude-files: ["uv.lock"]
below: 10
- label: "size-S"
size:
exclude-files: ["poetry.lock"]
exclude-files: ["uv.lock"]
above: 9
below: 100
- label: "size-M"
size:
exclude-files: ["poetry.lock"]
exclude-files: ["uv.lock"]
above: 100
below: 500
- label: "size-L"
size:
exclude-files: ["poetry.lock"]
exclude-files: ["uv.lock"]
above: 499
below: 1000
- label: "size-XL"
size:
exclude-files: ["poetry.lock"]
exclude-files: ["uv.lock"]
above: 999
- label: "airflow"
files:
Expand Down
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ add diagrams or images if necessary. It'll help the reviewer_ -->
- [ ] Did you make sure the branch is up-to-date with the `dev` branch?
- [ ] Did you write any new necessary tests?
- [ ] Did you make sure the changes pass local tests (`make test`)?
- [ ] Did you make sure the changes pass pre-commit rules (e.g `poetry run pre-commit run --all-files`)?
- [ ] Did you make sure the changes pass pre-commit rules (e.g `uv run pre-commit run --all-files`)?
42 changes: 26 additions & 16 deletions .github/workflows/artifact.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
name: Build and Push to Artifact Registry

"on":
workflow_dispatch:
push:
branches: ["*"]
tags: ["*"]
branches:
- "*"
tags:
- "v[0-9]+.[0-9]+.[0-9]+-*"

env:
PROJECT_ID: open-targets-genetics-dev
REGION: europe-west1
GAR_LOCATION: europe-west1-docker.pkg.dev/open-targets-genetics-dev
REPOSITORY: gentropy-app
PYTHON_VERSION_DEFAULT: "3.10.8"
PYTHON_VERSION_DEFAULT: "3.11.11"

jobs:
build-push-artifact:
Expand All @@ -37,7 +40,7 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: "Docker auth"
run: |-
run: |
gcloud auth configure-docker ${{ env.REGION }}-docker.pkg.dev --quiet

- name: Quick Docker build (gentropy only, AMD64 only, with layer cache)
Expand All @@ -50,36 +53,43 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Build and push gentropy image
# skip the `v` at the beginning of the tag for docker image tags
- name: Create a docker tag
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
id: docker-tag
shell: bash
env:
TAG: ${{ github.ref_name }}
run: |
TAG=$(echo $TAG | sed 's/^v//')
echo "TAG=$TAG" >> $GITHUB_ENV
echo "The tag for this build is $TAG"

- name: Build and push gentropy image
if: steps.docker-tag.outcome == 'success'
uses: docker/build-push-action@v6
with:
platforms: linux/amd64,linux/arm64
push: true
tags: "${{ env.GAR_LOCATION }}/${{ env.REPOSITORY }}/gentropy:${{ github.ref_name }}"
tags: "${{ env.GAR_LOCATION }}/${{ env.REPOSITORY }}/gentropy:${{ env.TAG }}"
context: .

- name: Build and push VEP image
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
if: steps.docker-tag.outcome == 'success'
uses: docker/build-push-action@v6
with:
platforms: linux/amd64
push: true
tags: "${{ env.GAR_LOCATION }}/${{ env.REPOSITORY }}/custom_ensembl_vep:${{ github.ref_name }}"
tags: "${{ env.GAR_LOCATION }}/${{ env.REPOSITORY }}/custom_ensembl_vep:${{ env.TAG }}"
context: .
file: "src/vep/Dockerfile"

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION_DEFAULT }}
- name: Install and configure Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
- name: Install uv
uses: astral-sh/setup-uv@v5

- name: Build and push spark cluster dependencies
run: |
make build
run: make build
34 changes: 15 additions & 19 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,47 +4,43 @@ name: Checks
pull_request:

env:
PYTHON_VERSION_DEFAULT: "3.10.8"
PYTHON_VERSION_DEFAULT: "3.11.11"

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
fail-fast: false
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.8
python-version: ${{ matrix.python-version }}
- name: Set up Java
uses: actions/setup-java@v4
with:
java-version: "8"
java-version: "11"
distribution: "temurin"
- name: Install and configure Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Load cached venv
id: cached-poetry-dependencies
id: cached-uv-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-${{ runner.os }}-${{ env.PYTHON_VERSION_DEFAULT }}-${{ hashFiles('**/poetry.lock') }}
- name: Validate project dependencies
run: poetry check
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('**/uv.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
- name: Install library
run: poetry install --no-interaction
if: steps.cached-uv-dependencies.outputs.cache-hit != 'true'
run: uv sync --all-groups
- name: Check dependencies
run: poetry run deptry .
run: uv run deptry .
- name: Run tests
run: poetry run pytest
run: uv run pytest
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
with:
Expand Down
28 changes: 10 additions & 18 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@ name: Release

"on":
push:
branches: ["main", "release/**"]
branches: ["main", "release/**", "dev"]

concurrency:
group: deploy
cancel-in-progress: false # prevent hickups with semantic-release

env:
PYTHON_VERSION_DEFAULT: "3.10.8"
PYTHON_VERSION_DEFAULT: "3.11.11"

jobs:
release:
Expand Down Expand Up @@ -40,9 +40,7 @@ jobs:

- name: Python Semantic Release
id: semrelease
# v9.6.0 is required due to the python v3.12 in the newer version of semantic release action which
# breaks the poetry build command.
uses: python-semantic-release/python-semantic-release@v9.6.0
uses: python-semantic-release/python-semantic-release@v9.16.1
with:
github_token: ${{ steps.trigger-token.outputs.token }}

Expand Down Expand Up @@ -121,25 +119,19 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION_DEFAULT }}
- name: Install and configure Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Load cached venv
id: cached-poetry-dependencies
id: cached-dependencies
uses: actions/cache@v4
with:
path: .venv
key: |
venv-${{ runner.os }}-\
${{ env.PYTHON_VERSION_DEFAULT }}-\
${{ hashFiles('**/poetry.lock') }}
${{ hashFiles('**/uv.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
- name: Install library
run: poetry install --without tests --no-interaction
if: steps.cached-dependencies.outputs.cache-hit != 'true'
run: uv sync --group docs
- name: Publish docs
run: poetry run mkdocs gh-deploy --force
run: uv run mkdocs gh-deploy --force
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,5 @@ site/
.coverage*
wandb/
hail*.log
.python-version
.idea
14 changes: 5 additions & 9 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
default_language_version:
python: python3.10
python: python3.11
ci:
autoupdate_commit_msg: "chore: pre-commit autoupdate"
autofix_commit_msg: "chore: pre-commit auto fixes [...]"
skip: [poetry-lock]
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.7.4
Expand Down Expand Up @@ -46,7 +45,7 @@ repos:
- id: python-check-blanket-noqa

- repo: https://github.com/hadialqattan/pycln
rev: v2.4.0
rev: v2.5.0
hooks:
- id: pycln
args: [--all]
Expand Down Expand Up @@ -101,10 +100,7 @@ repos:
rev: 0.5.9
hooks:
- id: pydoclint

- repo: https://github.com/python-poetry/poetry
rev: "1.8.0"
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.5.22
hooks:
- id: poetry-check
- id: poetry-lock
args: ["--no-update"]
- id: uv-lock
1 change: 0 additions & 1 deletion .python-version

This file was deleted.

25 changes: 9 additions & 16 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,27 +1,20 @@
FROM python:3.10-bullseye

RUN apt-get update \
&& apt-get clean \
&& apt-get install -y openjdk-11-jdk \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get clean && \
apt-get install -y openjdk-11-jdk && \
rm -rf /var/lib/apt/lists/*

ENV POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_IN_PROJECT=1 \
POETRY_VIRTUALENVS_CREATE=1 \
POETRY_CACHE_DIR=/tmp/poetry_cache \
JAVA_HOME=/usr
ENV JAVA_HOME=/usr

RUN pip install poetry==1.7.1
RUN pip install uv
WORKDIR /app

COPY pyproject.toml poetry.lock ./
COPY pyproject.toml uv.lock ./
RUN touch README.md

RUN poetry config installer.max-workers 10
RUN poetry install --without dev,docs,tests --no-root --no-interaction --no-ansi -vvv && rm -rf $POETRY_CACHE_DIR
RUN uv sync

COPY src ./src

RUN poetry install --without dev,docs,tests

ENTRYPOINT ["poetry", "run", "gentropy"]
ENTRYPOINT ["uv", "run", "gentropy"]
Loading
Loading