Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update code quality configuration #555

Merged
merged 6 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 32 additions & 69 deletions .github/workflows/pytest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,7 @@ jobs:
- "3.12"
- "3.13" # Latest supported by ixmp
gams-version:
# Version used until 2024-07; disabled
# - 25.1.1
# First version including a macOS arm64 distribution
- 43.4.1

# commented: force a specific version of pandas, for e.g. pre-release
# testing
# pandas-version:
# - ""
# - "==2.0.0rc0"
- "43.4.1" # First version including a macOS arm64 distribution

exclude:
# Specific version combinations that are invalid / not to be used
Expand All @@ -55,29 +46,29 @@ jobs:

runs-on: ${{ matrix.os }}
name: ${{ matrix.os }}-py${{ matrix.python-version }}
# commented: use with "pandas-version" in the matrix, above
# name: ${{ matrix.os }}-py${{ matrix.python-version }}-pandas${{ matrix.pandas-version }}

steps:
- uses: actions/checkout@v4
with:
fetch-depth: ${{ env.depth }}
fetch-tags: true

- uses: actions/setup-python@v5
- name: TEMPORARY Work around actions/checkout#2041
run: git fetch --tags

- name: Set up uv, Python
uses: astral-sh/setup-uv@v5
with:
cache-dependency-glob: "**/pyproject.toml"
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: "**/pyproject.toml"

- uses: ts-graphviz/setup-graphviz@v2
# TEMPORARY Work around ts-graphviz/setup-graphviz#630
if: ${{ ! startswith(matrix.os, 'macos-') }}
# Work around ts-graphviz/setup-graphviz#630
if: matrix.os != 'macos-13'

- uses: r-lib/actions/setup-r@v2
id: setup-r
with:
r-version: "4.4.1"
with: { r-version: "4.4.1" }

- name: Cache GAMS installer and R packages
uses: actions/cache@v4
Expand All @@ -95,45 +86,21 @@ jobs:
license: ${{ secrets.GAMS_LICENSE }}

- name: Set RETICULATE_PYTHON
# Use the environment variable set by the setup-python action, above.
run: echo "RETICULATE_PYTHON=$pythonLocation" >> $GITHUB_ENV
# Retrieve the Python executable set up above
run: echo "RETICULATE_PYTHON=$(uv python find)" >> $GITHUB_ENV
shell: bash

- name: Install Python package and dependencies
# [docs] contains [tests], which contains [report,tutorial]
run: |
pip install .[docs]
# commented: use with "pandas-version" in the matrix, above
# pip install --upgrade pandas${{ matrix.pandas-version }}
# TEMPORARY With Python 3.13 pyam-iamc resolves to 1.3.1, which in turn
# limits pint < 0.17. Override. cf. iiasa/ixmp#544
pip install --upgrade pint
- name: Install the package and dependencies
# [docs] requires [tests] which requires [report,tutorial]
run: uv pip install .[docs]

- name: Install R dependencies and tutorial requirements
# Workaround for https://github.com/actions/runner-images/issues/11137
if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' }}
run: |
install.packages(c("remotes", "Rcpp"))
remotes::install_cran(
c("IRkernel", "reticulate"),
dependencies = TRUE,
# force = TRUE,
)
reticulate::py_config()
shell: Rscript {0}
- name: "Install libpng-dev" # for R 'png', required by reticulate
if: startsWith(matrix.os, 'ubuntu-')
run: sudo apt install libpng-dev

- name: Install R dependencies and tutorial requirements
if: ${{ ! (matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12') }}
run: |
install.packages(c("remotes", "Rcpp"))
remotes::install_cran(
c("IRkernel", "reticulate"),
dependencies = TRUE,
# force = TRUE,
)
install.packages(c("IRkernel", "reticulate"))
# commented: for debugging
# print(reticulate::py_config())
Expand All @@ -142,23 +109,20 @@ jobs:
IRkernel::installspec()
shell: Rscript {0}

- name: Run test suite using pytest
- name: Run tests
run: |
pytest ixmp \
uv run --no-sync \
pytest ixmp \
-m "not performance" \
--color=yes -rA --verbose \
--color=yes --durations=20 -rA --verbose \
--cov-report=xml \
--numprocesses=auto --dist=loadgroup
shell: bash

- name: Upload test coverage to Codecov.io
uses: codecov/codecov-action@v5
# FIXME Limit runtime until
# https://github.com/codecov/codecov-action/issues/1316 is resolved
timeout-minutes: 1
continue-on-error: true
with:
token: ${{ secrets.CODECOV_TOKEN }} # required
token: ${{ secrets.CODECOV_TOKEN}}

pre-commit:
name: Code quality
Expand All @@ -167,12 +131,11 @@ jobs:

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }

- name: Force recreation of pre-commit virtual environment for mypy
if: github.event_name == 'schedule'
run: gh cache list -L 999 | cut -f2 | grep pre-commit | xargs -I{} gh cache delete "{}" || true
env: { GH_TOKEN: "${{ github.token }}" }

- uses: pre-commit/action@v3.0.1
- uses: astral-sh/setup-uv@v5
with: { cache-dependency-glob: "**/pyproject.toml" }
- uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit|${{ env.UV_PYTHON }}|${{ hashFiles('.pre-commit-config.yaml') }}
lookup-only: ${{ github.event_name == 'schedule' }} # Set 'true' to recreate cache
- run: uvx pre-commit run --all-files --color=always --show-diff-on-failure
15 changes: 4 additions & 11 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,27 +1,20 @@
repos:
- repo: local
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.14.1
hooks:
- id: mypy
name: mypy
always_run: true
require_serial: true
pass_filenames: false

language: python
entry: bash -c ". ${PRE_COMMIT_MYPY_VENV:-/dev/null}/bin/activate 2>/dev/null; mypy $0 $@"
additional_dependencies:
- mypy >= 1.9.0
- genno
- GitPython
- nbclient
- pandas-stubs
- pytest
- sphinx
- Sphinx
- werkzeug
- xarray
args: ["."]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.3.4
rev: v0.9.1
hooks:
- id: ruff
- id: ruff-format
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ See [`doc/README.rst`](doc/README.rst) for further details.

## License

Copyright © 2017–2024 IIASA Energy, Climate, and Environment (ECE) program
Copyright © 2017–2025 IIASA Energy, Climate, and Environment (ECE) program

`ixmp` is licensed under the Apache License, Version 2.0 (the "License"); you
may not use the files in this repository except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# -- Project information ---------------------------------------------------------------

project = "ixmp"
copyright = "2017–2024, IIASA Energy, Climate, and Environment (ECE) program"
copyright = "2017–%Y, IIASA Energy, Climate, and Environment (ECE) program"
author = "ixmp Developers"


Expand Down
5 changes: 2 additions & 3 deletions ixmp/backend/jdbc.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ def _domain_enum(domain):
return domain_enum.valueOf(domain.upper())
except java.IllegalArgumentException:
domains = ", ".join([d.name().lower() for d in domain_enum.values()])
raise ValueError(f"No such domain: {domain}, " f"existing domains: {domains}")
raise ValueError(f"No such domain: {domain}, existing domains: {domains}")


def _unwrap(v):
Expand Down Expand Up @@ -886,8 +886,7 @@ def clone(
# Raise exceptions for limitations of JDBCBackend
if not isinstance(platform_dest._backend, self.__class__):
raise NotImplementedError( # pragma: no cover
f"Clone between {self.__class__} and"
f"{platform_dest._backend.__class__}"
f"Clone between {self.__class__} and{platform_dest._backend.__class__}"
)
elif platform_dest._backend is not self:
package = s.__class__.__module__.split(".")[0]
Expand Down
7 changes: 3 additions & 4 deletions ixmp/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def report(context, config, key):

if not context:
raise click.UsageError(
"give either --url, --platform or --dbprops " "before command report"
"give either --url, --platform or --dbprops before command report"
)

# Instantiate the Reporter with the Scenario loaded by main()
Expand Down Expand Up @@ -227,8 +227,7 @@ def import_group(context):
"""
if not context or "scen" not in context:
raise click.UsageError(
"give --url, or --platform, --model, and "
"--scenario, before command import"
"give --url, or --platform, --model, and --scenario, before command import"
)


Expand Down Expand Up @@ -407,7 +406,7 @@ def list_scenarios(context, **kwargs):

if not context:
raise click.UsageError(
"give either --url, --platform or --dbprops " "before command list"
"give either --url, --platform or --dbprops before command list"
)

print(
Expand Down
5 changes: 2 additions & 3 deletions ixmp/tests/backend/test_jdbc.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,8 +373,7 @@ def test_verbose_exception(test_mp, exception_verbose_true):

exc_msg = exc_info.value.args[0]
assert (
"There exists no Scenario 'foo|bar' "
"(version: -1) in the database!" in exc_msg
"There exists no Scenario 'foo|bar' (version: -1) in the database!" in exc_msg
)
assert "at.ac.iiasa.ixmp.database.DbDAO.getRunId" in exc_msg
assert "at.ac.iiasa.ixmp.Platform.getScenario" in exc_msg
Expand Down Expand Up @@ -595,7 +594,7 @@ def test_reload_cycle(
mp = ixmp.Platform(**platform_args)

# Load existing Scenario
s0 = ixmp.Scenario(mp, model="foo", scenario=f"bar {i-1}", version=1)
s0 = ixmp.Scenario(mp, model="foo", scenario=f"bar {i - 1}", version=1)

memory_usage(f"pass {i} -- platform instantiated")

Expand Down
8 changes: 4 additions & 4 deletions ixmp/tests/core/test_scenario.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ def test_init_set(self, scen):
# Add set on a locked scenario
with pytest.raises(
RuntimeError,
match="This Scenario cannot be edited" ", do a checkout first!",
match="This Scenario cannot be edited, do a checkout first!",
):
scen.init_set("foo")

Expand Down Expand Up @@ -399,7 +399,7 @@ def test_excel_io(self, scen, scen_empty, tmp_path, caplog):

# With init_items=False, can't be read into an empty Scenario.
# Exception raised is the first index set, alphabetically
with pytest.raises(ValueError, match="no set 'i'; " "try init_items=True"):
with pytest.raises(ValueError, match="no set 'i'; try init_items=True"):
scen_empty.read_excel(tmp_path)

# File can be read with init_items=True
Expand Down Expand Up @@ -441,7 +441,7 @@ def test_excel_io(self, scen, scen_empty, tmp_path, caplog):

# Fails with add_units=False
with pytest.raises(
ValueError, match="The unit 'pounds' does not exist" " in the database!"
ValueError, match="The unit 'pounds' does not exist in the database!"
):
s.read_excel(tmp_path, init_items=True)

Expand Down Expand Up @@ -623,7 +623,7 @@ def test_set(scen_empty) -> None:
scen.add_set("i", ["i9", "extra"], ["i9 comment"])
# Missing element in the index set
with pytest.raises(
ValueError, match="The index set 'i' does not have an " "element 'bar'!"
ValueError, match="The index set 'i' does not have an element 'bar'!"
):
scen.add_set("foo", "bar")

Expand Down
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,9 @@ exclude_also = [
omit = ["ixmp/util/sphinx_linkcode_github.py"]

[tool.mypy]
exclude = [
"build/",
files = [
"doc",
"ixmp",
]

[[tool.mypy.overrides]]
Expand Down
Loading
Loading