Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update-python-resources: add option to ignore errors #19252

Merged

Conversation

gromgit
Copy link
Contributor

@gromgit gromgit commented Feb 6, 2025

This is particularly useful for third-party Python formulae that have a ton of resources, not all of which may adhere to homebrew/core's strict policies. See #19240 for context.

I've also added logic that ignores --ignore-errors on homebrew/core, although I personally think this new behavior is also useful for mainline formula creation.

Before: error out on a single non-conforming resource, zero resource blocks added to formula, scary stacktrace (in debug mode).

After: all conforming resources added, all non-conforming resources identified in comments, error message at end, brew exits non-zero without scary stacktrace:-

% brew update-python-resources --ignore-errors gromgit/test/auto-coder || echo OOPS
==> Retrieving PyPI dependencies for "auto-coder==0.1.243"...
==> Retrieving PyPI dependencies for excluded ""...
==> Getting PyPI info for "aiohappyeyeballs==2.4.4"
[200+ resource lines elided]
==> Getting PyPI info for "zhipuai==2.1.5.20250106"
==> Updating resource blocks
Error: Unable to resolve some dependencies. Please check /opt/homebrew/Library/Taps/gromgit/homebrew-test/Formula/auto-coder.rb for RESOURCE-ERROR comments.
OOPS

% brew cat gromgit/test/auto-coder | ggrep -C10 RESOURCE-ERROR
  license "Apache-2.0"

  depends_on "python@3.11"

  # Additional dependency
  # resource "" do
  #   url ""
  #   sha256 ""
  # end

  # RESOURCE-ERROR: Unable to resolve "azure-cognitiveservices-speech==1.42.0" (no suitable source distribution on PyPI)
  # RESOURCE-ERROR: Unable to resolve "ray==2.42.0" (no suitable source distribution on PyPI)

  resource "aiohappyeyeballs" do
    url "https://files.pythonhosted.org/packages/7f/55/e4373e888fdacb15563ef6fa9fa8c8252476ea071e96fb46defac9f18bf2/aiohappyeyeballs-2.4.4.tar.gz"
    sha256 "5fdd7d87889c63183afc18ce9271f9b0a7d32c2303e394468dd45d514a757745"
  end

  resource "aiohttp" do
    url "https://files.pythonhosted.org/packages/37/4b/952d49c73084fb790cb5c6ead50848c8e96b4980ad806cf4d2ad341eaa03/aiohttp-3.11.12.tar.gz"
    sha256 "7603ca26d75b1b86160ce1bbe2787a0b706e592af5b2504e12caa88a217767b0"
  end
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

This is particularly useful for third-party Python formulae that have a ton of resources, not all of which may adhere to homebrew/core's strict policies. See Homebrew#19240 for context.

I've also added logic that ignores `--ignore-errors` on `homebrew/core`, although I personally think this new behavior is also useful for mainline formula creation.

Before: error out on a single non-conforming resource, zero resource blocks added to formula, scary stacktrace.

After: all conforming resources added, all non-conforming resources identified in comments, error message at end, `brew` exits non-zero without scary stacktrace:-
```
% brew update-python-resources --ignore-errors gromgit/test/auto-coder || echo OOPS
==> Retrieving PyPI dependencies for "auto-coder==0.1.243"...
==> Retrieving PyPI dependencies for excluded ""...
==> Getting PyPI info for "aiohappyeyeballs==2.4.4"
[200+ resource lines elided]
==> Getting PyPI info for "zhipuai==2.1.5.20250106"
==> Updating resource blocks
Error: Unable to resolve some dependencies. Please check /opt/homebrew/Library/Taps/gromgit/homebrew-test/Formula/auto-coder.rb for RESOURCE-ERROR comments.
OOPS

% brew cat gromgit/test/auto-coder | ggrep -C10 RESOURCE-ERROR
  license "Apache-2.0"

  depends_on "python@3.11"

  # Additional dependency
  # resource "" do
  #   url ""
  #   sha256 ""
  # end

  # RESOURCE-ERROR: Unable to resolve "azure-cognitiveservices-speech==1.42.0" (no suitable source distribution on PyPI)
  # RESOURCE-ERROR: Unable to resolve "ray==2.42.0" (no suitable source distribution on PyPI)

  resource "aiohappyeyeballs" do
    url "https://files.pythonhosted.org/packages/7f/55/e4373e888fdacb15563ef6fa9fa8c8252476ea071e96fb46defac9f18bf2/aiohappyeyeballs-2.4.4.tar.gz"
    sha256 "5fdd7d87889c63183afc18ce9271f9b0a7d32c2303e394468dd45d514a757745"
  end

  resource "aiohttp" do
    url "https://files.pythonhosted.org/packages/37/4b/952d49c73084fb790cb5c6ead50848c8e96b4980ad806cf4d2ad341eaa03/aiohttp-3.11.12.tar.gz"
    sha256 "7603ca26d75b1b86160ce1bbe2787a0b706e592af5b2504e12caa88a217767b0"
  end
```
Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thanks @gromgit!

Only thing before merging: I'd almost be tempted to say this should be default behaviour? Thoughts @cho-m and @woodruffw?

@woodruffw
Copy link
Member

LGTM overall! My 0.02c is that this should be non-default since it results in a non-functional formula definition (well-formed, but won't run correctly due to missing deps that need to be fixed manually).

OTOH we could make it the default, but add an audit rule that detects RESOURCE-ERROR: and fails the CI. No strong opinions either way 🙂

@MikeMcQuaid MikeMcQuaid added this pull request to the merge queue Feb 6, 2025
Merged via the queue into Homebrew:master with commit bd7a3cc Feb 6, 2025
28 checks passed
@gromgit gromgit deleted the update-python-resources/ignore-errors branch February 7, 2025 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants