Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(iast): fix import loop [backport 2.20] #12129

Open
wants to merge 4 commits into
base: 2.20
Choose a base branch
from

Conversation

github-actions[bot]
Copy link
Contributor

Backport 03e8375 from #12095 to 2.20.

The usage of callonce in this function (that is now used in IAST and SCA) was triggering an import loop:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/utils/cache.py", line 124, in _
    retval, exc = f.__callonce_result__  # type: ignore[attr-defined]
                  ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'function' object has no attribute '__callonce_result__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/telemetry/writer.py", line 622, in periodic
    self._app_dependencies_loaded_event(newly_imported_deps)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/telemetry/writer.py", line 420, in _app_dependencies_loaded_event
    packages = update_imported_dependencies(self._imported_dependencies, newly_imported_deps)
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/telemetry/data.py", line 76, in update_imported_dependencies
    dists = get_module_distribution_versions(module_name)
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/packages.py", line 72, in get_module_distribution_versions
    pkgs = get_package_distributions()
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/utils/cache.py", line 135, in _
    raise exc
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/module.py", line 252, in _find_first_hook
    callable(cond) and cond(module.__name__)
                       ~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/ddtrace/appsec/_iast/_ast/ast_patching.py", line 486, in _should_iast_patch
    if _is_first_party(module_name):
       ~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/ddtrace/appsec/_iast/_ast/ast_patching.py", line 467, in _is_first_party
    _IMPORTLIB_PACKAGES = set(get_package_distributions())
                              ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/utils/cache.py", line 135, in _
    raise exc
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/utils/cache.py", line 127, in _
    retval = f()
  File "/usr/local/lib/python3.13/site-packages/ddtrace/internal/packages.py", line 58, in get_package_distributions
    return importlib_metadata.packages_distributions()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/importlib/metadata/__init__.py", line 1045, in packages_distributions
    for pkg in _top_level_declared(dist) or _top_level_inferred(dist):
                                            ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/importlib/metadata/__init__.py", line 1088, in _top_level_inferred
    opt_names = set(map(_get_toplevel_name, always_iterable(dist.files)))
                                                            ^^^^^^^^^^
  File "/usr/local/lib/python3.13/importlib/metadata/__init__.py", line 534, in files
    make_files(
    ~~~~~~~~~~^
        self._read_files_distinfo()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        or self._read_files_egginfo_installed()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        or self._read_files_egginfo_sources()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/importlib/metadata/_functools.py", line 102, in wrapper
    return func(param, *args, **kwargs)
  File "/usr/local/lib/python3.13/importlib/metadata/__init__.py", line 527, in make_files
    return starmap(make_file, csv.reader(lines))
                              ^^^^^^^^^^
AttributeError: partially initialized module 'csv' from '/usr/local/lib/python3.13/csv.py' has no attribute 'reader' (most likely due to a circular import)

APPSEC-56526

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com>
(cherry picked from commit 03e8375)
@github-actions github-actions bot added the ASM Application Security Monitoring label Jan 28, 2025
@github-actions github-actions bot requested review from a team as code owners January 28, 2025 18:16
@datadog-dd-trace-py-rkomorn
Copy link

datadog-dd-trace-py-rkomorn bot commented Jan 28, 2025

Datadog Report

Branch report: backport-12095-to-2.20
Commit report: e92f863
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1468 Skipped, 5m 7.55s Total duration (35m 41.84s time saved)

@pr-commenter
Copy link

pr-commenter bot commented Jan 28, 2025

Benchmarks

Benchmark execution time: 2025-02-03 08:47:45

Comparing candidate commit e92f863 in PR branch backport-12095-to-2.20 with baseline commit 1b2ec1c in branch 2.20.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.

@gnufede gnufede enabled auto-merge (squash) January 29, 2025 07:34
Copy link
Contributor Author

CODEOWNERS have been resolved as:

releasenotes/notes/fix-asm-package-distributions-import-loop-11693f2a26032774.yaml  @DataDog/apm-python
ddtrace/appsec/_iast/_ast/ast_patching.py                               @DataDog/asm-python
ddtrace/internal/packages.py                                            @DataDog/apm-core-python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ASM Application Security Monitoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants