Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolved race condition during Redis backend teardown #418

Merged
merged 1 commit into from
Sep 20, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 21 additions & 7 deletions src/pytest_celery/vendors/redis/backend/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,30 @@

import gc

from celery.result import AsyncResult

Check warning on line 11 in src/pytest_celery/vendors/redis/backend/api.py

View check run for this annotation

Codecov / codecov/patch

src/pytest_celery/vendors/redis/backend/api.py#L11

Added line #L11 was not covered by tests

from pytest_celery.api.backend import CeleryTestBackend


class RedisTestBackend(CeleryTestBackend):
def teardown(self) -> None:
# When a test that has a AsyncResult object is finished
# there's a race condition between the AsyncResult object
# and the Redis container. The AsyncResult object tries
# to release the connection but the Redis container has already
# exited. This causes a warning to be logged. To avoid this
# warning to our best effort we force a garbage collection here.
gc.collect(1)
"""When a test that has a AsyncResult object is finished there's a race
condition between the AsyncResult object and the Redis container.

The AsyncResult object tries to release the connection but the
Redis container has already exited.
"""
# First, force a garbage collection to clean up unreachable objects
gc.collect()

# Next, find all live AsyncResult objects and clean them up
async_results = [obj for obj in gc.get_objects() if isinstance(obj, AsyncResult)]

for async_result in async_results:
try:

Check warning on line 31 in src/pytest_celery/vendors/redis/backend/api.py

View check run for this annotation

Codecov / codecov/patch

src/pytest_celery/vendors/redis/backend/api.py#L31

Added line #L31 was not covered by tests
# Remove the backend reference to prevent interaction with Redis
async_result.backend = None
except Exception:
pass

Check warning on line 35 in src/pytest_celery/vendors/redis/backend/api.py

View check run for this annotation

Codecov / codecov/patch

src/pytest_celery/vendors/redis/backend/api.py#L33-L35

Added lines #L33 - L35 were not covered by tests

super().teardown()