Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Use env custom file if it exists for tests #3094

Closed
wants to merge 12 commits into from

Conversation

hubertdeng123
Copy link
Member

Tests in relay/snuba likely weren't being properly run with the pre-submit images that were built. This led to an event today where a change led to self-hosted e2e test breakage in relay on master that was not caught on the PR. I believe this should fix that issue. I suspect the custom env file for every commit on a PR wasn't being used for e2e tests that is built here: https://github.com/getsentry/action-self-hosted-e2e-tests/blob/main/action.yml#L64

@hubertdeng123 hubertdeng123 requested a review from azaslavsky May 29, 2024 17:08
Copy link

codecov bot commented May 29, 2024

Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time.

❌ Failed Test Results:

Completed 10 tests with 1 failed, 9 passed and 0 skipped.

View the full list of failed tests
Test Description Failure message
Testsuite:
pytest

Test name:
_integration-test.test_run::test_receive_transaction_events

Envs:
- default
client_login = (<httpx.Client object at 0x7f918c6b8850>, <Response [200 OK]>)

#x1B[0m#x1B[94mdef#x1B[39;49;00m #x1B[92mtest_receive_transaction_events#x1B[39;49;00m(client_login):
client, _ = client_login
#x1B[94mwith#x1B[39;49;00m sentry_sdk.init(
dsn=get_sentry_dsn(client), profiles_sample_rate=#x1B[94m1.0#x1B[39;49;00m, traces_sample_rate=#x1B[94m1.0#x1B[39;49;00m
):

#x1B[94mdef#x1B[39;49;00m #x1B[92mplaceholder_fn#x1B[39;49;00m():
#x1B[96msum#x1B[39;49;00m = #x1B[94m0#x1B[39;49;00m
#x1B[94mfor#x1B[39;49;00m i #x1B[95min#x1B[39;49;00m #x1B[96mrange#x1B[39;49;00m(#x1B[94m5#x1B[39;49;00m):
#x1B[96msum#x1B[39;49;00m += i
time.sleep(#x1B[94m0.25#x1B[39;49;00m)

#x1B[94mwith#x1B[39;49;00m sentry_sdk.start_transaction(op=#x1B[33m"#x1B[39;49;00m#x1B[33mtask#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, name=#x1B[33m"#x1B[39;49;00m#x1B[33mTest Transactions#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m):
placeholder_fn()
poll_for_response(
#x1B[33mf#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m#x1B[33m{#x1B[39;49;00mSENTRY_TEST_HOST#x1B[33m}#x1B[39;49;00m#x1B[.../organizations/sentry/events/&dataset=profiles&field=profile.id&project=1&statsPeriod=1h#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m,
client,
#x1B[94mlambda#x1B[39;49;00m x: #x1B[96mlen#x1B[39;49;00m(json.loads(x)[#x1B[33m"#x1B[39;49;00m#x1B[33mdata#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m]) > #x1B[94m0#x1B[39;49;00m,
)
> poll_for_response(
#x1B[33mf#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m#x1B[33m{#x1B[39;49;00mSENTRY_TEST_HOST#x1B[33m}#x1B[39;49;00m#x1B[.../organizations/sentry/events/&dataset=spansIndexed&field=id&project=1&statsPeriod=1h#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m,
client,
#x1B[94mlambda#x1B[39;49;00m x: #x1B[96mlen#x1B[39;49;00m(json.loads(x)[#x1B[33m"#x1B[39;49;00m#x1B[33mdata#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m]) > #x1B[94m0#x1B[39;49;00m,
)

#x1B[1m#x1B[31m_integration-test/test_run.py#x1B[0m:381:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

request = 'http://localhost:.../organizations/sentry/events/&dataset=spansIndexed&field=id&project=1&statsPeriod=1h'
client = <httpx.Client object at 0x7f918c6b8850>
validator = <function test_receive_transaction_events.<locals>.<lambda> at 0x7f918c53ff40>

#x1B[0m#x1B[94mdef#x1B[39;49;00m #x1B[92mpoll_for_response#x1B[39;49;00m(
request: #x1B[96mstr#x1B[39;49;00m, client: httpx.Client, validator: Callable = #x1B[94mNone#x1B[39;49;00m
) -> httpx.Response:
#x1B[94mfor#x1B[39;49;00m i #x1B[95min#x1B[39;49;00m #x1B[96mrange#x1B[39;49;00m(TIMEOUT_SECONDS):
response = client.get(
request, follow_redirects=#x1B[94mTrue#x1B[39;49;00m, headers={#x1B[33m"#x1B[39;49;00m#x1B[33mReferer#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m: SENTRY_TEST_HOST}
)
#x1B[94mif#x1B[39;49;00m response.status_code == #x1B[94m200#x1B[39;49;00m:
#x1B[94mif#x1B[39;49;00m validator #x1B[95mis#x1B[39;49;00m #x1B[94mNone#x1B[39;49;00m #x1B[95mor#x1B[39;49;00m validator(response.text):
#x1B[94mbreak#x1B[39;49;00m
time.sleep(#x1B[94m1#x1B[39;49;00m)
#x1B[94melse#x1B[39;49;00m:
> #x1B[94mraise#x1B[39;49;00m #x1B[96mAssertionError#x1B[39;49;00m(
#x1B[33m"#x1B[39;49;00m#x1B[33mtimeout waiting for response status code 200 or valid data#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m
)
#x1B[1m#x1B[31mE AssertionError: timeout waiting for response status code 200 or valid data#x1B[0m

#x1B[1m#x1B[31m_integration-test/test_run.py#x1B[0m:40: AssertionError

@hubertdeng123
Copy link
Member Author

CI failing as it is waiting for #3092

@hubertdeng123 hubertdeng123 marked this pull request as draft May 29, 2024 18:56
@github-actions github-actions bot locked and limited conversation to collaborators Jun 30, 2024
@hubertdeng123 hubertdeng123 deleted the hubertdeng123/use-env-custom-when-exists branch November 26, 2024 00:15
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

1 participant