[Segment Replication] Fix flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT #10721

Rishikesh1159 · 2023-10-18T20:39:24Z

Description

This PR fixes flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT by moving index creation out of setup.

Related Issues

Resolves #9499

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

github-actions · 2023-10-18T20:47:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28327/
CommitID: 10b94f1
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

github-actions · 2023-10-18T21:17:39Z

Compatibility status:

Checks if related components are compatible with change 7850dc8

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/performance-analyzer.git]

github-actions · 2023-10-19T13:13:21Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28434/
CommitID: 5b45a48
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

gbbafna · 2023-10-19T14:24:27Z

* What went wrong:
Execution failed for task ':distribution:bwc:staged:buildBwcLinuxTar'.
> Building 2.11.0 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/staged/build/bwc/checkout-2.11/distribution/archives/linux-tar/build/distributions/opensearch-min-2.11.0-SNAPSHOT-linux-x64.tar.gz

mch2 · 2023-10-19T15:57:01Z

@Rishikesh1159 How was createIndex in setup causing failures?

Rishikesh1159 · 2023-10-19T16:42:12Z

@Rishikesh1159 How was createIndex in setup causing failures?

@mch2 I was not able to repro this but from the logs I see errors might be happening because of index not being closed, few errors from logs seems like they are affect of previous test's operation. I don't have conclusive evidence that this would resolve as it is not reproducible so added test logging to catch this if it happens next time.

mch2 · 2023-10-19T20:51:26Z

@Rishikesh1159 How was createIndex in setup causing failures?

@mch2 I was not able to repro this but from the logs I see errors might be happening because of index not being closed, few errors from logs seems like they are affect of previous test's operation. I don't have conclusive evidence that this would resolve as it is not reproducible so added test logging to catch this if it happens next time.

This doesn't make much sense to me, @Before is executed before each test, and with suite scope the indices are all wiped in an @After test.

github-actions · 2023-10-19T21:19:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/28497/
CommitID: 0e62cbc

codecov · 2023-10-19T21:21:43Z

Codecov Report

Attention: 165 lines in your changes are missing coverage. Please review.

Comparison is base (36bd674) 71.27% compared to head (7850dc8) 71.44%.
Report is 7 commits behind head on main.

Files	Patch %	Lines
...org/opensearch/index/mapper/NumberFieldMapper.java	43.62%	61 Missing and 23 partials ⚠️
.../document/SortedUnsignedLongDocValuesSetQuery.java	15.09%	43 Missing and 2 partials ⚠️
.../main/java/org/apache/lucene/util/LongHashSet.java	39.62%	21 Missing and 11 partials ⚠️
...ch/common/xcontent/JsonToStringXContentParser.java	81.25%	1 Missing and 2 partials ⚠️
.../org/opensearch/http/netty4/Netty4HttpChannel.java	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10721      +/-   ##
============================================
+ Coverage     71.27%   71.44%   +0.17%     
- Complexity    59176    59316     +140     
============================================
  Files          4909     4911       +2     
  Lines        278429   278666     +237     
  Branches      40460    40528      +68     
============================================
+ Hits         198443   199095     +652     
+ Misses        63528    63018     -510     
- Partials      16458    16553      +95

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2023-10-20T16:56:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28603/
CommitID: e39b853
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T18:25:49Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28617/
CommitID: e39b853
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-23T04:19:47Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28751/
CommitID: e39b853
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

peternied · 2023-11-02T22:47:27Z

...c/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationSuiteIT.java

 import org.junit.Before;

 @OpenSearchIntegTestCase.ClusterScope(scope = OpenSearchIntegTestCase.Scope.SUITE, minNumDataNodes = 2)
+@TestLogging(reason = "Getting trace logs from replication package", value = "org.opensearch.indices.replication:TRACE")


I am happy to merge a change that adds more logging so we can better root cause a source of failure; however, I don't think the others changes should be merged.

+1 - @Rishikesh1159 mind making this update and we can add the test logging?

opensearch-trigger-bot · 2023-12-03T15:19:21Z

This PR is stalled because it has been open for 30 days with no activity.

github-actions · 2024-01-05T00:25:21Z

✅ Gradle check result for 7850dc8: SUCCESS

mch2 · 2024-01-29T18:35:19Z

closing this in favor of #11977

move index creation out of setup.

10b94f1

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

github-actions bot added bug Something isn't working flaky-test Random test failure that succeeds on second run Indexing:Replication Issues and PRs related to core replication framework eg segrep v2.11.0 Issues and PRs related to version 2.11.0 labels Oct 18, 2023

Adding test logging.

5b41cd0

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

Poojita-Raj approved these changes Oct 18, 2023

View reviewed changes

kotwanikunal approved these changes Oct 18, 2023

View reviewed changes

Merge branch 'opensearch-project:main' into fix-9499

5b45a48

Rishikesh1159 requested a review from abbashus as a code owner October 19, 2023 12:51

RamakrishnaChilaka approved these changes Oct 19, 2023

View reviewed changes

Merge branch 'opensearch-project:main' into fix-9499

0e62cbc

Merge branch 'opensearch-project:main' into fix-9499

e39b853

peternied self-assigned this Nov 2, 2023

peternied reviewed Nov 2, 2023

View reviewed changes

opensearch-trigger-bot bot added the stalled Issues that have stalled label Dec 3, 2023

Merge branch 'opensearch-project:main' into fix-9499

7850dc8

Rishikesh1159 requested review from adnapibar and tlfeng as code owners January 4, 2024 23:34

opensearch-trigger-bot bot removed the stalled Issues that have stalled label Jan 5, 2024

davidlago unassigned peternied Jan 16, 2024

mch2 closed this Jan 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Segment Replication] Fix flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT #10721

[Segment Replication] Fix flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT #10721

Rishikesh1159 commented Oct 18, 2023

github-actions bot commented Oct 18, 2023

github-actions bot commented Oct 18, 2023 •

edited

Loading

github-actions bot commented Oct 19, 2023

gbbafna commented Oct 19, 2023

mch2 commented Oct 19, 2023

Rishikesh1159 commented Oct 19, 2023 •

edited

Loading

mch2 commented Oct 19, 2023

github-actions bot commented Oct 19, 2023

codecov bot commented Oct 19, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 23, 2023

peternied Nov 2, 2023

mch2 Jan 4, 2024

opensearch-trigger-bot bot commented Dec 3, 2023

github-actions bot commented Jan 5, 2024

mch2 commented Jan 29, 2024

[Segment Replication] Fix flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT #10721

[Segment Replication] Fix flaky test org.opensearch.indices.replication.SegmentReplicationSuiteIT #10721

Conversation

Rishikesh1159 commented Oct 18, 2023

Description

Related Issues

Check List

github-actions bot commented Oct 18, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 18, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Oct 19, 2023

Gradle Check (Jenkins) Run Completed with:

gbbafna commented Oct 19, 2023

mch2 commented Oct 19, 2023

Rishikesh1159 commented Oct 19, 2023 • edited Loading

mch2 commented Oct 19, 2023

github-actions bot commented Oct 19, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Oct 19, 2023 • edited Loading

Codecov Report

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 23, 2023

Gradle Check (Jenkins) Run Completed with:

peternied Nov 2, 2023

Choose a reason for hiding this comment

mch2 Jan 4, 2024

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Dec 3, 2023

github-actions bot commented Jan 5, 2024

mch2 commented Jan 29, 2024

github-actions bot commented Oct 18, 2023 •

edited

Loading

Rishikesh1159 commented Oct 19, 2023 •

edited

Loading

codecov bot commented Oct 19, 2023 •

edited

Loading