Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build Issue Summary for 2023-12-04 #3560

Closed
github-actions bot opened this issue Dec 4, 2023 · 10 comments
Closed

Build Issue Summary for 2023-12-04 #3560

github-actions bot opened this issue Dec 4, 2023 · 10 comments
Assignees
@adamfarley
Labels
aarch Issues that affect or relate to the aarch ARCHITECTURE aix Issues that affect or relate to the AIX OS alpine-linux Issues that affect or relate to the Alpine LINUX OS arm Issues that affect or relate to the ARM OS macos Issues that affect or relate to the MAC OS weekly-build-triage windows Issues that affect or relate to the WINDOWS OS

Comments

@github-actions
Copy link

github-actions bot commented Dec 4, 2023 •
edited by adamfarley
Loading

Date Day JDK Version Pipeline Pass/Build Fail/Test Fail Pass/Preventable/Unpreventable
2023/12/02 Saturday JDK8 Link 9/0/2 9/0/2
2023/12/02 Saturday JDK11 Link 8/0/3 8/0/3
2023/12/03 Sunday JDK17 Link 8/0/3 8/0/3
2023/12/03 Sunday JDK21 Link 8/0/3 8/1/2
2023/11/30 Thursday JDK22 Link 8/0/3 8/2/1

Failed Builds

End of list
@adamfarley adamfarley self-assigned this Dec 4, 2023
@github-actions github-actions bot added aarch Issues that affect or relate to the aarch ARCHITECTURE aix Issues that affect or relate to the AIX OS alpine-linux Issues that affect or relate to the Alpine LINUX OS arm Issues that affect or relate to the ARM OS macos Issues that affect or relate to the MAC OS windows Issues that affect or relate to the WINDOWS OS labels Dec 4, 2023
@adamfarley
Copy link
Contributor

Note: Mac's excessive output may be due to an Xcode version change. Exploring that here.

E.g. https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-mac-aarch64-temurin/293/

@adamfarley
Copy link
Contributor

adamfarley commented Dec 4, 2023 •
edited
Loading

PROCESSCATCH went rogue and killed every job at 17:58:38, even if they were part of another job.

https://ci.adoptium.net/job/Test_openjdk11_hs_sanity.external_x86-64_linux/613/consoleFull

Killed examples:

Update: Issue resolved by removing a misplaced tag from the machine hosting all the affected containers.

TLDR: If ProcessCatch spots a problem while we're running a test on a host machine directly (not inside a container), it'll kill everything it can find; this includes anything running in containers.

@adamfarley
Copy link
Contributor

Two tests failed because they couldn't download renaissance.jar for performance runs, as part of (or just after) the liberty setup.

URL seems fine now, so assuming it's a temporary networking/server issue until future failures prove otherwise.

@adamfarley
Copy link
Contributor

adamfarley commented Dec 5, 2023 •
edited
Loading 

Exception: org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for test-docker-centos8-armv8-1; test-docker-centos8-armv8-1 was marked offline: Connection was broken

Network bugs infra issue.

Exception: org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for test-docker-ubuntu1804-armv8l-4; test-docker-ubuntu1804-armv8l-4 was marked offline: Connection was broken

Network bugs infra issue.

https://ci.adoptium.net/job/Test_openjdk21_hs_sanity.perf_x86-64_linux/141/console
https://ci.adoptium.net/job/Test_openjdk21_hs_extended.perf_x86-64_linux/29/console
https://ci.adoptium.net/job/Test_openjdk17_hs_extended.perf_x86-64_linux/147/console
https://ci.adoptium.net/job/Test_openjdk22_hs_sanity.perf_x86-64_linux/25/
https://ci.adoptium.net/job/Test_openjdk22_hs_extended.perf_x86-64_linux/15/

java.io.IOException: No space left on device

test-equinix_esxi-ubuntu2204-x64-1 issue is here.

https://ci.adoptium.net/job/Test_openjdk17_hs_sanity.system_x86-64_linux/383/console

12:17:54  make[1]: *** [compile.mk:45: compile] Terminated
12:17:54  make: *** [makefile:67: compile] Terminated
12:17:54  Terminated

Another PROCESSCATCH issue?

@adamfarley
Copy link
Contributor

adamfarley commented Dec 5, 2023 •
edited
Loading

https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_s390x_linux_testList_1/24/consoleFull
https://ci.adoptium.net/job/Test_openjdk22_hs_extended.openjdk_s390x_linux_testList_1/7/consoleFull

java.nio.file.FileSystemException: /proc/sys/fs/binfmt_misc: No such device

This Red Hat site says that binfmt_misc is "is a kernel feature which allows invoking almost every program by simply typing its name in the shell.".
Mossing kernal pieces seems bad. Will raise an infra issue. - Update, issue raised.

@adamfarley
Copy link
Contributor

adamfarley commented Dec 5, 2023 •
edited
Loading

https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_x86-64_mac_testList_1/31/console

Cannot contact test-orka-macos14-x64-96vzx: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel

Will note in the infra networking issue. Update: done.

@adamfarley
Copy link
Contributor

https://ci.adoptium.net/job/Test_openjdk21_hs_special.functional_x86-64_linux/27/console

23:47:08  make[1]: *** [settings.mk:356: special.functional-..] Terminated
23:47:08  make: *** [makefile:54: _special.functional] Terminated
23:47:08  Terminated
23:47:08  make[2]: *** [/home/jenkins/workspace/Test_openjdk21_hs_special.functional_x86-64_linux/aqa-tests/TKG/../TKG/settings.mk:356: special.functional-functional] Terminated
23:47:08  make[3]: *** [/home/jenkins/workspace/Test_openjdk21_hs_special.functional_x86-64_linux/aqa-tests/TKG/../TKG/settings.mk:356: special.functional-MBCS_Tests] Terminated
23:47:08  make[4]: *** [/home/jenkins/workspace/Test_openjdk21_hs_special.functional_x86-64_linux/aqa-tests/TKG/../TKG/settings.mk:356: special.functional-annotation] Terminated
23:47:08  make[5]: *** [autoGen.mk:59: MBCS_Tests_annotation_ko_KR_linux_0] Terminated

Another PROCESSCATCH issue?

@adamfarley
Copy link
Contributor

adamfarley commented Dec 5, 2023 •
edited
Loading

https://ci.adoptium.net/job/Test_openjdk22_hs_extended.openjdk_s390x_linux_testList_0/7/consoleFull

Tests ran very slow and were aborted due to timeout. Apparently this is par for the course fort this test set. Past timeouts occurred on:

And the sole pass happened on test-marist-rhel8-s390x-2.

Issue raised: adoptium/ci-jenkins-pipelines#852

@adamfarley
Copy link
Contributor

https://ci.adoptium.net/job/Test_openjdk22_hs_extended.openjdk_s390x_linux_testList_2/7/consoleFull

Many, many lines of this:
20:01:44  java.nio.channels.IllegalBlockingModeException, sleeping ...
20:01:44  
20:01:44  TEST RESULT: Error. Agent communication error: java.io.EOFException; check console log for any additional details
20:01:44  --------------------------------------------------
20:01:44  -----------------------------------
20:01:44  jdk_nio_1_FAILED

@adamfarley
Copy link
Contributor

https://ci.adoptium.net/job/Test_openjdk22_hs_extended.openjdk_ppc64_aix_testList_2/13/consoleFull

20:33:29  Cancelling nested steps due to timeout
20:33:29  Sending interrupt signal to process

@adamfarley adamfarley mentioned this issue Dec 5, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adamfarley adamfarley

Labels
aarch Issues that affect or relate to the aarch ARCHITECTURE aix Issues that affect or relate to the AIX OS alpine-linux Issues that affect or relate to the Alpine LINUX OS arm Issues that affect or relate to the ARM OS macos Issues that affect or relate to the MAC OS weekly-build-triage windows Issues that affect or relate to the WINDOWS OS
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@adamfarley