Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'run_lastz' failure #1610

Open
francicco opened this issue Feb 12, 2025 · 2 comments
Open

'run_lastz' failure #1610

francicco opened this issue Feb 12, 2025 · 2 comments

Comments

@francicco
Copy link

Hi,

I've been trying to align these genomes for a while now. After few days last crashes and does not recover, even if I try to restart cactus.

I'm been trying to align just the subset of genomes where lastz fails and it works.

So I'm wondering what's the problem.

This is the log
cactus.log

Thank you al lot
F

@glennhickey
Copy link
Collaborator

Looks like it's timing out. Please see DataBiosphere/toil#5177, #1554 and
https://github.com/ComparativeGenomicsToolkit/cactus/blob/master/doc/progressive.md#running-on-the-ucsc-prism-cluster

@francicco
Copy link
Author

Sorted that error!!! I'm now getting this one

it says No space left on device but I do have space... I don't understand.

[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'export_hal' kind-export_hal/instance-5l8xtc0x v20
Exit reason: FAILED
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'export_hal' kind-export_hal/instance-5l8xtc0x v21
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] Log from job "kind-export_hal/instance-5l8xtc0x" follows:
=========>
	[2025-02-14T17:11:54+0000] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
	[2025-02-14T17:11:54+0000] [MainThread] [I] [toil] Running Toil version 7.0.0-d569ea5711eb310ffd5703803f7250ebf7c19576 on host bp1-compute019.data.bp.acrc.priv.
	[2025-02-14T17:11:54+0000] [MainThread] [I] [toil.worker] Working on job 'export_hal' kind-export_hal/instance-5l8xtc0x v20
	[2025-02-14T17:11:55+0000] [MainThread] [I] [toil.worker] Loaded body Job('export_hal' kind-export_hal/instance-5l8xtc0x v20) from description 'export_hal' kind-export_hal/instance-5l8xtc0x v20
	[2025-02-14T17:11:56+0000] [MainThread] [I] [cactus.shared.common] Running the command ['halAppendCactusSubtree', '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.c2h', '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.fa', '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.newick', '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal']
	[2025-02-14T17:11:56+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:11:56.306996: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal"
	[2025-02-14T17:13:29+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:13:29+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:13:29.091939: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal" in 92.7827 seconds with job-memory 14.0 Gi
	[2025-02-14T17:13:30+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:13:30.515494: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups EcitonOocer,Lasius"
	[2025-02-14T17:18:04+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:18:04+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:18:04.406740: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups EcitonOocer,Lasius" in 273.8881 seconds with job-memory 14.0 Gi
	[2025-02-14T17:18:05+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:18:05.707282: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Ponerinae"
	[2025-02-14T17:21:53+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:21:53+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:21:53.825318: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Ponerinae" in 228.1159 seconds with job-memory 14.0 Gi
	[2025-02-14T17:21:55+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:21:55.622157: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Dqua,Lnig"
	[2025-02-14T17:26:57+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:26:57+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:26:57.168354: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Dqua,Lnig" in 301.544 seconds with job-memory 14.0 Gi
	[2025-02-14T17:26:58+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:26:58.618077: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Lnig,Lful"
	[2025-02-14T17:31:02+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:31:02+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:31:02.797418: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups Lnig,Lful" in 244.1765 seconds with job-memory 14.0 Gi
	[2025-02-14T17:31:03+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:31:03.944589: Running the command: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups EcitonOocer,Ponerinae"
	[2025-02-14T17:33:39+0000] [MainThread] [W] [toil.lib.humanize] Deprecated toil method.  Please use "toil.lib.conversions.bytes2human()" instead."
	[2025-02-14T17:33:39+0000] [MainThread] [I] [toil-rt] 2025-02-14 17:33:39.108398: Successfully ran: "halAppendCactusSubtree /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.c2h /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.fa /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.newick /tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal --outgroups EcitonOocer,Ponerinae" in 155.1619 seconds with job-memory 14.0 Gi
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files:
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-kfpn2ap7/file-62099fdb49f24198bd60453d781588af/Anc00.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-kfpn2ap7/file-e6ef627edde14ab69d409cc8984b8bfc/Anc00.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Anc00.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-6xu72_hp/file-905e8aab479649f9ad4fb5dab0022e68/Ponerinae.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-6xu72_hp/file-302c49576d0b438c85bad480b357cf8e/Ponerinae.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/Ponerinae.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-majoyn7x/file-ba0324a44b7b43ff8923123322b13698/EcitOocePseuDoliMyrmForm.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-majoyn7x/file-3c2df9519e294a1d8dae0702fca195a6/EcitOocePseuDoliMyrmForm.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitOocePseuDoliMyrmForm.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-8gutec0g/file-478d55e0daef4bcebc6db4c54f3f5acf/OdonHarp.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-8gutec0g/file-81a974c985244c7697d7f81cebcd6720/OdonHarp.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/OdonHarp.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-mk13xwbq/file-f3b6bee37b5e4cf495b4d558b25f3945/EcitonOocer.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-mk13xwbq/file-b6b911b3d71e44a0993d7592541d5d2f/EcitonOocer.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/EcitonOocer.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-_hg9id1s/file-04b8bf85dcf34a40990abd796c7ae43b/PseuDoliMyrmForm.c2h' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.c2h'
	[2025-02-14T17:33:51+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-_hg9id1s/file-638e105035914390b00c4b4735eda7b0/PseuDoliMyrmForm.c2h.fa' to path '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseuDoliMyrmForm.hal.fa'
	[2025-02-14T17:33:51+0000] [MainThread] [C] [toil.worker] Worker crashed with traceback:
	Traceback (most recent call last):
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/worker.py", line 438, in workerScript
	    job._runner(jobGraph=None, jobStore=job_store, fileStore=fileStore, defer=defer)
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/job.py", line 2984, in _runner
	    returnValues = self._run(jobGraph=None, fileStore=fileStore)
	                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/job.py", line 2895, in _run
	    return self.run(fileStore)
	           ^^^^^^^^^^^^^^^^^^^
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/job.py", line 3158, in run
	    rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs)
	             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/cactus/progressive/cactus_progressive.py", line 246, in export_hal
	    job.fileStore.readGlobalFile(results[genome_name][0], sub_hal_path)
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/fileStores/nonCachingFileStore.py", line 145, in readGlobalFile
	    self.jobStore.read_file(fileStoreID, localFilePath, symlink=symlink)
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/jobStores/fileJobStore.py", line 589, in read_file
	    atomic_copy(jobStoreFilePath, local_path, executable=executable)
	  File "/user/work/tk19812/software/cactus-bin-v2.9.3/venv-cactus-v2.9.3/lib/python3.12/site-packages/toil/lib/io.py", line 148, in atomic_copy
	    shutil.copyfile(src_path, dest_path_tmp)
	  File "/software/local/languages/miniforge3/envs/bioconda/lib/python3.12/shutil.py", line 273, in copyfile
	    _fastcopy_sendfile(fsrc, fdst)
	  File "/software/local/languages/miniforge3/envs/bioconda/lib/python3.12/shutil.py", line 164, in _fastcopy_sendfile
	    raise err from None
	  File "/software/local/languages/miniforge3/envs/bioconda/lib/python3.12/shutil.py", line 150, in _fastcopy_sendfile
	    sent = os.sendfile(outfd, infd, offset, blocksize)
	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	OSError: [Errno 28] No space left on device: '/user/work/tk19812/HoneyPotProject/Formicidae.MultiNodeCactus/jobStore/files/for-job/kind-cactus_cons/instance-eygqxs1g/file-a51ef02e3f7249a2b0e2d076595aeddb/PseudomyrmecinaeDolichoderinae.c2h' -> '/tmp/toilwf-b7658dee667457a3a90b80f11f6848ba/9bc4/job/tmplljaglwf/PseudomyrmecinaeDolichoderinae.hal.c2h.fa1ce052-226f-4fdb-afdb-31168579daf3.tmp.c2h'
	
	[2025-02-14T17:33:51+0000] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host bp1-compute019.data.bp.acrc.priv
<=========
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] The batch system left a non-empty file /user/work/tk19812/HoneyPotProject/Formicidae.MultiNodeCactus/toil_c7683400-1c48-4200-9f94-af9bb2679904.1.11780073.err.log:
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] Log from job "kind-export_hal/instance-5l8xtc0x" follows:
=========>
	cpu-bind=MASK - bp1-compute019, task  0  0 [2420416]: mask 0x20000002 set
<=========
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] The batch system left an empty file /user/work/tk19812/HoneyPotProject/Formicidae.MultiNodeCactus/toil_c7683400-1c48-4200-9f94-af9bb2679904.1.11780073.out.log
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] The batch system left a non-empty file /user/work/tk19812/HoneyPotProject/Formicidae.MultiNodeCactus/toil_c7683400-1c48-4200-9f94-af9bb2679904.1.11781884.err.log:
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] Log from job "kind-export_hal/instance-5l8xtc0x" follows:
=========>
	cpu-bind=MASK - bp1-compute019, task  0  0 [2431703]: mask 0x20000002000 set
<=========
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.leader] The batch system left an empty file /user/work/tk19812/HoneyPotProject/Formicidae.MultiNodeCactus/toil_c7683400-1c48-4200-9f94-af9bb2679904.1.11781884.out.log
[2025-02-14T17:33:54+0000] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'export_hal' kind-export_hal/instance-5l8xtc0x v21 with ID kind-export_hal/instance-5l8xtc0x to 1

Cheers
F

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants