Error with running bbmap.sh during the stitched-contig chimera test for some genes.

[Oulo]

Kindly assist. I don't know what I am doing wrong. hybpiper assembly keeps getting stuck for some genes in a few sequences. Other sequences run succesfully. Here is an example of a log file showing the error.

2023-03-08 18:44:05,284 - assemble.py - 6559 - exonerate - DEBUG - prefix is: 6559/ERR7620880
2023-03-08 18:44:05,285 - assemble.py - 6559 - exonerate - DEBUG - perform_stitched_contig_chimera_test is: True
2023-03-08 18:44:05,285 - assemble.py - 6559 - exonerate - DEBUG - path_to_interleaved_fasta is: 6559/6559_interleaved.fasta
2023-03-08 18:44:05,285 - exonerate_hits.py - 6559 - initial_exonerate - DEBUG - Exonerate command is: exonerate -m protein2genome --showalignment yes --showvulgar no -V 0 --refine full --ryo ">%ti,%qi,%qab,%qae,%pi,(%tS),%tab,%tae\n%tcs\n" 6559/6559_target.fasta 6559/6559_contigs.fasta > 6559/ERR7620880/exonerate_results.fasta
2023-03-08 18:44:05,836 - exonerate_hits.py - 6559 - initial_exonerate - DEBUG - Exonerate ran with --refine
2023-03-08 18:44:05,841 - exonerate_hits.py - 6559 - parse_exonerate_and_get_stitched_contig - DEBUG - no_stitched_contig is: False
2023-03-08 18:44:05,842 - exonerate_hits.py - 6559 - _parse_searchio_object - DEBUG - Number of HSPs before filtering via percent_similarity: 4
2023-03-08 18:44:05,842 - exonerate_hits.py - 6559 - _parse_searchio_object - DEBUG - Number of HSPs after filtering via percent_similarity: 4
2023-03-08 18:44:07,329 - exonerate_hits.py - 6559 - _stitched_contig_chimera_warning - INFO -
[ERROR]: Running bbmap.sh during the stitched-contig chimera test failed for gene 6559.
No stitched-contig chimera test will be performed. See the *.log file in the
sample directory for error details.
2023-03-08 18:44:07,329 - exonerate_hits.py - 6559 - _stitched_contig_chimera_warning - DEBUG - bbmap_command FAILED. Output is: Command 'bbmap.sh -Xmx1000m -t=1 ref=6559/ERR7620880/chimera_test_stitched_contig.fasta in=6559/6559_interleaved.fasta out=6559/ERR7620880/chimera_test_stitched_contig.sam interleaved=t pairedonly=t mappedonly=t maxindel=0 strictmaxindel=t nodisk=t subfilter=7 ambiguous=toss' returned non-zero exit status 1.
2023-03-08 18:44:07,329 - exonerate_hits.py - 6559 - _stitched_contig_chimera_warning - DEBUG - bbmap_command stdout is: [0.150s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Error occurred during initialization of VM
Cannot create VM thread. Out of system resources.

2023-03-08 18:44:07,329 - exonerate_hits.py - 6559 - _stitched_contig_chimera_warning - DEBUG - bbmap_command stderr is: /home/millicent/anaconda3/envs/hybpiper/bin/bbmap.sh: fork: retry: No child processes
java -ea -Xmx1000m -Xms1000m -cp /home/millicent/anaconda3/envs/hybpiper/opt/bbmap-38.96-1/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 -Xmx1000m -t=1 ref=6559/ERR7620880/chimera_test_stitched_contig.fasta in=6559/6559_interleaved.fasta out=6559/ERR7620880/chimera_test_stitched_contig.sam interleaved=t pairedonly=t mappedonly=t maxindel=0 strictmaxindel=t nodisk=t subfilter=7 ambiguous=toss

:

[chrisjackson-pellicle]

Hi @Oulo,

Looks like an error thrown by JAVA when running bbmap.sh. Can you upload your full log file here, please? Also, you could try reducing the number of threads you're running HybPiper with (parameter --cpu, see https://github.com/mossmatters/HybPiper/wiki/Full-pipeline-parameters#10-hybpiper-assemble), and/or the memory used by bbmap.sh (parameter --bbmap_memory).

Cheers,

Chris

[Oulo]

logfile.txt
here is the entire logfile after another trial

[chrisjackson-pellicle]

Hi @Oulo,

A few things:

1. In the log file you uploaded above, you've specified --bbmap_memory 5, which is only 5 MB of RAM. This leads to the bbbmap.sh error ("This program ran out of memory", etc) you're seeing in the log file. This error is different to the one in the log file fragment you posted in your first comment above. Try increasing the --bbmap_memory value from the default of 1000 (i.e. 1GB RAM) to 2000, while keeping the --cpu value low.

2. In your opening comment on this issue, you said that HybPiper was "getting stuck for some genes". However, in the full log file you uploaded I see that HybPiper finished successfully, and simply skipped the stitched-contig chimera test when it couldn't run bbmap.sh (see the log message " Running bbmap.sh during the stitched-contig chimera test failed for gene 5716", etc). Is this also what happened with the HybPiper run from which you posted a log fragment above?

Cheers,

Chris

[Oulo]

Sorry about that, here is the logfile for what I meant in the opening comment
logfile.txt

[chrisjackson-pellicle]

I think that's the same log you uploaded previously.