Use XZ for LZMA compression in CRAM

[athos]

Currently, CRAM files generated by the CRAM writer containing LZMA-compressed blocks cannot be read correctly by samtools, resulting in an error.

Repro

(require '[cljam.io.cram :as cram])

(with-open [r (cram/reader "test-resources/cram/medium.cram"
                           {:reference "hg19.fa"})
            w (cram/writer "medium_lzma.cram"
                           {:reference "hg19.fa"
                            :tag-compressor-overrides (constantly :lzma)})]
  (cram/write-header w (cram/read-header r))
  (cram/write-alignments w (cram/read-alignments r) (cram/read-header r)))

$ samtools view -T hg19.fa medium_lzma.cram
[E::lzma_mem_inflate] LZMA decode failure (error 7)
[E::cram_next_slice] Failure to decode slice
samtools view: error reading file "medium_lzma.cram"
$

Cause

According to the CRAM specification, the LZMA compression method used in CRAM is actually based on the XZ format:

CRAM uses the xz Stream format to encapsulate this algorithm, as defined in https://tukaani.org/xz/xz-file-format.txt.
https://github.com/samtools/hts-specs/blob/ebebbc8c2910ef2d4c5e7119c6f9ffac3bb6a0cb/CRAMv3.tex#L2503

In fact, htsjdk uses XZCompressorOutputStream/XZCompressorInputStream from commons-compress when the LZMA method is specified.

On the other hand, the current cljam CRAM writer uses LZMACompressorOutputStream for LZMA compression. This mismatch causes the generated CRAM files to be unreadable by samtools or htsjdk.

Change

The PR replaces the use of LZMACompressorOutputStream/LZMACompressorInputStream with XZCompressorOutputStream/XZCompressorInputStream for LZMA compression and decompression.

I have confirmed that with this change, CRAM files generated by the CRAM writer containing LZMA-compressed blocks can be successfully read by samtools without errors.

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 89.72%. Comparing base (b0b8cb4) to head (421df41).

Files	Patch %	Lines
src/cljam/io/cram/decode/structure.clj	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #321   +/-   ##
=======================================
  Coverage   89.72%   89.72%           
=======================================
  Files         101      101           
  Lines        9129     9129           
  Branches      480      480           
=======================================
  Hits         8191     8191           
  Misses        458      458           
  Partials      480      480           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[matsutomo81]

Thank you for your work on this. 🙏
LGTM 👍

[alumi]

Thank you for the quick fix! 👍

Since I was not very familiar with the XZ and LZMA formats, I did some research.
XZ is a format where the data block is enclosed by a header that starts with fd 37 7a 58 5a 00 and a footer that ends with 59 5a.
LZMA is a format with a header that begins with 5d followed by the compression level (00 00 80 00).
I confirmed that the CRAM file outputted by the modified code indeed has the XZ structure and can be read by samtools.

[athos]

Thank you for reviewing the change and clarifying the difference between those formats! That's helpful as I haven't dug that deep into it.