From c2fad3dd88e46c659abcce4d9b7628119b761a6b Mon Sep 17 00:00:00 2001 From: Dongjoon Hyun Date: Wed, 3 Jan 2024 06:43:46 -0800 Subject: [PATCH] ORC-1568: Use `readDiskRanges` if `orc.use.zerocopy` is enabled ### What changes were proposed in this pull request? This PR aims to use legacy `readDiskRanges` method if `orc.use.zerocopy` is enabled. ### Why are the changes needed? Since `orc.use.zerocopy` is disabled by default, Apache ORC will use `Vectored IO` still after this PR. The main background of this PR is that we had better keep the original behavior until Apache ORC take advantages of new `Vectored IO` API with ZeroCopy code path. We can improve later at Apache ORC 2.1.x. Note that `ZeroCopy` reader is known to be used by `Apache Hive LLAP` (`LlapOrcCacheLoader`, `LlapRecordReaderUtil`, `OrcEncodedDataReader`) like the following. However, `Apache Hive 4.0.0` is incompatible with Apache ORC 2.0.0 because it still requires Java 8 minimum support. - https://github.com/apache/hive/blob/b33b3d3454cc9c65a1879c68679f33f207f21c0e/llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/LlapOrcCacheLoader.java#L92 ```java boolean useZeroCopy = (daemonConf != null) && OrcConf.USE_ZEROCOPY.getBoolean(daemonConf); ``` ### How was this patch tested? Pass the CIs. Closes #1723 from dongjoon-hyun/ORC-1568. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun --- .../src/java/org/apache/orc/impl/RecordReaderUtils.java | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java b/java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java index eae78858cb..717e497377 100644 --- a/java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java +++ b/java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java @@ -107,7 +107,12 @@ public OrcProto.StripeFooter readStripeFooter(StripeInformation stripe) throws I public BufferChunkList readFileData(BufferChunkList range, boolean doForceDirect ) throws IOException { - RecordReaderUtils.readDiskRangesVectored(file, range, doForceDirect); + if (zcr == null) { + RecordReaderUtils.readDiskRangesVectored(file, range, doForceDirect); + } else { + RecordReaderUtils.readDiskRanges(file, zcr, range, doForceDirect, + minSeekSize, minSeekSizeTolerance); + } return range; }