Skip to content

Commit

Permalink
apacheGH-39363: [C++] Use Cast() instead of CastTo() for Parquet (apa…
Browse files Browse the repository at this point in the history
…che#39364)

### Rationale for this change

Remove legacy code

### What changes are included in this PR?

Replace the legacy scalar CastTo implementation for Parquet.

### Are these changes tested?

Yes. It is passed by existing all test cases for Parquet.

### Are there any user-facing changes?

Maybe, Yes. 

There is a dependency on the Parquet schema that the user handles. There may be a problem if the user has to deal with a type for which Casting is not implemented. However, in this case, it should be treated as a new issue with an implementation that improves the `Cast` compute kernel.

* Closes: apache#39363

Authored-by: Hyunseok Seo <hsseo0501@gmail.com>
Signed-off-by: mwish <maplewish117@gmail.com>
  • Loading branch information
llama90 authored and clayburn committed Jan 23, 2024
1 parent 7c5cd7e commit a11eba0
Showing 1 changed file with 7 additions and 4 deletions.
11 changes: 7 additions & 4 deletions cpp/src/arrow/dataset/file_parquet.cc
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
#include <utility>
#include <vector>

#include "arrow/compute/cast.h"
#include "arrow/compute/exec.h"
#include "arrow/dataset/dataset_internal.h"
#include "arrow/dataset/parquet_encryption_config.h"
Expand Down Expand Up @@ -58,6 +59,8 @@ using parquet::arrow::SchemaField;
using parquet::arrow::SchemaManifest;
using parquet::arrow::StatisticsAsScalars;

using compute::Cast;

namespace {

parquet::ReaderProperties MakeReaderProperties(
Expand Down Expand Up @@ -370,12 +373,12 @@ std::optional<compute::Expression> ParquetFileFragment::EvaluateStatisticsAsExpr
return std::nullopt;
}

auto maybe_min = min->CastTo(field.type());
auto maybe_max = max->CastTo(field.type());
auto maybe_min = Cast(min, field.type());
auto maybe_max = Cast(max, field.type());

if (maybe_min.ok() && maybe_max.ok()) {
min = maybe_min.MoveValueUnsafe();
max = maybe_max.MoveValueUnsafe();
min = maybe_min.MoveValueUnsafe().scalar();
max = maybe_max.MoveValueUnsafe().scalar();

if (min->Equals(*max)) {
auto single_value = compute::equal(field_expr, compute::literal(std::move(min)));
Expand Down

0 comments on commit a11eba0

Please sign in to comment.