Skip to content

Commit

Permalink
[SPARK-51241][SQL][TESTS] Add test cases with ignore nulls for ANY_VALUE
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
This PR proposes to add test cases with ignore nulls for `ANY_VALUE`.

### Why are the changes needed?
According to the discussion at #49981 (comment)

### Does this PR introduce _any_ user-facing change?
'No'.

### How was this patch tested?
GA

### Was this patch authored or co-authored using generative AI tooling?
'No'.

Closes #49990 from beliefer/SPARK-51241_new.

Authored-by: beliefer <beliefer@163.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
  • Loading branch information
beliefer authored and MaxGekk committed Feb 18, 2025
1 parent aa37f89 commit 6dbd12a
Showing 1 changed file with 7 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import org.apache.spark.SparkThrowable
import org.apache.spark.sql.catalyst.FunctionIdentifier
import org.apache.spark.sql.catalyst.analysis.{UnresolvedAttribute, _}
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.aggregate.{First, Last}
import org.apache.spark.sql.catalyst.expressions.aggregate.{AnyValue, First, Last}
import org.apache.spark.sql.catalyst.plans.logical.{OneRowRelation, Project}
import org.apache.spark.sql.catalyst.util.{DateTimeTestUtils, IntervalUtils}
import org.apache.spark.sql.catalyst.util.DateTimeConstants._
Expand Down Expand Up @@ -1150,13 +1150,18 @@ class ExpressionParserSuite extends AnalysisTest {
assertEqual(complexName2.quotedString, UnresolvedAttribute(Seq("fo``o", "ba``r")))
}

test("SPARK-19526 Support ignore nulls keywords for first and last") {
test("SPARK-19526: Support ignore nulls keywords for first and last") {
assertEqual("first(a ignore nulls)", First($"a", true).toAggregateExpression())
assertEqual("first(a)", First($"a", false).toAggregateExpression())
assertEqual("last(a ignore nulls)", Last($"a", true).toAggregateExpression())
assertEqual("last(a)", Last($"a", false).toAggregateExpression())
}

test("SPARK-51241: Add test cases with ignore nulls for ANY_VALUE") {
assertEqual("any_value(a ignore nulls)", AnyValue($"a", true).toAggregateExpression())
assertEqual("any_value(a)", AnyValue($"a", false).toAggregateExpression())
}

test("timestamp literals") {
DateTimeTestUtils.outstandingZoneIds.foreach { zid =>
withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> zid.getId) {
Expand Down

0 comments on commit 6dbd12a

Please sign in to comment.