Skip to content

Commit

Permalink
[SPARK-51119][SQL][FOLLOW-UP] Readers on executors resolving EXISTS_D…
Browse files Browse the repository at this point in the history
…EFAULT should not call catalogs

### What changes were proposed in this pull request?
 Code cleanup for #49840.  Literal#fromSQL should be the inverse of Literal#sql.  The cast handling is an artifact of the calling ResolveDefaultColumns object that adds the cast when making EXISTS_DEFAULT, so its handling is moved to ResolveDefaultColumns as well.

### Why are the changes needed?
Code cleanup

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Existing tests

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #49881 from szehon-ho/SPARK-51119-follow.

Authored-by: Szehon Ho <szehon.apache@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information
szehon-ho authored and cloud-fan committed Feb 11, 2025
1 parent e7709f0 commit 5135329
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,9 @@ object Literal {
s"but class ${Utils.getSimpleName(value.getClass)} found.")
}

/**
* Inverse of [[Literal.sql]]
*/
def fromSQL(sql: String): Expression = {
CatalystSqlParser.parseExpression(sql).transformUp {
case u: UnresolvedFunction =>
Expand All @@ -278,10 +281,6 @@ object Literal {
assert(u.orderingWithinGroup.isEmpty)
assert(!u.isInternal)
FunctionRegistry.builtin.lookupFunction(FunctionIdentifier(u.nameParts.head), u.arguments)
} match {
case c: Cast if c.needsTimeZone =>
c.withTimeZone(SQLConf.get.sessionLocalTimeZone)
case e: Expression => e
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -320,14 +320,20 @@ object ResolveDefaultColumns extends QueryErrorsBase
}

/**
* Analyze EXISTS_DEFAULT value. This skips some steps of analyze as most of the
* analysis has been done before.
* Analyze EXISTS_DEFAULT value. EXISTS_DEFAULT value was created from CURRENT_DEFAQULT
* via [[analyze]] and thus this can skip most of those steps.
*/
private def analyzeExistenceDefaultValue(field: StructField): Expression = {
val defaultSQL = field.metadata.getString(EXISTS_DEFAULT_COLUMN_METADATA_KEY)

// Parse the expression.
val expr = Literal.fromSQL(defaultSQL)
val expr = Literal.fromSQL(defaultSQL) match {
// EXISTS_DEFAULT will have a cast from analyze() due to coerceDefaultValue
// hence we need to add timezone to the cast if necessary
case c: Cast if c.needsTimeZone =>
c.withTimeZone(SQLConf.get.sessionLocalTimeZone)
case e: Expression => e
}

// Check invariants
if (expr.containsPattern(PLAN_EXPRESSION)) {
Expand Down

0 comments on commit 5135329

Please sign in to comment.