DAT-19308 - feat(databricks): ensure essential table properties Delta.*

[CharlesQueiroz]

Overview

This PR addresses the handling of essential Delta table properties in the Databricks extension, specifically ensuring critical properties are always included in table creation statements regardless of user input.

Problem

Previously, when users specified custom TBLPROPERTIES during table creation, the system would not guarantee the presence of essential Delta properties.

Solution

The implementation introduces a property management system that:

1. Defines a set of essential Delta properties that must always be present
2. Provides default values for these properties
3. Implements property merging logic that:
   • Preserves user-specified values for essential properties when provided
   • Automatically adds missing essential properties with default values
   • Maintains any additional custom properties specified by users

Key Changes

• Created constants for essential properties and their default values
• Implemented property processing logic in processTableProperties method
• Added test coverage for various property combinations
• Ensured backward compatibility with existing table creation workflows

Testing

The changes are covered by unit tests that verify:

• Default property inclusion
• Custom property preservation
• Mixed property scenarios
• Various value types (strings, numbers, booleans)
• Edge cases with empty or null properties

[SvampX]

Looks good to me.
Had a concern about wrapping of properties key/value in quotes, but acccording to tests logic uses existing user specified approach and default values are wrapped.

[SvampX]

@CharlesQueiroz we need to update expected SQL-s, see failed harness tests.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
88.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud