[FEA] Add Validation utilities to user_tools package #1549
Labels
feature request
New feature or request
user_tools
Scope the wrapper module running CSP, QualX, and reports (python)
Is your feature request related to a problem? Please describe.
We need some utilities to validate whether 2 tables are equal.
For a started, given 2 warehouses, and pairs of tables, the scripts will compare the schemas, rows, columns and generate report what is broken.
Describe the solution you'd like
Initially we can revisit the implementation in datavalidation directory and consider moving what is applicable to the user_Tools package.
The first step will be the python scripts dataset and metadata validations.
The second step is to to provide a class to configure and wrap those 2 scripts to be submitted as Spark jobs.
Additional context
The text was updated successfully, but these errors were encountered: