Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

databricks: print_dbt_project_evaluator_issues - quoting #399

Closed
5 tasks
bram-83 opened this issue Dec 4, 2023 · 3 comments · Fixed by #400
Closed
5 tasks

databricks: print_dbt_project_evaluator_issues - quoting #399

bram-83 opened this issue Dec 4, 2023 · 3 comments · Fixed by #400
Labels
bug Something isn't working

Comments

@bram-83
Copy link

bram-83 commented Dec 4, 2023

Describe the bug

Printing the results does not work when the databricks schema contains special characters

Steps to reproduce

using special characters in your schema: for example ch-123

Expected results

the expected behaviour = printing the results

Actual results

on-run-end failed, error:

[INVALID_IDENTIFIER] The identifier ch-51278 is invalid. Please, consider quoting it with back-quotes as ch-51278.(line 4, pos 28)
== SQL ==
/* {"app": "dbt", "dbt_version": "1.7.3", "dbt_databricks_version": "1.7.2", "databricks_sql_connector_version": "2.9.3", "profile_name": "user", "target_name": "qas", "connection_name": "master"} */

    select * from qas.ch-51278.fct_test_coverage

----------------------------^^^

Screenshots and log output

System information

The contents of your packages.yml file:
packages:

  • package: dbt-labs/dbt_external_tables
    version: 0.8.7
  • package: dbt-labs/dbt_utils
    version: [">=1.0.0", "<2.0.0"]
  • package: dbt-labs/dbt_project_evaluator
    version: 0.8.0

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • trino/starburst
  • [ x] other (specify: databricks)

The output of dbt --version:
Running with dbt=1.7.3
Registered adapter: databricks=1.7.2

Additional context

dbt_project_evaluator > macros > on-run-end > print_dbt_project_evaluator_issues
set db_schema: quote a lowercase the three level namespace

@bram-83 bram-83 added the bug Something isn't working label Dec 4, 2023
@b-per
Copy link
Collaborator

b-per commented Dec 4, 2023

Hi @bram-83 , thanks for raising this.

As we don't use ref in the current code, it doesn't take in account the quoting config that might be set in dbt_project.yml.

What I would recommend is doing a copy of the macro and changing this line from

{% set model_schema = model_details.schema %}

to

{% set model_schema = "`" ~ model_details.schema ~ "`"  %}

You can then update you on-run-end to point to your own macro instead of the package one.

If this works, we could update the macro in the package and add an optional parameter to add quoting.

Let me know how it goes.

@bram-83
Copy link
Author

bram-83 commented Dec 8, 2023

Hi @b-per,
Thank you for the tip. I quoted the schema & catalog name.
I't working fine now, thanks,
image

@b-per
Copy link
Collaborator

b-per commented Dec 11, 2023

When we do a new release you will be able to use print_dbt_project_evaluator_issues(quote='`')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants