Skip to content

Latest commit

 

History

History
130 lines (90 loc) · 3.78 KB

README.md

File metadata and controls

130 lines (90 loc) · 3.78 KB

An unaffiliated python interface for dbt Cloud APIs

Coverage Package version Downloads

Documentation: https://dbtc.dpguthrie.com

Interactive Demo: https://dpguthrie-dbtc-streamlit-home-yy7c0b.streamlit.app/

Source Code: https://github.com/dpguthrie/dbtc

V2 Docs: https://docs.getdbt.com/dbt-cloud/api-v2

V3 Docs (Unofficial): https://documenter.getpostman.com/view/14183654/UVsSNiXC

Overview

dbtc is an unaffiliated python interface to various dbt Cloud API endpoints.

This library acts as a convenient interface to two different APIs that dbt Cloud offers:

  • Cloud API: This is a REST API that exposes endpoints that allow users to programatically create, read, update, and delete resources within their dbt Cloud Account.
  • Metadata API: This is a GraphQL API that exposes metadata generated from a job run within dbt Cloud.

Requirements

Python 3.7+

  • Requests - The elegant and simple HTTP library for Python, built for human beings.
  • sgqlc - Simple GraphQL Client
  • Typer - Library for building CLI applications

Installation

pip install dbtc

Basic Usage

Python

The interface to both APIs are located in the dbtCloudClient class.

The example below shows how you use the cloud property on an instance of the dbtCloudClient class to to access a method, trigger_job_from_failure, that allows you to restart a job from its last point of failure.

from dbtc import dbtCloudClient

# Assumes that DBT_CLOUD_SERVICE_TOKEN env var is set
client = dbtCloudClient()

account_id = 1
job_id = 1
payload = {'cause': 'Restarting from failure'}

run = client.cloud.trigger_job_from_failure(
    account_id,
    job_id,
    payload,
    should_poll=False,
)

# This returns a dictionary containing two keys
run['data']
run['status']

Similarly, use the metadata property to retrieve information about certain resources within your project - the example below shows how to retrieve metadata from models related to the most recent run for a given job_id.

from dbtc import dbtCloudClient

client = dbtCloudClient()

job_id = 1

models = client.metadata.get_models(job_id)

# Models nested inside a couple keys
models['data']['models']

# This is a list
models['data']['models'][0]

CLI

The CLI example below will map to the python cloud example above:

dbtc trigger-job-from-failure \
    --account-id 1 \
    --job-id 1 \
    --payload '{"cause": "Restarting from failure"}' \
    --no-should-poll

Similarly, for the metadata example above:

dbtc get-models --job-id 1

If not setting your service token as an environment variable, do the following:

dbtc --token this_is_my_token get_models --job-id 1

License

This project is licensed under the terms of the MIT license.