Should project evaluator track progress over time?

[jschintz-nytimes]

Describe the feature

Allow a configuration that tracks progress & changes to a projects health by capturing snapshots or metrics overtime.

Getting users to onboard to project evaluator is hard - its even harder to show if material progress is being made. Give a carrot for people to substantiate just how much work they've done.

Describe alternatives you've considered

1. For anything in the marts or tests - take a snapshot. This feels like too much data. I'll never get into the weeds with which models had a PK a quarter ago.
2. Capture a summary of each test into a singular table each time this runs.
3. Maybe this should just be a feature for Explorer.
4. Maybe not - is this just overkill to satisfy people who like #s?

Additional context

This is looking for ways to track progress - becomes teams aren't making much progress.
It should help managers & governance figure out if progress is being made

Who will this benefit?

Devs - they deserve more credit for the hard work they do.

Are you interested in contributing this feature?

I sure am!

[b-per]

Yes, that would be a great way to use the package.

But, I am wondering, should this be baked into this project itself?

Or is too specific to individual use cases and means that we can't provide a single generic solution to it.

Or maybe we just provide an example of query for people to create themselves?

My initial thought would be that someone could create an incremental model (or a snapshot) in their own project that would query whatever fct_... table of this package, and add info about the date, the project etc...

It could be:

-- model dbt_project_evaluator_hist

{{
    config(
        materialized='incremental',
        full_refresh = false
    )
}}


with num_issues_fct_direct_join_to_source as (
select count(*) as cnt from {{ ref("fct_direct_join_to_source") }} 
)

, num_issues_fct_staging_dependent_on_marts_or_intermediate as (
select count(*) as cnt from {{ ref("fct_staging_dependent_on_marts_or_intermediate") }} 
)

...

select 
  {{ run_started_at }} as run_started_at,
  {{ project_name }} as dbt_project_name,
  {{ env_var('DBT_CLOUD_PROJECT_ID','') }} as dbt_cloud_project_id,
  {{ env_var('DBT_CLOUD_PR_ID','') }} as dbt_cloud_pr_id,
...
  fct_direct_join_to_source.cnt as cnt_issues_fct_direct_join_to_source,
  fct_staging_dependent_on_marts_or_intermediate.cnt as cnt_issues_fct_staging_dependent_on_marts_or_intermediate,
...
from num_issues_fct_direct_join_to_source cross join num_issues_fct_staging_dependent_on_marts_or_intermediate