support reusable templates in recipes

[trey-stafford]

• We want to be able to support re-use of "templates" that we define in recipes. E.g., the ADC's viz workflow
• Support the use of a reusable template in a new ogdc recipe. Update the recipe meta yaml spec to support such input.
• Start with something simple. Maybe our "add compression and overviews" template, as this is a standard qgreenland packaging step for rasters.

This will establish the foundation for work on #38

[rushirajnenuji]

Thank you @trey-stafford, it makes sense to make re-usable templates that we can use for each recipes.

Sharing the default Viz workflow config as below:

{
  "version": null,
  "dir_geotiff": "geotiff",
  "dir_web_tiles": "web_tiles",
  "dir_3dtiles": "3dtiles",
  "dir_staged": "staged",
  "dir_input": "input",
  "dir_footprints": "footprints",
  "filename_staging_summary": "staging_summary.csv",
  "filename_rasterization_events": "rasterization_events.csv",
  "filename_rasters_summary": "rasters_summary.csv",
  "filename_config": "config.json",
  "ext_web_tiles": ".png",
  "ext_input": ".shp",
  "ext_staged": ".gpkg",
  "ext_footprints": ".gpkg",
  "prop_centroid_x": "staging_centroid_x",
  "prop_centroid_y": "staging_centroid_y",
  "prop_area": "staging_area",
  "prop_tile": "staging_tile",
  "prop_centroid_tile": "staging_centroid_tile",
  "prop_filename": "staging_filename",
  "prop_identifier": "staging_identifier",
  "prop_centroid_within_tile": "staging_centroid_within_tile",
  "prop_duplicated": "staging_duplicated",
  "input_crs": null,
  "simplify_tolerance": 0.0001,
  "tms_id": "WGS1984Quad",
  "tile_path_structure": [
    "style",
    "tms",
    "z",
    "x",
    "y"
  ],
  "z_range": [
    0,
    13
  ],
  "tile_size": [
    256,
    256
  ],
  "statistics": [
    {
      "name": "polygon_count",
      "weight_by": "count",
      "property": "centroids_per_pixel",
      "aggregation_method": "sum",
      "resampling_method": "sum",
      "val_range": [
        0,
        null
      ],
      "nodata_val": 0,
      "nodata_color": "#ffffff00"
    },
    {
      "name": "coverage",
      "weight_by": "area",
      "property": "area_per_pixel_area",
      "aggregation_method": "sum",
      "resampling_method": "average",
      "val_range": [
        0,
        1
      ],
      "nodata_val": 0,
      "nodata_color": "#ffffff00"
    }
  ],
  "geometricError": null,
  "z_coord": 0,
  "deduplicate_at": null,
  "deduplicate_method": null,
  "deduplicate_keep_rules": [],
  "deduplicate_overlap_tolerance": 0.5,
  "deduplicate_overlap_both": true,
  "deduplicate_centroid_tolerance": null,
  "deduplicate_distance_crs": "EPSG:3857",
  "deduplicate_clip_to_footprint": true,
  "deduplicate_clip_method": "intersects"
}

Note: the above config is for the entire workflow, and can be broken down to a generalized recipe config template. And it might evolve to include additional metadata as @alonakos makes progress on getting the WMTS and STAC support.

Looking forward to discussing this further.

[trey-stafford]

Thanks @rushirajnenuji for the viz workflow config example! I think this will be most useful when we start working on #38 . I'm using this issue to focus on a mechanism for defining and using reusable templates with recipes. This foundational work can then be leveraged for implementing the viz workflow template in #38 .