Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dataset ingest JSONs #187

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions datasets/critical_habitats/raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/critical_habitats/2023",
"path": "/geo-datasets/data/rasters/critical_habitats",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "None",
"file_mask": "YYYY/critical_habitats.tif",
"name": "critical_habitats_v2",
"title": "Global Critical Habitats Screening - Version 2.0",
"description": "The International Finance Corporation's (IFC's) Performance Standard 6 (PS6) is one of the most influential biodiversity standards of current time, particularly within large-scale infrastructure and the extractive sector. Within PS6, biodiversity significance is articulated through the concept of 'Critical Habitat,' a definition developed by the IFC and detailed through criteria aligned with those that support internationally accepted biodiversity designations. Critical Habitat criteria identify habitats of significant importance to threatened, endemic, congregatory and migratory species, threatened or unique ecosystems, and key evolutionary processes. PS6 requires projects to achieve net gains in the biodiversity values for which Critical Habitat is identified.",
Expand Down
33 changes: 17 additions & 16 deletions datasets/cru_ts/ingest/yearly_pre_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/cru_ts/cru_ts_4.07/yearly/pre/mean",
"path": "/geo-datasets/data/rasters/cru_ts/cru_ts_4.07/yearly/pre/mean",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "cru.pre.YYYY.tif",
"name": "cru_ts_407_pre_yearly_mean",
"title": "Yearly Mean Precipitation - CRU TS (v4.07)",
"description": "Yearly mean precipitation from Climate Research Unit (CRU) in millimeters. Version 4.07",
"details": "Created by aggregating (mean) monthly precipitation data from CRU TS",
"details": "Created by aggregating (mean) monthly precipitation data from CRU TS",
"version": "4.07",
"options": {
"resolution": 0.5,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "precipitation in millimeters"
"resolution": 0.5,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "precipitation in millimeters"
},
"extras": {
"citation": "Harris, I., Osborn, T.J., Jones, P. et al. Version 4 of the CRU TS monthly high-resolution gridded multivariate climate dataset. Sci Data 7, 109 (2020). https://doi.org/10.1038/s41597-020-0453-3",
"sources_web": "https://crudata.uea.ac.uk/cru/data/hrg/",
"sources_name": "Climate Research Unit",
"tags": ["cru", "precipitation", "environment", "yearly"]
"citation": "Harris, I., Osborn, T.J., Jones, P. et al. Version 4 of the CRU TS monthly high-resolution gridded multivariate climate dataset. Sci Data 7, 109 (2020). https://doi.org/10.1038/s41597-020-0453-3",
"sources_web": "https://crudata.uea.ac.uk/cru/data/hrg/",
"sources_name": "Climate Research Unit",
"tags": ["cru", "precipitation", "environment", "yearly"]
},
"active": 1
}
}
31 changes: 16 additions & 15 deletions datasets/cru_ts/ingest/yearly_tmp_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/cru_ts/cru_ts_4.07/yearly/tmp/mean",
"path": "/geo-datasets/data/rasters/cru_ts/cru_ts_4.07/yearly/tmp/mean",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "cru.tmp.YYYY.tif",
"name": "cru_ts_407_tmp_yearly_mean",
"title": "Yearly Mean Temperature - CRU TS (v4.07)",
"description": "Yearly mean temperature from Climate Research Unit (CRU) in degrees Celsius. Version 4.07",
"details": "Created by aggregating (mean) monthly mean daily temperature data from CRU TS",
"details": "Created by aggregating (mean) monthly mean daily temperature data from CRU TS",
"version": "4.07",
"options": {
"resolution": 0.5,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum temperature value per unit of analysis",
"mean" : "average temperature value per unit of analysis",
"min" : "minimum temperature value per unit of analysis"
},
"factor": 1,
"variable_description": "degrees Celsius"
"resolution": 0.5,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum temperature value per unit of analysis",
"mean" : "average temperature value per unit of analysis",
"min" : "minimum temperature value per unit of analysis"
},
"factor": 1,
"variable_description": "degrees Celsius"
},
"extras": {
"citation": "Harris, I., Osborn, T.J., Jones, P. et al. Version 4 of the CRU TS monthly high-resolution gridded multivariate climate dataset. Sci Data 7, 109 (2020). https://doi.org/10.1038/s41597-020-0453-3",
"sources_web": "https://crudata.uea.ac.uk/cru/data/hrg/",
"sources_name": "Climate Research Unit",
"tags": ["cru", "temperature", "environment", "yearly"]
"citation": "Harris, I., Osborn, T.J., Jones, P. et al. Version 4 of the CRU TS monthly high-resolution gridded multivariate climate dataset. Sci Data 7, 109 (2020). https://doi.org/10.1038/s41597-020-0453-3",
"sources_web": "https://crudata.uea.ac.uk/cru/data/hrg/",
"sources_name": "Climate Research Unit",
"tags": ["cru", "temperature", "environment", "yearly"]
},
"active": 1
}
7 changes: 4 additions & 3 deletions datasets/distance_to_country_border/raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/distance_to/geoBoundaries_borders",
"path": "/geo-datasets/data/rasters/distance_to_country_borders/geoboundaries_borders_distance.tif",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "None",
"name": "dist_to_geoBoundaries_borders",
"file_mask": null,
"name": "dist_to_geoboundaries_borders",
"title": "Distance to Country Borders (geoBoundaries)",
"description": "Distance to country borders, measured in meters. Derived using geoBoundaries ADM0 (Country) boundaries.",
"details": "",
Expand Down
3 changes: 2 additions & 1 deletion datasets/esa_landcover/raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/esa_landcover",
"path": "/geo-datasets/data/rasters/esa_landcover",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "esa_lc_YYYY.tif",
Expand Down
33 changes: 17 additions & 16 deletions datasets/gpm/monthly_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/gpm/v07b/monthly",
"path": "/geo-datasets/data/rasters/gpm/v07b/monthly",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "gpm_precipitation_YYYY_MM.tif",
"name": "gpm_precipitation_v07b_monthly",
"title": "Monthly IMERG (GPM) precipitation (v07B)",
"description": "Monthly IMERG GPM (v07B) precipitation - research version",
"details": "Created by extracting monthly IMERG V07B GPM data",
"details": "Created by extracting monthly IMERG V07B GPM data",
"version": "v07B",
"options": {
"resolution": 0.1,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "0.001 millimeters per hour"
"resolution": 0.1,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "0.001 millimeters per hour"
},
"extras": {
"citation": "G. Huffman, D. Bolvin, D. Braithwaite, K. Hsu, R. Joyce, P. Xie, 2014: Integrated Multi-satellitE Retrievals for GPM (IMERG), V07B. NASA's Precipitation Processing Center, accessed 27 June, 2024, arthurhouftps.pps.eosdis.nasa.gov",
"sources_web": "https://gpm.nasa.gov/data/directory",
"sources_name": "NASA Goddard Space Flight Center",
"tags": ["nasa","precipitation", "gpm","environment", "monthly"]
"citation": "G. Huffman, D. Bolvin, D. Braithwaite, K. Hsu, R. Joyce, P. Xie, 2014: Integrated Multi-satellitE Retrievals for GPM (IMERG), V07B. NASA's Precipitation Processing Center, accessed 27 June, 2024, arthurhouftps.pps.eosdis.nasa.gov",
"sources_web": "https://gpm.nasa.gov/data/directory",
"sources_name": "NASA Goddard Space Flight Center",
"tags": ["nasa","precipitation", "gpm","environment", "monthly"]
},
"active": 1
}
}
33 changes: 17 additions & 16 deletions datasets/gpm/yearly_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/gpm/v07b/yearly/mean",
"path": "/geo-datasets/data/rasters/gpm/v07b/yearly/mean",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "gpm_precipitation_YYYY.tif",
"name": "gpm_precipitation_v07b_yearly_mean",
"title": "Yearly IMERG (GPM) precipitation (v07B)",
"description": "Yearly IMERG GPM (v07B) precipitation - Created by aggregating (mean) the monthly GPM precipitation",
"details": "Created by aggregatubg monthly IMERG V07B GPM data",
"details": "Created by aggregatubg monthly IMERG V07B GPM data",
"version": "v07B",
"options": {
"resolution": 0.1,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "0.001 millimeters per hour"
"resolution": 0.1,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum precipitation value per unit of analysis",
"mean" : "average precipitation value per unit of analysis",
"min" : "minimum precipitation value per unit of analysis"
},
"factor": 1,
"variable_description": "0.001 millimeters per hour"
},
"extras": {
"citation": "G. Huffman, D. Bolvin, D. Braithwaite, K. Hsu, R. Joyce, P. Xie, 2014: Integrated Multi-satellitE Retrievals for GPM (IMERG), V07B. NASA's Precipitation Processing Center, accessed 27 June, 2024, arthurhouftps.pps.eosdis.nasa.gov",
"sources_web": "https://gpm.nasa.gov/data/directory",
"sources_name": "NASA Goddard Space Flight Center",
"tags": ["nasa","precipitation", "gpm","environment", "yearly"]
"citation": "G. Huffman, D. Bolvin, D. Braithwaite, K. Hsu, R. Joyce, P. Xie, 2014: Integrated Multi-satellitE Retrievals for GPM (IMERG), V07B. NASA's Precipitation Processing Center, accessed 27 June, 2024, arthurhouftps.pps.eosdis.nasa.gov",
"sources_web": "https://gpm.nasa.gov/data/directory",
"sources_name": "NASA Goddard Space Flight Center",
"tags": ["nasa","precipitation", "gpm","environment", "yearly"]
},
"active": 1
}
}
39 changes: 20 additions & 19 deletions datasets/malaria_atlas_project/ingest.json
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/malaria_atlas_project/pf_incidence_rate/",
"path": "/geo-datasets/data/rasters/malaria_atlas_project/pf_incidence_rate/",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "Global_Pf_Incidence_Rate_YYYY.tif",
"name": "map_pf_incidence_rate",
"title": "Malaria Atlas Project - Malaria (Pf) Incidence Rate",
"description": "Number of newly diagnosed Malaria (Plasmodium falciparum) cases per 1,000 population, in a given year",
"details": "",
"title": "Malaria Atlas Project - Malaria (Pf) Incidence Rate",
"description": "Number of newly diagnosed Malaria (Plasmodium falciparum) cases per 1,000 population, in a given year",
"details": "",
"version": 202206,
"options": {
"resolution": 0.041666667,
"extract_types": ["mean", "min", "max", "count"],
"extract_types_info" : {
"max" : "maximum value measured within each unit of analysis",
"mean" : "average value measured within each unit of analysis",
"min" : "minimum value measured within each unit of analysis",
"count" : "total count of pixels per unit of analysis"
},
"factor": 1,
"variable_description": "rate per 1,000 people",
"resolution": 0.041666667,
"extract_types": ["mean", "min", "max", "count"],
"extract_types_info" : {
"max" : "maximum value measured within each unit of analysis",
"mean" : "average value measured within each unit of analysis",
"min" : "minimum value measured within each unit of analysis",
"count" : "total count of pixels per unit of analysis"
},
"factor": 1,
"variable_description": "rate per 1,000 people",
"pixel_check": false
},
"extras": {
"citation": "Hay, S.I. and Snow, R.W., 2006. The Malaria Atlas Project: developing global maps of malaria risk. PLoS medicine, 3(12), p.e473.",
"sources_web": "https://data.malariaatlas.org/",
"sources_name": "Malaria Atlas Project",
"tags": ["malaria", "health", "welfare"]
"citation": "Hay, S.I. and Snow, R.W., 2006. The Malaria Atlas Project: developing global maps of malaria risk. PLoS medicine, 3(12), p.e473.",
"sources_web": "https://data.malariaatlas.org/",
"sources_name": "Malaria Atlas Project",
"tags": ["malaria", "health", "welfare"]
},
"active": 1
}
}
33 changes: 17 additions & 16 deletions datasets/modis_lst/ingest/061/yearly_night_ingest.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/MODIS/terra/MOLT/MOD11C3.061/yearly/night/mean",
"path": "/geo-datasets/data/rasters/MODIS/terra/MOLT/MOD11C3.061/yearly/night/mean",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "modis_lst_night_cmg_YYYY.tif",
"name": "modis_lst_mod11c3_061_night_annual_mean",
"title": "Annual nighttime land surface temperature - MODIS v6.1",
"description": "Annual nighttime land surface temperature in Kelvins. Created by aggregating (mean) month data from MOD11C3.061 CMG LST product",
"details": "Production of the previous version (V6) Terra MODIS Land Surface Temperature and Emissivity (LST&E) data products was discontinued on November 15, 2022, due to significant loss of data retrieval following the Constellation Exit Maneuvers.",
"details": "Production of the previous version (V6) Terra MODIS Land Surface Temperature and Emissivity (LST&E) data products was discontinued on November 15, 2022, due to significant loss of data retrieval following the Constellation Exit Maneuvers.",
"version": "MOD11C3.061",
"options": {
"resolution": 0.05,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum land surface temperature value per unit of analysis",
"mean" : "average land surface temperature value per unit of analysis",
"min" : "minimum land surface temperature value per unit of analysis"
},
"factor": 1,
"variable_description": "temperature in Kelvins"
"resolution": 0.05,
"extract_types": ["max", "mean", "min"],
"extract_types_info" : {
"max" : "maximum land surface temperature value per unit of analysis",
"mean" : "average land surface temperature value per unit of analysis",
"min" : "minimum land surface temperature value per unit of analysis"
},
"factor": 1,
"variable_description": "temperature in Kelvins"
},
"extras": {
"citation": "Wan, Z., Hook, S., Hulley, G. (2015). MOD11C3 MODIS/Terra Land Surface Temperature/Emissivity Monthly L3 Global 0.05Deg CMG V061 [Data set]. NASA EOSDIS LP DAAC. doi: 10.5067/MODIS/MOD11C3.061",
"sources_web": "https://lpdaac.usgs.gov/products/mod11c3v061/",
"sources_name": "Land Processes Distributed Active Archive Center",
"tags": ["nasa","earthdata", "modis", "lst", "land surface temperature", "environment", "yearly"]
"citation": "Wan, Z., Hook, S., Hulley, G. (2015). MOD11C3 MODIS/Terra Land Surface Temperature/Emissivity Monthly L3 Global 0.05Deg CMG V061 [Data set]. NASA EOSDIS LP DAAC. doi: 10.5067/MODIS/MOD11C3.061",
"sources_web": "https://lpdaac.usgs.gov/products/mod11c3v061/",
"sources_name": "Land Processes Distributed Active Archive Center",
"tags": ["nasa","earthdata", "modis", "lst", "land surface temperature", "environment", "yearly"]
},
"active": 1
}
}
3 changes: 2 additions & 1 deletion datasets/oco2/ingest/monthly_xco2_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/gesdisc/OCO2_L2_Lite_FP.11.1r/xco2/month_interp",
"path": "/geo-datasets/data/rasters/gesdisc/OCO2_L2_Lite_FP.11.1r/xco2/month_interp",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "xco2_YYYYMM.tif",
Expand Down
3 changes: 2 additions & 1 deletion datasets/oco2/ingest/yearly_xco2_raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/gesdisc/OCO2_L2_Lite_FP.11.1r/xco2/year_interp",
"path": "/geo-datasets/data/rasters/gesdisc/OCO2_L2_Lite_FP.11.1r/xco2/year_interp",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "xco2_YYYY.tif",
Expand Down
27 changes: 14 additions & 13 deletions datasets/plad/raster_ingest.json
Original file line number Diff line number Diff line change
@@ -1,27 +1,28 @@
{
"base": "/sciclone/aiddata10/REU/geo/data/rasters/plad/v70",
"path": "/geo-datasets/data/rasters/plad/v70",
"is_global": true,
"type": "raster",
"file_extension": ".tif",
"file_mask": "leader_birthplace_data_YYYY.tif",
"name": "plad_v70_leader_birthplace",
"title": "Presence of a political leader's birthplace (ADM1/ADM2 use only)",
"description": "The Political Leaders Affiliation Database (PLAD) contains information on the birthplaces of the effective leaders of 177 countries around the world in the 1989-2023 period. Birthplace coordinates with precision at the ADM2 level or finer were rasterized to support aggregation in GeoQuery at the ADM2 (or coarser) units.",
"details": "Note: more than one birthplace may exist in a given year during transition years. However, the data is only binary, indicating the presence of birthplace of any leader during the given year.",
"details": "Note: more than one birthplace may exist in a given year during transition years. However, the data is only binary, indicating the presence of birthplace of any leader during the given year.",
"version": "7.0",
"options": {
"resolution": 0.05,
"extract_types": ["max"],
"extract_types_info" : {
"max" : "presence of a leader birthplace in unit of analysis"
},
"factor": 1,
"variable_description": "binary indicating presence of a leader's birthplace"
"resolution": 0.05,
"extract_types": ["max"],
"extract_types_info" : {
"max" : "presence of a leader birthplace in unit of analysis"
},
"factor": 1,
"variable_description": "binary indicating presence of a leader's birthplace"
},
"extras": {
"citation": "Bomprezzi, Pietro; Dreher, Axel; Fuchs, Andreas; Hailer, Teresa; Kammerlander, Andreas; Kaplan, Lennart; Marchesi, Silvia; Masi, Tania; Robert, Charlotte; Unfried, Kerstin, 2020, The Political Leaders Affiliation Database (PLAD), https://doi.org/10.7910/DVN/YUS575, Harvard Dataverse, V7",
"sources_web": "https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YUS575",
"sources_name": "The Political Leaders Affiliation Database (PLAD)",
"tags": ["political", "leaders", "affiliation", "birthplace", "influence", "yearly"]
"citation": "Bomprezzi, Pietro; Dreher, Axel; Fuchs, Andreas; Hailer, Teresa; Kammerlander, Andreas; Kaplan, Lennart; Marchesi, Silvia; Masi, Tania; Robert, Charlotte; Unfried, Kerstin, 2020, The Political Leaders Affiliation Database (PLAD), https://doi.org/10.7910/DVN/YUS575, Harvard Dataverse, V7",
"sources_web": "https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YUS575",
"sources_name": "The Political Leaders Affiliation Database (PLAD)",
"tags": ["political", "leaders", "affiliation", "birthplace", "influence", "yearly"]
},
"active": 1
}
Loading