Scripts for processing of historical and future hazards data. For more information see the wiki.
As of January 2022, we have calculated a total of 10 hazard indices. The descriptions of these are available in the wiki, under hazard definitions. The indices are as follows,
- Drought stress
- NDD: number of dry days
- NDWS: number of soil moisture stress days
- TAI: Thornthwaite's aridity index
- Heat stress
- NTx35: number of heat stress (Tmax > 35ºC) days for crops
- NTx40: number of extreme heat stress (Tmax > 40ºC) days for crops
- HSM_NTx35: number of heat stress days for maize during the crop growing season
- HSH: human heat stress index
- THI: cattle thermal humidity index
- Waterlogging and flooding
- NDWL0: number of days with soil waterlogging at moisture content at start of saturation or above.
- NDWL50: number of days with soil waterlogging at moisture content 50% between field capacity and saturation.
These data are available for two SSPs (SSP2-4.5, SSP5=8.5), two periods (2021-2040, 2041-2060), and five GCMs (ACCESS-ESM1-5, MPI-ESM1-2-HR, EC-Earth3, INM-CM5-0, MRI-ESM2-0) in addition to the multi-model ensemble mean.
While the Atlas is under development, the data is internally available to Atlas scientists in the EiA server under ~/common_data/atlas_hazards/cmip6. The data are also available in the AWS S3 and Google Cloud buckets.
This repository is divided into 7 basic processes, each available under specific folders containing the code for processing all Atlas hazard layers. Each folder contains various functions designed to work as generically as possible so the workflow can be replicated. The below provides an overview of each process, and some of the functions therein.
To get access to the code it is best to first clone the repository locally, as most of the scripts require small modifications (e.g., local folders). To clone the repository, use
git clone https://github.com/AdaptationAtlas/hazards.gitFor hazards calculations, right now we require five datasets, namely, CHIRPS/CHIRTS (from the Climate Hazards Group), AgERA5 (from the EU Copernicus service), CMIP6 projections, and crop calendars. For each dataset, we have built a script / function that helps download the dataset based on specified dates and/or spatial domain. For instance, for CHIRPS, the following code will download a global CHIRPS data layer (in GeoTiff format)
getChirps(date = as.Date("1981-01-01""))An interesting function we built is the downloadCMIP6 function, which leverages work done by the SantanderMetGroup on the CMIP6 Atlas. This function allows downloading raw CMIP6 data in an automated way, without having to pass through the ESFG CMIP6 portal.
cmip6_data <- downloadCMIP6(ds_name = "CMIP6_ACCESS-ESM1-5_scenario_r1i1p1f1",
rcp = "ssp246",
varname = "tasmax",
years.hist = 1995:2014,
years.rcp = 2021:2060,
lons = c(-23, 59),
lats = c(-37, 40),
basedir = my_local_dir)Contains functions to create various masks for Africa that are needed for the post-processing of the hazard layers. There is also a function to pre-process crop calendar data and make it compatible (resolution and extent), with the rest of the data layers.
The two main scripts here are bc_calc_anomalies.R and getDailyFutureData.R. These scripts help first calculating monthly climate anomalies, and then apply these to the historical climate data. An example is provided here,
#historical climatology
his_clm <- calc_climatology(data_file = "CMIP6_ACCESS-ESM1-5_historical_r1i1p1f1__Africa_daily.tif",
period = 1995:2014,
sce_lab = "historical",
gcm_name = "CMIP6_ACCESS-ESM1-5",
varname = "tasmax",
mth_dir = "my_folder_with_monthly_annual_files",
clm_dir = "my_folder_with_climatology_files")
#future climatology
rcp_clm <- calc_climatology(data_file = "CMIP6_ACCESS-ESM1-5_ssp245_r1i1p1f1__Africa_daily.tif",
period = 2021:2040,
sce_lab = "ssp245",
gcm_name = "CMIP6_ACCESS-ESM1-5",
varname = "tasmax",
mth_dir = "my_folder_with_monthly_annual_files",
clm_dir = "my_folder_with_climatology_files")
#interpolate anomalies
rcp_anom <- intp_anomalies(his_clm = "my_folder_with_climatology_files_hist",
rcp_clm = "my_folder_with_climatology_files_rcp",
anom_dir = "my_output_folder_for_anomalies",
ref = reference_raster_object,
gcm_name = "CMIP6_ACCESS-ESM1-5",,
rcp = "ssp245",
varname = "tasmax",
period = 2021:2040)Now produce the daily future data:
get_daily_future_data(gcm = "ACCESS-ESM1-5",
ssp = "ssp245",
var = "tasmax",
prd = "2021_2040")This set of functions calculates each index in spatial form, taking as input the downscaled / bias-corrected CMIP6 daily data. A script is provided for each hazard index. Note that these scripts contain both paths, execution loops (or purrr::map()), and functions. So some modifications will be in order for them to work in a new computing environment than ours. Most functions take the year and month as input, and in some cases other additional parameters. For example, for NTx40, we built the function calc_ntx(), which is used as follows:
gcm <- "ACCESS-ESM1-5"
ssp <- "ssp245"
prd <- "2021_2040"
calc_ntx(yr=2021, mn="01", thr=40)Each of the indices has a meta-data function, which uses the Atlas metadata functions. This function will calculate long-term statistics (using the function continuous_map() in script calc_LongTermStats.R). It will also generate discrete maps of hazard severity levels (function category_map() in script calc_discreteMaps.R), and create the actual meta-data and bucket uploadable data files. These scripts need to be sourced (i.e., they are not functions).
These scripts will automatically upload the processed output into the AWS S3 and Google Buckets. They use the Linux system library gsutil and the R package aws.s3.