Merge branch 'master' into isotopes

CHANGELOG.md

@@ -5,14 +5,18 @@
 ### New Datasets
 * Add precipitation isotope data from BCO (#152). By [Hauke Schulz](https://github.com/observingClouds)
 * Add water vapor isotope data from BCO (#153). By [Hauke Schulz](https://github.com/observingClouds)
+* Add EUREC4A-MIP boundary conditions from ERA5 and COSMO for current and pseudo-global warming experiment (#135). By [torresalavez](https://github.com/torresalavez) and [Hauke Schulz](https://github.com/observingClouds)
 * Add cloud droplet size distribution from specMACS instrument. By [Veronika Pörtge](https://github.com/vpoertge)
+* Add LICHT lidar data from RV Meteor as provided by the MPI-TCO catalog (#12). By [Leif Denby](https://github.com/leifdenby), [Hauke Schulz](https://github.com/observingClouds), [Tobias Kölling](https://github.com/d70-t) and [Nina Robbins](https://github.com/ninarobbins)
 ### Updated Datasets
 * Updated RV Meteor cloud radar data (LIMRAD94) to version 1.2, in which the variable hydrometeor_mask was added which tells you whether there is a signal in the processed reflectivity or not
+* Add gridded POLDIRAD product with kerchunk references to provide a continous dataset (#160). By [Hauke Schulz](https://github.com/observingClouds)
 ### Removed Datasets
 ### Fixes
 ### Internal Changes
 * update IPFS version in CI to 0.23.0 to improve access times and make weekly_test more reliable (#149). By [Hauke Schulz](https://github.com/observingClouds)
 * add citation recommendations (#155). By [Hauke Schulz](https://github.com/observingClouds)
+* install dependencies via python eurec4a package. By [Tobias Kölling](https://github.com/d70-t)
 
 ## 1.0.0
 

Meteor/main.yaml

@@ -31,3 +31,10 @@ sources:
     driver: opendap
     description: Surface heat fluxes measured on-board R/V Meteor
     metadata: {}
+
+  LICHT_LIDAR:
+    description: Observations made with the LICHT LIDAR on R/V Meteor
+    args:
+      path: "https://tcodata.mpimet.mpg.de/METEOR/EUREC4A/catalog.yaml"
+    driver: intake.catalog.local.YAMLFileCatalog
+    metadata: {}

README.md

@@ -17,7 +17,7 @@ To use you will need to install `intake`, `xarray`, `intake-xarray`, `zarr`,
 `pydap`, `requests`, `s3fs` and `ipfsspec`
 
 ```bash
-pip install intake xarray intake-xarray zarr pydap s3fs requests ipfsspec
+pip install "intake<2.0.0" xarray intake-xarray zarr pydap s3fs requests ipfsspec
 ```
 
 **Or**, if you feel courageous (and want the newest updates), you can also install the [`requirements.txt`](requirements.txt) directly:

Simulations/EUREC4A-MIP/control/main.yaml

@@ -0,0 +1,11 @@
+plugins:
+  source:
+     - module: intake_xarray
+
+sources:
+  setup:
+    args:
+      path: "{{CATALOG_DIR}}/setup/main.yaml"
+    description: "Simulation setups"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/control/setup/bc.yaml

@@ -0,0 +1,22 @@
+plugins:
+  source:
+    - module: intake_xarray
+
+sources:
+
+  ERA5:
+    driver: zarr
+    description: Boundary conditions for storm resolving models
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/CTRL_ERA5/CTRL_ERA5.ref
+      consolidated: false
+    metadata: {}
+
+  COSMO:
+    args:
+      path: "{{CATALOG_DIR}}/cosmo.yaml"
+    description: "Boundary conditions for large eddy models"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/control/setup/cosmo.yaml

@@ -0,0 +1,25 @@
+plugins:
+  source:
+    - module: intake_xarray
+
+sources:
+
+  2D:
+    driver: zarr
+    description: 2D fields
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/COSMO_CTRL_BC_2D/COSMO_CTRL_BC_2D.ref
+      consolidated: false
+    metadata: {}
+
+  3D:
+    driver: zarr
+    description: 2D fields
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/COSMO_CTRL_BC_3D/COSMO_CTRL_BC_3D.ref
+      consolidated: false
+    metadata: {}

Simulations/EUREC4A-MIP/control/setup/main.yaml

@@ -0,0 +1,11 @@
+plugins:
+  source:
+     - module: intake_xarray
+
+sources:
+  boundary_conditions:
+    args:
+      path: "{{CATALOG_DIR}}/bc.yaml"
+    description: "Boundary conditions"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/main.yaml

@@ -0,0 +1,18 @@
+plugins:
+  source:
+     - module: intake_xarray
+
+sources:
+  control:
+    args:
+      path: "{{CATALOG_DIR}}/control/main.yaml"
+    description: "EUREC4A-MIP simulations based on realistic boundary conditions"
+    driver: yaml_file_cat
+    metadata: {}
+
+  warming:
+    args:
+      path: "{{CATALOG_DIR}}/warming/main.yaml"
+    description: "EUREC4A-MIP simulations based on 4K warming"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/warming/main.yaml

@@ -0,0 +1,11 @@
+plugins:
+  source:
+     - module: intake_xarray
+
+sources:
+  setup:
+    args:
+      path: "{{CATALOG_DIR}}/setup/main.yaml"
+    description: "Simulation setups"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/warming/setup/bc.yaml

@@ -0,0 +1,22 @@
+plugins:
+  source:
+    - module: intake_xarray
+
+sources:
+
+  ERA5:
+    driver: zarr
+    description: Boundary conditions for storm resolving models
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/PGW+4K/PGW+4K.ref
+      consolidated: false
+    metadata: {}
+
+  COSMO:
+    args:
+      path: "{{CATALOG_DIR}}/cosmo.yaml"
+    description: "Boundary conditions for large eddy models"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/EUREC4A-MIP/warming/setup/cosmo.yaml

@@ -0,0 +1,25 @@
+plugins:
+  source:
+    - module: intake_xarray
+
+sources:
+
+  2D:
+    driver: zarr
+    description: 2D fields
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/COSMO_PGW_BC_2D/COSMO_PGW_BC_2D.ref
+      consolidated: false
+    metadata: {}
+
+  3D:
+    driver: zarr
+    description: 2D fields
+    args:
+      urlpath: reference://
+      storage_options:
+        fo: https://swift.dkrz.de/v1/dkrz_0913c8f3-e7b6-4f94-9221-06880d4ccfea/COSMO_PGW_BC_3D/COSMO_PGW_BC_3D.ref
+      consolidated: false
+    metadata: {}

Simulations/EUREC4A-MIP/warming/setup/main.yaml

@@ -0,0 +1,11 @@
+plugins:
+  source:
+     - module: intake_xarray
+
+sources:
+  boundary_conditions:
+    args:
+      path: "{{CATALOG_DIR}}/bc.yaml"
+    description: "Boundary conditions"
+    driver: yaml_file_cat
+    metadata: {}

Simulations/main.yaml

@@ -23,3 +23,10 @@ sources:
     description: 'Grids of simulations'
     driver: yaml_file_cat
     metadata: {}
+
+  EUREC4A_MIP:
+    args:
+      path: "{{CATALOG_DIR}}/EUREC4A-MIP/main.yaml"
+    description: "EUREC4A Model intercomparison (MIP) simulations and setup"
+    driver: yaml_file_cat
+    metadata: {}

barbados/BCO/bco.yaml

@@ -44,6 +44,13 @@ sources:
     driver: yaml_file_cat
     metadata: {}
 
+  POLDIRAD:
+    args:
+      path: "{{CATALOG_DIR}}/poldirad.yaml"
+    description: 'C-band Polarimetric Doppler radar (POLDIRAD)'
+    driver: yaml_file_cat
+    metadata: {}
+
   radar_reflectivity:
     description: Radar reflectivities measured with the Ka-Band radar at the Barbados Cloud Observatory.
     driver: zarr

barbados/BCO/poldirad.yaml

@@ -0,0 +1,15 @@
+plugins:
+  source:
+    - module: intake_xarray
+
+sources:
+  gridded:
+    driver: zarr
+    description: Poldirad measurements on Barbados during EUREC4A - Gridded data
+    args:
+      urlpath: reference::ipfs://QmRq663s4S9pwmRpbovzZJ7dRVXyoriSdBALxo7oCxGVUG
+      consolidated: false
+    metadata:
+      doi: https://doi.org/10.25326/217
+      maintainer: Martin Hagen <martin.hagen[at]dlr.de>, Florian Ewald <florian.ewald[at]dlr.de>
+      note: This catalog entry references the original data given at the DOI but includes additional processing steps to provide a joint dataset. Please access the original dataset to check if potential issues persist.

requirements.txt

@@ -1,11 +1 @@
-# the most recent version of intake-xarray (v3.2.2) only supports the OPeNDAP
-# servers by URS and ESGF, the fork bellow allows for connecting to general OPeNDAP servers
-intake[dataframe] # since intake 0.6.1 to_dask() doesn't work anymore without the [dataframe] specification due to a missing msgpack dependency
-xarray
-zarr
-fsspec>=0.7.4
-netcdf4!=1.5.4,!=1.5.5,!=1.5.5.1,!=1.6.0,!=1.6.1,!=1.6.2,!=1.6.3,!=1.6.4 # this is due to https://github.com/Unidata/netcdf4-python/issues/1052 and a CURL error in 1.6.0 (https://github.com/Unidata/netcdf-c/issues/2459) and https://github.com/eurec4a/eurec4a-intake/issues/121
-s3fs
-ipfsspec
-requests
-intake-xarray>=0.3.2
+eurec4a[data]>=0.0.6