Merge pull request #206 from ecmwf/release/0.4.0

Release/0.4.0

.github/workflows/ci.yml

@@ -52,6 +52,7 @@ jobs:
         environment-name: DEVELOP
         channels: conda-forge
         cache-env: true
+        cache-env-key: ubuntu-latest-3.10
         extra-specs: |
           python=3.10
     - name: Install package
@@ -61,6 +62,34 @@ jobs:
       run: |
         make unit-tests
 
+  unit-tests-no-eccodes:
+    name: unit-tests (3.10)
+    if: ${{ !github.event.pull_request.head.repo.fork && github.event.action != 'labeled' || github.event.label.name == 'approved-for-ci' }}
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        ref: ${{ github.event.pull_request.head.sha || github.ref }}
+    - name: Install Conda environment with Micromamba
+      uses: mamba-org/provision-with-micromamba@v14
+      with:
+        environment-file: tests/environment-unit-tests.yml
+        environment-name: DEVELOP
+        channels: conda-forge
+        cache-env: true
+        cache-env-key: ubuntu-latest-3.10-no-eccodes
+        extra-specs: |
+          python=3.10
+    - name: Install package
+      run: |
+        python -m pip install --no-deps -e .
+        micromamba remove eccodes
+    - name: Run tests without eccodes
+      run: |
+        python -m pytest -v -m 'no_eccodes'
+
+
   type-check:
     needs: [unit-tests]
     if: ${{ !github.event.pull_request.head.repo.fork && github.event.action != 'labeled' || github.event.label.name == 'approved-for-ci' }}

.pre-commit-config.yaml

@@ -15,7 +15,7 @@ repos:
   hooks:
   - id: isort
 - repo: https://github.com/psf/black
-  rev: 23.3.0
+  rev: 23.9.1
   hooks:
   - id: black
 - repo: https://github.com/keewis/blackdoc

.readthedocs.yml

@@ -1,6 +1,9 @@
 version: 2
 formats: []
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.9"
 python:
-  version: "3.8"
   install:
   - requirements: docs/requirements.txt

README.md

@@ -1,18 +1,17 @@
 # earthkit-data
 
-![earthkit-data](docs/_static/earthkit-data.png)
+<img src="docs/_static/earthkit-data.png" width="120">
 
+[![PyPI version fury.io](https://badge.fury.io/py/earthkit-data.svg)](https://pypi.python.org/pypi/earthkit-data/)
 [![PyPI pyversions](https://img.shields.io/pypi/pyversions/earthkit-data.svg)](https://pypi.python.org/pypi/earthkit-data/)
 
+**DISCLAIMER**
+
+> This project is in the **BETA** stage of development. Please be aware that interfaces and functionality may change as the project develops. If this software is to be used in operational systems you are **strongly advised to use a released tag in your system configuration**, and you should be willing to accept incoming changes and bug fixes that require adaptations on your part. ECMWF **does use** this software in operations and abides by the same caveats.
+
 A format-agnostic interface for geospatial data with a focus on meteorology and
 climate science.
 
-> :warning: **DISCLAIMER**
->
-> This project is **BETA** and will be **Experimental** for the foreseeable future.
-> Interfaces and functionality are likely to change, and the project itself may be scrapped.
-> **DO NOT** use this software in any project/software that is operational.
-
 ## Documentation
 
 The documentation can be found at https://earthkit-data.readthedocs.io/.

docs/api.rst

@@ -30,6 +30,11 @@ BUFR
 - :py:class:`~data.readers.bufr.bufr.BUFRList`
 - :py:class:`~data.readers.bufr.bufr.BUFRMessage`
 
+CSV
+----
+
+- :py:class:`~data.readers.csv.CSVReader`
+
 Other
 --------
 

docs/conf.py

@@ -35,6 +35,7 @@
     "nbsphinx",
     "sphinx.ext.autodoc",
     "sphinx.ext.napoleon",
+    "sphinx.ext.intersphinx",
     "autoapi.extension",
     "earthkit.data.sphinxext.xref",
     "earthkit.data.sphinxext.module_output",
@@ -109,6 +110,9 @@
 }
 
 
+intersphinx_mapping = {"pandas": ("https://pandas.pydata.org/docs/", None)}
+
+
 def setup(app):
     from skip_api_rules import _skip_api_items
 

docs/examples.rst

@@ -15,6 +15,9 @@ Here is a list of example notebooks to illustrate how to use earthkit-data.
     examples/cds.ipynb
     examples/ecmwf_open_data.ipynb
     examples/fdb.ipynb
+    examples/mars.ipynb
+    examples/polytope.ipynb
+    examples/wekeo.ipynb
 
 
 .. toctree::
@@ -33,6 +36,7 @@ Here is a list of example notebooks to illustrate how to use earthkit-data.
     examples/grib_tar.ipynb
     examples/grib_url.ipynb
     examples/grib_to_netcdf.ipynb
+    examples/numpy_fieldlist.ipynb
 
 .. toctree::
     :maxdepth: 1
@@ -73,4 +77,3 @@ Here is a list of example notebooks to illustrate how to use earthkit-data.
     examples/cache.ipynb
     examples/projection.ipynb
     examples/metadata.ipynb
-    examples/numpy_fieldlist.ipynb

docs/examples/cds.ipynb

@@ -211,7 +211,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.8.18"
   }
  },
  "nbformat": 4,

docs/examples/grib_lat_lon_value.ipynb

@@ -13,7 +13,7 @@
    "id": "c74a86a6-79ae-44bc-a9c9-8cc8a3aaa484",
    "metadata": {},
    "source": [
-    "We read a GRIB file containing pressure level data. First we ensure the example file is available."
+    "In this example we will work with pressure level GRIB data. First we ensure the example file is available."
    ]
   },
   {
@@ -32,7 +32,7 @@
    "id": "ae4ff8c4-d87c-4d57-83a5-38e1e2545556",
    "metadata": {},
    "source": [
-    "We create a fieldlist with the temperature fields only."
+    "We will only use the temperature fields, so we extract them from the data."
    ]
   },
   {
@@ -198,7 +198,7 @@
    "id": "c7ec94f3-4e68-4380-a88d-128b2fb46cd8",
    "metadata": {},
    "source": [
-    "The simplest way to access the latitudes, longitudes and values is to use the *data()* method. It works both on fields and fieldlists."
+    "The simplest way to access the latitudes, longitudes and values is using the *data()* method. It works both on fields and fieldlists."
    ]
   },
   {
@@ -311,7 +311,7 @@
    "id": "a21f46b4-6c79-4306-a005-390ef2a02173",
    "metadata": {},
    "source": [
-    "*data()* only works on a fieldlist if all the fields has the same grid. The first two elements of the resulting ndarray are the latitude and longitude arrays (shared between fields), while the rest of the elements are the value arrays per field. Since we have six fields in our data the size of the first axis of the resulting ndarray is 2+6=8."
+    "*data()* only works on a fieldlist if all the fields have the same grid. The first two elements of the resulting ndarray are the latitude and longitude arrays (shared between fields), while the rest of the elements are the value arrays per field. Since we have six fields in our data the size of the first axis of the resulting ndarray is 2+6=8."
    ]
   },
   {
@@ -582,7 +582,7 @@
    "id": "392bdc90-0e15-415f-892b-7ebbf86cbc8c",
    "metadata": {},
    "source": [
-    "*to_latlon()* only works on a fieldlist if all the fields has the same grid. In this case it returns the same dict that we would get for any of the fields:"
+    "*to_latlon()* only works on a fieldlist if all the fields have the same grid. In this case it returns the same dict that we would get for any of the fields:"
    ]
   },
   {
@@ -679,7 +679,7 @@
    "id": "3bbbaad8-f9b7-4faa-b9ad-7a696f391b40",
    "metadata": {},
    "source": [
-    "For *to_numpy()* we can set the array type with the *dtype* keyword both for fields and fieldlists:"
+    "For all the methods above we can set the array type with the *dtype* keyword both for fields and fieldlists:"
    ]
   },
   {
@@ -708,11 +708,35 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 18,
    "id": "c89a473d-c6fd-4400-bfbc-da35bdcf8dc3",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "llv = ds.data(dtype=np.float32)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "1e888bd9-48ec-4d18-8ef3-ee6d4fd107c0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([ 90.     ,   0.     , 272.56418, 272.53918, 271.26532, 255.84306,\n",
+       "       244.00323, 226.65315], dtype=float32)"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llv[:,0,0]"
+   ]
   }
  ],
  "metadata": {