Feature/refactor streaming

[sandorkertesz]

Fixes #364

This PR:

• addsbatched() and group_by() for stream and fieldlist like objects
• adds theread_all option for from_source() when stream=True
• removes the batch_size and group_by options from from_source (This is a breaking change!)

Examples

ds = from_source("url", "http://..../my_data.grib", stream=True)

for f in ds:
     # f is a field

# at this point ds consumed the stream

ds1 = from_source("file", "my_local_data.grib")
ds2 = from_source("url", "http://..../my_data.grib", stream=True)

for f in ds1.batched(2):
     # f is now a Fieldlist with 2 Fields

for f in ds2.batched(2):
     # f is now a Fieldlist with 2 Fields

ds1 = from_source("file", "my_local_data.grib")
ds2 = from_source("url", "http://..../my_data.grib", stream=True)

for f in ds1.group_by("level"):
     # f is a Fieldlist

for f in ds2.group_by("level"):
     # f is a Fieldlist

ds = from_source("url", "http://..../my_data.grib", stream=True, read_all=True)

# ds is a Fieldlist in memory, so all these work
len(ds)
r = ds.sel(param="t")

for f in ds:
     # f isa Field

for f in ds.batched(2):
     # f is a Fieldlist with 2 Fields

Documentation

• batched: https://earthkit-data--371.org.readthedocs.build/en/371/_api/data/core/fieldlist/index.html#data.core.fieldlist.FieldList.batched
• group_by https://earthkit-data--371.org.readthedocs.build/en/371/_api/data/core/fieldlist/index.html#data.core.fieldlist.FieldList.group_by
• streams: https://earthkit-data--371.org.readthedocs.build/en/371/guide/streams.html

Notebooks

https://earthkit-data--371.org.readthedocs.build/en/371/examples/data_from_stream.html
https://earthkit-data--371.org.readthedocs.build/en/371/examples/url_stream.html
https://earthkit-data--371.org.readthedocs.build/en/371/examples/fdb.html

[iainrussell]

Apart from that suggestion (what do you think @sandorkertesz?), I think this looks really really nice - a good and more intuitive way of doing things.

[codecov-commenter]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

[tlmquintino]

Looks good. Thanks for clearing up the interface.