upgrades for model v2 (#29)

* update rest endpoints, remove v1 geometry * remove cache kwarg * allow printing urls, parse datetimes * update forecast and stats plots * remove samples, fix plots * drop compute from analyze function names * use dictionary comprehension for return periods * move plots to subpackage * use anon requests and check directory exists for table cache * correct handling requests without a reach_id * remove partially implemented plots * correct downloads * update downloading metadata tables * consolidate table download functions * update gitignore * correct parsing dates, calculate simple forecast * update recognized date formats * remove jinja2 dependency * correct plots * specify release candidate version for publishing early versions * forecast stats analysis function * allow retrieving forecasts from aws * filter statistics to complete years * correct dataframe filters * update geoglows capitalization * remove units references * revise location for downloading metadata table * change download protocol from s3 to https * update docs
geoglows · Apr 9, 2024 · 42365c3 · 42365c3
1 parent e9b463a
commit 42365c3
Show file tree

Hide file tree

Showing 53 changed files with 2,097 additions and 2,260 deletions.
diff --git a/.gitignore b/.gitignore
@@ -6,4 +6,5 @@ build
 dev.ipynb
 geoglows.egg-info
 dist
-.pypirc
+.pypirc
+*.parquet
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-# GEOGloWS
+# GEOGLOWS
 [![Conda Version](https://img.shields.io/conda/vn/conda-forge/geoglows.svg)](https://img.shields.io/conda/vn/conda-forge/geoglows.svg)
 [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/geoglows.svg)](https://anaconda.org/conda-forge/geoglows)
 [![PyPI](https://img.shields.io/pypi/v/geoglows)](https://pypi.org/project/geoglows)
@@ -16,8 +16,6 @@ Also available on pypi
 pip install geoglows
 ```
 
-Tools to access and process data coming from the GEOGloWS initiative.
+Tools to access and process data coming from the GEOGLOWS initiative.
 
 https://geoglows.readthedocs.io
-
-The sample data are from ReachID 13073600 which is on the mississippi river
diff --git a/docs/api-documentation.rst b/docs/api-documentation.rst
@@ -7,19 +7,18 @@ There are 3 modules in the geoglows package.
 .. toctree::
    :maxdepth: 3
 
-   api-documentation/streamflow
+   api-documentation/data
    api-documentation/bias
    api-documentation/plots
-   api-documentation/analysis
-   api-documentation/examples
+   api-documentation/analyze
 
 
 FAQ
 ~~~
 
 How do I save streamflow data to csv?
 -------------------------------------
-By default, the results of most of the `geoglows.streamflow` functions return a pandas DataFrame. You can save those to
+By default, the results of most of the `geoglows.data` functions return a pandas DataFrame. You can save those to
 a csv, json, pickle, or other file. For example, save to csv with the dataframe's ``.to_csv()`` method.
 
 .. code-block:: python

diff --git a/docs/api-documentation/analysis.rst b/docs/api-documentation/analysis.rst
diff --git a/docs/api-documentation/analyze.rst b/docs/api-documentation/analyze.rst
@@ -0,0 +1,11 @@
+================
+geoglows.analyze
+================
+
+Analyze
+~~~~~~~
+Functions which post process results from the streamflow data service into additional, useful products
+
+.. automodule:: geoglows.analyze
+    :members: gumbel1, simple_forecast, forecast_stats, daily_averages, monthly_averages, annual_averages, daily_stats,
+              daily_variance, daily_flow_anomaly, return_periods, low_return_periods
diff --git a/docs/api-documentation/data.rst b/docs/api-documentation/data.rst
@@ -0,0 +1,30 @@
+=============
+geoglows.data
+=============
+
+The data module provides functions for requesting forecasted and historical data river discharge simulations.
+The data can be retrieved from the REST data service hosted by ECMWF or it can be retrieved from the repository sponsored
+by the AWS Open Data Program. The speed and reliability of the AWS source is typically better than the REST service.
+
+In general, each function requires a river ID. The name for the ID varies based on the streams network dataset. It is called
+LINKNO in GEOGLOWS which uses the TDX-Hydro streams dataset. This is the same as a reach_id or common id (COMID) used previously.
+To find a LINKNO (river ID number), please refer to https://data.geoglows.org and browse the tutorials.
+
+Forecasted Streamflow
+---------------------
+
+Historical Simulation
+---------------------
+
+.. automodule:: geoglows.data
+    :members:
+        retrospective, return_periods, annual_averages, monthly_averages, daily_averages,
+    :noindex:
+
+GEOGLOWS Model Utilities
+------------------------
+
+.. automodule:: geoglows.data
+    :members:
+        metadata_tables
+    :noindex:
diff --git a/docs/api-documentation/examples.rst b/docs/api-documentation/examples.rst
diff --git a/docs/api-documentation/streamflow.rst b/docs/api-documentation/streamflow.rst
diff --git a/docs/dev-notes.rst b/docs/dev-notes.rst
diff --git a/docs/index.rst b/docs/index.rst
@@ -7,22 +7,22 @@ geoglows
 .. image:: https://anaconda.org/geoglows/geoglows/badges/latest_release_date.svg
         :target: https://anaconda.org/geoglows/geoglows
 
-The geoglows python package is intended to promote access to data, API's, and code developed for the `GEOGloWS ECMWF Streamflow Model <https://geoglows.ecmwf.int>`_.
-Read more about GEOGloWS at `<https://geoglows.org>`_
+The geoglows Python package enables access to data, API's, and code developed for the `GEOGLOWS Streamflow Model <https://geoglows.ecmwf.int>`_.
+Read more about GEOGLOWS at `<https://geoglows.org>`_
 
 Demos
 =====
 These links will be maintained to reference the most updated versions of the tutorials.
 The tutorials are GitHub Gists which you can copy and launch in a Google Collaboratory setting directly from the GitHub.
 
 - Retrieve & plot GEOGloWS model data: `<https://gist.github.com/rileyhales/873896e426a5bd1c4e68120b286bc029>`_
-- Finding Stream ID #'s programatically: `<https://gist.github.com/rileyhales/ad92d1fce3aa36ef5873f2f7c2632d31>`_
+- Finding Stream ID #'s programmatically: `<https://gist.github.com/rileyhales/ad92d1fce3aa36ef5873f2f7c2632d31>`_
 - Bias Evaluation and Calibration at a point: `<https://gist.github.com/rileyhales/d5290e12b5858d59960d0898fbd0ed69>`_
 - Generate/Download High Res Plot Images: `<https://gist.github.com/rileyhales/9b5bbb0c5f307eb14b9f1ced39d641e4>`_
 
 About GEOGloWS ECMWF Streamflow
 ===============================
-GEOGloWS ECMWF Streamflow Project: This project provides access to the results of a hydrologic model that is run each
+GEOGLOWS ECMWF Streamflow Project: This project provides access to the results of a hydrologic model that is run each
 day. The model is based on a group of unique weather forecasts, known as an ensemble, from ECMWF. Each unique
 precipitation forecast, known as an ensemble member, produces a unique streamflow forecast. There are 52 members of the
 ensemble that drives the model each day. The ERA-5 historical precipitation dataset to also used to produce a
@@ -34,5 +34,4 @@ hindcasted streamflow on each river. `Read more here <https://geoglows.ecmwf.int
     :maxdepth: 1
 
     api-documentation
-    dev-notes
     license
diff --git a/docs/license.rst b/docs/license.rst
@@ -3,7 +3,7 @@ License
 
 BSD 3-Clause License
 
-Copyright (c) 2021, Riley Hales
+Copyright (c) 2024, Riley Hales
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/environment.yaml b/environment.yaml
@@ -0,0 +1,19 @@
+name: pygeoglows
+channels:
+  - conda-forge
+dependencies:
+  - python>=3
+  - dask
+  - fastparquet
+  - requests
+  - pandas
+  - plotly>=5
+  - jinja2
+  - shapely>=2
+  - scipy>=1
+  - s3fs
+  - numpy>=1
+  - hydrostats
+  - HydroErr
+  - xarray
+  - zarr
diff --git a/geoglows/__init__.py b/geoglows/__init__.py
@@ -1,10 +1,16 @@
 import geoglows.bias
-import geoglows.plots
-import geoglows.streamflow
-import geoglows.analysis
-import geoglows.examples
+import geoglows._plots as plots
+import geoglows.data
+import geoglows.analyze
+import geoglows.streams
+import geoglows.tables
 
-__all__ = ['bias', 'plots', 'streamflow', 'analysis']
-__version__ = '0.27.1'
+from ._constants import METADATA_TABLE_LOCAL_PATH as METADATA_TABLE_PATH
+
+__all__ = [
+    'bias', 'plots', 'data', 'analyze', 'streams', 'tables',
+    'METADATA_TABLE_PATH'
+]
+__version__ = '1.0.0'
 __author__ = 'Riley Hales'
 __license__ = 'BSD 3-Clause Clear License'
diff --git a/geoglows/_constants.py b/geoglows/_constants.py
@@ -0,0 +1,3 @@
+import os
+
+METADATA_TABLE_LOCAL_PATH = os.path.join(os.path.dirname(__file__), 'data', 'metadata-tables.parquet')
diff --git a/geoglows/_plots/__init__.py b/geoglows/_plots/__init__.py
@@ -0,0 +1,4 @@
+from .plots import *
+
+
+__all__ = []
diff --git a/geoglows/_plots/format_tools.py b/geoglows/_plots/format_tools.py
@@ -0,0 +1,28 @@
+from plotly.offline import plot as offline_plot
+
+
+def build_title(main_title, plot_titles: list):
+    if plot_titles is not None:
+        main_title += '<br>'.join(plot_titles)
+    return main_title
+
+
+def return_period_plot_colors():
+    return {
+        '2 Year': 'rgba(254, 240, 1, .4)',
+        '5 Year': 'rgba(253, 154, 1, .4)',
+        '10 Year': 'rgba(255, 56, 5, .4)',
+        '20 Year': 'rgba(128, 0, 246, .4)',
+        '25 Year': 'rgba(255, 0, 0, .4)',
+        '50 Year': 'rgba(128, 0, 106, .4)',
+        '100 Year': 'rgba(128, 0, 246, .4)',
+    }
+
+
+def plotly_figure_to_html_plot(figure, include_plotlyjs: bool = False, ) -> str:
+    return offline_plot(
+        figure,
+        config={'autosizable': True, 'responsive': True},
+        output_type='div',
+        include_plotlyjs=include_plotlyjs
+    )
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		import os

		METADATA_TABLE_LOCAL_PATH = os.path.join(os.path.dirname(__file__), 'data', 'metadata-tables.parquet')