spil server side: updates and doc

README.md

@@ -275,6 +275,13 @@ It is possible to run actions on the currently selected Sid.
 
 **spil_ui** is a separate repository (in the process of being released).   
 
+## REST API
+
+Spil can run server side.
+
+![](docs/img/rest-small.png)
+
+*(REST API and docker under development)*
 
 ## Flexible and configurable
 

docker-compose.yml

@@ -9,13 +9,14 @@ services:
     ports:
       - "80:80"
 
+    # you can replace "./spil_hamlet_conf" by an actual config
     volumes:
       - ./spil_hamlet_conf:/spil_conf
 
-    # for development: edit the config will reload the app
+    # for development: this is needed to make hot reload work (see below)
     environment:
       - WATCHFILES_FORCE_POLLING=true
 
-    # development options for config hot reload.
+    # for development: runs with hot reload, reloads server when the config folder changes
     command: ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80", "--reload", "--reload-dir", "/spil_conf"]
-    # https://www.uvicorn.org/settings/
+    # See https://www.uvicorn.org/settings/

docs/client_server.md

@@ -0,0 +1,61 @@
+# Spil network deployment
+
+Client / Server deployment of Spil is still experimental, and work in progress.
+
+## Server side Spil
+
+A fastapi powered Spil REST API is currently under development.
+
+It allows access to the Crud interface via a rest api.
+- /find/{config}/{sid}
+- /get/{config}/{sid}
+- /write/{config}/{sid}
+
+![](img/rest.png)
+
+### Running the API
+
+- Either from the cloned repo:  
+  (includes creation of test data)  
+```shell
+git clone https://github.com/MichaelHaussmann/spil.git
+cd spil
+python -c "import spil;import hamlet_scripts.save_examples_to_mock_fs as mfs;mfs.run()"
+cd spil_server/fastapi/app
+uvicorn main:app --reload
+```
+
+The API will run on [http://127.0.0.1:8000/find](http://127.0.0.1:8000/find) and [http://127.0.0.1:8000/get](http://127.0.0.1:8000/get)    
+Example: [http://127.0.0.1:8000/find/all/hamlet/a/**/maya?asset=*&task=rig&version=v001](http://127.0.0.1:8000/find/all/hamlet/a/**/maya?asset=*&task=rig&version=v001)  
+
+
+- Using Docker compose
+```shell
+git clone https://github.com/MichaelHaussmann/spil.git
+cd spil
+docker compose up -d
+```
+This will build `spil/spil_server/docker/Dockerfile`
+And run as defined in `spil/docker-compose.yml`  
+
+The API will run on [http://127.0.0.1/find](http://127.0.0.1/find) and [http://127.0.0.1/get](http://127.0.0.1/get)    
+Example: [http://127.0.0.1/find/all/hamlet/a/**/maya?asset=*&task=rig&version=v001](http://127.0.0.1/find/all/hamlet/a/**/maya?asset=*&task=rig&version=v001)  
+
+
+### Production config
+
+Check out comments in the Dockerfile `spil/spil_server/docker/Dockerfile`  
+and the docker-compose `spil/docker-compose.yml`  
+It is possible to map a production configuration onto the docker config folder. 
+
+
+## Client Side Spil
+
+A FindInSpilRest Finder is also in development. 
+
+It is able to consume the Spil rest API.
+This Finder can replace any other finder, and be used without any change in the code.
+
+**This is not currently implemented**
+
+

docs/index.rst

@@ -18,6 +18,7 @@
    installation
    configuration
    configuration_advanced
+   client_server
 
 .. toctree::
    :caption: User Guides

docs/installation.md

@@ -76,3 +76,9 @@ Found: hamlet/a/char/gertrude
 
 You are all set, and can experiment with Spil.
 
+## Server Side Install
+
+Work in progress Docker and REST API.  
+See [Spil network deployment](client_server.md)
+
+

docs/overview.md

@@ -273,6 +273,13 @@ It is possible to run actions on the currently selected Sid.
 
 **spil_ui** is a separate repository (in the process of being released).   
 
+## REST API
+
+Spil can run server side.
+
+![](img/rest-small.png)
+
+*(REST API and docker under development)*
 
 ## Flexible and configurable
 

spil_hamlet_conf/spil_rest_conf.py

@@ -0,0 +1,35 @@
+# type: ignore
+"""
+Demo configuration for the rest API.
+
+Defines Finder and Getter instances for routes formatted as:
+"/find/{config}/{search:path}"
+where "config" is the name of the config.
+
+"""
+from spil import FindInList
+from spil import FindInAll, GetFromAll, FindInPaths, GetFromPaths
+# from spil_plugins.sg.get_sg import GetFromSG
+# from spil_plugins.sg.find_sg import FindInSG
+from spil_hamlet_conf.hamlet_scripts.example_sids import sids
+
+
+# Example:
+# "/find/all/hamlet/*"
+# will call FindInAll().find()
+finder_config = {
+    # 'sg': FindInSG(),
+    'all': FindInAll(),
+    'paths': FindInPaths(),
+    'ls': FindInList(list(sids))
+}
+
+
+# Example:
+# "/get/all/hamlet/*"
+# will call GetFromAll().get()
+getter_config = {
+    # 'sg': GetFromSG(),
+    'all': GetFromAll(),
+    'paths': GetFromPaths()
+}

spil_server/docker/Dockerfile

@@ -2,7 +2,6 @@
 # Build:                docker build -t spil/spil -f spil_server/docker/Dockerfile .
 # Run with python:      docker run -ti spil/spil python
 
-#FROM python:3.11
 FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
 
 COPY ./spil_server/docker/requirements.txt /spil/requirements.txt
@@ -19,6 +18,7 @@ ENV PYTHONPATH="${PYTHONPATH}:/spil:/spil_conf"
 COPY ./spil_hamlet_conf /spil/spil_hamlet_conf
 
 # Init test data
+# Only used for builtin demo. Can be removed for real usage.
 # PYTHONDONTWRITEBYTECODE avoids to have __pycache__ in the Image
 RUN export PYTHONDONTWRITEBYTECODE=1 && python -c "import spil;import hamlet_scripts.save_examples_to_mock_fs as mfs;mfs.run()"
 

spil_server/fastapi/app/main.py

@@ -1,44 +1,71 @@
 """
-uvicorn main:app --reload
+This file is part of SPIL, The Simple Pipeline Lib.
+
+(C) copyright 2019-2023 Michael Haussmann, spil@xeo.info
+
+SPIL is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+
+SPIL is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more details.
+
+You should have received a copy of the GNU Lesser General Public License along with SPIL.
+If not, see <https://www.gnu.org/licenses/>.
 """
 from __future__ import annotations
-from typing import Any, Optional, List
-
 from pathlib import Path
 from fastapi import FastAPI
 from starlette.requests import Request
 
+"""
+
+
+* This is work in progress *
+* Not production ready *
+
+
+https://fastapi.tiangolo.com/deployment/docker/ (includes a nice intro to dockers)
+https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker
+
+Mount volume in Windows:
+https://medium.com/@kale.miller96/how-to-mount-your-current-working-directory-to-your-docker-container-in-windows-74e47fa104d7
+
+uvicorn main:app --reload
+"""
 try:
     import spil
 except ImportError:
     root = Path(__file__).resolve().parent.parent.parent.parent
-    import sys
+    import sys  # fmt: skip
     sys.path.append(root.as_posix())
 
-from spil import Sid, Finder, Getter, FindInList
-from spil import FindInAll, GetFromAll, FindInPaths, GetFromPaths
-# from spil_plugins.sg.get_sg import GetFromSG
-# from spil_plugins.sg.find_sg import FindInSG
-from spil_hamlet_conf.hamlet_scripts.example_sids import sids
+from spil import Finder, Getter
+
+try:
+    import spil_rest_conf as rest_conf  # type: ignore
+except ImportError:
+    from spil.conf import sid_conf_import_error_message  # fmt: skip
+    problem = sid_conf_import_error_message.format(module="spil_rest_conf")
+    print(problem)
+    rest_conf = None
 
-app = FastAPI()
 
-finder_config = {
-    # 'sg': FindInSG(),
-    'all': FindInAll(),
-    'paths': FindInPaths(),
-    'ls': FindInList(list(sids))
-}
+app = FastAPI()
 
-getter_config = {
-    # 'sg': GetFromSG(),
-    'all': GetFromAll(),
-    'paths': GetFromPaths()
-}
 
 @app.get("/find/{config}/{search:path}")
 def find(config: str, search: str, request: Request):
-    finder: Finder | None = finder_config.get(config)
+    """
+    Returns Sids for named "config" and given "search" sid.
+
+    Args:
+        config: name of a config
+        search: search sid
+        request: optional request (is automatically handled)
+
+    Returns:
+        the retrieved sids, as json list.
+
+    """
+    finder: Finder | None = rest_conf.finder_config.get(config)
     if not finder:
         print(f"Finder not found for {config}")
         return []
@@ -50,9 +77,21 @@ def find(config: str, search: str, request: Request):
     for sid in finder.find(search):
         yield {"sid": sid.uri}
 
+
 @app.get("/get/{config}/{search:path}")
 def get(config: str, search: str, request: Request):
-    getter: Getter | None = getter_config.get(config)
+    """
+    Returns Sids and data for named "config" and given "search" sid.
+
+    Args:
+        config: name of a config
+        search: search sid
+        request: optional request (is automatically handled)
+
+    Returns:
+        the retrieved data, as json list.
+    """
+    getter: Getter | None = rest_conf.getter_config.get(config)
     if not getter:
         print(f"Finder not found for {config}")
         return []