Merge pull request #2459 from VOLTTRON/rtd_restructure

RTD to develop restructure

docs/source/agent-framework/agents-overview.rst

@@ -0,0 +1,43 @@
+.. _Agent-Framework:
+
+===============
+Agents Overview
+===============
+
+Agents in VOLTTRON can be loosely defined as software modules communicating on the platform which perform some function
+on behalf of the user.  Agents may perform a huge variety of tasks, but common use cases involve data collection,
+control of ICS and IOT devices, and various platform management tasks.  Agents implemented using the VOLTTRON agent
+framework inherit a number of capabilities, including message bus connectivity and agent lifecycle.
+
+Agents deployed on VOLTTRON can perform one or more roles which can be broadly classified into the following groups:
+
+-  Platform Agents: Agents which are part of the platform and provide a service to other agents. Examples are the
+   Actuator and Master Driver agents which serve as interfaces between control agents and drivers.
+-  Control Agents: These agents implement algorithms to control the devices of interest and interact with other
+   resources to achieve some goal.
+-  Service Agents: These agents perform various data collection or platform management services.  Agents in this
+   category include weather service agents which collect weather data from remote sources or operations agents which
+   help users maintain situational awareness of their deployment.
+-  Cloud Agents: These agents represent a remote application which needs access to the messages and data on the
+   platform. This agent would subscribe to topics of interest to the remote application and would also allow it publish
+   data to the platform.
+
+The platform includes some valuable services which can be leveraged by agents:
+
+-  Message Bus: All agents and services publish and subscribe to topics on the message bus. This provides a single
+   interface that abstracts the details of devices and agents from each other. Components in the platform basically
+   produce and consume events.
+-  Configuration Store: Using the configuration store, agent operations can be altered ad-hoc without significant
+   disruption or downtime.
+-  Historian Framework: Historian agents automatically collect data from a subset of topics on the message bus and store
+   them in a data store of choice.  Currently SQL, MongoDB, CrateDB and other historians exist, and more can be
+   developed to fit the needs of a deployment by inheriting from the base historian.  The base historian has been
+   developed to be fast and reliable, and to handle many common pitfalls of data collection over a network.
+-  Weather Information: These agents periodically retrieve data from the a remote weather API then format the
+   response and publish it to the platform message bus on a weather topic.
+-  Device interfaces: Drivers publish device data onto the message bus and send control signals issued from control
+   agents to the corresponding device.  Drivers are capable of handling the locking of devices to prevent multiple
+   conflicting directives.
+-  Application Scheduling: This service allows the scheduling of agents’ access to devices in order to prevent conflicts.
+-  Logging service: Agents can publish arbitrary strings to a logging topic and this service will push them to a
+   historian for later analysis.

docs/source/core_services/control/Agent-Execution-Environment.rst → docs/source/agent-framework/aip.rst

@@ -1,3 +1,10 @@
+.. _Agent-Instantiation-and-Packaging:
+
+=======================================
+AIP - Agent Instantiation and Packaging
+=======================================
+
+
 .. _Agent-Execution-Environment:
 
 Used Environmental Variables
@@ -6,3 +13,6 @@ Used Environmental Variables
 - **AGENT_VIP_IDENTITY** - The router address an agent will attempt to connect to.
 - **AGENT_CONFIG** - The path to a configuration file to use during agent launch.
 - **VOLTTRON_HOME** - The home directory where the volttron instances is located.
+
+Documentation coming soon!
+

docs/source/agent-framework/core-service-agents/dnp3_and_mesa/dnp3-agent.rst

@@ -0,0 +1,197 @@
+.. _DNP3-Agent:
+
+==========
+DNP3 Agent
+==========
+
+`DNP3 <https://en.wikipedia.org/wiki/DNP3>`_ (Distributed Network Protocol) is a set of communications protocols that
+are widely used by utilities such as electric power companies, primarily for
+`SCADA <https://en.wikipedia.org/wiki/SCADA>`_ purposes.  It was adopted in 2010 as
+`IEEE Std 1815-2010 <http://ieeexplore.ieee.org/document/5518537/?reload=true>`_,
+later updated to `1815-2012 <https://standards.ieee.org/findstds/standard/1815-2012.html>`_.
+
+VOLTTRON's DNP3 Agent is an implementation of a DNP3 Outstation as specified in IEEE Std 1815-2012.  It engages in
+bidirectional network communications with a DNP3 Master, which might be located at a power utility.
+
+Like some other VOLTTRON protocol agents (e.g. IEEE2030_5Agent), the DNP3 Agent can optionally be front-ended by a DNP3
+device driver running under VOLTTRON's MasterDriverAgent.  This allows a DNP3 Master to be treated like any other device
+in VOLTTRON's ecosystem.
+
+The VOLTTRON DNP3 Agent implementation of an Outstation is built on PyDNP3, an open-source library from Kisensum
+containing Python language bindings for Automatak's C++ `opendnp3 <https://www.automatak.com/opendnp3/>`_ library, the
+de facto reference implementation of DNP3.
+
+The DNP3 Agent exposes DNP3 application-layer functionality, creating an extensible base from which specific custom
+behavior can be designed and supported.  By default, the DNP3 Agent acts as a simple transfer agent, publishing data
+received from the Master on the VOLTTRON Message Bus, and responding to RPCs from other VOLTTRON agents by sending data
+to the Master.
+
+
+Requirements
+============
+
+PyDNP3 can be installed in an activated environment with:
+
+.. code-block:: bash
+
+    pip install pydnp3
+
+
+RPC Calls
+---------
+
+The DNP3 Agent exposes the following VOLTTRON RPC calls:
+
+.. code-block:: python
+
+    def get_point(self, point_name):
+        """
+            Look up the most-recently-received value for a given output point.
+
+        @param point_name: The point name of a DNP3 PointDefinition.
+        @return: The (unwrapped) value of a received point.
+        """
+
+    def get_point_by_index(self, group, index):
+        """
+            Look up the most-recently-received value for a given point.
+
+        @param group: The group number of a DNP3 point.
+        @param index: The index of a DNP3 point.
+        @return: The (unwrapped) value of a received point.
+        """
+
+    def get_points(self):
+        """
+            Look up the most-recently-received value of each configured output point.
+
+        @return: A dictionary of point values, indexed by their VOLTTRON point names.
+        """
+
+    def set_point(self, point_name, value):
+        """
+            Set the value of a given input point.
+
+        @param point_name: The point name of a DNP3 PointDefinition.
+        @param value: The value to set. The value's data type must match the one in the DNP3 PointDefinition.
+        """
+
+    def set_points(self, point_list):
+        """
+            Set point values for a dictionary of points.
+
+        @param point_list: A dictionary of {point_name: value} for a list of DNP3 points to set.
+        """
+
+    def config_points(self, point_map):
+        """
+            For each of the agent's points, map its VOLTTRON point name to its DNP3 group and index.
+
+        @param point_map: A dictionary that maps a point's VOLTTRON point name to its DNP3 group and index.
+        """
+
+    def get_point_definitions(self, point_name_list):
+        """
+            For each DNP3 point name in point_name_list, return a dictionary with each of the point definitions.
+
+            The returned dictionary looks like this:
+
+            {
+                "point_name1": {
+                    "property1": "property1_value",
+                    "property2": "property2_value",
+                    ...
+                },
+                "point_name2": {
+                    "property1": "property1_value",
+                    "property2": "property2_value",
+                    ...
+                }
+            }
+
+            If a definition cannot be found for a point name, it is omitted from the returned dictionary.
+
+        :param point_name_list: A list of point names.
+        :return: A dictionary of point definitions.
+        """
+
+
+Pub/Sub Calls
+-------------
+
+The DNP3 Agent uses two topics when publishing data to the VOLTTRON message bus:
+
+ * **Point Values (default topic: `dnp3/point`)**: As the DNP3 Agent communicates with the Master,
+   it publishes received point values on the VOLTTRON message bus.
+
+ * **Outstation status (default topic: dnp3/status)**: If the status of the DNP3 Agent outstation
+   changes, for example if it is restarted, it publishes its new status on the VOLTTRON message bus.
+
+
+Data Dictionary of Point Definitions
+------------------------------------
+
+The DNP3 Agent loads and uses a data dictionary of point definitions, which are maintained by agreement between the
+(DNP3 Agent) Outstation and the DNP3 Master.  The data dictionary is stored in the agent's registry.
+
+
+Current Point Values
+--------------------
+
+The DNP3 Agent tracks the most-recently-received value for each point definition in its data dictionary, regardless of
+whether the point value's source is a VOLTTRON RPC call or a message from the DNP3 Master.
+
+
+Agent Configuration
+-------------------
+
+The DNP3Agent configuration file specifies the following fields:
+
+ - **local_ip**: (string) Outstation's host address (DNS resolved).  Default: ``0.0.0.0``.
+ - **port**: (integer) Outstation's port number - the port that the remote endpoint (Master) is listening on.  Default:
+   20000.
+ - **point_topic**: (string) VOLTTRON message bus topic to use when publishing DNP3 point values.  Default:
+   ``dnp3/point``.
+ - **outstation_status_topic**: (string) Message bus topic to use when publishing outstation status.  Default:
+   ``dnp3/outstation_status``.
+ - **outstation_config**: (dictionary) Outstation configuration parameters. All are optional. Parameters include:
+
+   - **database_sizes**: (integer) Size of each outstation database buffer. Default: 10.
+   - **event_buffers**: (integer) Size of the database event buffers.  Default: 10.
+   - **allow_unsolicited**: (boolean) Whether to allow unsolicited requests.  Default: ``True``.
+   - **link_local_addr**: (integer) Link layer local address.  Default: 10.
+   - **link_remote_addr**: (integer) Link layer remote address.  Default: 1.
+   - **log_levels**: (list) List of bit field names (`OR'd` together) that filter what gets logged by DNP3.  Default:
+     ``NORMAL``. Possible values: ``ALL``, ``ALL_APP_COMMS``, ``ALL_COMMS``, ``NORMAL``, ``NOTHING``.
+   - **threads_to_allocate**: (integer) Threads to allocate in the manager's thread pool.  Default: 1.
+
+A sample DNP3 Agent configuration file is available in `services/core/DNP3Agent/dnp3agent.config`.
+
+
+VOLTTRON DNP3 Device Driver
+===========================
+
+VOLTTRON's DNP3 device driver exposes `get_point`/`set_point` RPC calls and scrapes for DNP3 points.
+
+The driver periodically issues DNP3Agent RPC calls to refresh its cached representation of DNP3 data.  It issues RPC
+calls to the DNP3 Agent as needed when responding to `get_point`, `set_point` and `scrape_all` calls.
+
+For information about the DNP3 driver, see :ref:`DNP3 Driver Configuration <DNP3-Driver-Config>`.
+
+
+Installing the DNP3 Agent
+=========================
+
+To install DNP3Agent, please consult the installation advice in `services/core/DNP3Agent/README.md`.  `README.md`
+specifies a default agent configuration, which can be overridden as needed.
+
+An agent installation script is available:
+
+.. code-block:: shell
+
+    $ export VOLTTRON_ROOT=<volttron github install directory>
+    $ cd $VOLTTRON_ROOT
+    $ source services/core/DNP3Agent/install_dnp3_agent.sh
+
+When installing the Mesa Agent, please note that the agent's point definitions must be loaded into the agent's config
+store.  See `install_dnp3_agent.sh` for an example of how to load them.