Merge branch 'develop' into rpc_inspection

.gitignore

@@ -28,3 +28,5 @@ __pycache__
 start-rabbitmq
 stop-rabbitmq
 rabbitmq.log
+.coverage
+htmlcov/

README.md

@@ -1,4 +1,5 @@
 ![image](docs/source/images/VOLLTRON_Logo_Black_Horizontal_with_Tagline.png)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/fcf58045b4804edf8f4d3ecde3016f76)](https://app.codacy.com/gh/VOLTTRON/volttron?utm_source=github.com&utm_medium=referral&utm_content=VOLTTRON/volttron&utm_campaign=Badge_Grade_Settings)
 
 VOLTTRON™ is an open source platform for distributed sensing and control. The
 platform provides services for collecting and storing data from buildings and
@@ -273,10 +274,10 @@ tail volttron.log
 Listener agent heartbeat publishes appear in the logs as:
 
 ```sh
-2016-10-17 18:17:52,245 (listeneragent-3.2 11367) listener.agent INFO: Peer: 'pubsub', Sender: 'listeneragent-3.2_1'
-:, Bus: u'', Topic: 'heartbeat/listeneragent-3.2_1', Headers:
-{'Date': '2016-10-18T01:17:52.239724+00:00', 'max_compatible_version': u'', 'min_compatible_version': '3.0'},
-Message: {'status': 'GOOD', 'last_updated': '2016-10-18T01:17:47.232972+00:00', 'context': 'hello'}
+2020-04-20 18:49:31,395 (listeneragent-3.3 13458) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.2_1:, Bus: , Topic: heartbeat/listeneragent-3.2_1, Headers: {'TimeStamp': '2020-04-20T18:49:31.393651+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
+'GOOD'
+2020-04-20 18:49:36,394 (listeneragent-3.3 13458) __main__ INFO: Peer: pubsub, Sender: listeneragent-3.2_1:, Bus: , Topic: heartbeat/listeneragent-3.2_1, Headers: {'TimeStamp': '2020-04-20T18:49:36.392294+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message:
+'GOOD'
 ```
 
 To top the platform run the following command:

ci-integration/virtualization/requirements_test.txt

@@ -4,4 +4,5 @@ pytest-timeout
 mock
 websocket-client
 numpy>1.13<2
-pandas
+pandas
+mysql-connector-python-rf

docs/source/community_resources/index.rst

@@ -12,19 +12,19 @@ Slack Channel
 ^^^^^^^^^^^^^
 
 volttron-community.slack.com is where the |VOLTTRON| community at large can ask questions and meet with others
-using |VOLTTRON| Signup via https://volttron-community.signup.team/
+using |VOLTTRON|.  To be added to Slack please email the VOLTTRON team at
+`volttron@pnnl.gov <mailto:volttron@pnnl.gov?subject=Subscribe%20To%20List>`__.
 
 Mailing List
 ^^^^^^^^^^^^
 
-Join the mailing list by emailing
-`volttron@pnnl.gov <mailto:volttron@pnnl.gov?subject=Subscribe%20To%20List>`__.
+Join the mailing list by emailing `volttron@pnnl.gov <mailto:volttron@pnnl.gov?subject=Subscribe%20To%20List>`__.
 
 Stack Overflow
 ^^^^^^^^^^^^^^
 
 The VOLTTRON community supports questions being asked and answered through Stack Overflow.  The questions tagged with
-the volttron tag can be found at http://stackoverflow.com/questions/tagged/volttron.
+the `volttron` tag can be found at http://stackoverflow.com/questions/tagged/volttron.
 
 Office Hours
 ^^^^^^^^^^^^

docs/source/core_services/control/AgentStatus.rst

@@ -21,9 +21,8 @@ platform along with their uuid, associated `tag <AgentTag>`__, and
    which was installed. Agents can be controlled with this using "--name
    ". Note, if multiple instances of a wheel are installed they will all
    have the same name and can be controlled as a group.
--  `TAG <AgentTag>`__ is a user provided tag which makes it simpler to
-   track and refer to agents. Using "--tag " agents can be controlled
-   using this
+-  `TAG <AgentTag>`__ is a user-provided tag which makes it simpler to
+   track and refer to agents. Agents can be controlled by using "--tag".
 -  PRI is the priority for agents which have been "enabled" using the
    ``vctl enable`` command. When enabled, agents will be
    automatically started in priority order along with the platform.

docs/source/core_services/drivers/driver_configuration/modbus-tk-driver.rst

@@ -145,8 +145,9 @@ Each row configures a register definition on the device.
       Default is FALSE.
     - **Default Value** (Optional) - The point's default value. If it is reverted by an agent, it changes back
       to this value. If this value is missing, it will revert to the last known value not set by an agent.
-    - **Transform** (Optional) - Scaling algorithm: scale(multiplier), scale_int(multiplier), mod10k(reverse),
-      or none. Default is an empty string.
+    - **Transform** (Optional) - Scaling algorithm: scale(multiplier), scale_int(multiplier), scale_reg(register_name),
+      scale_reg_power10(register_name), scale_decimal_int_signed(multiplier), mod10k(reverse),
+      mod10k64(reverse), mod10k48(reveres) or none. Default is an empty string.
     - **Table** (Optional) - Standard modbus table name defining how information is stored in slave device.
       There are 4 different tables:
 

docs/source/core_services/drivers/driver_configuration/the-energy-detective-driver.rst

@@ -0,0 +1,140 @@
+.. _The-Energy-Detective-Driver:
+
+The Energy Detective Meter Driver
+------------------------------------
+
+
+Introduction
+------------
+
+The TED-Pro is an energy monitoring system that can measure energy consumption
+of multiple mains and supports submetering of individual circuits. 
+This driver connects to a TED Pro Energy Control Center (ECC) and can collect
+information from multiple Measuring Transmiting Units (MTUs) and Spyder submetering
+devices connected to the ECC.
+
+configuration
+-------------
+
+The TED Pro device interface is configured as follows. You'll need the ip address
+or hostname of the ECC on a network segment accessible from the Volttron instance, 
+if configured to use a port other than 80, you can provide it as shown below,
+following a colon after the host address. 
+
+.. code-block:: json
+
+    {
+        "driver_type": "ted_meter", 
+        "driver_config": {
+            "device_address": "192.168.1.100:8080", 
+            "username": "username", 
+            "password": "password", 
+            "scrape_spyder": true, 
+            "track_totalizers": true
+        }
+    }
+
+Parameters
+**********
+
+    - **username** - Username if the TED Pro is configured with Basic Authentication
+    - **password** - Password if the TED Pro is configured with Basic Authentication
+    - **device_address** - Hostname or IP address of the TED Pro ECC, a non-standard port can be included if needed
+    - **scrape_spyder** - Default true, enables or disables collection of the submetering data from spyder devices 
+    connected to the TED Pro
+    - **track_totalizers** - Default true, enables or disables tracking of lifetime totals in the VOLTTRON Driver
+
+.. note::
+
+    The TED Pro does not expose its internal lifetime totalized metering, instead offering month to date (MTD)
+    and daily totals (TDY). Using the "track_totalizers" setting, the ted-meter driver will attempt to maintain
+    monotonically increasing lifetime totalizers. To do so, it must retain state regarding the running total and
+    the last read value. The driver makes use of the VOLTTRON Config subsystem to store this state.
+    To reset these totals, delete the state/ted_meter/<device_path> config from the master driver config store and restart the 
+    master driver.
+
+.. note::
+
+    This driver does not make use of the registry config. Because it is able to determine the configuration
+    of the TED Pro Device via the API, it simply creates registers for each data source on the TED Pro
+
+
+.. note::
+
+    This driver is internally aware of the appropriate HayStack Tags for its registers, however, the 
+    MasterDriver Framework makes no provision for publishing those tags during a scrape. Therefore,
+    integration of the tagging data is left to the end user.
+
+Examples
+********
+
+|TED Pro showing spyder outputs|
+
+The above configuration in the TED will result in the following scrape from the ted-meter driver on the message bus:
+
+.. code-block:: text
+
+    [
+        {
+            'mtu-1/load_kva': 0.271,
+            'mtu-1/load_kw': 0.203,
+            'mtu-1/phase_angle': 195,
+            'mtu-1/phase_current-a': '0',
+            'mtu-1/phase_current-b': '0',
+            'mtu-1/phase_current-c': '0',
+            'mtu-1/phase_voltage-a': '0',
+            'mtu-1/phase_voltage-b': '0',
+            'mtu-1/phase_voltage-c': '0',
+            'mtu-1/power_factor': 0.749,
+            'mtu-1/voltage': 121.30000000000001,
+            'spyder-1/AHU/load': 0.0,
+            'spyder-1/AHU/mtd': 0.0,
+            'spyder-1/AHU/mtd_totalized': 0.0,
+            'spyder-1/C/U/load': 0.0,
+            'spyder-1/C/U/mtd': 0.0,
+            'spyder-1/C/U/mtd_totalized': 0.0,
+            'spyder-1/Fridge/load': 0.0,
+            'spyder-1/Fridge/mtd': 0.056,
+            'spyder-1/Fridge/mtd_totalized': 0.056,
+            'spyder-1/HW/load': 0.0,
+            'spyder-1/HW/mtd': 0.14400000000000002,
+            'spyder-1/HW/mtd_totalized': 0.14400000000000002,
+            'spyder-1/Toaster/load': 0.0,
+            'spyder-1/Toaster/mtd': 0.24,
+            'spyder-1/Toaster/mtd_totalized': 0.24,
+            'system/mtd': 0.652,
+            'system/mtd_totalized': 0.652
+        },
+        {
+            'mtu-1/load_kva': {'type': 'integer', 'tz': u'', 'units': 'kVA'},
+            'mtu-1/load_kw': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'mtu-1/phase_angle': {'type': 'integer', 'tz': u'', 'units': 'degrees'},
+            'mtu-1/phase_current-a': {'type': 'integer', 'tz': u'', 'units': 'Amps'},
+            'mtu-1/phase_current-b': {'type': 'integer', 'tz': u'', 'units': 'Amps'},
+            'mtu-1/phase_current-c': {'type': 'integer', 'tz': u'', 'units': 'Amps'},
+            'mtu-1/phase_voltage-a': {'type': 'integer', 'tz': u'', 'units': 'Volts'},
+            'mtu-1/phase_voltage-b': {'type': 'integer', 'tz': u'', 'units': 'Volts'},
+            'mtu-1/phase_voltage-c': {'type': 'integer', 'tz': u'', 'units': 'Volts'},
+            'mtu-1/power_factor': {'type': 'integer', 'tz': u'', 'units': 'ratio'},
+            'mtu-1/voltage': {'type': 'integer', 'tz': u'', 'units': 'Volts'},
+            'spyder-1/AHU/load': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'spyder-1/AHU/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/AHU/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/C/U/load': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'spyder-1/C/U/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/C/U/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/Fridge/load': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'spyder-1/Fridge/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/Fridge/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/HW/load': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'spyder-1/HW/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/HW/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/Toaster/load': {'type': 'integer', 'tz': u'', 'units': 'kW'},
+            'spyder-1/Toaster/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'spyder-1/Toaster/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'system/mtd': {'type': 'integer', 'tz': u'', 'units': 'kWh'},
+            'system/mtd_totalized': {'type': 'integer', 'tz': u'', 'units': 'kWh'}
+        }
+    ]
+
+.. |TED Pro showing spyder outputs| image:: ../files/ted-spyders.png

docs/source/devguides/agent_development/Agent-Development.rst

@@ -167,16 +167,16 @@ store values and setting up a configuration handler.
 Values in the default config can be built into the agent or come from the
 packaged configuration file. The subscribe method tells our agent which function
 to call whenever there is a new or updated config file. For more information
-on using the configuration store see :doc:`Agent Configuration Store <Agent-Configuration-Store>`
+on using the configuration store see :doc:`Agent Configuration Store <Agent-Configuration-Store>`.
 
-`_create_subscriptions` (covered in the next section) will use the value in self.setting2
+`_create_subscriptions` (covered in the next section) will use the value in `self.setting2`
 to create a new subscription.
 
 Setting up a Subscription
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The Agent creates a subscription using the value of self.setting2 in the method
-`_create_subscription`. The messages for this subscription hare handeled with
+The Agent creates a subscription using the value of `self.setting2` in the method
+`_create_subscription`. The messages for this subscription are handled with
 the `_handle_publish` method:
 
 ::
@@ -191,20 +191,21 @@ the `_handle_publish` method:
 
         def _handle_publish(self, peer, sender, bus, topic, headers,
                                     message):
+            #By default no action is taken.
             pass
 
 Agent Lifecycle Events
 ^^^^^^^^^^^^^^^^^^^^^^
 
-Methods may be setup to be called at agent startup and shudown:
+Methods may be setup to be called at agent startup and shutdown:
 
 ::
 
         @Core.receiver("onstart")
         def onstart(self, sender, **kwargs):
             """
-            This is method is called once the Agent has successfully connected to the platform.
-            This is a good place to setup subscriptions if they are not dynamic or
+            This method is called once the Agent has successfully connected to the platform.
+            This is a good place to setup subscriptions if they are not dynamic or to
             do any other startup activities that require a connection to the message bus.
             Called after any configurations methods that are called at startup.
 
@@ -224,15 +225,15 @@ Methods may be setup to be called at agent startup and shudown:
             """
             pass
 
-As the comment mentions. With the new configuration store feature `onstart` methods
+As the comment mentions, with the new configuration store feature `onstart` methods
 are mostly unneeded. However this code does include an example of how to do a Remote
-Proceedure Call to another agent.
+Procedure Call to another agent.
 
-Agent Remote Proceedure Calls
+Agent Remote Procedure Calls
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-An agent may receive commands from other agents via a Remote Proceedure Call or RPC for short.
-This is done with the @RPC.export decorattor:
+An agent may receive commands from other agents via a Remote Procedure Call (RPC).
+This is done with the `@RPC.export` decorator:
 
 ::
 
@@ -248,7 +249,7 @@ This is done with the @RPC.export decorattor:
 Packaging Configuration
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-The wizard will automatically create a setup.py file. This file sets up the
+The wizard will automatically create a `setup.py` file. This file sets up the
 name, version, required packages, method to execute, etc. for the agent based on
 your answers to the wizard. The packaging process will also use this
 information to name the resulting file.
@@ -288,20 +289,20 @@ information to name the resulting file.
 Launch Configuration
 ~~~~~~~~~~~~~~~~~~~~
 
-In TestAgent, the wizard will automatically create a file called "config".
+In TestAgent, the wizard will automatically create a JSON file called "config".
 It contains configuration information for the agent. This file contains
-examples every datatype supported by the configuration system:
+examples of every datatype supported by the configuration system:
 
 ::
 
     {
       # VOLTTRON config files are JSON with support for python style comments.
       "setting1": 2, #Integers
-      "setting2": "some/random/topic2", #strings
+      "setting2": "some/random/topic2", #Strings
       "setting3": true, #Booleans: remember that in JSON true and false are not capitalized.
       "setting4": false,
       "setting5": 5.1, #Floating point numbers.
-      "setting6": [1,2,3,4], # Lists
+      "setting6": [1,2,3,4], #Lists
       "setting7": {"setting7a": "a", "setting7b": "b"} #Objects
     }
 
@@ -316,8 +317,8 @@ To install the agent the platform must be running. Start the platform with the c
 ``./start-volttron``
 
 .. note:: If you are not in an activated environment, this script will start
-    the platform running in the background in the correct environment, however
-    the environment will not be activated for you, you must activate it yourself.
+    the platform running in the background in the correct environment. However
+    the environment will not be activated for you; you must activate it yourself.
 
 Now we must install it into the platform. Use the following command to install it and add a tag for easily referring to
 the agent. From the project directory, run the following command:
@@ -331,16 +332,16 @@ This will result in output similar to the following:
 
 .. code-block:: bash
 
-      AGENT                    IDENTITY           TAG       STATUS          HEALTH
-    e testeragent-0.5          testeragent-0.5_1  testagent
+      AGENT                    IDENTITY           TAG       PRI
+  df  testeragent-0.5          testeragent-0.5_1  testagent
 
-Where the number or letter is the unique portion of the full uuid for the agent. AGENT is
+The initial number or letter is a unique portion of the full UUID for the agent. AGENT is
 the "name" of the agent based on the contents of its class name and the version in its setup.py. IDENTITY is the
 agent's identity in the platform. This is automatically assigned based on class name and instance number. This agent's
-ID is _1 because it is the first instance. TAG is the name we assigned in the command above. HEALTH
-is the current health of the agent as reported by the agents health subsystem. 
+ID is _1 because it is the first instance. TAG is the name we assigned in the command above. PRI is the priority for
+agents which have been "enabled" using the ``vctl enable`` command.
 
-When using lifecycle commands on agents, they can be referred to be UUID (default) or AGENT (name) or TAG.
+When using lifecycle commands on agents, they can be referred to by the UUID (default) or AGENT (name) or TAG.
 
 
 Testing the Agent
@@ -352,28 +353,30 @@ From the Command Line
 To test the agent, we will start the platform (if not already running), launch the agent, and
 check the log file.
 
--  With the VOLTTRON environment activated, start the platform by
-   running (if needed):
+With the VOLTTRON environment activated, start the platform by running (if needed):
 
 ``./start-volttron``
 
--  Launch the agent by <uuid> using the result of the list command:
+You can launch the agent in three ways, all of which you can find by using the
+``vctl list`` command:
+
+-  By using the <uuid>:
 
 ``vctl start <uuid>``
 
--  Launch the agent by name with:
+-  By name:
 
 ``vctl start --name testeragent-0.1``
 
--  Launch the agent by tag with:
+-  By tag:
 
 ``vctl start --tag testagent``
 
--  Check that it is :ref:`running <AgentStatus>`:
+Check that it is :ref:`running <AgentStatus>`:
 
 ``vctl status``
 
--  Start the ListenerAgent as in :ref:`Building VOLTTRON <Building-VOLTTRON>`
+-  Start the ListenerAgent as in :ref:`Building VOLTTRON <Building-VOLTTRON>`.
 -  Check the log file for messages indicating the TestAgent is receiving
    the ListenerAgents messages: