24-NN-II-Backprop.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    },
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mp\n",
    "from IPython.display import Image, HTML\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#  NN II -- Compute Graph, Backprop and Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "In this lecture we'll gradually build out a light weight neural network training framework reminiscent of PyTorch.\n",
    "\n",
    "We build:\n",
    "* A simple neural network engine we call `Value` class that wraps numbers and math operators and includes useful attributes and methods for implementing _forward pass_ and _back propagation_. _(~63 lines of code)_\n",
    "\n",
    "We'll provide (and explain)\n",
    "* A simple _compute graph_ visualization function. _(34 lines of code)_\n",
    "* A small set of helper functions that easily define a neuron, layer and multi-layer perceptron (MLP). _(84 lines of code)_\n",
    "\n",
    "With that we can implement a neural network training loop, and see how similar it is to a PyTorch implementation.\n",
    "\n",
    "---\n",
    "The code is based on Andrej Karpathy's [micrograd](https://github.com/karpathy/micrograd)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Neuron and Neural Networks (Recap)\n",
    "\n",
    "Now let's switch gears a bit to define an _artificial neuron_. For better or worse\n",
    "it is named after and loosely modeled on a biological neuron.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<!-- Image Credit \"https://cs231n.github.io/neural-networks-1/\"-->\n",
    "\n",
    "<center>\n",
    "    \n",
    "<img src=\"figs/NN-figs/neuron.png\" width=\"75%\">\n",
    "    \n",
    "</center> \n",
    "\n",
    "From [cs231n](https://cs231n.github.io/neural-networks-1/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "* The dendrites carry impulses from other neurons of different distances.\n",
    "* Once the collective firing rate of the impulses exceed a certain threshold, the neuron fires its own pulse through the axon to other neurons\n",
    "\n",
    "There are companies trying to mimic this impulse (i.e. spiking) based neuron in silicon -- so called _neuromorphic computing_.\n",
    "\n",
    "See for example [Neuromorphic Computing](https://en.wikipedia.org/wiki/Neuromorphic_engineering) or [Spiking Neural Network](https://en.wikipedia.org/wiki/Spiking_neural_network)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Some examples of companies and projects are Intel's [Loihi](https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html) and startups such as GrAI Matter Labs [VIP processor](https://www.graimatterlabs.ai/product)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Artificial Neuron\n",
    "\n",
    "<!-- Image Credit \"https://cs231n.github.io/neural-networks-1/\"-->\n",
    "\n",
    "<center>\n",
    "    \n",
    "<img src=\"figs/NN-figs/neuron_model.jpeg\" width=\"75%\">\n",
    "    \n",
    "</center> \n",
    "\n",
    "From [cs231n](https://cs231n.github.io/neural-networks-1/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The more common artifical neuron \n",
    "* collects one or more inputs, \n",
    "* each multiplied by a unique weight\n",
    "* sums the weighted inputs\n",
    "* adds a bias\n",
    "* then finally usually applies a nonlinear activation function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Relating Back to Earlier Lectures\n",
    "\n",
    "__Question__\n",
    "\n",
    "What does \n",
    "\n",
    "$$\\sum_i w_i x_i + b$$\n",
    "\n",
    "remind you of?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Answer__\n",
    "\n",
    "How about multiple regression in the Linear Regression lecture?\n",
    "\n",
    "$$y = \\beta_0 + \\beta_1 u + \\beta_2 v$$\n",
    "\n",
    "We just renamed things: $\\beta_0 \\gets b$, $\\beta_1 \\gets w_0$, $\\beta_2 \\gets w_1$, $u \\gets x_0$ and $v \\gets x_1$ for $i \\in [0,1]$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "> So multiple linear regression can be viewed as one linear neuron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Activation function is typically some nonlinear function that compresses the input in some way. Historically, it's been the sigmoid and $\\tanh()$ functions. See for example [Hyperbolic Functions](https://en.wikipedia.org/wiki/Hyperbolic_functions#Tanh)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "hide_input": true,
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHFCAYAAAAOmtghAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABTqUlEQVR4nO3deVxUZd8G8GtmGIYdBZRFEXFDDbcwE5fUVJRMLVssy7S0J1MzxbdS60mtt8eyUl/LpcX0aXPJpZUSyn1NkXLfF1RABGWHYZb7/QOZRAYcxhnOzJnr+/nwgbnnPoffPQeGi3OfRSGEECAiIiKSCaXUBRARERHZEsMNERERyQrDDREREckKww0RERHJCsMNERERyQrDDREREckKww0RERHJCsMNERERyQrDDREREckKww2Rk9u7dy8efvhhNGnSBBqNBsHBwYiNjcXUqVMr9evduzd69+4tTZEWOn/+PBQKBVasWHHbvrNmzYJCobBovW+88QaaNGkCNzc31KtX786KvEPffvstFixYYPY5hUKBWbNm1Wk9RHLkJnUBRGS9X375BUOGDEHv3r0xd+5chIaGIiMjA/v378eqVavw4YcfmvouXrxYwkotExoait27d6N58+Y2W+cPP/yAd955B6+//jri4+Oh0Whstm5rfPvttzh8+DAmT55c5bndu3ejcePGdV8Ukcww3BA5sblz5yIyMhIbN26Em9s/v85PPPEE5s6dW6lv27Zt67q8WtNoNOjatatN13n48GEAwKRJk9CwYUObrtvWbD12IlfFaSkiJ5aTk4OgoKBKwaaCUln519vctNSlS5fw6KOPwtfXF/Xq1cNTTz2Fffv2VZkaGj16NHx8fHD8+HEMGDAA3t7eCA0NxbvvvgsA2LNnD3r06AFvb2+0atUK//3vf6vUc/jwYQwdOhT169eHh4cHOnbsWKVfddNSv/zyCzp27AiNRoPIyEh88MEHFr0+TZs2xRtvvAEACA4OrjTtU90UUNOmTTF69GjT4xUrVkChUGDz5s148cUXERQUhMDAQAwbNgzp6elVlv/2228RGxsLHx8f+Pj4oGPHjli2bBmA8m3wyy+/4MKFC1AoFKaPCuZqsuR127JlCxQKBVauXInXX38dYWFh8PPzQ79+/XDixAmLXisiOWG4IXJisbGx2Lt3LyZNmoS9e/dCp9NZvGxRURH69OmDzZs347333sOaNWsQHByM4cOHm+2v0+kwbNgwDBo0CD/88APi4+Mxffp0zJgxA6NGjcJzzz2HDRs2ICoqCqNHj0ZKSopp2RMnTqBbt244cuQIFi5ciPXr16Nt27YYPXp0lT1Mt/rjjz8wdOhQ+Pr6YtWqVXj//fexZs0aLF++/LZj3LBhA8aMGQMA+O2337B7926MHTvW4tfoZmPHjoVarca3336LuXPnYsuWLXj66acr9XnzzTfx1FNPISwsDCtWrMCGDRswatQoXLhwAUD51GD37t0REhKC3bt3mz6qU9vXbcaMGbhw4QI+//xzfPrppzh16hQGDx4Mg8Fg1ZiJnJYgIqeVnZ0tevToIQAIAEKtVotu3bqJOXPmiIKCgkp9e/XqJXr16mV6vGjRIgFA/Prrr5X6vfDCCwKAWL58ualt1KhRAoBYt26dqU2n04kGDRoIAOLAgQOm9pycHKFSqURCQoKp7YknnhAajUakpaVV+l7x8fHCy8tL5ObmCiGEOHfuXJXvfe+994qwsDBRUlJiasvPzxcBAQHCkrewmTNnCgDi6tWrldoBiJkzZ1bpHxERIUaNGmV6vHz5cgFAjB8/vlK/uXPnCgAiIyNDCCHE2bNnhUqlEk899VSN9QwaNEhERESYfe7Wmix93TZv3iwAiAceeKBSvzVr1ggAYvfu3TXWRCQ33HND5MQCAwOxfft27Nu3D++++y6GDh2KkydPYvr06WjXrh2ys7OrXXbr1q3w9fXFwIEDK7U/+eSTZvsrFAo88MADpsdubm5o0aIFQkND0alTJ1N7QEAAGjZsaNpbAQCbNm1C3759ER4eXmmdo0ePRnFxcbV7L4qKirBv3z4MGzYMHh4epnZfX18MHjy42rHZw5AhQyo9bt++PQCYxpmcnAyDwYAJEybY7HvW9nW7XY1EroLhhkgGOnfujNdeew3fffcd0tPTMWXKFJw/f77GKZ+cnBwEBwdXaTfXBgBeXl6VAgYAuLu7IyAgoEpfd3d3lJaWVvpeoaGhVfqFhYWZnjfn+vXrMBqNCAkJqfKcuTZ7CgwMrPS44qyrkpISAMDVq1cBwKZnO9X2dbtdjUSuguGGSGbUajVmzpwJ4J8zhcwJDAzElStXqrRnZmbavKbAwEBkZGRUaa84IDcoKMjscvXr14dCoTBb053WqdFooNVqq7RXF7Rup0GDBgDKD9K2FWtfNyJXx3BD5MTM/eEDgGPHjgH45z98c3r16oWCggL8+uuvldpXrVpluwJv6Nu3LzZt2lTl7KIvv/wSXl5e1Z4C7e3tjS5dumD9+vWV9gQVFBTgp59+uqOamjZtioMHD1Zq27RpEwoLC61aX1xcHFQqFZYsWVJjP41GY/GeFGtfNyJXx+vcEDmxAQMGoHHjxhg8eDBat24No9GIv/76Cx9++CF8fHzw8ssvV7vsqFGjMH/+fDz99NP43//9X7Ro0QK//vorNm7cCKDqqeR3YubMmfj555/Rp08fvPnmmwgICMA333yDX375BXPnzoW/v3+1y7799tsYOHAg+vfvj6lTp8JgMOC9996Dt7c3rl27ZnVNI0eOxL///W+8+eab6NWrF44ePYqPP/64xlpq0rRpU8yYMQNvv/02SkpK8OSTT8Lf3x9Hjx5FdnY2Zs+eDQBo164d1q9fjyVLliAmJgZKpRKdO3c2u847ed2IXBnDDZETe+ONN/DDDz9g/vz5yMjIgFarRWhoKPr164fp06ejTZs21S7r7e2NTZs2YfLkyXj11VehUCgQFxeHxYsX44EHHrDpbQqioqKwa9cuzJgxAxMmTEBJSQnatGmD5cuXV7qmjDn9+/fH999/jzfeeAPDhw9HSEgIxo8fj5KSElNgsMYrr7yC/Px8rFixAh988AG6dOmCNWvWYOjQoVav86233kLLli3x0Ucf4amnnoKbmxtatmyJSZMmmfq8/PLLOHLkCGbMmIG8vDwIISCEMLu+O3ndiFyZQlT3W0VELuk///kP3njjDaSlpfFWAETklLjnhsiFffzxxwCA1q1bQ6fTYdOmTVi4cCGefvppBhsicloMN0QuzMvLC/Pnz8f58+eh1WrRpEkTvPbaa6ZbFhAROSNOSxEREZGs8FRwIiIikhWGGyIiIpIVhhsiIiKSFZc7oNhoNCI9PR2+vr5QKBRSl0NEREQWEEKgoKAAYWFht73IqMuFm/T09Cp32CUiIiLncPHixdteqsLlwo2vry+A8hfHz8/PpuvW6XRISkpCXFwc1Gq1TdftCOQ+PkD+Y+T4nJ/cx8jxOT97jTE/Px/h4eGmv+M1cblwUzEV5efnZ5dw4+XlBT8/P1n+0Mp9fID8x8jxOT+5j5Hjc372HqMlh5TwgGIiIiKSFYYbIiIikhWGGyIiIpIVhhsiIiKSFYYbIiIikhWGGyIiIpIVhhsiIiKSFYYbIiIikhWGGyIiIpIVhhsiIiKSFUnDzbZt2zB48GCEhYVBoVDg+++/v+0yW7duRUxMDDw8PNCsWTMsXbrU/oUSERGR05A03BQVFaFDhw74+OOPLep/7tw5PPDAA+jZsydSU1MxY8YMTJo0CevWrbNzpUREROQsJL1xZnx8POLj4y3uv3TpUjRp0gQLFiwAALRp0wb79+/HBx98gEceecROVRIREdUdIQSM4qbPEBCi/DmjKP9a3OhX/hnlDajcV9y0vn++Lu9zc4ebn6tSy41nb13nrfXeTK/XI1dr8XDtwqnuCr57927ExcVVahswYACWLVsGnU5n9u6jWq0WWu0/r3J+fj6A8ruW6nQ6m9ZXsT5br9dRyH18gPzHyPE5P7mP8U7HJ4RAic6AIq0BxToDSspufOjKP8r0RpTqjCjVG6DVG6HVGVFmMEKnL/9cZhDQGYzlH/ryr/VGgTKDEXqDgN5ohMEooDcKGG760BsFjEYBg6j4DNNjg7E8cBiFgFEI6HQqTNv/O4yiclip+FoO/NQqPPagff7GWsKpwk1mZiaCg4MrtQUHB0Ov1yM7OxuhoaFVlpkzZw5mz55dpT0pKQleXl52qTM5Odku63UUch8fIP8xcnzOT+5j/G1jMgp0QIEOKNQrUKQDCvVAkU6BYj1QYkD5Z70CJQag1ABob3wIKKQu/zYUgNEo4XevmqAUt3xR0yto7rlb29RK2/+MFhcXW9zXqcINACgUlV/Cit1ht7ZXmD59OhISEkyP8/PzER4ejri4OPj5+dm0Np1Oh+TkZPTv39/sXiRnJ/fxAfIfI8fn/OQwRoNRID2vBBevlSA9rxQZN31cyS9F+rVCFOnvLKAoFICXWgUPtQqe7ip4qpXwVKugUavg4aaEh1oFdzcl3N2U0Lgp4a4q/9pdpYRapYD6xmM3pQJuKgXclOXtbkoFVMryx6obj5WKf9pVSgWUCtz4XP6hUpb/jVIpFDAY9Ni9ayfu69kDarUaSoUCCgXKPwNQKsr73txW/udNceM5QHEjSpR/Xf6F4qbHCoXCFDYq/jRW9zfSHuz1M1ox82IJpwo3ISEhyMzMrNSWlZUFNzc3BAYGml1Go9FAo9FUaVer1XZ7Y7Dnuh2B3McHyH+MHJ/zc4YxFmr1OHWlACevFOBEZiHOZRfiQk4xLl4vhs5Q0/xL+R9iN6UCgT7uCPTWoL63GvW93BHg7Y56nmr4VXx4qOHn4QZfDzW8NSr4eLjBR+MGT7WqTv+gW0qn0+GUB9C0gZ/Db787Zeuf0dqsy6nCTWxsLH766adKbUlJSejcubPsf0iIiBxZfqkOhy7l4e9Lufj7Yi4OX87H5dySavu7q5QID/BEWD1PNKrniVB/T4TV80CgtxtOpP6JYQ/0QwM/LyiVjhdQyPFJGm4KCwtx+vRp0+Nz587hr7/+QkBAAJo0aYLp06fj8uXL+PLLLwEA48aNw8cff4yEhAQ8//zz2L17N5YtW4aVK1dKNQQiIpeUXajFnrM52H0mB3vPXcPprEKz/Rr6ahAV4otWwb5o1sAbkYHeiAjyRoifB1RmgotOp0PBSSDA253BhqwmabjZv38/+vTpY3pccWzMqFGjsGLFCmRkZCAtLc30fGRkJBITEzFlyhQsWrQIYWFhWLhwIU8DJyKyM73BiP0XruP3o1ew7dRVnLxSNcyEB3iifeN66Ni4HqIb+aN1iC/qe7tLUC25OknDTe/evaucH3+zFStWVGnr1asXDhw4YMeqiIgIALR6AzYfv4qko5nYfDwL14srn4rbOsQXsc0DEdssEDER9RHoU/X4RiIpONUxN0REZF9CCPx9KQ/rUi7hx7/TkVfyT6Cp56XG/VEN0bdNMGKbByKAe2XIQTHcEBERCkp1WL3vIlbtu1jp+JkQPw880C4U/dsG456m9eGm4v2WyfEx3BARubD03BKs2HUeK/emoUCrBwB4qJUYeFcIHolpjG7Ng8we+EvkyBhuiIhc0Jmrhfjoj1P4+WAG9MbyYx9bNPTBc90jMbhDKHw9eHkNcl4MN0RELiSroBT/9/sprNp3EYYboSa2WSCevy8SvVs15OnXJAsMN0RELqBIq8en287is+1nUVxmAAD0a9MQL/dthXaN/SWujsi2GG6IiGTu96NX8Pr3h3AlXwsA6BBeDzPiW+PeZuZvW0Pk7BhuiIhk6lpRGWb/dAQ//JUOAGgS4IXXBrbGA+1CHPK+S0S2wnBDRCQzQggkHsrEmz8cRk5RGZQK4Pn7mmFKv1bwUKukLo/I7hhuiIhkpKTMgNc3HML61MsAgFbBPnj/0Q7oEF5P2sKI6hDDDRGRTKTlFOOFr1NwLCMfKqUCE3o3x4T7W0Djxr015FoYboiIZGDziSxMXvUX8kp0CPJxx0dP3o3Y5jxgmFwTww0RkRMTQuDjTacx7/eTEALoGF4PS56+G6H+nlKXRiQZhhsiIidlMAr8+4fD+HZvGgDgqXub4M3BbTkNRS6P4YaIyAnpDEZMX/cXfvgrHQoF8PbQaDzdNULqsogcAsMNEZGTKTMAE1b+hc0nsuGmVGD+8I4Y3CFM6rKIHAbDDRGREynU6vHJcSVO52dD46bE0qdj0Kd1Q6nLInIoDDdERE6iVGfA818dwOl8Jbw1Knwx6h7eQoHIDIYbIiInYDAKTF71F/ZfyIWHSuCrZzvj7qYMNkTmKKUugIiIaiaEwKwfj+C3I5lQqxQYG2VEu0a8kzdRdRhuiIgc3KLNp/HVngtQKIAPH22Hlv5C6pKIHBrDDRGRA1uz7yI+SDoJAJj5YFvER4dIXBGR42O4ISJyULvP5GD6hkMAgBd7N8fo7pESV0TkHBhuiIgcUFZBKV5amQqDUeChjmF4dUCU1CUROQ2GGyIiB6M3GDFpZSqyC7WICvbFnGHtoVAopC6LyGkw3BAROZgFv5/CnrPX4O2uwuKn74anO+8VRVQbDDdERA5k84ksfLz5NABgziPt0byBj8QVETkfhhsiIgeRnluChNV/AQBGdo3AEN4visgqDDdERA7AeOMKxNeLdWjXyB9vPNhG6pKInBbDDRGRA/h67wX8eb78OJtFI+6Gxo3H2RBZi+GGiEhil64X471fjwMApsW3RpNAL4krInJuDDdERBISQmDGhsMoKjPgnqb18dS9EVKXROT0GG6IiCS0/sBlbDt5Fe5uSrz7SHsolbyeDdGdYrghIpLI1QIt3vr5KADg5b4tedo3kY0w3BARSWTWj0eQV6JD21A//Ou+ZlKXQyQbDDdERBL449gV/HIoAyqlAnMfbQ+1im/HRLbC3yYiojqmMxjxTuIxAMCYHpGIbuQvcUVE8sJwQ0RUx1b+mYazV4sQ4O2Oife3kLocItlhuCEiqkP5pTos+P0UAGByv5bw81BLXBGR/DDcEBHVoUWbT+NaURmaNfDGk12aSF0OkSwx3BAR1ZGL14qxfMd5AMCM+DY8iJjITvibRURUR+ZuPIEygxGxzQLRt01Dqcshki2GGyKiOpCadh0//Z0OhQJ4fVAbKBS8EjGRvTDcEBHZmRACcxLLb4w5rFNjnvpNZGcMN0REdrb7TA7+PH8N7m5K/M+AVlKXQyR7DDdERHb20abTAIAn7glHqL+nxNUQyR/DDRGRHe0/fw27z+ZArVLghV7NpS6HyCUw3BAR2dHHm8v32jxyd2M0qse9NkR1geGGiMhODl3Kw5YTV6FUAC/25l4borrCcENEZCcfbSq/zcLQjo0QEegtcTVEroPhhojIDo5n5iPp6BUoFMB47rUhqlMMN0REdrBo8xkAQHx0CFoG+0pcDZFrYbghIrKxM1cL8fPBdADAxD4tJa6GyPUw3BAR2diyHecgBNC3dUO0DfOTuhwil8NwQ0RkQ3nFOmw4cBkA8Px9zSSuhsg1MdwQEdnQmv0XUaIzoHWIL+6NDJC6HCKXJHm4Wbx4MSIjI+Hh4YGYmBhs3769xv7ffPMNOnToAC8vL4SGhuLZZ59FTk5OHVVLRFQ9g1Hgyz3nAQCjujXlnb+JJCJpuFm9ejUmT56M119/HampqejZsyfi4+ORlpZmtv+OHTvwzDPPYMyYMThy5Ai+++477Nu3D2PHjq3jyomIqtp8PAsXr5XA31ONhzo2krocIpclabiZN28exowZg7Fjx6JNmzZYsGABwsPDsWTJErP99+zZg6ZNm2LSpEmIjIxEjx498MILL2D//v11XDkRUVX/3X0eADD8nnB4uqukLYbIhblJ9Y3LysqQkpKCadOmVWqPi4vDrl27zC7TrVs3vP7660hMTER8fDyysrKwdu1aDBo0qNrvo9VqodVqTY/z8/MBADqdDjqdzgYj+UfF+my9Xkch9/EB8h8jx2c/Z64WYfupbCgVwJOdG9mtBm5D5yb38QH2G2Nt1qcQQgibfncLpaeno1GjRti5cye6detmav/Pf/6D//73vzhx4oTZ5dauXYtnn30WpaWl0Ov1GDJkCNauXQu1Wm22/6xZszB79uwq7d9++y28vLxsMxgicnlrzyqx/YoS7eobMba1UepyiGSnuLgYI0aMQF5eHvz8ar7EgmR7bircesCdEKLag/COHj2KSZMm4c0338SAAQOQkZGBV155BePGjcOyZcvMLjN9+nQkJCSYHufn5yM8PBxxcXG3fXFqS6fTITk5Gf379682bDkzuY8PkP8YOT77KCjVY0bKVgAG/M/Qe9CteaDdvhe3oXOT+/gA+42xYubFEpKFm6CgIKhUKmRmZlZqz8rKQnBwsNll5syZg+7du+OVV14BALRv3x7e3t7o2bMn/vd//xehoaFVltFoNNBoNFXa1Wq13X6w7LluRyD38QHyHyPHZ1s//HkJRWUGtGzog/uiguvkLCluQ+cm9/EBth9jbdYl2QHF7u7uiImJQXJycqX25OTkStNUNysuLoZSWblklar8oD2JZteIyMUZjQJf7r4AAHiGp38TOQRJz5ZKSEjA559/ji+++ALHjh3DlClTkJaWhnHjxgEon1J65plnTP0HDx6M9evXY8mSJTh79ix27tyJSZMmoUuXLggLC5NqGETkwvaeu4Zz2UXw0bhhWCee/k3kCCQ95mb48OHIycnBW2+9hYyMDERHRyMxMREREREAgIyMjErXvBk9ejQKCgrw8ccfY+rUqahXrx7uv/9+vPfee1INgYhc3HcpFwEAgzuEwlsj+WGMRAQHOKB4/PjxGD9+vNnnVqxYUaXtpZdewksvvWTnqoiIbq9Qq8evh8qPG3w0JlziaoioguS3XyAicla/HExHic6AZg28cXeTelKXQ0Q3MNwQEVnpu/2XAACPxYTzQGIiB8JwQ0RkhTNXC7H/wnWolAo8cjcPJCZyJAw3RERWWJtSvtemV6sGaOjnIXE1RHQzhhsioloyGAXWH6iYkmoscTVEdCuGGyKiWtp26iqu5GtR30uNvm3MX1GdiKTDcENEVEtrbxxIPLRjI7i78W2UyNHwt5KIqBauF5Uh+egVAMBjnTklReSIGG6IiGrhh78uo8xgRNtQP9wV5i91OURkBsMNEVEt/PB3OgDgUR5ITOSwGG6IiCx08VoxUtNyoVQAD3YIlbocIqoGww0RkYV+PpgBALg3MhANfXltGyJHxXBDRGShnw+WT0kN7hAmcSVEVBOGGyIiC5y5Wogj6flwUyowMDpE6nKIqAYMN0REFvj57/IpqR4tgxDg7S5xNURUE4YbIqLbEELgp4opqfackiJydAw3RES3ceJKAU5nFcLdTYn+d/F2C0SOjuGGiOg2frpxbZs+UQ3g56GWuBoiuh2GGyKiGggh8NON420e5JQUkVNguCEiqsHBS3lIu1YMT7UKfds0lLocIrIAww0RUQ0qrm3Tr20wvNzdJK6GiCzBcENEVA2jUZiuSjy4PW+3QOQsGG6IiKqRejEXGXml8NW4oVdUA6nLISILMdwQEVUj6UgmAOD+Ng2hcVNJXA0RWYrhhojIDCEENt4IN3FtebsFImfCcENEZMaprEKczymGu5sSvTklReRUGG6IiMzYeLh8r03PFkHw1vAsKSJnwnBDRGRG0tErAIA43m6ByOkw3BAR3eJybgkOXc6DUgH0a8NwQ+RsGG6IiG6RfONA4s4RAQj00UhcDRHVFsMNEdEtNh7hlBSRM2O4ISK6yfWiMvx5/hoAYMBdPAWcyBkx3BAR3eSP41kwGAXahPohPMBL6nKIyAoMN0REN/nnwn2ckiJyVgw3REQ3lJQZsP3UVQCckiJyZgw3REQ3bD15FaU6I8IDPNEm1FfqcojISgw3REQ3JB39515SCoVC4mqIyFoMN0REAAxGgS0nyqek+vN4GyKnxnBDRATg70u5uFZUBl8PN8RE1Je6HCK6Aww3REQANh3LAgD0atUAahXfGomcGX+DiYgAbDpeHm7ub91Q4kqI6E4x3BCRy8vMK8XRjHwoFOV7bojIuTHcEJHLq9hr0zG8Hm+USSQDDDdE5PIqwk1fTkkRyQLDDRG5tFKdATtPZwMA+jDcEMkCww0RubS9566hRGdAiJ8H2ob6SV0OEdkAww0RubRNx64AAPq0bsCrEhPJBMMNEbksIQQ2nag4BZxXJSaSC4YbInJZZ64W4uK1Eri7KdG9RaDU5RCRjTDcEJHLqjhLqmuzQHi5u0lcDRHZCsMNEbmsP47xFHAiOWK4ISKXlFeiw/4L1wHwlgtEcsNwQ0QuaefpbBiMAi0a+iA8wEvqcojIhhhuiMglbT1xFQDvJUUkRww3RORyhBDYepLhhkiuGG6IyOWcyipEZn4pPNRKdIkMkLocIrIxycPN4sWLERkZCQ8PD8TExGD79u019tdqtXj99dcREREBjUaD5s2b44svvqijaolIDiqmpO6NDISHWiVxNURka5Je2GH16tWYPHkyFi9ejO7du+OTTz5BfHw8jh49iiZNmphd5vHHH8eVK1ewbNkytGjRAllZWdDr9XVcORE5s22nOCVFJGeShpt58+ZhzJgxGDt2LABgwYIF2LhxI5YsWYI5c+ZU6f/bb79h69atOHv2LAICynclN23atC5LJiInV1ymx96z1wAA9zHcEMmSZOGmrKwMKSkpmDZtWqX2uLg47Nq1y+wyP/74Izp37oy5c+fiq6++gre3N4YMGYK3334bnp6eZpfRarXQarWmx/n5+QAAnU4HnU5no9HAtM6bP8uN3McHyH+MHB+w89RVlBmMaFTPA03quTvda8Ft6NzkPj7AfmOszfqsCjfnz5/H9u3bcf78eRQXF6NBgwbo1KkTYmNj4eHhYdE6srOzYTAYEBxc+WZ1wcHByMzMNLvM2bNnsWPHDnh4eGDDhg3Izs7G+PHjce3atWqPu5kzZw5mz55dpT0pKQleXva5tkVycrJd1uso5D4+QP5jdOXxrTunBKBEhKYYv/76a90VZWOuvA3lQO7jA2w/xuLiYov71ircfPvtt1i4cCH+/PNPNGzYEI0aNYKnpyeuXbuGM2fOwMPDA0899RRee+01REREWLROhUJR6bEQokpbBaPRCIVCgW+++Qb+/v4Ayqe2Hn30USxatMjs3pvp06cjISHB9Dg/Px/h4eGIi4uDn5+fpUO3iE6nQ3JyMvr37w+1Wm3TdTsCuY8PkP8YOT5gwYIdAIrx1P2dENfW+e4Ezm3o3OQ+PsB+Y6yYebGExeHm7rvvhlKpxOjRo7FmzZoqB/xqtVrs3r0bq1atQufOnbF48WI89thj1a4vKCgIKpWqyl6arKysKntzKoSGhqJRo0amYAMAbdq0gRACly5dQsuWLasso9FooNFoqrSr1Wq7/WDZc92OQO7jA+Q/Rlcd38VrxTiXUww3pQI9o4Kd+jVw1W0oF3IfH2D7MdZmXRafCv72229j//79mDhxotkzmTQaDXr37o2lS5fi2LFjtz3Q193dHTExMVV2WyUnJ6Nbt25ml+nevTvS09NRWFhoajt58iSUSiUaN25s6VCIyEVVXLjv7ib14ech7z8sRK7M4nAzaNAgi1caFBSEe+6557b9EhIS8Pnnn+OLL77AsWPHMGXKFKSlpWHcuHEAyqeUnnnmGVP/ESNGIDAwEM8++yyOHj2Kbdu24ZVXXsFzzz1X7QHFREQVTFcljuJZUkRyZtVF/JYtW2a2Xa/XY/r06RavZ/jw4ViwYAHeeustdOzYEdu2bUNiYqLpeJ2MjAykpaWZ+vv4+CA5ORm5ubno3LkznnrqKQwePBgLFy60ZhhE5ELK9EbsPpMDALivJcMNkZxZdbbU1KlTkZiYiM8++8x0vZnjx49jxIgRyMvLM3uNmuqMHz8e48ePN/vcihUrqrS1bt3aJY4yJyLbOpB2HYVaPQK93XFXmG1PJiAix2LVnpvU1FRcuXIF7dq1Q3JyMhYtWoS7774b0dHR+Ouvv2xcIhHRndt2Y0rqvlYNoFSaPyOTiOTBqj03kZGR2LZtG6ZMmYKBAwdCpVLhyy+/xBNPPGHr+oiIbGKrKdwESVwJEdmb1TfO/Pnnn7Fy5Up069YN9erVw2effYb09HRb1kZEZBM5hVocSS+/RkaPFjzehkjurAo3L7zwAh5//HG8+uqr2LZtGw4ePAiNRoN27dphzZo1tq6RiOiO7DidDQBoE+qHBr5Vr3tFRPJi1bTUzp07sXfvXnTo0AEAEBISgsTERCxatAjPPfccHn/8cZsWSUR0J3acKg83PVtySorIFVgVblJSUsxe9XfChAno16/fHRdFRGQrQghsZ7ghcilWTUuZCzYVoqKirC6GiMjWzlwtRGZ+KdzdlLinaYDU5RBRHbA43AwcOBC7du26bb+CggK89957WLRo0R0VRkRkC9tOlu+1uTcyAB5qlcTVEFFdsHha6rHHHsPjjz8OX19fDBkyBJ07d0ZYWBg8PDxw/fp1HD16FDt27EBiYiIefPBBvP/++/asm4jIIhUHE/dowSkpIldhcbgZM2YMRo4cibVr12L16tX47LPPkJubCwBQKBRo27YtBgwYgJSUFE5NEZFDKNMbseds+S0XevB4GyKXUasDit3d3TFixAiMGDECAJCXl4eSkhIEBgbK/tbtROR8DqRdR3GZAUE+7mgTwlsuELkKq86WquDv7w9/f39b1UJEZFMVp4B3bxHEWy4QuRCLw82PP/5o8UqHDBliVTFERLa0/VT5LRd68i7gRC7F4nDz0EMPVXqsUCgghKj0uILBYLjzyoiI7kBucRkOXs4DwIOJiVyNxaeCG41G00dSUhI6duyIX3/9Fbm5ucjLy0NiYiLuvvtu/Pbbb/asl4jIIjtP50AIoFWwD0L8PaQuh4jqkFXH3EyePBlLly5Fjx49TG0DBgyAl5cX/vWvf+HYsWM2K5CIyBo7TpdPSfFGmUSux6orFJ85c8bsgcT+/v44f/78ndZERHRHhBCmi/fxlgtErseqcHPPPfdg8uTJyMjIMLVlZmZi6tSp6NKli82KIyKyxoVrxbicWwK1SoF7m/GWC0Suxqpw88UXXyArKwsRERFo0aIFWrRogSZNmiAjIwPLli2zdY1ERLWy83T5hftiIurDy/2OrnhBRE7Iqt/6Fi1a4ODBg0hOTsbx48chhEDbtm3Rr1+/SmdNERFJYeeZawB4CjiRq7L6XxqFQoG4uDjExcXZsh4iojtiEMDus+XhhqeAE7kmi8PNwoUL8a9//QseHh5YuHBhjX0nTZp0x4UREVkjrRAo1Orh76lGdCNeQZ3IFVkcbubPn4+nnnoKHh4emD9/frX9FAoFww0RSeZkXvnUeLfmgVDxlgtELsnicHPu3DmzXxMROZITueXnSfAu4ESuy6qzpW4mhKh0GwYiIqkUavU4V1j+dU9evI/IZVkdbr788ku0a9cOnp6e8PT0RPv27fHVV1/ZsjYiolr58/x1GIUC4fU90STQS+pyiEgiVp0tNW/ePPz73//GxIkT0b17dwghsHPnTowbNw7Z2dmYMmWKreskIrqtiuvbdGseKHElRCQlq8LNRx99hCVLluCZZ54xtQ0dOhR33XUXZs2axXBDRJLYdaY83HRvzqsSE7kyq6alMjIy0K1btyrt3bp1q3RLBiKiupKZV4rTV4uggEBsM+65IXJlVoWbFi1aYM2aNVXaV69ejZYtW95xUUREtbXjdPmNMsO9gXpeaomrISIpWTUtNXv2bAwfPhzbtm1D9+7doVAosGPHDvzxxx9mQw8Rkb3tOHUVANCqHs/eJHJ1Vu25eeSRR7B3714EBQXh+++/x/r16xEUFIQ///wTDz/8sK1rJCKqkRACO24cTNzan+GGyNVZfW+pmJgYfP3117ashYjIKieuFCC7UAtPtRKRvnqpyyEiiVkdbgAgKysLWVlZMBqNldrbt29/R0UREdXGjlPlx9vc07Q+3JRXJK6GiKRmVbhJSUnBqFGjcOzYsSpXJ1YoFDAYDDYpjojIEttvhJvuzQOBPIYbIldnVbh59tln0apVKyxbtgzBwcFQKHhzOiKShlZvwJ/nrgEov3jf2QMSF0REkrMq3Jw7dw7r169HixYtbF0PEVGtpFy4jhKdAUE+GkQF++Cs1AURkeSsOluqb9+++Pvvv21dCxFRrVVMSfVsGcS9yEQEwMo9N59//jlGjRqFw4cPIzo6Gmp15QtmDRkyxCbFERHdzo6bwg0REWBluNm1axd27NiBX3/9tcpzPKCYiOrKtaIyHE7PAwD0aMFwQ0TlrJqWmjRpEkaOHImMjAwYjcZKHww2RFRXdp7OhhBA6xBfNPTzkLocInIQVoWbnJwcTJkyBcHBwbauh4jIYhVTUtxrQ0Q3syrcDBs2DJs3b7Z1LUREFhNCYPuN+0n1bNVA4mqIyJFYdcxNq1atMH36dOzYsQPt2rWrckDxpEmTbFIcEVF1zmYXIT2vFO4qJbo0DZC6HCJyIFafLeXj44OtW7di69atlZ5TKBQMN0Rkd9tPlu+1uSeyPjzdVRJXQ0SOxOqL+BERSWnH6YrjbTglRUSVWXXMDRGRlHQGI3afyQHA69sQUVVW7blJSEgw265QKODh4YEWLVpg6NChCAjgPDgR2V5qWi6KygwI9HZH21A/qcshIgdjVbhJTU3FgQMHYDAYEBUVBSEETp06BZVKhdatW2Px4sWYOnUqduzYgbZt29q6ZiJycRVnSXVvEQSlkrdcIKLKrJqWGjp0KPr164f09HSkpKTgwIEDuHz5Mvr3748nn3wSly9fxn333YcpU6bYul4iItP9pHpwSoqIzLAq3Lz//vt4++234ef3z+5gPz8/zJo1C3PnzoWXlxfefPNNpKSk2KxQIiIAyCvW4eClXAA83oaIzLMq3OTl5SErK6tK+9WrV5Gfnw8AqFevHsrKyu6sOiKiW+w6kw2jAFo09EGov6fU5RCRA7J6Wuq5557Dhg0bcOnSJVy+fBkbNmzAmDFj8NBDDwEA/vzzT7Rq1cqWtRIRYVvFVYm514aIqmHVAcWffPIJpkyZgieeeAJ6vb58RW5uGDVqFObPnw8AaN26NT7//HPbVUpELk8Iga0nysNNL95ygYiqYVW48fHxwWeffYb58+fj7NmzEEKgefPm8PHxMfXp2LGjrWokIgIAnLlaWH7LBTcl7o0MlLocInJQVoWbCj4+Pmjfvr2taiEiqtGWG3tt7o0M4C0XiKhaFoebYcOGYcWKFfDz88OwYcNq7Lt+/XqLC1i8eDHef/99ZGRk4K677sKCBQvQs2fP2y63c+dO9OrVC9HR0fjrr78s/n5E5Ly23TgFnFNSRFQTi8ONv78/FAqF6WtbWL16NSZPnozFixeje/fu+OSTTxAfH4+jR4+iSZMm1S6Xl5eHZ555Bn379sWVK1dsUgsRObZSnQF7z5bfcoHhhohqYnG4Wb58uenrxYsXw2g0wtvbGwBw/vx5fP/992jTpg0GDBhg8TefN28exowZg7FjxwIAFixYgI0bN2LJkiWYM2dOtcu98MILGDFiBFQqFb7//nuLvx8ROa+9565BqzcizN8DLRr63H4BInJZVp8K/tVXXwEAcnNz0bVrV3z44Yd46KGHsGTJEovWUVZWhpSUFMTFxVVqj4uLw65du6pdbvny5Thz5gxmzpxpTelE5KQqzpK6r1UD015kIiJzrDqg+MCBA6ZTvteuXYvg4GCkpqZi3bp1ePPNN/Hiiy/edh3Z2dkwGAwIDg6u1B4cHIzMzEyzy5w6dQrTpk3D9u3b4eZmWelarRZardb0uOIigzqdDjqdzqJ1WKpifbZer6OQ+/gA+Y/Rmce35UT5hUO7Nw+otn5nHp+l5D5Gjs/52WuMtVmfVeGmuLgYvr6+AICkpCQMGzYMSqUSXbt2xYULF2q1rlv/AxNCmP2vzGAwYMSIEZg9e3atLg44Z84czJ49u0p7UlISvLy8alWrpZKTk+2yXkch9/EB8h+js43vmhY4m+0GJQQKz6Qg8TZvM842PmvIfYwcn/Oz9RiLi4st7mtVuGnRogW+//57PPzww9i4caPpBplZWVmV7jdVk6CgIKhUqip7abKysqrszQGAgoIC7N+/H6mpqZg4cSIAwGg0QggBNzc3JCUl4f7776+y3PTp05GQkGB6nJ+fj/DwcMTFxVlcq6V0Oh2Sk5PRv39/qNVqm67bEch9fID8x+is41u17xJw4Cg6NamPR4d0qbafs46vNuQ+Ro7P+dlrjBUzL5awKty8+eabGDFiBKZMmYK+ffsiNjYWQPnekE6dOlm0Dnd3d8TExCA5ORkPP/ywqT05ORlDhw6t0t/Pzw+HDh2q1LZ48WJs2rQJa9euRWRkpNnvo9FooNFoqrSr1Wq7/WDZc92OQO7jA+Q/Rmcb384z1wAAvaMaWlS3s43PGnIfI8fn/Gw9xtqsy6pw8+ijj6JHjx7IyMhAhw4dTO19+/atFFRuJyEhASNHjkTnzp0RGxuLTz/9FGlpaRg3bhyA8r0uly9fxpdffgmlUono6OhKyzds2BAeHh5V2olIPnQGI3aeLr++zX08BZyILGD1FYpDQkIQEhJSqa1Ll+p3F5szfPhw5OTk4K233kJGRgaio6ORmJiIiIgIAEBGRgbS0tKsLZGIZCA1LRcFWj0CvN3RrpFtrrFFRPJ2R7dfsIXx48dj/PjxZp9bsWJFjcvOmjULs2bNsn1RROQwtp385y7gSiVPASei27PqOjdERHVl641wc19LTkkRkWUYbojIYV0t0OJweh4AoGerIImrISJnwXBDRA5ry4ksCAG0a+SPhr4eUpdDRE6C4YaIHNbmG1cl7tO6ocSVEJEzYbghIodUpjdi28nyU8DvZ7gholpguCEih7T//DUUavUI8nFHe54CTkS1wHBDRA5p0/HyKaneUQ15CjgR1QrDDRE5pE03jrfhlBQR1RbDDRE5nPPZRTh7tQhuSgV6tOQp4ERUOww3RORwKqakukQGwM9D3jcXJCLbY7ghIoezmVNSRHQHGG6IyKEUavXYe/YaAF7fhoisw3BDRA5lx6lslBmMiAj0QrMgb6nLISInxHBDRA5l8/F/pqQUCp4CTkS1x3BDRA7DaBQ83oaI7hjDDRE5jCPp+cgq0MLLXYUukQFSl0NETorhhogcRsUp4D1aBEHjppK4GiJyVgw3ROQwko9lAgD6tuGUFBFZj+GGiBzCpevFOHw5H0oF0K9NsNTlEJETY7ghIoeQdOQKAKBz0wAE+mgkroaInBnDDRE5hKSj5VNScW2514aI7gzDDRFJ7lpRGf48V35V4gF3hUhcDRE5O4YbIpLcH8euwCiAtqF+CA/wkrocInJyDDdEJLmNN463ibuLU1JEdOcYbohIUsVlemw/dRUAp6SIyDYYbohIUttOXoVWb0R4gCdah/hKXQ4RyQDDDRFJquIU8AFtQ3ijTCKyCYYbIpKMzmDE78cqjrfhlBQR2QbDDRFJZu/Za8gv1SPQ2x0xEfWlLoeIZILhhogkU3Hhvn5tgqFSckqKiGyD4YaIJGE0in+Ot4nmKeBEZDsMN0Qkib8u5SIzvxTe7ip0ax4kdTlEJCMMN0QkiZ/+TgcA9G8bDA+1SuJqiEhOGG6IqM4ZjAK/HMwAADzYPkziaohIbhhuiKjO7Tt/DVkFWvh5uKFnK05JEZFtMdwQUZ2rmJIaGB0CjRunpIjIthhuiKhO6QxG/Hq4/BTwwR04JUVEtsdwQ0R1ateZHFwrKkOgtztimwVKXQ4RyRDDDRHVqYopqfh2IXBT8S2IiGyP7yxEVGe0egM2VkxJ8SwpIrIThhsiqjNbT1xFgVaPYD8N7mkaIHU5RCRTDDdEVGd+vunaNkreS4qI7IThhojqRHGZHslHy+8lxbOkiMieGG6IqE5sOp6FEp0B4QGe6NDYX+pyiEjGGG6IqE788Ff5WVIPtg+DQsEpKSKyH4YbIrK77EItNh/PAgA81LGRxNUQkdwx3BCR3X2fehl6o0CHxv6ICvGVuhwikjmGGyKyKyEEvtt/CQDwaOdwiashIlfAcENEdnXoch5OXCmAu5sSQ3jhPiKqAww3RGRXFXttBtwVAn8vtcTVEJErYLghIrsp1Rnww1+XAQCPd24scTVE5CoYbojIbpKOXkF+qR5h/h7o1jxI6nKIyEUw3BCR3Xy3/yIA4JGYxlDxdgtEVEcYbojILtJzS7DjdDYA4NEYTkkRUd1huCEiu1h/4BKEALpEBiAi0FvqcojIhTDcEJHNCSGwNqX8LKnHuNeGiOqY5OFm8eLFiIyMhIeHB2JiYrB9+/Zq+65fvx79+/dHgwYN4Ofnh9jYWGzcuLEOqyUiS+w5ew3nc4rh7a7CA+1CpS6HiFyMpOFm9erVmDx5Ml5//XWkpqaiZ8+eiI+PR1pamtn+27ZtQ//+/ZGYmIiUlBT06dMHgwcPRmpqah1XTkQ1+e+u8wCAoZ0awVvjJm0xRORyJA038+bNw5gxYzB27Fi0adMGCxYsQHh4OJYsWWK2/4IFC/Dqq6/innvuQcuWLfGf//wHLVu2xE8//VTHlRNRdS7nliDpaCYAYFRsU2mLISKXJFm4KSsrQ0pKCuLi4iq1x8XFYdeuXRatw2g0oqCgAAEBAfYokYis8PWeCzAKILZZIG+SSUSSkGx/cXZ2NgwGA4KDgyu1BwcHIzMz06J1fPjhhygqKsLjjz9ebR+tVgutVmt6nJ+fDwDQ6XTQ6XRWVF69ivXZer2OQu7jA+Q/RnuPr1RnwKo/y6eVn763cZ2/jnLffoD8x8jxOT97jbE261MIIYRNv7uF0tPT0ahRI+zatQuxsbGm9nfeeQdfffUVjh8/XuPyK1euxNixY/HDDz+gX79+1fabNWsWZs+eXaX922+/hZeXl/UDIKIq9mQpsPKMCvXdBf59twEqXrePiGykuLgYI0aMQF5eHvz8/GrsK9mem6CgIKhUqip7abKysqrszbnV6tWrMWbMGHz33Xc1BhsAmD59OhISEkyP8/PzER4ejri4uNu+OLWl0+mQnJyM/v37Q62W3w0C5T4+QP5jtOf4hBBYungPgAKM7d0Kg3tG2nT9lpD79gPkP0aOz/nZa4wVMy+WkCzcuLu7IyYmBsnJyXj44YdN7cnJyRg6dGi1y61cuRLPPfccVq5ciUGDBt32+2g0Gmg0mirtarXabj9Y9ly3I5D7+AD5j9Ee49t3/hqOZRZA46bEiHubSvr6yX37AfIfI8fn/Gw9xtqsS9JzNBMSEjBy5Eh07twZsbGx+PTTT5GWloZx48YBKN/rcvnyZXz55ZcAyoPNM888g//7v/9D165dTXt9PD094e/vL9k4iAhYceP074c6NkJ9b3dpiyEilyZpuBk+fDhycnLw1ltvISMjA9HR0UhMTERERAQAICMjo9I1bz755BPo9XpMmDABEyZMMLWPGjUKK1asqOvyieiGjLwS/Hb4xunf3ZpKWwwRuTzJr641fvx4jB8/3uxztwaWLVu22L8gIqq1b/akwWAU6NI0AG3DbHssGxFRbUl++wUicm5FWj2+3nsBAPfaEJFjYLghojvy9Z4LyC3WITLIGwOjQ6Quh4iI4YaIrFeqM+Cz7WcBAON7N4dKyQvbEJH0GG6IyGqr/kxDdmEZGtXzxEOdGkldDhERAIYbIrKSVm/AJ9vK99q82Ls51Cq+nRCRY+C7ERFZZf2By8jIK0WwnwaPxjSWuhwiIhOGGyKqNb3BiMVbTgMA/nVfc3ioVRJXRET0D4YbIqq1H/9Ox8VrJQj0dseILk2kLoeIqBKGGyKqFYNRYNHm8r02Y3s2g6c799oQkWNhuCGiWvn5YDrOXC2Cv6caT3flXhsicjwMN0RksVKdAXN/OwEAeL5nJHw95H1XYyJyTgw3RGSx5TvP43JuCUL9PTCmRzOpyyEiMovhhogsklOoxeIbx9q8MiCKx9oQkcNiuCEiiyz4/RQKtHpEN/LDQx15NWIiclwMN0R0W6ezCvDtn2kAgNcfaAsl7yFFRA6M4YaIbmtO4nEYjAL92gQjtnmg1OUQEdWI4YaIarTrdDb+OJ4FN6UC0x9oLXU5RES3xXBDRNXSGYx46+ejAICn7m2C5g18JK6IiOj2GG6IqFqfbjuL45kFqOelxsv9WkldDhGRRRhuiMis01mF+L8/TgEA3nywLQK83SWuiIjIMgw3RFSF0Sgwbd1BlOmN6NWqAR7uxFO/ich5MNwQURVf772A/Reuw9tdhXcejoZCwVO/ich5MNwQUSWXrhfjvV+PAwBei2+NxvW9JK6IiKh2GG6IyEQIgdc3HEZRmQGdI+rj6XsjpC6JiKjWGG6IyGTN/ovYevIq3N2UeO/R9rwSMRE5JYYbIgIAHMvIx5s/HAEATOnXite0ISKnxXBDRCgo1WHCNwegvXF21Av3NZO6JCIiqzHcELk4IQSmrz+Es9lFCPX3wPzhHTkdRUROjeGGyMV9tecCfj6YATelAh+PuJsX6yMip8dwQ+TC/r6Yi7dv3DtqWnxrxETUl7giIqI7x3BD5KKy8ksx/psD0BkEBt4VgjE9IqUuiYjIJtykLoCI6l5BqQ6jlqfgcm4JIoO8Mfex9rwKMRHJBvfcELkYvRGYsPJvHMvIR5CPO/77bBf4eailLouIyGa454bIhRiNAl+fViI15xq83VVY8WwXNAnk7RWISF6454bIRQghMOe3E0jNUcJNqcDSkTGIbuQvdVlERDbHcEPkAoQQ+L8/TmHF7jQAwLvDotGzZQOJqyIisg9OSxHJnBAC7/xyDJ/vOAcAGBphwNAOoRJXRURkPww3RDJmMArMWH8Iq/dfBADMiI9CcO4RiasiIrIvTksRyVSZ3ohJK1Oxev9FKBXA3Efb49luEVKXRURkd9xzQyRD+aU6vPRtKraevAq1SoGFT3RCfLtQ6HQ6qUsjIrI7hhsimTl5pQAvfJWCc9lF8FAr8cnIzujVigcPE5HrYLghkpGfD6bj1bUHUVxmQJi/B5Y8HYMO4fWkLouIqE4x3BDJgN5gxHu/Hcdn28vPiOreIhALn+iEQB+NxJUREdU9hhsiJ3fySgFeXXsQf13MBQCM69Uc/xPXCm4qni9ARK6J4YbISekMRnyy9QwW/nEaZQYjfDVumPtoe8S34zVsiMi1MdwQOaEj6Xl45buDOJqRDwDo27oh3nm4HUL8PSSujIhIegw3RE4kK78U838/hdX70mAUQD0vNWYNvgtDO4ZBoVBIXR4RkUNguCFyAoVaPT7degafbT+HEp0BADCoXShmDbkLDXx50DAR0c0YbogcWEGpDqv3XcSSLWeQU1QGALi7ST3MeKANOjcNkLg6IiLHxHBD5IAy8kqwYud5fLs3DQVaPQAgMsgbrw2MwoC7QjgFRURUA4YbIgdhNAr8ef4aVv2Zhp8PZkBvFACA5g288XzPZngkpjHUPL2biOi2GG6IJHYhpwjrDlzG+gOXcOl6iam9a7MAPN+zGfpENYRSyT01RESWYrghqmNCCJy5Woiko1eQfPQKUtNyTc/5atzwYIdQPNmlCdo3ridZjUREzozhhqgOFGn12Hf+Gnaezsbvx7JwLrvI9JxCAfRoEYRHYxpjwF0h8FCrJKyUiMj5MdwQ2UFucRn+vpSHP8/lYPeZHBy8lGc6hgYA3FVKxDYPRP+2wejfNhjBfrz4HhGRrTDcEN0BIQSuFmpx6kohjmXk4+9LeTh4KRcXcoqr9G1c3xNdmwWiT1RD9IpqAB8Nf/2IiOyB765EFijVGXDxWjHO5xTjQk4RzmUX4XRWIU5eKcD1Yp3ZZSKDvNEpvB66Ng9EbLNAhAd41XHVRESuSfJws3jxYrz//vvIyMjAXXfdhQULFqBnz57V9t+6dSsSEhJw5MgRhIWF4dVXX8W4cePqsGKSEyEE8kp0uFqgxdUCLTJyi7EtXYHUxOO4UlCG9NwSpOeV4mqBttp1KBRA00BvtAr2QfvG9dC+sT/aN6oHfy91HY6EiIgqSBpuVq9ejcmTJ2Px4sXo3r07PvnkE8THx+Po0aNo0qRJlf7nzp3DAw88gOeffx5ff/01du7cifHjx6NBgwZ45JFHJBgBSUkIAa3eiOIyA4rL9CgpM6BQq0eRtvxzoVaPwlId8kv1yC/RIb9Uh7wSHa4X63C9qAzXi3XILS6rdCxMORVwIa3K9/PVuKFpkDciAr3QNNAbkUHeiArxRYuGPjwImIjIgUgabubNm4cxY8Zg7NixAIAFCxZg48aNWLJkCebMmVOl/9KlS9GkSRMsWLAAANCmTRvs378fH3zwgeThxmAUuJxbgmta4HJuCdzcyqcqxK1/N820CVTtVNFHmB4L0+N/lheV+v3z9U3toupjoxA3+oublhMwipueF+XPG2/qX6bT4fB1BTTHsqBQqWA0lj9vEOLG1wKGG5/1xvI2g1HAIACD0Qi9UcBgENAZBXQGI/QGI3QGAb3RCL1BoOzGY53eiDKDEWV6I7R6w43P5R+lOgNKdQZo9UaU6AxmX19r+Huq0cBXgyBvNcryc9CpdSTCA7wRWs8Tjep5IqyeJ+p7qXllYCIiJyBZuCkrK0NKSgqmTZtWqT0uLg67du0yu8zu3bsRFxdXqW3AgAFYtmwZdDod1Oqq0wBarRZa7T9TCvn5+QAAnU4Hnc78sRLWuFqgRe8PtwNww+wD2222XsejwmfH/5K6iCrc3ZTwUqvgrVHB290NPh5u8HZXwVvjBn9PN/h6qOHv4QZfDzfU93JHfW91+WcvNep5uUPjVn7lX51Oh+TkZPTv26zKz5Ner5diaDZV8TNvy599RyL38QHyHyPH5/zsNcbarE+ycJOdnQ2DwYDg4OBK7cHBwcjMzDS7TGZmptn+er0e2dnZCA0NrbLMnDlzMHv27CrtSUlJ8PKy3QGe+WWAWml+asKS//UVVb7450tzy9/cX3Fru+KfrxW3tN/8WFHTY0Xltoqvlbc8V/5YQHmjTam48XHz17e0qW76UCrLP7spRHmb6THgpgTUypu/FlDfaFMrAXcl4K4q/3zbC/gaABTd+MgBrqP8ozrJycm3WaFz4/icn9zHyPE5P1uPsbi46lmo1ZH8gOJbd/MLIWrc9W+uv7n2CtOnT0dCQoLpcX5+PsLDwxEXFwc/Pz9ryzbrkUE3/uvv39/sXiRnZ9qrIdPxAfIfI8fn/OQ+Ro7P+dlrjBUzL5aQLNwEBQVBpVJV2UuTlZVVZe9MhZCQELP93dzcEBgYaHYZjUYDjUZTpV2tVtvtB8ue63YEch8fIP8xcnzOT+5j5Picn63HWJt1SXaLYXd3d8TExFTZbZWcnIxu3bqZXSY2NrZK/6SkJHTu3Fn2PyRERERkGcnCDQAkJCTg888/xxdffIFjx45hypQpSEtLM123Zvr06XjmmWdM/ceNG4cLFy4gISEBx44dwxdffIFly5bhf/7nf6QaAhERETkYSY+5GT58OHJycvDWW28hIyMD0dHRSExMREREBAAgIyMDaWn/XG8kMjISiYmJmDJlChYtWoSwsDAsXLhQ8tPAiYiIyHFIfkDx+PHjMX78eLPPrVixokpbr169cODAATtXRURERM5K0mkpIiIiIltjuCEiIiJZYbghIiIiWWG4ISIiIllhuCEiIiJZYbghIiIiWWG4ISIiIllhuCEiIiJZYbghIiIiWZH8CsV1TQgBoHa3TreUTqdDcXEx8vPzZXkjT7mPD5D/GDk+5yf3MXJ8zs9eY6z4u13xd7wmLhduCgoKAADh4eESV0JERES1VVBQAH9//xr7KIQlEUhGjEYj0tPT4evrC4VCYdN15+fnIzw8HBcvXoSfn59N1+0I5D4+QP5j5Picn9zHyPE5P3uNUQiBgoIChIWFQams+agal9tzo1Qq0bhxY7t+Dz8/P9n+0ALyHx8g/zFyfM5P7mPk+JyfPcZ4uz02FXhAMREREckKww0RERHJCsONDWk0GsycORMajUbqUuxC7uMD5D9Gjs/5yX2MHJ/zc4QxutwBxURERCRv3HNDREREssJwQ0RERLLCcENERESywnBDREREssJwUwvvvPMOunXrBi8vL9SrV89sn7S0NAwePBje3t4ICgrCpEmTUFZWVuN6tVotXnrpJQQFBcHb2xtDhgzBpUuX7DCC2tmyZQsUCoXZj3379lW73OjRo6v079q1ax1WbrmmTZtWqXXatGk1LiOEwKxZsxAWFgZPT0/07t0bR44cqaOKa+f8+fMYM2YMIiMj4enpiebNm2PmzJm3/Zl05G24ePFiREZGwsPDAzExMdi+fXuN/bdu3YqYmBh4eHigWbNmWLp0aR1VWntz5szBPffcA19fXzRs2BAPPfQQTpw4UeMy1f2eHj9+vI6qttysWbOq1BkSElLjMs60/QDz7ykKhQITJkww29/Rt9+2bdswePBghIWFQaFQ4Pvvv6/0vLXvh+vWrUPbtm2h0WjQtm1bbNiwwaZ1M9zUQllZGR577DG8+OKLZp83GAwYNGgQioqKsGPHDqxatQrr1q3D1KlTa1zv5MmTsWHDBqxatQo7duxAYWEhHnzwQRgMBnsMw2LdunVDRkZGpY+xY8eiadOm6Ny5c43LDhw4sNJyiYmJdVR17b311luVan3jjTdq7D937lzMmzcPH3/8Mfbt24eQkBD079/fdN8yR3L8+HEYjUZ88sknOHLkCObPn4+lS5dixowZt13WEbfh6tWrMXnyZLz++utITU1Fz549ER8fj7S0NLP9z507hwceeAA9e/ZEamoqZsyYgUmTJmHdunV1XLlltm7digkTJmDPnj1ITk6GXq9HXFwcioqKbrvsiRMnKm2vli1b1kHFtXfXXXdVqvPQoUPV9nW27QcA+/btqzS+5ORkAMBjjz1W43KOuv2KiorQoUMHfPzxx2aft+b9cPfu3Rg+fDhGjhyJv//+GyNHjsTjjz+OvXv32q5wQbW2fPly4e/vX6U9MTFRKJVKcfnyZVPbypUrhUajEXl5eWbXlZubK9RqtVi1apWp7fLly0KpVIrffvvN5rXfibKyMtGwYUPx1ltv1dhv1KhRYujQoXVT1B2KiIgQ8+fPt7i/0WgUISEh4t133zW1lZaWCn9/f7F06VI7VGh7c+fOFZGRkTX2cdRt2KVLFzFu3LhKba1btxbTpk0z2//VV18VrVu3rtT2wgsviK5du9qtRlvKysoSAMTWrVur7bN582YBQFy/fr3uCrPSzJkzRYcOHSzu7+zbTwghXn75ZdG8eXNhNBrNPu9M2w+A2LBhg+mxte+Hjz/+uBg4cGCltgEDBognnnjCZrVyz40N7d69G9HR0QgLCzO1DRgwAFqtFikpKWaXSUlJgU6nQ1xcnKktLCwM0dHR2LVrl91rro0ff/wR2dnZGD169G37btmyBQ0bNkSrVq3w/PPPIysry/4FWum9995DYGAgOnbsiHfeeafGKZtz584hMzOz0vbSaDTo1auXw22v6uTl5SEgIOC2/RxtG5aVlSElJaXSaw8AcXFx1b72u3fvrtJ/wIAB2L9/P3Q6nd1qtZW8vDwAsGh7derUCaGhoejbty82b95s79KsdurUKYSFhSEyMhJPPPEEzp49W21fZ99+ZWVl+Prrr/Hcc8/d9kbNzrL9bmbt+2F129WW76EMNzaUmZmJ4ODgSm3169eHu7s7MjMzq13G3d0d9evXr9QeHBxc7TJSWbZsGQYMGIDw8PAa+8XHx+Obb77Bpk2b8OGHH2Lfvn24//77odVq66hSy7388stYtWoVNm/ejIkTJ2LBggUYP358tf0rtsmt29kRt5c5Z86cwUcffYRx48bV2M8Rt2F2djYMBkOtXntzv5PBwcHQ6/XIzs62W622IIRAQkICevTogejo6Gr7hYaG4tNPP8W6deuwfv16REVFoW/fvti2bVsdVmuZe++9F19++SU2btyIzz77DJmZmejWrRtycnLM9nfm7QcA33//PXJzc2v8h9CZtt+trH0/rG672vI91OXuCn6rWbNmYfbs2TX22bdv322PMalgLp0LIW6b2m2xjKWsGfOlS5ewceNGrFmz5rbrHz58uOnr6OhodO7cGREREfjll18wbNgw6wu3UG3GN2XKFFNb+/btUb9+fTz66KOmvTnVuXXb2HN7mWPNNkxPT8fAgQPx2GOPYezYsTUuK/U2rEltX3tz/c21O5qJEyfi4MGD2LFjR439oqKiEBUVZXocGxuLixcv4oMPPsB9991n7zJrJT4+3vR1u3btEBsbi+bNm+O///0vEhISzC7jrNsPKP+HMD4+vtLe/Fs50/arjjXvh/Z+D3X5cDNx4kQ88cQTNfZp2rSpResKCQmpckDU9evXodPpqqTUm5cpKyvD9evXK+29ycrKQrdu3Sz6vrVlzZiXL1+OwMBADBkypNbfLzQ0FBERETh16lStl7XGnWzTijOCTp8+bTbcVJzZkZmZidDQUFN7VlZWtdvYHmo7xvT0dPTp0wexsbH49NNPa/396nobmhMUFASVSlXlv7uaXvuQkBCz/d3c3GoMr1J76aWX8OOPP2Lbtm1o3LhxrZfv2rUrvv76aztUZlve3t5o165dtT9Xzrr9AODChQv4/fffsX79+lov6yzbz9r3w+q2qy3fQ10+3AQFBSEoKMgm64qNjcU777yDjIwM04ZOSkqCRqNBTEyM2WViYmKgVquRnJyMxx9/HACQkZGBw4cPY+7cuTap61a1HbMQAsuXL8czzzwDtVpd6++Xk5ODixcvVvrht6c72aapqakAUG2tkZGRCAkJQXJyMjp16gSgfF5969ateO+996wr2Aq1GePly5fRp08fxMTEYPny5VAqaz8bXdfb0Bx3d3fExMQgOTkZDz/8sKk9OTkZQ4cONbtMbGwsfvrpp0ptSUlJ6Ny5s1U/y/YmhMBLL72EDRs2YMuWLYiMjLRqPampqZJuK0tptVocO3YMPXv2NPu8s22/my1fvhwNGzbEoEGDar2ss2w/a98PY2NjkZycXGnPeVJSkm3/obfZocku4MKFCyI1NVXMnj1b+Pj4iNTUVJGamioKCgqEEELo9XoRHR0t+vbtKw4cOCB+//130bhxYzFx4kTTOi5duiSioqLE3r17TW3jxo0TjRs3Fr///rs4cOCAuP/++0WHDh2EXq+v8zGa8/vvvwsA4ujRo2afj4qKEuvXrxdCCFFQUCCmTp0qdu3aJc6dOyc2b94sYmNjRaNGjUR+fn5dln1bu3btEvPmzROpqani7NmzYvXq1SIsLEwMGTKkUr+bxyeEEO+++67w9/cX69evF4cOHRJPPvmkCA0NdbjxCVF+5l2LFi3E/fffLy5duiQyMjJMHzdzlm24atUqoVarxbJly8TRo0fF5MmThbe3tzh//rwQQohp06aJkSNHmvqfPXtWeHl5iSlTpoijR4+KZcuWCbVaLdauXSvVEGr04osvCn9/f7Fly5ZK26q4uNjU59Yxzp8/X2zYsEGcPHlSHD58WEybNk0AEOvWrZNiCDWaOnWq2LJlizh79qzYs2ePePDBB4Wvr69stl8Fg8EgmjRpIl577bUqzznb9isoKDD9rQNges+8cOGCEMKy98ORI0dWOqNx586dQqVSiXfffVccO3ZMvPvuu8LNzU3s2bPHZnUz3NTCqFGjBIAqH5s3bzb1uXDhghg0aJDw9PQUAQEBYuLEiaK0tNT0/Llz56osU1JSIiZOnCgCAgKEp6enePDBB0VaWlodjqxmTz75pOjWrVu1zwMQy5cvF0IIUVxcLOLi4kSDBg2EWq0WTZo0EaNGjXKo8VRISUkR9957r/D39xceHh4iKipKzJw5UxQVFVXqd/P4hCg//XHmzJkiJCREaDQacd9994lDhw7VcfWWWb58udmf2Vv/r3Gmbbho0SIREREh3N3dxd13313pNOlRo0aJXr16Veq/ZcsW0alTJ+Hu7i6aNm0qlixZUscVW666bXXzz9+tY3zvvfdE8+bNhYeHh6hfv77o0aOH+OWXX+q+eAsMHz5chIaGCrVaLcLCwsSwYcPEkSNHTM87+/arsHHjRgFAnDhxospzzrb9Kk5Vv/Vj1KhRQgjL3g979epl6l/hu+++E1FRUUKtVovWrVvbPMwphLhxdBYRERGRDPBUcCIiIpIVhhsiIiKSFYYbIiIikhWGGyIiIpIVhhsiIiKSFYYbIiIikhWGGyIiIpIVhhsiIiKSFYYbIiIikhWGGyIiIpIVhhsicnpXr15FSEgI/vOf/5ja9u7dC3d3dyQlJUlYGRFJgfeWIiJZSExMxEMPPYRdu3ahdevW6NSpEwYNGoQFCxZIXRoR1TGGGyKSjQkTJuD333/HPffcg7///hv79u2Dh4eH1GURUR1juCEi2SgpKUF0dDQuXryI/fv3o3379lKXREQS4DE3RCQbZ8+eRXp6OoxGIy5cuCB1OUQkEe65ISJZKCsrQ5cuXdCxY0e0bt0a8+bNw6FDhxAcHCx1aURUxxhuiEgWXnnlFaxduxZ///03fHx80KdPH/j6+uLnn3+WujQiqmOcliIip7dlyxYsWLAAX331Ffz8/KBUKvHVV19hx44dWLJkidTlEVEd454bIiIikhXuuSEiIiJZYbghIiIiWWG4ISIiIllhuCEiIiJZYbghIiIiWWG4ISIiIllhuCEiIiJZYbghIiIiWWG4ISIiIllhuCEiIiJZYbghIiIiWWG4ISIiIln5f+8NyfIsn/eoAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "x = np.linspace(-10, 10, 100)\n",
    "y = sigmoid(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.title('Sigmoid function')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('sigmoid(x)')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The hyperbolic tangent, $\\tanh$, is basically the sigmoid function shifted and scaled to a range of [-1,1]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "hide_input": true,
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAksAAAHFCAYAAADi7703AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQlUlEQVR4nO3deVxU9f4/8NfMMAwgiwIyQKKgkai4ggua21VwyWxXr8VtUbte7ZbRciPrm7b5s03KsrIsTEq5ZWZezUDLLXBDQDMlFxAVUFBh2JwZZs7vD2RyBIYBYQ5n5vV8PObhzJnPObzPuxFfnTnnc2SCIAggIiIiogbJxS6AiIiIqD1jWCIiIiKygGGJiIiIyAKGJSIiIiILGJaIiIiILGBYIiIiIrKAYYmIiIjIAoYlIiIiIgsYloiIiIgsYFgiIklJS0vDokWLUFpa2qY/Jy8vDzKZDO+8847V6zz22GOYOHFis3/W9u3b4e7ujvPnzzd7XSJqewxLRCQpaWlpWLx4cZuHpebKzMzE6tWr8frrrzd73XHjxmHIkCF48cUX26AyIrpZDEtERK3g//2//4chQ4YgMjKyRevPnz8fX3/9Nc6ePdvKlRHRzWJYIiLJWLRoEZ577jkAQEhICGQyGWQyGXbs2IHk5GTExMQgICAArq6u6NWrF1544QVUVlaabeORRx6Bu7s7Tp48icmTJ8Pd3R1BQUF45plnoNVqG/y57733HkJCQuDu7o6oqCjs3bvX7P0LFy5gw4YNiI2NNVs+d+5cuLi4ICMjw7TMaDRi3LhxUKvVKCwsNC2/88474e7ujs8+++ymekRErY9hiYgkY/bs2fj3v/8NAPj++++Rnp6O9PR0DBo0CCdOnMDkyZOxatUqbN26FQsWLMB///tf3HnnnfW2o9frMXXqVIwbNw4bN27EY489hmXLlmHp0qX1xn700UdITU1FQkICvv76a1RWVmLy5MkoKyszjUlJSYFer8fYsWPN1k1ISECvXr0wbdo009eGixcvxo4dO5CUlISAgADTWGdnZwwfPhybN29ujVYRUWsSiIgk5O233xYACLm5uY2OMRqNgl6vF3bu3CkAELKzs03vPfzwwwIA4b///a/ZOpMnTxZ69uxpep2bmysAEPr27SvU1NSYlu/fv18AIKxdu9a07F//+pfg6uoqGI3GerWcOHFC8PT0FO6++25h27ZtglwuF1566aUG6164cKEgl8uFioqKJvtARLbDI0tEZBdOnz6NmTNnwt/fHwqFAkqlEqNHjwYAHDt2zGysTCard8SpX79+OHPmTL3t3nHHHVAoFGbjAJiNLSgoQOfOnSGTyeqtf+utt+Kzzz7DDz/8gClTpmDkyJFYtGhRg/vg5+cHo9GIoqIi63aaiGzCSewCiIhuVkVFBUaOHAkXFxe8/vrruO222+Dm5oazZ8/i3nvvRXV1tdl4Nzc3uLi4mC1TqVS4evVqvW37+PjUGwfAbJvV1dX1tne9O+64A2q1GhcuXEBcXJxZ+Lpe3TZurJeIxMWwRESS98svv6CgoAA7duwwHU0CYLPpBXx9fXHo0KFG3587dy7Ky8vRp08fPPnkkxg5ciQ6depUb9zly5dN2yOi9oNfwxGRpDR0ZKfu66+69+p8+umnNqkpLCwMly5dMjvpu87nn3+OpKQkfPjhh/jxxx9RWlqKRx99tMHtnD59Gj4+PlCr1W1dMhE1A8MSEUlK3759AQDvv/8+0tPTcfDgQfTr1w+dOnXC3LlzsWHDBvzvf//D3//+d2RnZ9ukpjFjxkAQBOzbt89s+ZEjR/Dkk0/i4YcfxqOPPoru3btj1apV2LhxIxISEuptZ+/evRg9enSD5z4RkXgYlohIUsaMGYP4+Hhs2rQJt99+OwYPHozc3Fxs3rwZbm5ueOihh/DYY4/B3d0dycnJNqlpxIgRCA4OxsaNG03LKisrMW3aNISEhGDFihWm5ffddx/mz5+P559/Hvv37zctP3XqFI4cOYIHH3zQJjUTkfVkgiAIYhdBRCR17777Lt544w2cP38erq6uzV7/5ZdfxldffYVTp07ByYmnkxK1JzyyRETUCubPnw8vLy989NFHzV63tLQUH330Ed58800GJaJ2iGGJiKgVuLi4YM2aNfVOMrdGbm4u4uPjMXPmzDaojIhuFr+GIyIiIrKAR5aIiIiILGBYIiIiIrKAYYmIiIjIAl520QqMRiMKCgrg4eHByeSIiIgkQhAElJeXIzAwEHJ548ePGJZaQUFBAYKCgsQug4iIiFrg7Nmz6NKlS6PvMyy1Ag8PDwC1zfb09BS5GvHp9XqkpKQgJiYGSqVS7HLsHvtte+y5bbHftuVI/dZoNAgKCjL9O94YhqVWUPfVm6enJ8MSav+iubm5wdPT0+7/orUH7Lftsee2xX7bliP2u6lTaHiCNxEREZEFDEtEREREFjAsEREREVnAsERERERkAcMSERERkQUMS0REREQWMCwRERERWcCwRERERGQBwxIRERGRBQxLRERERBZIKizt2rULd955JwIDAyGTyfDDDz80uc7OnTsREREBFxcXdO/eHZ988km9MevXr0fv3r2hUqnQu3dvbNiwoQ2qJyIiIimSVFiqrKxE//798eGHH1o1Pjc3F5MnT8bIkSORmZmJF198EU8++STWr19vGpOeno7p06cjNjYW2dnZiI2NxbRp07Bv37622g0iIiKSEEndSHfSpEmYNGmS1eM/+eQTdO3aFQkJCQCAXr164eDBg3jnnXdw3333AQASEhIQHR2N+Ph4AEB8fDx27tyJhIQErF27ttX3gYiIqL0QBAGCABgFAQJq/9TpDdAZgCpdDZyMMgh14wAIAgABqH1V+1q4flum53+NueGP2m3AfBumem6o7Xod3ZzhrhIntkgqLDVXeno6YmJizJZNmDABq1atgl6vh1KpRHp6Op5++ul6Y+oCVkO0Wi20Wq3ptUajAVB7p2a9Xt96OyBRdT1gL2yD/bY99ty22nO/jUYBVXoDqnUGVOkNqNIaUK03oEpXu0xbY4C2xgidwQhtjRFafe1zXU3tQ1tjhN5ghN4ooMZgRI1BQI1RqF1mEFBj/GuZURBgMN7wEAQYjbXvCwJqX18LQAajAKNQGzqMggCD6flffzbOCc/t/8VWbbTKa1N7Y8bgLq26TWs/U3YdloqKiqBWq82WqdVq1NTUoKSkBAEBAY2OKSoqanS7S5YsweLFi+stT0lJgZubW+sUbwdSU1PFLsGhsN+2x57bVlv2+6oB0OhqH+U1MlTXANU1QFWNDFUG1Ht9tQbQGgG9UdZmNUmdDMINr82fNNQ5WUPPrz05+vsRbCk+3Gr1AUBVVZVV4+w6LAGATGb+n6PusN71yxsac+Oy68XHxyMuLs70WqPRICgoCDExMfD09GyNsiVNr9cjNTUV0dHRUCqVYpdj99hv22PPbetm+62p1iPvUhXOXK5CYdlVXCzXorhce+1PHYortKjSGW6qRpkMcFMq4OqsgJuzwvRc5SSHykkBZye56aFyksNZce1PJzmUCjmc5DIoFTI4Xf9cLjdbprj+IfvruVwGOMnlkMlgek8mA+TXxtQ9l8tq/72TX3stu/Zahr9ey2VATU0Nfv31V/ztb2PhrFRCBvOxMhmu/Vn772TdMqD+v6ftXd03Q02x67Dk7+9f7wjRxYsX4eTkBB8fH4tjbjzadD2VSgWVSlVvuVKp5C/O67AftsV+2x57bluW+l2prUFuSSXyLlUit7gSuZcqkVdSibxLVbhcqbNq+x2cFfDzdIFPB2d0dFPC01UJL1clPF1q/zQ93JTwcHFCB2en2mDk7AQXpVxyQaExer0eLgqgYwdXu/98W7t/dh2WoqKisGnTJrNlKSkpiIyMNDUoKioKqampZuctpaSkYPjw4TatlYiIrFNjMCLnQjmyzpYiK78UWWdLcbK4AoKFc3A6e6gQ4tMBXTq5orOnCn4eLvDzUNU+PGufdxDp5GFq/yT1yaioqMDJkydNr3Nzc5GVlQVvb2907doV8fHxOH/+PL766isAwNy5c/Hhhx8iLi4Oc+bMQXp6OlatWmV2ldtTTz2FUaNGYenSpbjrrruwceNGbNu2DXv27LH5/hERUX2FZVeReUmGw1tzcPi8BkfOl+Gq3lhvnE8HZwT7dkCwTweE+LqZngf7dhDtKiqyD5L69Bw8eBBjx441va47b+jhhx9GYmIiCgsLkZ+fb3o/JCQEW7ZswdNPP42PPvoIgYGB+OCDD0zTBgDA8OHDsW7dOrz00kt4+eWX0aNHDyQnJ2Po0KG22zEiIjIRBAG/n9fg56NFSPmjCH9eqACgAP48YxrjoXJC/6COGHDt0T+oIzp71D89gqg1SCosjRkzpt68C9dLTEyst2z06NE4dOiQxe3ef//9uP/++2+2PCIiaqEagxH7cy8j5Y8LSDlahIKyq6b3FHIZAlyNGNUnCIO6+WBAkBe6+7pDLrePc4So/ZNUWCIiIvtxVW/Azj+L8fPRIvxy/CJKq/6a88ZVqcCYnp0R00eNkT288duvqZg8ubfdn3BM7RPDEhER2VRZlR5J+87gy9/yUFLx1wS/ndyUGN9LjQl9/HF7qC9clAoA7XMySnIsDEtERGQT50ursWp3LtYdyDfNa+Tv6YLJfQMQ00eNyG6d4KSQ1C1LyUEwLBERUZv6o0CDlbtOYdPhQhiu3WMjzN8D/xzdHVP6BULJgETtHMMSERG1OkEQkHbqEj7ZeQq7T5SYlg/v4YN/ju6BUaG+djOJI9k/hiUiImpVp4or8ML6wziQdwVA7S007ugXiMdHdkffLl4iV0fUfAxLRETUKgxGAZ/vPo13U/+ErsYIF6UcMwZ3xazbQxDkzZuMk3QxLBER0U07ebEcz357GFlnSwEAo27rjCX39sUtHV3FLYyoFTAsERFRi9UYjPhsdy6Wbas9muShcsLLU3rjgcguPCeJ7AbDEhERtcifF8rx3LfZyD5XBgAY27Mz3ry3LwK8eDSJ7AvDEhERNUuNwYhPd53G+9tOQGcwwsPFCa/c2Qf3DbqFR5PILjEsERGR1YrLtZj91UFkXzs3aVyYH964py/8vVzELYyoDTEsERGRVc6XViP28304XVIJL1clFk3tjbsH8GgS2T+GJSIialJuSSUe+nwfzpdW45aOrkiaPRQhvh3ELovIJhiWiIjIouNFGjz0+X6UVGjR3bcDkmYPRSCnBCAHwrBERESNyjpbioe/2I+yaj16BXjiq8eGoLOHSuyyiGyKYYmIiBqUfuoSZq8+gEqdAQO7dkTiI0Pg5aYUuywim2NYIiKien45fgH/SjoEbY0Rw3v44LN/RKKDiv9kkGPiJ5+IiMxsyi7A08lZqDEKGN9LjQ9nDoSLUiF2WUSiYVgiIiKT5AP5eOH7IxAE4K4BgXjngf5QKuRil0UkKoYlIiICAGw5Uoj/rD8CAPj7kK54/e5wKOScQ4mIYYmIiHC6uALPf3cYAPBwVDcsmtqHk00SXcNjq0REDu6q3oB5Xx9ChbYGQ4K98fKU3gxKRNdhWCIicnCvbDyK40Xl8HV3xvKZA+HEc5SIzPBvBBGRA/su4xySD56FTAa8P2Mg1J68IS7RjRiWiIgcVE5ROV76ofaE7qfH34YRt/qKXBFR+8SwRETkgCq0NfjX1xm4qjdiZKgvnhh7q9glEbVbDEtERA5GEATEf38Ep4sr4e/pgoTpAyDnFAFEjWJYIiJyMEn78rEpuwBOchk+enAgfNx5Y1wiSxiWiIgcyJFzZXht0x8AgP9MDENEN2+RKyJq/xiWiIgcRFmVHvO+yYDOYERMbzVmjwwRuyQiSZBcWFqxYgVCQkLg4uKCiIgI7N69u9GxjzzyCGQyWb1Hnz59TGMSExMbHHP16lVb7A4RkU0IgoBnv8vG2cvVCPJ2xdsP9OfEk0RWklRYSk5OxoIFC7Bw4UJkZmZi5MiRmDRpEvLz8xsc//7776OwsND0OHv2LLy9vfHAAw+YjfP09DQbV1hYCBcXzjVCRPYjaV8+Uv+4AGeFHCtmRsDLVSl2SUSSIamw9N5772HWrFmYPXs2evXqhYSEBAQFBeHjjz9ucLyXlxf8/f1Nj4MHD+LKlSt49NFHzcbJZDKzcf7+/rbYHSIimyir0uPdlBwAQPzkMPTt4iVyRUTSIpkb6ep0OmRkZOCFF14wWx4TE4O0tDSrtrFq1SqMHz8e3bp1M1teUVGBbt26wWAwYMCAAXjttdcwcODARrej1Wqh1WpNrzUaDQBAr9dDr9dbu0t2q64H7IVtsN+2J7Wev78tB6VVeoT6dcCMiEDJ1F1Hav2WOkfqt7X7KJmwVFJSAoPBALVabbZcrVajqKioyfULCwvx008/4ZtvvjFbHhYWhsTERPTt2xcajQbvv/8+RowYgezsbISGhja4rSVLlmDx4sX1lqekpMDNza0Ze2XfUlNTxS7BobDftieFnhdXA6uzFQBkGOejQcrPW8UuqcWk0G974gj9rqqqsmqcZMJSnRtPSBQEwaqTFBMTE9GxY0fcfffdZsuHDRuGYcOGmV6PGDECgwYNwvLly/HBBx80uK34+HjExcWZXms0GgQFBSEmJgaenp7N2Bv7pNfrkZqaiujoaCiVPC+irbHftielnj+xNgsG4SJGhfrgmZkRYpfTIlLqtz1wpH7XfTPUFMmEJV9fXygUinpHkS5evFjvaNONBEHAF198gdjYWDg7O1scK5fLMXjwYJw4caLRMSqVCipV/UnclEql3X+wmoP9sC322/bae8/3517Gz39chFwGvDSlT7uu1Rrtvd/2xhH6be3+SeYEb2dnZ0RERNQ7LJiamorhw4dbXHfnzp04efIkZs2a1eTPEQQBWVlZCAgIuKl6iYjEZDQKeH1z7eSTM4Z0xW1qD5ErIpIuyRxZAoC4uDjExsYiMjISUVFRWLlyJfLz8zF37lwAtV+PnT9/Hl999ZXZeqtWrcLQoUMRHh5eb5uLFy/GsGHDEBoaCo1Ggw8++ABZWVn46KOPbLJPRERt4cfsAhw+VwZ3lROeHn+b2OUQSZqkwtL06dNx6dIlvPrqqygsLER4eDi2bNliurqtsLCw3pxLZWVlWL9+Pd5///0Gt1laWorHH38cRUVF8PLywsCBA7Fr1y4MGTKkzfeHiKgtVOsMWLr1OADgX2N6oLMH7/1GdDMkFZYAYN68eZg3b16D7yUmJtZb5uXlZfFs92XLlmHZsmWtVR4RkehW7TmNwrKruKWjK2bdzluaEN0syZyzRERETbtYfhUf7zgFAHh+Yk+4KBUiV0QkfQxLRER2ZFnqn6jUGTAgqCOm9g8Uuxwiu8CwRERkJ44XaZB84CwA4OUpvXijXKJWwrBERGQHBEHAG5uPwSgAd/QNQEQ3b7FLIrIbDEtERHZgx5/F2H2iBM4KOf4zMUzscojsCsMSEZHE1RiMeGPzMQDAoyOC0dWH96gkak0MS0REErfuwFmcvFiBTm5KzBt7q9jlENkdhiUiIgmrMRhNUwUsGH8bvFzt+15eRGJgWCIikrDUPy7gfGk1vDs4Y/rgILHLIbJLDEtERBL2ZVoeAGDmkK6cgJKojTAsERFJ1NGCMuzPvQwnuQwPDesmdjlEdothiYhIohJ/ywMATOobAH8vF3GLIbJjDEtERBJ0qUKLjdkFAIBHhgeLWwyRnWNYIiKSoLX786GrMaJ/Fy8M6tpR7HKI7BrDEhGRxOgNRqzZewYA8MiIYN4DjqiNMSwREUnM1t+LcEGjha+7CpP7BohdDpHdY1giIpKYL3/LBQA8NKwrVE6cLoCorTEsERFJSPbZUhzKL4VSIcPMoV3FLofIITAsERFJyOprk1BO6RcIPw9OF0BkCwxLREQScbH8KjYd5nQBRLbGsEREJBHf7MuH3iBgUNeO6B/UUexyiBwGwxIRkQRoawxI2psPAHh0RIjI1RA5FoYlIiIJ2HKkECUVWqg9VZgY7i92OUQOhWGJiKidEwQBX167D1zssG5QKvirm8iW+DeOiKidO5RfisPnyuDsJMffh3C6ACJbY1giImrnEq9NF3BX/0D4uKvELYbIATEsERG1Y0VlV/HTkUIAtfeBIyLbY1giImrHkvaeQY1RwJAQb/QJ9BK7HCKHxLBERNRO6Q1GrDtwbboATkJJJBqGJSKiduq3kyUoqdDBp4MzonurxS6HyGExLBERtVM/Ztfe2mRy3wA4cboAItFI7m/fihUrEBISAhcXF0RERGD37t2Njt2xYwdkMlm9x/Hjx83GrV+/Hr1794ZKpULv3r2xYcOGtt4NIiKLruoNSDl6AQBw14BAkashcmySCkvJyclYsGABFi5ciMzMTIwcORKTJk1Cfn6+xfVycnJQWFhoeoSGhpreS09Px/Tp0xEbG4vs7GzExsZi2rRp2LdvX1vvDhFRo349fhEV2hrc0tEVg7p2ErscIocmqbD03nvvYdasWZg9ezZ69eqFhIQEBAUF4eOPP7a4np+fH/z9/U0PhUJhei8hIQHR0dGIj49HWFgY4uPjMW7cOCQkJLTx3hARNW5jVu1XcFP6B0Aul4lcDZFjcxK7AGvpdDpkZGTghRdeMFseExODtLQ0i+sOHDgQV69eRe/evfHSSy9h7NixpvfS09Px9NNPm42fMGGCxbCk1Wqh1WpNrzUaDQBAr9dDr9dbu0t2q64H7IVtsN+219Y9L7+qxy85FwEAk/v4Ofx/W37GbcuR+m3tPkomLJWUlMBgMECtNr8iRK1Wo6ioqMF1AgICsHLlSkRERECr1WLNmjUYN24cduzYgVGjRgEAioqKmrVNAFiyZAkWL15cb3lKSgrc3Nyau2t2KzU1VewSHAr7bXtt1fP9F2XQ1SigdhWQe2gP8nhgCQA/47bmCP2uqqqyapxkwlIdmcz8t4YgCPWW1enZsyd69uxpeh0VFYWzZ8/inXfeMYWl5m4TAOLj4xEXF2d6rdFoEBQUhJiYGHh6ejZrf+yRXq9HamoqoqOjoVQqxS7H7rHfttfWPf9udQaAS5gRdSvuGNuj1bcvNfyM25Yj9bvum6GmSCYs+fr6QqFQ1Dvic/HixXpHhiwZNmwYkpKSTK/9/f2bvU2VSgWVqv79mZRKpd1/sJqD/bAt9tv22qLnJRVapJ2+DAC4Z1AQ/5teh59x23KEflu7f5I5wdvZ2RkRERH1DgumpqZi+PDhVm8nMzMTAQEBptdRUVH1tpmSktKsbRIRtZYtRwphMAro18ULwb4dxC6HiCChI0sAEBcXh9jYWERGRiIqKgorV65Efn4+5s6dC6D267Hz58/jq6++AlB7pVtwcDD69OkDnU6HpKQkrF+/HuvXrzdt86mnnsKoUaOwdOlS3HXXXdi4cSO2bduGPXv2iLKPROTYfrx2FdzU/pxbiai9kFRYmj59Oi5duoRXX30VhYWFCA8Px5YtW9CtWzcAQGFhodmcSzqdDs8++yzOnz8PV1dX9OnTB5s3b8bkyZNNY4YPH45169bhpZdewssvv4wePXogOTkZQ4cOtfn+EZFjO3elCgfPXIFMBtzJsETUbkgqLAHAvHnzMG/evAbfS0xMNHv9/PPP4/nnn29ym/fffz/uv//+1iiPiKjFNmUXAgCGhnhD7ekicjVEVEcy5ywREdm7unvB3TXgFpErIaLrMSwREbUDJy+W41ihBkqFDJPC/cUuh4iuw7BERNQO1J3YPSq0Mzq6OYtcDRFdj2GJiEhkgiBg47Wv4KYO4IndRO0NwxIRkcgOnyvDmUtVcFUqML6X9ZPsEpFtMCwREYms7sTu8b3V6KCS3EXKRHaPYYmISEQGo4D/HeZElETtGcMSEZGI9uVewgWNFp4uThh1m6/Y5RBRAxiWiIhEtOnaV3CT+wZA5aQQuRoiagjDEhGRSHQ1Rmw5UgSAX8ERtWcMS0REItl9ohhl1Xr4eagwtLuP2OUQUSMYloiIRLLx2kSUd/QLgEIuE7kaImoMwxIRkQiqdQak/nEBAO8FR9TeMSwREYlg94liVOsN6NLJFf27eIldDhFZwLBERCSC7ccuAgDG91JDJuNXcETtGcMSEZGNGY0CfsmpDUvjevmJXA0RNYVhiYjIxo6cL0NxuRbuKicMDeFVcETtHcMSEZGNbT9We2L3qNt84ezEX8NE7R3/lhIR2di2a+crjQtTi1wJEVmDYYmIyIYKSqvxR6EGMhkwpmdnscshIiswLBER2dAvx2uPKg3q2gk+7iqRqyEiazAsERHZUN35SrwKjkg6GJaIiGykSleD305dAlA7vxIRSQPDEhGRjew5UQJdjRFB3q4I9XMXuxwishLDEhGRjWy/7io4ztpNJB0MS0RENsBZu4mki2GJiMgGOGs3kXQxLBER2QBn7SaSLv6NJSKyAc7aTSRdDEtERG2Ms3YTSRvDEhFRG+Os3UTSJrmwtGLFCoSEhMDFxQURERHYvXt3o2O///57REdHo3PnzvD09ERUVBR+/vlnszGJiYmQyWT1HlevXm3rXSEiB8FZu4mkTVJhKTk5GQsWLMDChQuRmZmJkSNHYtKkScjPz29w/K5duxAdHY0tW7YgIyMDY8eOxZ133onMzEyzcZ6enigsLDR7uLi42GKXiMjOcdZuIulzEruA5njvvfcwa9YszJ49GwCQkJCAn3/+GR9//DGWLFlSb3xCQoLZ6zfffBMbN27Epk2bMHDgQNNymUwGf3//Nq2diBxT3azdXTpx1m4iqZJMWNLpdMjIyMALL7xgtjwmJgZpaWlWbcNoNKK8vBze3t5myysqKtCtWzcYDAYMGDAAr732mlmYupFWq4VWqzW91mg0AAC9Xg+9Xm/tLtmtuh6wF7bBfttec3q+7Y8iAMDYnp1RU1PTpnXZK37GbcuR+m3tPkomLJWUlMBgMECtNj+MrVarUVRUZNU23n33XVRWVmLatGmmZWFhYUhMTETfvn2h0Wjw/vvvY8SIEcjOzkZoaGiD21myZAkWL15cb3lKSgrc3NyasVf2LTU1VewSHAr7bXtN9dwoAFsPKwDI4F6Wiy1bTtumMDvFz7htOUK/q6qqrBonmbBU58b7KQmCYNU9ltauXYtFixZh48aN8PP76yTLYcOGYdiwYabXI0aMwKBBg7B8+XJ88MEHDW4rPj4ecXFxptcajQZBQUGIiYmBp6dnc3fJ7uj1eqSmpiI6OhpKpVLscuwe+2171vb88LkyaPbuQweVAk9MG8/JKFuIn3HbcqR+130z1BTJhCVfX18oFIp6R5EuXrxY72jTjZKTkzFr1ix8++23GD9+vMWxcrkcgwcPxokTJxodo1KpoFLVv/xXqVTa/QerOdgP22K/ba+pnu88UXti9+jbOqODK6cMuFn8jNuWI/Tb2v2TzP/mODs7IyIiot5hwdTUVAwfPrzR9dauXYtHHnkE33zzDe64444mf44gCMjKykJAQMBN10xEjq1u1u6/cdZuIkmTzJElAIiLi0NsbCwiIyMRFRWFlStXIj8/H3PnzgVQ+/XY+fPn8dVXXwGoDUr/+Mc/8P7772PYsGGmo1Kurq7w8vICACxevBjDhg1DaGgoNBoNPvjgA2RlZeGjjz4SZyeJyC4Ulv01a/dYztpNJGmSCkvTp0/HpUuX8Oqrr6KwsBDh4eHYsmULunXrBgAoLCw0m3Pp008/RU1NDebPn4/58+eblj/88MNITEwEAJSWluLxxx9HUVERvLy8MHDgQOzatQtDhgyx6b4RkX3ZfoyzdhPZC0mFJQCYN28e5s2b1+B7dQGozo4dO5rc3rJly7Bs2bJWqIyI6C+ctZvIfkjmnCUiIqngrN1E9oVhiYiolf128hJn7SayIwxLREStbOeftecrje3pZ9U8cETUvjEsERG1sl1/lgConV+JiKSPYYmIqBXllVQi/3IVlAoZonr4iF0OEbUChiUiola060QxACCiWyd0UEnugmMiagDDEhFRK9r1Z21YGsWv4IjsBsMSEVEr0dUYkX5tyoBRoQxLRPaCYYmIqJVknLmCSp0Bvu7O6B3gKXY5RNRKGJaIiFpJ3flKI0M7Qy7nlAFE9oJhiYiolfx1vpKvyJUQUWtiWCIiagXF5VocLdAAqD2yRET2g2GJiKgV7DlZe1SpT6AnfN1VIldDRK2JYYmIqBXUzdrNKQOI7A/DEhHRTTIaBey+dnI3pwwgsj8MS0REN+mPQg1KKnTo4KxARLdOYpdDRK2MYYmI6CbVTRkQ1cMHzk78tUpkb/i3mojoJvEWJ0T2jWGJiOgmVGprkHHmCgCer0RkrxiWiIhuQvqpS9AbBHT1dkOwbwexyyGiNsCwRER0E+rOV+Ks3UT2i2GJiOgmmM5X4ldwRHaLYYmIqIXOXK5C3qUqOMlliOrhI3Y5RNRGGJaIiFpoz4naWbsHdesEDxelyNUQUVthWCIiaqHdJy8BAEZzygAiu8awRETUAjVGYO/pywB4vhKRvXNq7go5OTlYu3Ytdu/ejby8PFRVVaFz584YOHAgJkyYgPvuuw8qFe+4TUT2La8cqNQZ4NPBGX0CPcUuh4jakNVHljIzMxEdHY3+/ftj165dGDx4MBYsWIDXXnsNDz30EARBwMKFCxEYGIilS5dCq9W2Zd1ERKI6Vlb76/P2UF/I5TKRqyGitmT1kaW7774bzz33HJKTk+Ht7d3ouPT0dCxbtgzvvvsuXnzxxVYpkoiovTleWhuQ+BUckf2zOiydOHECzs7OTY6LiopCVFQUdDrdTRVGRNReXarQ4lxlbVgayckoieye1V/DWROUAKCqqqpZ44mIpGbPqdoTu3v5e8DPw0XkaoiorbXoargxY8bg3Llz9Zbv27cPAwYMuNmaLFqxYgVCQkLg4uKCiIgI7N692+L4nTt3IiIiAi4uLujevTs++eSTemPWr1+P3r17Q6VSoXfv3tiwYUNblU9EdqBufqWRoZyIksgRtCgseXp6ol+/fli3bh0AwGg0YtGiRRg1ahSmTp3aqgVeLzk5GQsWLMDChQuRmZmJkSNHYtKkScjPz29wfG5uLiZPnoyRI0ciMzMTL774Ip588kmsX7/eNCY9PR3Tp09HbGwssrOzERsbi2nTpmHfvn1tth9EJF1Go4A9p2rnVxp5K7+CI3IEzZ46AAB+/PFHfPLJJ5g9ezZ+/PFH5OXlIT8/H5s3b8b48eNbu0aT9957D7NmzcLs2bMBAAkJCfj555/x8ccfY8mSJfXGf/LJJ+jatSsSEhIAAL169cLBgwfxzjvv4L777jNtIzo6GvHx8QCA+Ph47Ny5EwkJCVi7dm2b7QsRSdOxIg1KKnRwlgsY1LWj2OUQkQ20KCwBwNy5c3HmzBksXboUTk5O2LFjB4YPH96atZnR6XTIyMjACy+8YLY8JiYGaWlpDa6Tnp6OmJgYs2UTJkzAqlWroNfroVQqkZ6ejqeffrremLqA1RCtVms2NYJGowEA6PV66PX65uyWXarrAXthG+y3bf167AIAINRLgEwwsO82wM+4bTlSv63dxxaFpStXrmD27NnYvn07Pv30U+zcuRMxMTF46623MG/evJZsskklJSUwGAxQq9Vmy9VqNYqKihpcp6ioqMHxNTU1KCkpQUBAQKNjGtsmACxZsgSLFy+utzwlJQVubm7W7pLdS01NFbsEh8J+28bGo3IAcoR5Cey5jbHftuUI/a67KK0pLQpL4eHhCAkJQWZmJkJCQjBnzhwkJydj3rx52Lx5MzZv3tySzVpFJjOf/E0QhHrLmhp/4/LmbjM+Ph5xcXGm1xqNBkFBQYiJiYGnJ2fy1ev1SE1NRXR0NJRK3ly0rbHftlOprcGz+38FICCso8Ce2wg/47blSP2u+2aoKS0KS3PnzsXChQshl/91fvj06dMxYsQIPProoy3ZZJN8fX2hUCjqHfG5ePFivSNDdfz9/Rsc7+TkBB8fH4tjGtsmAKhUqgZv6aJUKu3+g9Uc7Idtsd9tL+PkZegNArp0ckVnl3L23MbYb9tyhH5bu38tuhru5ZdfNgtKdbp06dJmh+2cnZ0RERFRb/upqamNnisVFRVVb3xKSgoiIyNNDWpsTFuef0VE0rTrz2IAwMhbfWDh4DMR2Rmrw1Jjl+c35vz5880upilxcXH4/PPP8cUXX+DYsWN4+umnkZ+fj7lz5wKo/XrsH//4h2l83UnocXFxOHbsGL744gusWrUKzz77rGnMU089hZSUFCxduhTHjx/H0qVLsW3bNixYsKDV6yciadtVN78SpwwgcihWh6XBgwdjzpw52L9/f6NjysrK8NlnnyE8PBzff/99qxR4venTpyMhIQGvvvoqBgwYgF27dmHLli3o1q0bAKCwsNAs1IWEhGDLli3YsWMHBgwYgNdeew0ffPCBadoAABg+fDjWrVuHL7/8Ev369UNiYiKSk5MxdOjQVq+fiKTr7OUq5JZUwkkuw7Dujd8fk4jsj9XnLB07dgxvvvkmJk6cCKVSicjISAQGBsLFxQVXrlzBH3/8gaNHjyIyMhJvv/02Jk2a1CYFz5s3r9Er7hITE+stGz16NA4dOmRxm/fffz/uv//+1iiPiOzUzmtfwQ3q2gkeLi2edYWIJMjqI0vnzp3D0qVLUVBQgE8++QS33XYbSkpKcOLECQDAgw8+iIyMDPz2229tFpSIiMRSd77SKN44l8jhWP2/RwMHDkRRURE6d+6MZ555BgcOHDBdUUZEZM/0BiPSrt3iZNRtnUWuhohszeojSx07dsTp06cBAHl5eTAajW1WFBFRe5KZX4oKbQ28OzgjPNBL7HKIyMasPrJ03333YfTo0QgICIBMJkNkZCQUCkWDY+tCFRGRPaj7Cu72W30hl8tgMIhcEBHZlNVhaeXKlbj33ntx8uRJPPnkk5gzZw48PDzasjYionZh14m685X4FRyRI2rWJR0TJ04EAGRkZOCpp55iWCIiu3e5Uocj58sAAKNCeXI3kSNq0fWvX375ZWvXQUTULu0+UQxBAML8PeDn6SJ2OUQkghbd7oSIyFHUza80ml/BETkshiUiokYIgoDd125xwvOViBwXwxIRUSOOFZajuFwLV6UCkcGdxC6HiETCsERE1Ii6q+CGdfeGyqnhqVKIyP4xLBERNWIXz1ciIjAsERE1qEpXg4N5VwDwfCUiR8ewRETUgL2nL0FnMKJLJ1eE+HYQuxwiEhHDEhFRA3b9+ddVcDKZTORqiEhMDEtERA2oO19pVCi/giNydAxLREQ3OHu5CqdLKqGQyzD8Vh+xyyEikTEsERHdoG7KgEFdO8LTRSlyNUQkNoYlIqIb8Cs4IroewxIR0XX0BiPSTl4CwCkDiKgWwxIR0XWyzpaiXFuDTm5KhN/iJXY5RNQOMCwREV2n7iu420M7QyHnlAFExLBERGTmr/OVfEWuhIjaC4YlIqJrLlfqcPh8GQCer0REf2FYIiK6Zs/JEggCEObvAbWni9jlEFE7wbBERHSN6Ss4HlUiouswLBERATAaBezIqQ1LoxmWiOg6DEtERACyz5WipEILD5UTBgd7i10OEbUjDEtERAC2H7sIABjVszOcnfirkYj+wt8IREQAth27AAAYF+YnciVE1N4wLBGRwztfWo3jReWQy4CxPRmWiMicZMLSlStXEBsbCy8vL3h5eSE2NhalpaWNjtfr9fjPf/6Dvn37okOHDggMDMQ//vEPFBQUmI0bM2YMZDKZ2WPGjBltvDdE1J78cu2oUkS3TujUwVnkaoiovZFMWJo5cyaysrKwdetWbN26FVlZWYiNjW10fFVVFQ4dOoSXX34Zhw4dwvfff48///wTU6dOrTd2zpw5KCwsND0+/fTTttwVImpntl07X2lcL7XIlRBRe+QkdgHWOHbsGLZu3Yq9e/di6NChAIDPPvsMUVFRyMnJQc+ePeut4+XlhdTUVLNly5cvx5AhQ5Cfn4+uXbualru5ucHf379td4KI2qVKbQ3ST10CAIzvxa/giKg+SYSl9PR0eHl5mYISAAwbNgxeXl5IS0trMCw1pKysDDKZDB07djRb/vXXXyMpKQlqtRqTJk3CK6+8Ag8Pj0a3o9VqodVqTa81Gg2A2q/+9Hp9M/bMPtX1gL2wDfb75uw8fhE6gxFBnVzRtaPKqj6y57bFftuWI/Xb2n2URFgqKiqCn1/9/+Pz8/NDUVGRVdu4evUqXnjhBcycOROenp6m5Q8++CBCQkLg7++P33//HfHx8cjOzq53VOp6S5YsweLFi+stT0lJgZubm1X1OAJLPaTWx363zDcn5QDk6K6qxE8//dSsddlz22K/bcsR+l1VVWXVOFHD0qJFixoMHdc7cOAAAEAmk9V7TxCEBpffSK/XY8aMGTAajVixYoXZe3PmzDE9Dw8PR2hoKCIjI3Ho0CEMGjSowe3Fx8cjLi7O9Fqj0SAoKAgxMTFmQcxR6fV6pKamIjo6GkqlUuxy7B773XJGo4DX3t4JQIfHJg7G8B4+Vq3HntsW+21bjtTvum+GmiJqWHriiSeavPIsODgYhw8fxoULF+q9V1xcDLXa8gmZer0e06ZNQ25uLn755Zcmw8ygQYOgVCpx4sSJRsOSSqWCSqWqt1ypVNr9B6s52A/bYr+bLzP/CkoqdPBQOSHqVj8omzkZJXtuW+y3bTlCv63dP1HDkq+vL3x9fZscFxUVhbKyMuzfvx9DhgwBAOzbtw9lZWUYPnx4o+vVBaUTJ07g119/hY9P0//XePToUej1egQEBFi/I0QkSaZZu2/jrN1E1DhJ/Hbo1asXJk6ciDlz5mDv3r3Yu3cv5syZgylTppid3B0WFoYNGzYAAGpqanD//ffj4MGD+Prrr2EwGFBUVISioiLodDoAwKlTp/Dqq6/i4MGDyMvLw5YtW/DAAw9g4MCBGDFihCj7SkS2s/143ZQBvAqOiBonibAE1F6x1rdvX8TExCAmJgb9+vXDmjVrzMbk5OSgrKwMAHDu3Dn8+OOPOHfuHAYMGICAgADTIy0tDQDg7OyM7du3Y8KECejZsyeefPJJxMTEYNu2bVAoFDbfRyKynfOl1ThWqOGs3UTUJElcDQcA3t7eSEpKsjhGEATT8+DgYLPXDQkKCsLOnTtbpT4ikhbO2k1E1pLMkSUiotbEWbuJyFoMS0TkcK6ftXtcGL+CIyLLGJaIyOHsOVkCncGIrt5uuNXPXexyiKidY1giIoez/dr5SuN6+Vk1sS0ROTaGJSJyKEajgF+OFwMAxvN8JSKyAsMSETmU7HOlKKnQwkPlhMHB3mKXQ0QSwLBERA7ll+OctZuImoe/KYjIofw1ZQCvgiMi6zAsEZHD4KzdRNQSDEtE5DA4azcRtQTDEhE5DM7aTUQtwbBERA6Bs3YTUUsxLBGRQ+Cs3UTUUgxLROQQOGs3EbUUwxIR2T2DUTDNr8RZu4mouRiWiMjupZ+6hJIKHTq5KTEkhLN2E1HzMCwRkd37Mfs8AGBS3wAoFfy1R0TNw98aRGTXtDUG/PR7EQBgav9AkashIiliWCIiu7YjpxjlV2vg7+mCIbxxLhG1AMMSEdm1H7MLAAB39g+AXM6r4Iio+RiWiMhuVWprTFMGTO1/i8jVEJFUMSwRkd1K/eMCruqNCPHtgPBbPMUuh4gkimGJiOzWxqzaq+Cm9g/kRJRE1GIMS0Rkl65U6rD7RAkAYOoAXgVHRC3HsEREdmnL74WoMQroE+iJHp15LzgiajmGJSKySz9m1V4Fx7mViOhmMSwRkd0pLKvG/rzLAIA7GZaI6CYxLBGR3flfdiEEARgS7I3Ajq5il0NEEsewRER2xzQRJU/sJqJWwLBERHbldHEFjpwvg0Iuw+Rwf7HLISI7wLBERHal7qjSyFBf+LirRK6GiOyBZMLSlStXEBsbCy8vL3h5eSE2NhalpaUW13nkkUcgk8nMHsOGDTMbo9Vq8e9//xu+vr7o0KEDpk6dinPnzrXhnhBRWxEEwRSWeBUcEbUWyYSlmTNnIisrC1u3bsXWrVuRlZWF2NjYJtebOHEiCgsLTY8tW7aYvb9gwQJs2LAB69atw549e1BRUYEpU6bAYDC01a4QURs5WqDB6eJKqJzkiOnDr+CIqHU4iV2ANY4dO4atW7di7969GDp0KADgs88+Q1RUFHJyctCzZ89G11WpVPD3b/iXZllZGVatWoU1a9Zg/PjxAICkpCQEBQVh27ZtmDBhQuvvDBG1mbqjSuN6+cFdJYlfb0QkAZL4bZKeng4vLy9TUAKAYcOGwcvLC2lpaRbD0o4dO+Dn54eOHTti9OjReOONN+Dn5wcAyMjIgF6vR0xMjGl8YGAgwsPDkZaW1mhY0mq10Gq1ptcajQYAoNfrodfrb2pf7UFdD9gL22C/axmNAn68di+4O8LVbdoP9ty22G/bcqR+W7uPkghLRUVFpoBzPT8/PxQVFTW63qRJk/DAAw+gW7duyM3Nxcsvv4y//e1vyMjIgEqlQlFREZydndGpUyez9dRqtcXtLlmyBIsXL663PCUlBW5ubs3YM/uWmpoqdgkOxdH7fUoDFGmc4KIQUH06A1vy2v5nOnrPbY39ti1H6HdVVZVV40QNS4sWLWowdFzvwIEDANDgHcMFQbB4J/Hp06ebnoeHhyMyMhLdunXD5s2bce+99za6XlPbjY+PR1xcnOm1RqNBUFAQYmJi4OnpaXF/HIFer0dqaiqio6OhVCrFLsfusd+1Xtn0B4BzmNzvFtw1JbxNfxZ7blvst205Ur/rvhlqiqhh6YknnsCMGTMsjgkODsbhw4dx4cKFeu8VFxdDrVZb/fMCAgLQrVs3nDhxAgDg7+8PnU6HK1eumB1dunjxIoYPH97odlQqFVSq+pckK5VKu/9gNQf7YVuO3G+9wYiffq/9HXH3wC4264Mj91wM7LdtOUK/rd0/UcOSr68vfH19mxwXFRWFsrIy7N+/H0OGDAEA7Nu3D2VlZRZDzY0uXbqEs2fPIiAgAAAQEREBpVKJ1NRUTJs2DQBQWFiI33//HW+99VYL9oiIxLDnZAmuVOnh6+6M4T18xC6HiOyMJKYO6NWrFyZOnIg5c+Zg79692Lt3L+bMmYMpU6aYndwdFhaGDRs2AAAqKirw7LPPIj09HXl5edixYwfuvPNO+Pr64p577gEAeHl5YdasWXjmmWewfft2ZGZm4qGHHkLfvn1NV8cRUfu3Kav2Krg7+gbASSGJX2tEJCGSOMEbAL7++ms8+eSTpivXpk6dig8//NBsTE5ODsrKygAACoUCR44cwVdffYXS0lIEBARg7NixSE5OhoeHh2mdZcuWwcnJCdOmTUN1dTXGjRuHxMREKBQK2+0cEbVYtc6An4/WXpAxlfeCI6I2IJmw5O3tjaSkJItjBEEwPXd1dcXPP//c5HZdXFywfPlyLF++/KZrJCLb25h1HpU6A4K8XTGoa6emVyAiaiYeryYiyRIEAYlpeQCAfwwLtngVKxFRSzEsEZFkpZ++hONF5XBVKjBtcJDY5RCRnWJYIiLJSvwtDwBwX8Qt8HK170uciUg8DEtEJElnL1dh27HauZUeGR4sbjFEZNcYlohIkr5Kz4NRAEaG+uJWP4+mVyAiaiGGJSKSnEptDdYdOAsAeHREsLjFEJHdY1giIsn5PvM8yq/WINjHDWNuq3+TbSKi1sSwRESSIggCEn/LBQA8PDwYcjmnCyCitsWwRESSsudkCU4VV8Jd5YT7I7qIXQ4ROQCGJSKSlC+vTRdwf0QXeLhwugAiansMS0QkGbkllfjl+EUAtV/BERHZAsMSEUnGV+l5AICxPTsjxLeDuMUQkcNgWCIiSSi/qse3B88BAB4dESJyNUTkSBiWiEgS1mecQ4W2Bj06d8DIUF+xyyEiB8KwRETtntEoYHX6GQDAIyNCIJNxugAish2GJSJq93b+WYzckkp4uDjh3oG3iF0OETkYhiUiave+TMsDAEyPDEIHlZO4xRCRw2FYIqJ27eTFCuz6sxgyGacLICJxMCwRUbu2+tpRpfG91AjydhO3GCJySAxLRNRulVXrsf5Q3XQBweIWQ0QOi2GJiNqtVbtPo0pnQE+1B6K6+4hdDhE5KIYlImqXCsuqsXL3aQDA09GhnC6AiETDsERE7dLbP+fgqt6IIcHemNDHX+xyiMiBMSwRUbtz5FwZvj90HgDw0pRePKpERKJiWCKidkUQBLy++Q8AwD0Db0G/Lh3FLYiIHB7DEhG1Kyl/XMC+3MtQOcnx3ISeYpdDRMSwRETth67GiCVbjgEAHh/VHYEdXUWuiIiIYYmI2pE1e88g71IVfN1V+OfoHmKXQ0QEgGGJiNqJ0iodPth+AgDwbMxtcOc94IionWBYIqJ24YPtJ1FWrUeYvwceiAwSuxwiIhOGJSIS3eniCnyVngcAWHhHLyjknCqAiNoPyYSlK1euIDY2Fl5eXvDy8kJsbCxKS0striOTyRp8vP3226YxY8aMqff+jBkz2nhviOh6/++n46gxChjbszNGhnYWuxwiIjOSOSlg5syZOHfuHLZu3QoAePzxxxEbG4tNmzY1uk5hYaHZ659++gmzZs3CfffdZ7Z8zpw5ePXVV02vXV15BQ6Rrew9fQkpf1yAQi7Di5N7iV0OEVE9kghLx44dw9atW7F3714MHToUAPDZZ58hKioKOTk56Nmz4blY/P3Nb5GwceNGjB07Ft27dzdb7ubmVm8sEbU9o/GvCSj/PiQIoWoPkSsiIqpPEmEpPT0dXl5epqAEAMOGDYOXlxfS0tIaDUvXu3DhAjZv3ozVq1fXe+/rr79GUlIS1Go1Jk2ahFdeeQUeHo3/0tZqtdBqtabXGo0GAKDX66HX65uza3aprgfshW1Iud8bMgvw+3kN3FVOeGJ0iGT2Qco9lyL227Ycqd/W7qMkwlJRURH8/PzqLffz80NRUZFV21i9ejU8PDxw7733mi1/8MEHERISAn9/f/z++++Ij49HdnY2UlNTG93WkiVLsHjx4nrLU1JS4ObmZlU9jsBSD6n1Sa3fOgPwRpYCgAx/U2uxb9d2sUtqNqn1XOrYb9tyhH5XVVVZNU7UsLRo0aIGQ8f1Dhw4AAAN3khTEASrb7D5xRdf4MEHH4SLi4vZ8jlz5pieh4eHIzQ0FJGRkTh06BAGDRrU4Lbi4+MRFxdneq3RaBAUFISYmBh4enpaVY890+v1SE1NRXR0NJRKpdjl2D2p9vutn/9EmS4Pt3R0wZuPjIBKqRC7JKtJtedSxX7bliP1u+6boaaIGpaeeOKJJq88Cw4OxuHDh3HhwoV67xUXF0OtVjf5c3bv3o2cnBwkJyc3OXbQoEFQKpU4ceJEo2FJpVJBpVLVW65UKu3+g9Uc7IdtSanf249dwGd78gAA/3dnH7i7uVheoZ2SUs/tAfttW47Qb2v3T9Sw5OvrC19f3ybHRUVFoaysDPv378eQIUMAAPv27UNZWRmGDx/e5PqrVq1CREQE+vfv3+TYo0ePQq/XIyAgoOkdIKJmO3elCnH/zQYAPDI8GBP68OIKImrfJDHPUq9evTBx4kTMmTMHe/fuxd69ezFnzhxMmTLF7OTusLAwbNiwwWxdjUaDb7/9FrNnz6633VOnTuHVV1/FwYMHkZeXhy1btuCBBx7AwIEDMWLEiDbfLyJHo6sxYv43mSir1qN/UEdOFUBEkiCJsATUXrHWt29fxMTEICYmBv369cOaNWvMxuTk5KCsrMxs2bp16yAIAv7+97/X26azszO2b9+OCRMmoGfPnnjyyScRExODbdu2QaGQzvkTRFLx5pZjyD5bCi9XJT6aORDOTpL5FUREDkwSV8MBgLe3N5KSkiyOEQSh3rLHH38cjz/+eIPjg4KCsHPnzlapj4gs23y4EIlpeQCAZdP7o0snXjlKRNLA/60jojZ3urgC/1l/GADwrzE98Lewpi/MICJqLxiWiKhNXdUbMO/rQ6jQ1mBIiDeeib5N7JKIiJqFYYmI2tQrG4/ieFE5fN2dsfzvA+Gk4K8dIpIW/tYiojbzXcY5JB88C5kMeH/GQKg9pTmfEhE5NoYlImoTx4s0eOmHIwCAp8ffhhG3Nj2nGhFRe8SwREStrkJbg3lfH8JVvREjQ33xxNhbxS6JiKjFGJaIqFVd1Rvw1NpMnC6uhL+nCxKmD4Bcbt09HImI2iPJzLNERO1fla4Gj3+VgT0nS+DsJMeHMwfCx73+fRSJiKSEYYmIWkVZtR6Pfrkfh/JL4easwOcPRyIy2FvssoiIbhrDEhHdtJIKLWJX7cexQg08XZyQ+NgQDOraSeyyiIhaBcMSEd2UgtJqPLRqH04XV8LXXYU1s4agV4Cn2GUREbUahiUiarG8kko8+Pk+nC+tRqCXC5JmD0X3zu5il0VE1KoYloioRXKKyvHQqn0oLtcixLcDkmYPxS0dXcUui4io1TEsEVGzZZ8txcNf7kdplR5h/h5YM2soOnvwqjcisk8MS0TULOmnLmH26gOo1BkwIKgjEh8djI5uzmKXRUTUZhiWiMgqV/UGvL/9BFbuOg2DUUBUdx989nAk3FX8NUJE9o2/5YioSZn5V/Dcd4dx8mIFAOCuAYFYel8/uCgVIldGRNT2GJaIqFFX9QYs2/YnPtt1GkYB8HVX4fW7wzEx3F/s0oiIbIZhiYgalHHmCp7/LhuniisBAPcMvAX/N6U3OnXg+UlE5FgYlojIzFW9Ae+m5ODzPbkQBKCzhwpv3tMX0b3VYpdGRCQKhiUiMjmQdxn/+e4wTpfUHk26d1Dt0SRe7UZEjoxhicjBCYKAXSdK8OnOU0g7dQkAoPZUYcm9ffG3MB5NIiJiWCJyUHqDEf87XIBPd57G8aJyAIBCLsO0yC54YVIveLkqRa6QiKh9YFgicjAV2hqs25+PL/bkoqDsKgDAzVmBvw/pisduD+EtS4iIbsCwROQgLpZfReJveUjaewaaqzUAaqcCeHREMB4a2g1ebjySRETUEIYlIjt2QXMVqX9cwM9Hi7D39CXoDQIAoLtvBzw+qjvuHngLJ5YkImoCwxKRnTldXIGfj15Ayh9FyMwvNXsvolsn/HNUd4zvpYZcLhOnQCIiiWFYIpI4g1HAmQrgvdQTSD1ebLolSZ2BXTtiQh9/xPRWo3tnd5GqJCKSLoYlIokpLKtGVn4pss6VIiu/FEfOl6FK5wQgFwCgVMgQ1cMXMb3ViOmthp+ni7gFExFJHMMSUTslCAIuV+qQc6EcWWdrg1H2uVJc0GjrjXVRCBgb5o+JfQMwNswPni48WZuIqLUwLBGJrKxKj9xLlcgtqUBuSRXySiqRd6kSuSWVKL921dr1FHIZeqo9MKBrRwwI6ojwAHfkHNiFKXf0h1LJkERE1NokE5beeOMNbN68GVlZWXB2dkZpaWmT6wiCgMWLF2PlypW4cuUKhg4dio8++gh9+vQxjdFqtXj22Wexdu1aVFdXY9y4cVixYgW6dOnShntDjkBXY0RxhRYXNVdxsVyLi+VaFF/3/GL5VZy/Uo0rVXqL2+nSyRX9u9QGo/5BHdH3Fi+4Ov91BZter8cJnqtNRNRmJBOWdDodHnjgAURFRWHVqlVWrfPWW2/hvffeQ2JiIm677Ta8/vrriI6ORk5ODjw8PAAACxYswKZNm7Bu3Tr4+PjgmWeewZQpU5CRkQGFgpdUOyJBEKAzGFGlNaBKb0C1rgaVWgOqdAZU62ufa67qUVZd+9BU66GprjG9vv5hLT8PFUJ8OyDEtwOCfTsg2Kf2eTcfN17aT0QkMsmEpcWLFwMAEhMTrRovCAISEhKwcOFC3HvvvQCA1atXQ61W45tvvsE///lPlJWVYdWqVVizZg3Gjx8PAEhKSkJQUBC2bduGCRMmtMm+WOtKpQ6VOvOvYQTB8jrXvy9AqLdcML0Wbnh93VrXjf3r+XXLBfPXRkG4tqx2u/qaGuSWA4fySyFXKCAI140RaqsyCgKMpuUCjMa6ZX8tNxhr1zEYBRiuvTYYa8fUGK79afxreY3BCL1RgL7GiBqjAL3BiBqDAL3x2p8GI3Q1RmhrjNDWGEzPr1+m1RtRpTfAYGyi0VZyksvQ2UMFPw8VOnu4wM+z9rmfhwv8PFQI6OiCYJ8O6KCSzF9FIiKHY7e/oXNzc1FUVISYmBjTMpVKhdGjRyMtLQ3//Oc/kZGRAb1ebzYmMDAQ4eHhSEtLazQsabVaaLV/nWSr0WgA1H4dotdbfzShKUu3HsO6A+dabXu25YSE3/eLXcRNc3aSw02pgKuzAm7XHq5KBTxdnODpqoSXqxIeLk7wclXC64Zl3h2c0clVacV8RsJNfW7q1m3Nzx5Zxp7bFvttW47Ub2v30W7DUlFREQBArTa/a7parcaZM2dMY5ydndGpU6d6Y+rWb8iSJUtMR7qul5KSAjc3t5st3eT8WTmUDfxD29Q/vbJGXsgaG9PAOvXGXlvW2PK61zLZdc+vPZHXvS8zXy4DIL/hT9m18TKZYPZaLrvhce09xbX1FNeey+W1fzrJALlMMC1XXLdcKQecrj2UMsBJLlz3HHCWAyoF4KyoXcciAUD1tcd1Txv/9LSd1NRUEX6qY2PPbYv9ti1H6HdVVZVV40QNS4sWLWowdFzvwIEDiIyMbPHPkMnM/7UTBKHeshs1NSY+Ph5xcXGm1xqNBkFBQYiJiYGnp2eLa73R5Fbbkm3p9XqkpqYiOjqaV2fZAPtte+y5bbHftuVI/a77ZqgpooalJ554AjNmzLA4Jjg4uEXb9vf3B1B79CggIMC0/OLFi6ajTf7+/tDpdLhy5YrZ0aWLFy9i+PDhjW5bpVJBpVLVW65UKu3+g9Uc7Idtsd+2x57bFvttW47Qb2v3T9Sw5OvrC19f3zbZdkhICPz9/ZGamoqBAwcCqL2ibufOnVi6dCkAICIiAkqlEqmpqZg2bRoAoLCwEL///jveeuutNqmLiIiIpEUy5yzl5+fj8uXLyM/Ph8FgQFZWFgDg1ltvhbt77f2uwsLCsGTJEtxzzz2QyWRYsGAB3nzzTYSGhiI0NBRvvvkm3NzcMHPmTACAl5cXZs2ahWeeeQY+Pj7w9vbGs88+i759+5qujiMiIiLHJpmw9H//939YvXq16XXd0aJff/0VY8aMAQDk5OSgrKzMNOb5559HdXU15s2bZ5qUMiUlxTTHEgAsW7YMTk5OmDZtmmlSysTERM6xRERERAAkFJYSExObnGNJuGESIplMhkWLFmHRokWNruPi4oLly5dj+fLlrVAlERER2Rt500OIiIiIHBfDEhEREZEFDEtEREREFjAsEREREVnAsERERERkAcMSERERkQUMS0REREQWMCwRERERWcCwRERERGSBZGbwbs/qZg7XaDQiV9I+6PV6VFVVQaPR2P0dq9sD9tv22HPbYr9ty5H6Xffv9o13ALkRw1IrKC8vBwAEBQWJXAkRERE1V3l5Oby8vBp9XyY0FaeoSUajEQUFBfDw8IBMJhO7HNFpNBoEBQXh7Nmz8PT0FLscu8d+2x57blvst205Ur8FQUB5eTkCAwMhlzd+ZhKPLLUCuVyOLl26iF1Gu+Pp6Wn3f9HaE/bb9thz22K/bctR+m3piFIdnuBNREREZAHDEhEREZEFDEvU6lQqFV555RWoVCqxS3EI7Lftsee2xX7bFvtdH0/wJiIiIrKAR5aIiIiILGBYIiIiIrKAYYmIiIjIAoYlIiIiIgsYlshmtFotBgwYAJlMhqysLLHLsUt5eXmYNWsWQkJC4Orqih49euCVV16BTqcTuzS7sWLFCoSEhMDFxQURERHYvXu32CXZpSVLlmDw4MHw8PCAn58f7r77buTk5IhdlsNYsmQJZDIZFixYIHYp7QLDEtnM888/j8DAQLHLsGvHjx+H0WjEp59+iqNHj2LZsmX45JNP8OKLL4pdml1ITk7GggULsHDhQmRmZmLkyJGYNGkS8vPzxS7N7uzcuRPz58/H3r17kZqaipqaGsTExKCyslLs0uzegQMHsHLlSvTr10/sUtoNTh1ANvHTTz8hLi4O69evR58+fZCZmYkBAwaIXZZDePvtt/Hxxx/j9OnTYpcieUOHDsWgQYPw8ccfm5b16tULd999N5YsWSJiZfavuLgYfn5+2LlzJ0aNGiV2OXaroqICgwYNwooVK/D6669jwIABSEhIELss0fHIErW5CxcuYM6cOVizZg3c3NzELsfhlJWVwdvbW+wyJE+n0yEjIwMxMTFmy2NiYpCWliZSVY6jrKwMAPhZbmPz58/HHXfcgfHjx4tdSrvCG+lSmxIEAY888gjmzp2LyMhI5OXliV2SQzl16hSWL1+Od999V+xSJK+kpAQGgwFqtdpsuVqtRlFRkUhVOQZBEBAXF4fbb78d4eHhYpdjt9atW4dDhw7hwIEDYpfS7vDIErXIokWLIJPJLD4OHjyI5cuXQ6PRID4+XuySJc3afl+voKAAEydOxAMPPIDZs2eLVLn9kclkZq8FQai3jFrXE088gcOHD2Pt2rVil2K3zp49i6eeegpJSUlwcXERu5x2h+csUYuUlJSgpKTE4pjg4GDMmDEDmzZtMvvHxGAwQKFQ4MEHH8Tq1avbulS7YG2/637JFRQUYOzYsRg6dCgSExMhl/P/i26WTqeDm5sbvv32W9xzzz2m5U899RSysrKwc+dOEauzX//+97/xww8/YNeuXQgJCRG7HLv1ww8/4J577oFCoTAtMxgMkMlkkMvl0Gq1Zu85GoYlalP5+fnQaDSm1wUFBZgwYQK+++47DB06FF26dBGxOvt0/vx5jB07FhEREUhKSnLoX3CtbejQoYiIiMCKFStMy3r37o277rqLJ3i3MkEQ8O9//xsbNmzAjh07EBoaKnZJdq28vBxnzpwxW/boo48iLCwM//nPfxz+60+es0RtqmvXrmav3d3dAQA9evRgUGoDBQUFGDNmDLp27Yp33nkHxcXFpvf8/f1FrMw+xMXFITY2FpGRkYiKisLKlSuRn5+PuXPnil2a3Zk/fz6++eYbbNy4ER4eHqbzwry8vODq6ipydfbHw8OjXiDq0KEDfHx8HD4oAQxLRHYlJSUFJ0+exMmTJ+uFUR5EvnnTp0/HpUuX8Oqrr6KwsBDh4eHYsmULunXrJnZpdqdueoYxY8aYLf/yyy/xyCOP2L4gcmj8Go6IiIjIAp71SURERGQBwxIRERGRBQxLRERERBYwLBERERFZwLBEREREZAHDEhEREZEFDEtEREREFjAsEREREVnAsERERERkAcMSERERkQUMS0RENyguLoa/vz/efPNN07J9+/bB2dkZKSkpIlZGRGLgveGIiBqwZcsW3H333UhLS0NYWBgGDhyIO+64AwkJCWKXRkQ2xrBERNSI+fPnY9u2bRg8eDCys7Nx4MABuLi4iF0WEdkYwxIRUSOqq6sRHh6Os2fP4uDBg+jXr5/YJRGRCHjOEhFRI06fPo2CggIYjUacOXNG7HKISCQ8skRE1ACdTochQ4ZgwIABCAsLw3vvvYcjR45ArVaLXRoR2RjDEhFRA5577jl89913yM7Ohru7O8aOHQsPDw/873//E7s0IrIxfg1HRHSDHTt2ICEhAWvWrIGnpyfkcjnWrFmDPXv24OOPPxa7PCKyMR5ZIiIiIrKAR5aIiIiILGBYIiIiIrKAYYmIiIjIAoYlIiIiIgsYloiIiIgsYFgiIiIisoBhiYiIiMgChiUiIiIiCxiWiIiIiCxgWCIiIiKygGGJiIiIyAKGJSIiIiIL/j8DwmsAANeaVgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(-5,5,0.2), np.tanh(np.arange(-5,5,0.2)))\n",
    "plt.title('tanh(x)')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('f(x)')\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A more common activation function these days and that is more efficient to implement is the _Rectified Linear Unit_ or _ReLU_.\n",
    "\n",
    "$$ \\textrm{ReLU}(x) = \\mathrm{max}(0, x) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "hide_input": true,
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAHFCAYAAADcytJ5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7SUlEQVR4nO3deXyU5b3+8WuSTCYLCQiBsIV9CRACCoi4IKggiAubtXU52mNtPaLVQwUEqoALEdCqrUdcarW1RSwBWQSVILKIGyCSsO8ECFtYkpBlMpl5fn8o+UlZTEIy9zMzn/frxR9z55lwzTchuXjuZ2YclmVZAgAAsKEw0wEAAADOh6ICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsi6IC4Ge9++67cjgc5X8iIiLUqFEj/fKXv9T27dsr/fmWLVsmh8Oh9PT08x7jcDj08MMPn/Nj6enpcjgcWrZs2Vkfu/766/Xggw9WOtPbb7+tJk2aqLCwsNL3BVBzKCoAKuydd97RV199pSVLlujhhx/W/PnzdfXVV+vEiROmo0mS5s2bp1WrVunJJ5+s9H3vvfdexcbGaurUqTWQDEBVUVQAVFhKSoquuOIK9enTR+PHj9cTTzyhI0eOaO7cuaajSZImT56sIUOGqEmTJpW+b0REhH73u9/plVdeUVFRUQ2kA1AVFBUAVda9e3dJ0uHDh8vX1qxZo1tvvVV169ZVVFSULr30Uv373/+u8Szr1q3Tt99+q3vuuad8zbIs3XTTTapXr56ys7PL14uKitSpUyd16NDhjK2eu+66S/n5+Zo5c2aN5wVQMRQVAFW2e/duSVK7du0kSZ9//rmuuuoqnTx5Uq+//rrmzZunrl276o477tC7775bo1k++ugjhYeHq3fv3uVrDodD7733nmJiYvSLX/xCHo9HkvTQQw9p9+7d+ve//63Y2Njy4xs2bKjk5GQtXLiwRrMCqLgI0wEABA6v16uysjKVlJRo1apVevbZZ9W7d2/deuutkn4oAJ06ddLSpUsVEfHDj5cbb7xRubm5GjdunP7rv/5LYWE18/+jr776Sm3btlWtWrXOWK9Xr55mzpypPn36aPTo0UpNTdXf//53/fWvf1Xnzp3P+jyXXXaZlixZUiMZAVQeRQVAhV1xxRVn3O7QoYPmzZuniIgI7dixQ1u2bNELL7wgSSorKys/7qabbtJHH32krVu3qkOHDjWSLScnRw0aNDjnx6666io999xzGjNmjFwul+6++27df//95zy2QYMGOnLkiMrKysrLFgBz2PoBUGH/+Mc/tHr1ai1dulS/+93vtHnzZv3qV7+S9P+vU3n88cfldDrP+PPQQw9JknJzcyv8d4WHh8vr9Z7zY6dLkNPpLF8rLi5WVFTUeT/fXXfdpcjISLndbo0aNeq8x0VFRcmyLJWUlFQ4K4Caw38XAFRYhw4dyi+g7du3r7xer/76178qPT29fBtl7NixGjp06Dnv3759+wr/XYmJiTpw4MA5P3Z6PTExsXwtISFBx48fP+fxXq9Xd911ly655BK5XC7df//9WrVqlSIjI8869vjx43K5XGdtIQEwg6ICoMqmTp2q2bNn66mnntKGDRvUtm1brV+/XpMnT77oz33DDTdozpw5Onr0qOrXr1++blmWZs2apRYtWqhNmzbl68nJyed9mvSECRO0cuVKLV68WLGxserdu7dGjRqlV1555axjd+3apY4dO150fgDVg6ICoMouueQSjR07VqNHj9aMGTP0xhtvaODAgbrxxht13333qUmTJjp+/Lg2b96s7777TrNmzTrj/l9//fU5P++1116rp556SgsWLFDPnj31xBNPqG3btjp06JDeeustrV69+qynPPfp00d/+9vftG3btvJnIUlSRkaG0tLS9OSTT+r666+XJKWlpenxxx9Xnz59NGTIkPJjfT6fvv322/NevwLAAAsAfsY777xjSbJWr1591seKi4utZs2aWW3btrXKysqs9evXW7/4xS+sBg0aWE6n02rYsKF13XXXWa+//nr5fT7//HNL0nn/fP7555ZlWdb27dutu+++22rUqJEVERFh1alTx+rfv7/12WefnZUjLy/PqlWrljV16tTytZycHKtBgwbWddddZ3m93vJ1n89n3XLLLVadOnWs3bt3l69/9tlnliRr7dq11TA1ANXBYVmWZawlAUA1euSRR/TZZ59p48aNcjgclb7/Pffco127dmnVqlU1kA5AVVBUAASNw4cPq127dnr77bc1fPjwSt13586d6tChg5YuXaqrr766hhICqCyengwgaCQmJupf//qXiouLK33f7Oxsvfrqq5QUwGY4owIAAGyLMyoAAMC2KCoAAMC2KCoAAMC2jL7g28SJEzVp0qQz1hITE3Xo0KEK3d/n8yknJ0dxcXFVeioiAADwP8uyVFBQoMaNG//sO6obf2XaTp06nfGW6uHh4RW+b05OjpKSkmoiFgAAqGH79u1T06ZNL3iM8aISERGhhg0bVum+cXFxkn54oPHx8dUZK2B5PB4tXrxY/fv3P+OdZVEzmLd/MW//Yt7+FUrzzs/PV1JSUvnv8QsxXlS2b9+uxo0by+VyqWfPnpo8ebJatWp1zmPdbrfcbnf57YKCAklSdHS0oqOj/ZLX7iIiIhQTE6Po6Oig/0a3A+btX8zbv5i3f4XSvD0ejyRV6LINo6+j8vHHH6uoqEjt2rXT4cOH9eyzz2rLli3auHGj6tWrd9bx57qmRZJmzJihmJgYf0QGAAAXqaioSHfeeafy8vJ+dkfEVi/4VlhYqNatW2v06NEaOXLkWR//zzMqp08d5ebmsvXzI4/Ho4yMDPXr1y/oG7kdMG//Yt7+xbz9K5TmnZ+fr4SEhAoVFeNbPz8VGxurzp07a/v27ef8uMvlksvlOmvd6XQG/Re1spiJfzFv/2Le/sW8/SsU5l2Zx2er11Fxu93avHmzGjVqZDoKAACwAaNF5fHHH9fy5cu1e/duffPNNxo+fLjy8/N17733mowFAABswujWz/79+/WrX/1Kubm5ql+/vq644gp9/fXXat68uclYAADAJowWlZkzZ5r86wEAgM3Z6hoVAACAn6KoAAAA26KoAAAA26KoAAAA26KoAAAA26KoAACAs1iWpaVbDsv0O+1QVAAAwFk+XHdA//3uGv33u6uNlhWKCgAAOMORghJNWrBJktS9RV05HA5jWSgqAACgnGVZenLuBuUVe5TSJF6/7d3KaB6KCgAAKLcw66A+3XhYEWEOTR3WRc5ws1WBogIAACRJx065NWHeRknSQ33bqGPjeMOJKCoAAOBHkxZs0rHCUrVPjNPDfduYjiOJogIAACQt3nhI89fnKMwhTbs9VZER9qgI9kgBAACMySvyaPzcDZKk3/ZurdSmdcwG+gmKCgAAIe6ZhZt0tMCtVvVj9dgNbU3HOQNFBQCAELZs6xGlr90vh0OaNjxVUc5w05HOQFEBACBEFZR4NHZOliTp11e2VLfmdQ0nOhtFBQCAEJX28RYdzCtRs7oxevzGdqbjnBNFBQCAEPTljlzN+CZbkjRlWKpiIiMMJzo3igoAACGmqLRMY+ZkSpLuvqKZerWuZzjR+VFUAAAIMVM/2ap9x4vVpE60nhjYwXScC6KoAAAQQlbvOa6/f7VHkjR5aGfVctlzy+c0igoAACGixOPVmPRMWZZ0e7emurZdfdORfhZFBQCAEPHSkm3alVuoBnEu/fHmjqbjVAhFBQCAEPD9vpN6a8UuSdJzQzqrdrTTcKKKoagAABDk3GVejU5fL58l3da1sfp1TDQdqcIoKgAABLn/W7pD2w6fUkKtSE24pZPpOJVCUQEAIIhtzMnTa8t2SpKevi1FdWMjDSeqHIoKAABByuP1adSsTJX5LA1MaaibOjcyHanSKCoAAASpN5bv1KaD+aoT49TTt6WYjlMlFBUAAILQtsMF+vNnOyRJE2/ppPpxLsOJqoaiAgBAkCnz+jRq1nqVen26PrmBbuva2HSkKqOoAAAQZP62arfW789TXFSEnhvSWQ6Hw3SkKqOoAAAQRHYdPaUXF2+TJD05qKMa1o4ynOjiUFQAAAgSPp+lMbMz5S7z6Zq2Cbq9e1PTkS4aRQUAgCDxj6/2aPWeE4qNDNfkAN/yOY2iAgBAENh3vEhTPtkqSXpiYLKS6sYYTlQ9KCoAAAQ4y/phy6fY41XPlnV1V8/mpiNVG4oKAAAB7v1v9+nLnccU5QzTlGGpCgsL/C2f0ygqAAAEsJyTxZq8aLMk6fH+7dUiIdZwoupFUQEAIEBZlqVxH2bplLtMlzWro19f1dJ0pGpHUQEAIEDN+e6Alm09qsiIME0d3kXhQbTlcxpFBQCAAHQkv0STFmyUJD12Q1u1aVDLcKKaQVEBACDAWJal8XM3KL+kTJ2b1NZvr2llOlKNoagAABBgPso8qIxNh+UMd2ja7amKCA/eX+fB+8gAAAhCx065NWH+D1s+I/q2UXLDeMOJahZFBQCAADJh/kYdLyxVcsM4PdSnjek4NY6iAgBAgPhkwyF9lHlQ4WEOTRveRZERwf9rPPgfIQAAQeBkUamenLdBkvS73q3UuWltw4n8g6ICAEAAePqjTTpa4Fbr+rH6/fVtTcfxG4oKAAA29/mWI5rz3QE5HNK027soyhluOpLfUFQAALCx/BKPxn2YJUm6/6qWuqzZJYYT+RdFBQAAG0tbtEUH80rUvF6M/tC/vek4fkdRAQDAplbtyNX732ZLkqYMS1V0ZOhs+ZxGUQEAwIYK3WUaMztTkvRfvZrrilb1DCcyg6ICAIANTft0q/afKFaTOtEaPSDZdBxjKCoAANjMt7uP690v90iSnh/WWbVcEWYDGURRAQDARko83vItnzu6J+matvUNJzKLogIAgI28/NkO7c4tVMP4KI2/uYPpOMbZpqikpaXJ4XDoscceMx0FAAAj9hRI73y5V5L03JAUxUc5DScyzxZFZfXq1XrzzTeVmppqOgoAAEa4y3x6f2e4fJY05NImur5DoulItmC8qJw6dUp33XWX3nrrLV1ySWi92h4AAKe9tmyXDhU7lFArUhNu6Wg6jm0Yv4x4xIgRGjRokG644QY9++yzFzzW7XbL7XaX387Pz5ckeTweeTyeGs0ZKE7PgXn4B/P2L+btX8zbfzbm5OuNFbslSU8ObKdYpyOo516Zx2a0qMycOVPfffedVq9eXaHj09LSNGnSpLPWFy9erJiYmOqOF9AyMjJMRwgpzNu/mLd/Me+a5fVJL2aFy2s51LWuT9r/vRbt/950rBpVVFRU4WONFZV9+/bp0Ucf1eLFixUVFVWh+4wdO1YjR44sv52fn6+kpCT1799f8fHxNRU1oHg8HmVkZKhfv35yOrkIq6Yxb/9i3v7FvP3j/5bt0oGiHaoT7dTwVsUhMe/TOyIVYayorF27VkeOHFG3bt3K17xer1asWKFXX31Vbrdb4eFnvqeBy+WSy+U663M5nc6g/6JWFjPxL+btX8zbv5h3zdl6qED/t2ynJOmpm5MVvn9dSMy7Mo/PWFG5/vrrlZWVdcbar3/9ayUnJ2vMmDFnlRQAAIJJmden0enr5fFauqFDom7u3FAf7zedyn6MFZW4uDilpKScsRYbG6t69eqdtQ4AQLB5+4vdWr8/T/FREXpuSIocDofpSLZk/OnJAACEmp1HT+nFjG2SpCdv7qjE+IpdqxmKjD89+aeWLVtmOgIAADXK67M0Jj1TpWU+9W5XX8O7NTUdydY4owIAgB/946s9WrP3hGIjw5U2tDNbPj+DogIAgJ9kHyvS1E+2SpLG3tRBTepEG05kfxQVAAD8wOezNGZ2poo9XvVqVU93Xt7MdKSAQFEBAMAP3l+dra92HVO0M1zPD+ussDC2fCqCogIAQA07cLJYaYu2SJJG3dhezevFGk4UOCgqAADUIMuyNG5Olk65y9S9+SW678oWpiMFFIoKAAA1KH3tfi3fdlSREWGaMjyVLZ9KoqgAAFBDDueX6JmPNkmSRvZrp9b1axlOFHgoKgAA1ADLsjT+ww3KLylTatPa+s3VLU1HCkgUFQAAasCCzINasvmwnOEOTRveRRHh/MqtCqYGAEA1yz3l1oR5GyRJj1zXVu0bxhlOFLgoKgAAVLMJ8zfqRJFHHRrF63/6tDYdJ6BRVAAAqEafbDiohZkHFR7m0LThqXKy5XNRmB4AANXkZFGp/jh3oyTpf65trZQmtQ0nCnwUFQAAqsnTCzYp95RbbRrU0iPXtzEdJyhQVAAAqAZLtxzWnHUHFOaQpg1PlSsi3HSkoEBRAQDgIuWXeDRuzg/P8rn/6pa6tNklhhMFD4oKAAAXKW3RZh3KL1GLejEa2a+96ThBhaICAMBF+GJ7rt7/dp8kacqwVEVHsuVTnSgqAABUUaG7TGNmZ0qS7u3VXD1b1TOcKPhQVAAAqKKpn2zRgZPFalInWqMHJJuOE5QoKgAAVME3u47p71/tlfTDlk+sK8JwouBEUQEAoJKKS73lWz6/7JGkq9smGE4UvCgqAABU0p8ytmrPsSI1jI/SuEEdTMcJahQVAAAq4bvsE3r7i92SpMlDUxQf5TScKLhRVAAAqKASj1ej0zPls6ShlzbRdcmJpiMFPYoKAAAV9Jel27XjyCkl1HLpqVs6mo4TEigqAABUQNb+PL2+fJck6dnBKaoTE2k4UWigqAAA8DNKy3walb5eXp+lQamNNCCloelIIYOiAgDAz5i+bKe2HCpQ3dhITbq1k+k4IYWiAgDABWw5lK9XP98uSZp4aycl1HIZThRaKCoAAJxHmdenUbMy5fFa6tcxUbekNjIdKeRQVAAAOI+3Vu5W1oE8xUdF6NnBKXI4HKYjhRyKCgAA57DjyCm9tGSbJOnJmzsqMT7KcKLQRFEBAOA/eH2WRqevV2mZT9e2q6/h3ZqajhSyKCoAAPyHd7/co++yT6qWK0KTh3Zmy8cgigoAAD+x91ihpn26RZI09qZkNakTbThRaKOoAADwI5/P0uj0TJV4fLqydT3deXkz05FCHkUFAIAf/evbbH2z+7iineF6fmgqWz42QFEBAEDS/hNFen7RZknS6AHt1axejOFEkCgqAADIsiyNnZOlwlKvuje/RPf2amE6En5EUQEAhLxZa/Zr5fZcuSLCNHV4qsLC2PKxC4oKACCkHcor0TMLN0mSRvZrp1b1axlOhJ+iqAAAQpZlWRr/YZYKSsrUpWlt3X91S9OR8B8oKgCAkDV/fY4+23JEznCHpt3eRRHh/Fq0G74iAICQdLTArQnzN0qSfn9dW7VLjDOcCOdCUQEAhKQJ8zfoZJFHHRvF68E+rU3HwXlQVAAAIWdR1kEtyjqkiDCHpt2eKidbPrbFVwYAEFJOFJbqqXkbJEn/06e1OjWubTgRLoSiAgAIKZMWbFTuqVK1bVBLD1/XxnQc/AyKCgAgZCzZdFhzv89RmEOadnsXuSLCTUfCz6CoAABCQl6xR+PnZkmSHrimlbom1TEbCBVCUQEAhITJCzfrcL5bLRNi9b/92pmOgwqiqAAAgt7K7Uf1wZp9cjikqcNTFeVkyydQUFQAAEHtlLtMT8z+Ycvn3l4t1KNFXcOJUBkUFQBAUJvy8RYdOFmspLrRGj2gvek4qCSKCgAgaH2965je+3qvJOn5oamKiYwwnAiVRVEBAASl4lKvxszOlCT96vJmuqpNguFEqAqKCgAgKL2weKv2HitSo9pRGndTsuk4qCKjRWX69OlKTU1VfHy84uPj1atXL3388ccmIwEAgsDavSf0t1W7JUmTh3ZWXJTTcCJUldGi0rRpUz3//PNas2aN1qxZo+uuu0633XabNm7caDIWACCAlXi8Gp2+XpYlDbusqfq2b2A6Ei6C0auKbrnlljNuP/fcc5o+fbq+/vprderUyVAqAEAg+/Nn27XzaKHqx7n01M0dTcfBRbLN5c9er1ezZs1SYWGhevXqdc5j3G633G53+e38/HxJksfjkcfj8UtOuzs9B+bhH8zbv5i3fwXivDccyNcbK3ZJkp6+pYNinIGTPxDnXVWVeYwOy7KsGszys7KystSrVy+VlJSoVq1amjFjhm666aZzHjtx4kRNmjTprPUZM2YoJiampqMCAGyszCe9kBWug0UOXVrPp/va+UxHwnkUFRXpzjvvVF5enuLj4y94rPGiUlpaquzsbJ08eVKzZ8/WX//6Vy1fvlwdO559uu5cZ1SSkpKUm5v7sw80VHg8HmVkZKhfv35yOrl4rKYxb/9i3v4VaPP+y9Kd+vPnO1U31qlFj1ylerGRpiNVSqDN+2Lk5+crISGhQkXF+NZPZGSk2rRpI0nq3r27Vq9erVdeeUVvvPHGWce6XC65XK6z1p1OZ9B/USuLmfgX8/Yv5u1fgTDvzQfz9dryH7d8bktRwzqxhhNVXSDM+2JV5vHZ7nVULMs646wJAAAXUub1aVT6epX5LN3YKVGDOjcyHQnVyOgZlXHjxmngwIFKSkpSQUGBZs6cqWXLlumTTz4xGQsAEEDeXLlLGw7kq3a0U8/cliKHw2E6EqqR0aJy+PBh3XPPPTp48KBq166t1NRUffLJJ+rXr5/JWACAALHjSIFeXrJdkvTUzR3VID7KcCJUN6NF5e233zb51wMAApjXZ2lUeqZKy3zq276+hl7WxHQk1ADbXaMCAEBFvLNqt9Zln1ScK0KTh3ZmyydIUVQAAAFnT26hXli8VZI0blAHNaodbTgRagpFBQAQUHw+S6NnZ6rE49NVberplz2STEdCDaKoAAACyr++2atvdx9XTGS4nh+aypZPkKOoAAACxr7jRUr7eIskacyAZCXV5e1Tgh1FBQAQECzL0rgPs1RU6tXlLerqniuam44EP6CoAAACwr/X7NPK7blyRYRpyvBUhYWx5RMKKCoAANs7lFeiZz/aLEn6Q/92apkQuO/lg8qhqAAAbO30lk+Bu0xdkuro/qtbmY4EP6KoAABsbe73B7R0yxFFhofpheGpCmfLJ6RQVAAAtnWkoEQT52+SJD16Q1u1TYwznAj+RlEBANjWhHkblVfsUafG8fptb7Z8QhFFBQBgSwszD+rjDYcUEebQ1OGpcobzKysU8VUHANjO8cJSPTVvgyTpoT6t1alxbcOJYApFBQBgO5MWbNSxwlK1T4zTw9e1NR0HBlFUAAC2krHpsOZ9n6MwhzR1eKoiI/hVFcr46gMAbCOvyKPxH2ZJkh7o3UpdkuqYDQTjKCoAANt4duEmHSlwq1VCrP73hnam48AGKCoAAFtYtvWIZq3dL8ePWz5RznDTkWADFBUAgHEFJR6Nm/PDls99V7ZQ9xZ1DSeCXVBUAADGPf/xFuXklSipbrRG3djedBzYCEUFAGDUlzty9a9vsiVJU4amKiYywnAi2AlFBQBgTFFpmcbMyZQk3dWzma5sk2A4EeyGogIAMGbap1u173ixGteO0hMDk03HgQ1RVAAARqzZc1zvfrlHkjR5aGfFRTnNBoItUVQAAH5X4vFqdHqmLEsa3q2p+rRvYDoSbKrSVyxt3bpV77//vlauXKk9e/aoqKhI9evX16WXXqobb7xRw4YNk8vlqomsAIAg8dKSbdqVW6gGcS49Oaij6TiwsQqfUVm3bp369eunLl26aMWKFerRo4cee+wxPfPMM7r77rtlWZbGjx+vxo0ba8qUKXK73TWZGwAQoNbvO6m3VuySJD03pLNqx7Dlg/Or8BmVwYMHa9SoUfrggw9Ut+75X4jnq6++0ksvvaQXX3xR48aNq5aQAIDg4C7zalT6evks6dYujdWvY6LpSLC5CheV7du3KzIy8meP69Wrl3r16qXS0tKLCgYACD7/t3SHth0+pXqxkZp4ayfTcRAAKrz1U5GSIklFRUWVOh4AEBo25uTptWU7JUlP35aiurH8nsDPq9Kzfvr06aP9+/eftf7NN9+oa9euF5sJABBkPF6fRqdnqsxnaUCnhrqpc0PTkRAgqlRU4uPjlZqaqpkzZ0qSfD6fJk6cqN69e+vWW2+t1oAAgMD35opd2piTr9rRTj09uJMcDofpSAgQVXpDhfnz5+v111/Xb37zG82fP1979uxRdna2Fi5cqBtuuKG6MwIAAti2wwV6Zcl2SdKEWzqqQVyU4UQIJFV+56cHH3xQe/fu1ZQpUxQREaFly5bpyiuvrM5sAIAA5/VZGp2eqVKvT9clN9CQS5uYjoQAU6WtnxMnTmjYsGGaPn263njjDf3iF79Q//799dprr1V3PgBAAPvbF7v1/b6TinNF6LkhKWz5oNKqdEYlJSVFLVu21Lp169SyZUs98MAD+uCDD/TQQw9p4cKFWrhwYXXnBAAEmN25hXph8VZJ0vhBHdSodrThRAhEVTqj8uCDD2rFihVq2bJl+dodd9yh9evX8/opAAD5fJbGpGfKXebT1W0SdEePJNOREKCqdEblySefPOd606ZNlZGRcVGBAACB772v9+rbPccVExmutKGd2fJBlVX4jEp2dnalPvGBAwcqHQYAEPj2HS/SlE+2SJKeGJispLoxhhMhkFW4qPTo0UMPPPCAvv322/Mek5eXp7feekspKSmaM2dOtQQEAAQOy7L0xJxMFZV6dXnLurq7Z3PTkRDgKrz1s3nzZk2ePFkDBgyQ0+lU9+7d1bhxY0VFRenEiRPatGmTNm7cqO7du2vatGkaOHBgTeYGANjQzNX7tGrHMUU5wzR1WKrCwtjywcWp8BmV/fv3a8qUKcrJydHrr7+udu3aKTc3V9u3//AiPnfddZfWrl2rVatWUVIAIATlnCzWcws3S5Ie799eLRJiDSdCMKjwGZVLL71Uhw4dUv369fWHP/xBq1evVr169WoyGwAgQFiWpfEfZumUu0yXNqujX1/V8ufvBFRAhc+o1KlTR7t27ZIk7dmzRz6fr8ZCAQACy4frDujzrUcVGR6macNTFc6WD6pJhc+oDBs2TNdee60aNWokh8Oh7t27Kzw8/JzHni40AIDgdyS/RJMWbJIkPXpDW7VpEGc4EYJJhYvKm2++qaFDh2rHjh36/e9/rwceeEBxcXwzAkAosyxLf5y7QXnFHqU0iddve7cyHQlBplIv+DZgwABJ0tq1a/Xoo49SVAAgxC3MOqjFmw4rIsyhacO7yBlepRc8B86rSq9M+84771R3DgBAgDleWKoJ8zZKkkb0baMOjeINJ0IwovoCAKpkwvyNOlZYquSGcRrRt43pOAhSFBUAQKV9uvGQFqzPUfiPWz6REfw6Qc3gOwsAUCl5RR79ce4GSdJve7dS56a1DSdCMKOoAAAq5ZmFm3S0wK1W9WP16PVtTcdBkKOoAAAq7POtR5S+dr8cDmna8FRFOc/9elpAdaGoAAAqpKDEo3FzsiRJ/31VS3VrXtdwIoQCigoAoELSPt6ig3klal4vRo/3b286DkIERQUA8LO+3JGrGd9kS5KeH5qq6Ei2fOAfFBUAwAUVuss0Zk6mJOnuK5qpV+t6hhMhlFBUAAAX9KclO7TveLGa1InWEwM7mI6DEGO0qKSlpalHjx6Ki4tTgwYNNHjwYG3dutVkJADAT+zMl977ccsnbWhn1XJV6Z1XgCozWlSWL1+uESNG6Ouvv1ZGRobKysrUv39/FRYWmowFAJBU4vFq5s5wWZb0i+5N1btdfdOREIKMVuNPPvnkjNvvvPOOGjRooLVr16p3796GUgEAJOmVpTt1pMShBnEujR/U0XQchChbncPLy8uTJNWte+7n5rvdbrnd7vLb+fn5kiSPxyOPx1PzAQPA6TkwD/9g3v7FvP1n/f48/W3VHknShJvaKSaCude0UPr+rsxjdFiWZdVglgqzLEu33XabTpw4oZUrV57zmIkTJ2rSpElnrc+YMUMxMTE1HREAQkKZT5qWGa5DxQ51S/Dpv9r6TEdCkCkqKtKdd96pvLw8xcfHX/BY2xSVESNGaOHChfriiy/UtGnTcx5zrjMqSUlJys3N/dkHGio8Ho8yMjLUr18/OZ1O03GCHvP2L+btHy8t2aHXlu9SvVin/tChWINvYt7+EErf3/n5+UpISKhQUbHF1s8jjzyi+fPna8WKFectKZLkcrnkcrnOWnc6nUH/Ra0sZuJfzNu/mHfN2XAgT2+s3C1JmnBzB1nZ3zFvPwuFeVfm8Rl91o9lWXr44Yc1Z84cLV26VC1btjQZBwBCmsfr0+j0THl9lgamNNTAlIamIwFmz6iMGDFCM2bM0Lx58xQXF6dDhw5JkmrXrq3o6GiT0QAg5LyxfKc2HcxXnRinnr4txXQcQJLhMyrTp09XXl6e+vTpo0aNGpX/+eCDD0zGAoCQs+1wgf782Q5J0sRbOql+3Nnb7IAJRs+o2OQ6XgAIaWVen0bNWq9Sr0/XJzfQbV0bm44ElOO9fgAgxP1t1W6t35+nuKgIPTeksxwOh+lIQDmKCgCEsF1HT+nFxdskSU8O6qiGtaMMJwLORFEBgBDl81kanZ4pd5lP17RN0O3dz//yEIApFBUACFH/+GqP1uw9odjIcKUNZcsH9kRRAYAQlH2sSFM+2SpJeuKmDmp6CW9DAnuiqABAiLEsS2NmZ6rY41XPlnV11+XNTEcCzouiAgAh5v1v9+mrXccU5QzTlGGpCgtjywf2RVEBgBBy4GSxJi/aLEl6vH97tUiINZwIuDCKCgCECMuyNG5Olk65y3RZszr69VW8vxrsj6ICACFi9ncHtHzbUUVGhGnq8C4KZ8sHAYCiAgAh4Eh+iZ5esFGS9NgNbdWmQS3DiYCKoagAQJCzLEvj525QfkmZOjeprd9e08p0JKDCKCoAEOQWZB5UxqbDcoY7NO32VEWE86MfgYPvVgAIYsdOuTVx/g9bPiP6tlFyw3jDiYDKoagAQBCbMH+jjheWKrlhnB7q08Z0HKDSKCoAEKQ+2XBIH2UeVHiYQ9OGd1FkBD/yEXj4rgWAIHSyqFR/nLtBkvS73q3UuWltw4mAqqGoAEAQevqjTco95Vbr+rH6/fVtTccBqoyiAgBB5vMtRzTnuwNyOKSpw7soyhluOhJQZRQVAAgi+SUejfswS5J0/1Ut1a35JYYTAReHogIAQSRt0WYdzCtR83ox+kP/9qbjABeNogIAQeKL7bl6/9t9kqQpw1IVHcmWDwIfRQUAgkChu0xPzMmUJN1zRXNd0aqe4URA9aCoAEAQmPbpVu0/UawmdaI1ZmCy6ThAtaGoAECA+3b3cb375R5J0vPDOquWK8JsIKAaUVQAIIAVl3o1ZvYPWz53dE/SNW3rG04EVC+KCgAEsJeWbNPu3EIlxrs0blAH03GAakdRAYAAtS77hP66cpckafKQzqod7TScCKh+FBUACEDuMq9Gp2fKZ0lDLm2i6zskmo4E1AiKCgAEoL98tkPbj5xSQi2Xnrq5o+k4QI2hqABAgNlwIE/Tl++UJD07uJMuiY00nAioORQVAAggHq9Po9Iz5fVZGtS5kQakNDIdCahRFBUACCCvL9upzQfzdUmMUxNv7WQ6DlDjKCoAECC2HS7Qn5dulyRNvLWT6se5DCcCah5FBQACQJnXp1Gz1svjtXRDh0Td2qWx6UiAX1BUACAAvP3Fbq3fn6e4qAg9NyRFDofDdCTALygqAGBzO4+e0osZ2yRJT97cUYnxUYYTAf5DUQEAG/P5LI1Jz1RpmU+929XX7d2amo4E+BVFBQBs7O9f7dGavScUGxmutKGd2fJByKGoAIBNZR8r0tRPtkqSxt7UQU3qRBtOBPgfRQUAbMiyLI2Znalij1dXtKqrOy9vZjoSYARFBQBsaMa32fpq1zFFO8M1ZViqwsLY8kFooqgAgM0cOFmstEVbJEmjbmyv5vViDScCzKGoAICNWJalcXOydMpdpm7NL9G9V7YwHQkwiqICADYy+7sDWr7tqCIjwjR1eKrC2fJBiKOoAIBNHM4v0dMLNkqSRvZrp9b1axlOBJhHUQEAG7AsS+M/3KD8kjKlNq2t31zd0nQkwBYoKgBgAwsyD2rJ5sNyhjs0bXgXRYTz4xmQKCoAYFzuKbcmzNsgSXrkurZq3zDOcCLAPigqAGDYhPkbdaLIo+SGcfqfPq1NxwFshaICAAZ9suGgFmYeVHiYQy/c3kVOtnyAM/AvAgAMOVlUqj/O/eFZPg9e20opTWobTgTYD0UFAAx5esEm5Z5yq02DWvr99W1NxwFsiaICAAYs3XJYc9YdUJhDmjY8Va6IcNORAFuiqACAn+WXeDRuzg/P8rn/6pa6tNklhhMB9kVRAQA/m7xwsw7ll6hFvRiN7NfedBzA1igqAOBHK7cf1czV+yRJU4d3UXQkWz7AhVBUAMBPCt1lemJ2liTp3l7NdXnLuoYTAfZHUQEAP5n6yRYdOFmsJnWiNXpAsuk4QEAwWlRWrFihW265RY0bN5bD4dDcuXNNxgGAGvPNrmP6+1d7JUlThqUq1hVhOBEQGIwWlcLCQnXp0kWvvvqqyRgAUKOKS70aMztTkvTLHkm6um2C4URA4DBa6QcOHKiBAweajAAANe5PGVu151iRGsZHadygDqbjAAEloM49ut1uud3u8tv5+fmSJI/HI4/HYyqWrZyeA/PwD+btX4E473X7TurtL3ZLkp6+tYOiwwMnfyDOO5CF0rwr8xgdlmVZNZilwhwOhz788EMNHjz4vMdMnDhRkyZNOmt9xowZiomJqcF0AFB5Hp80LTNch4sd6p7g0z1tfaYjAbZQVFSkO++8U3l5eYqPj7/gsQFVVM51RiUpKUm5ubk/+0BDhcfjUUZGhvr16yen02k6TtBj3v4VaPP+U8Z2TV+xWwm1IvXxI1epToz9M/9UoM070IXSvPPz85WQkFChohJQWz8ul0sul+usdafTGfRf1MpiJv7FvP0rEOadtT9Pb36xR5L07ODOql87cM/6BsK8g0kozLsyj4/XUQGAalZa5tOo9PXy+iwNSm2kASkNTUcCApbRMyqnTp3Sjh07ym/v3r1b33//verWratmzZoZTAYAVTd92U5tOVSgS2KcmnRrJ9NxgIBmtKisWbNGffv2Lb89cuRISdK9996rd99911AqAKi6LYfy9ern2yVJE2/tpIRaZ29XA6g4o0WlT58+ssm1vABw0cq8Po2alSmP11K/jom6tUtj05GAgMc1KgBQTd5auVtZB/IUHxWhZwenyOFwmI4EBDyKCgBUgx1HTumlJdskSU/e3FGJ8VGGEwHBgaICABfJ67M0On29Sst8urZdfQ3v1tR0JCBoUFQA4CK9++UefZd9UrVcEZo8tDNbPkA1oqgAwEXYe6xQ0z7dIkkae1OymtSJNpwICC4UFQCoIp/P0uj0TJV4fOrVqp5+1YPXfwKqG0UFAKroX99m65vdxxXtDNeUYakKC2PLB6huFBUAqIL9J4r0/KLNkqRRN7ZXs3qB+14+gJ1RVACgkizL0tg5WSos9ap780t035UtTEcCghZFBQAqadaa/Vq5PVeuiDBNHc6WD1CTKCoAUAmH8kr0zMJNkqSR/dqpVf1ahhMBwY2iAgAVZFmWxn+YpYKSMnVpWlv3X93SdCQg6FFUAKCC5q/P0WdbjsgZ7tC027soIpwfoUBN418ZAFTA0QK3JszfKEn6/XVt1S4xznAiIDRQVACgAibM36CTRR51bBSvB/u0Nh0HCBkUFQD4GYuyDmpR1iFFhDk0dXiqnGz5AH7DvzYAuIAThaV6at4GSdL/9GmtlCa1DScCQgtFBQAuYNKCjco9Vaq2DWrp4evamI4DhByKCgCcx5JNhzX3+xyFOaRpt3eRKyLcdCQg5FBUAOAc8oo9Gj83S5L0wDWt1DWpjtlAQIiiqADAOUxeuFmH891qmRCr/+3XznQcIGRRVADgP6zYdlQfrNknh0OaOjxVUU62fABTKCoA8BOn3GUaO+eHLZ97e7VQjxZ1DScCQhtFBQB+YsrHW3TgZLGS6kZr9ID2puMAIY+iAgA/+nrXMb339V5J0vNDUxUTGWE4EQCKCgBIKi71aszsTEnSry5vpqvaJBhOBECiqACAJOmFxVu191iRGtWO0ribkk3HAfAjigqAkLd27wn9bdVuSdLkoZ0VF+U0nAjAaRQVACGtxOPV6PT1sixp2GVN1bd9A9ORAPwERQVASPvzZ9u182ih6se59OTNHUzHAfAfKCoAQlbW/jy9sWKXJOm5wSmqExNpOBGA/0RRARCSSst8GpW+Xl6fpVu6NFb/Tg1NRwJwDhQVACHptWU7tOVQgerFRmriLR1NxwFwHhQVACFn88F8vbp0hyRp0m2dVK+Wy3AiAOdDUQEQUsq8Po1Oz1SZz9KNnRI1qHMj05EAXABFBUBIeXPlLmUdyFPtaKeeuS1FDofDdCQAF0BRARAydhwp0MtLtkuSnrq5oxrERxlOBODnUFQAhASvz9Lo9EyVlvnUp319Db2sielIACqAogIgJLyzare+yz6pOFeE0oZ2ZssHCBAUFQBBb09uoV5YvFWSNG5QBzWqHW04EYCKoqgACGo+n6UxszNV4vHpqjb19MseSaYjAagEigqAoPavb/bqm93HFRMZrueHprLlAwQYigqAoLXveJHSPt4iSRozIFlJdWMMJwJQWRQVAEHJsiyN+zBLRaVe9Whxie65ornpSACqgKICICjNWrNfK7fnyhURpinDUhUWxpYPEIgoKgCCzqG8Ej2zcJMk6Q/926lV/VqGEwGoKooKgKBiWZbGf5ilgpIydUmqo/uvbmU6EoCLQFEBEFTmfZ+jz7YcUWR4mKYNT1U4Wz5AQKOoAAgaRwvcmrhgoyTp99e3UbvEOMOJAFwsigqAoPHUvA06WeRRx0bx+t21rU3HAVANKCoAgsKirIP6eMMhRYQ5NO32VDnD+fEGBAP+JQMIeMcLS/Xk3A2SpIf6tFanxrUNJwJQXSgqAALepAUbdaywVO0Sa2nEdW1MxwFQjSgqAAJaxqbDmvd9jsIc0tThXeSKCDcdCUA1oqgACFh5RR6N/zBLkvTANa3UNamO2UAAqh1FBUDAenbhJh0pcKtVQqz+t18703EA1ACKCoCAtHzbUc1au18OhzR1eKqinGz5AMGIogIg4BSUlGns7ExJ0r29Wqh7i7qGEwGoKRQVAAFn2uJtyskrUVLdaI0e0N50HAA1yHhRee2119SyZUtFRUWpW7duWrlypelIAGxse55D76/eL0maMjRVMZERhhMBqElGi8oHH3ygxx57TOPHj9e6det0zTXXaODAgcrOzjYZC4BNFZWW6f2dP/zYurNnM13ZJsFwIgA1zeh/Rf70pz/p/vvv129+8xtJ0ssvv6xPP/1U06dPV1pamrFcRaVlOl5YauzvvxhlZWU67pYOnCxWRITHdJygx7z9a/rnO3TM7VCj2lEaOzDZdBwAfmCsqJSWlmrt2rV64oknzljv37+/vvzyy3Pex+12y+12l9/Oz8+XJHk8Hnk81fdL4tOsg/rfWVnV9vn8L0KTvmMLzX+Yt79NHNROUeGq1n/3ONvp+TJn/wileVfmMRorKrm5ufJ6vUpMTDxjPTExUYcOHTrnfdLS0jRp0qSz1hcvXqyYmJhqy7b+mENOh/HLdwD8J4d0VaKlkt3fadFu02FCR0ZGhukIISUU5l1UVFThY41fheZwOM64bVnWWWunjR07ViNHjiy/nZ+fr6SkJPXv31/x8fHVlukmSeOr7bP5l8fjUUZGhvr16yen02k6TtBj3v7FvP2LeftXKM379I5IRRgrKgkJCQoPDz/r7MmRI0fOOstymsvlksvlOmvd6XQG/Re1spiJfzFv/2Le/sW8/SsU5l2Zx2dsfyMyMlLdunU76xRXRkaGrrzySkOpAACAnRjd+hk5cqTuuecede/eXb169dKbb76p7OxsPfjggyZjAQAAmzBaVO644w4dO3ZMTz/9tA4ePKiUlBQtWrRIzZs3NxkLAADYhPGLaR966CE99NBDpmMAAAAb4jm4AADAtigqAADAtigqAADAtigqAADAtigqAADAtigqAADAtigqAADAtigqAADAtigqAADAtoy/Mu3FsCxLUuXeLjrYeTweFRUVKT8/P+jffdMOmLd/MW//Yt7+FUrzPv17+/Tv8QsJ6KJSUFAgSUpKSjKcBAAAVFZBQYFq1659wWMcVkXqjE35fD7l5OQoLi5ODofDdBxbyM/PV1JSkvbt26f4+HjTcYIe8/Yv5u1fzNu/QmnelmWpoKBAjRs3VljYha9CCegzKmFhYWratKnpGLYUHx8f9N/odsK8/Yt5+xfz9q9QmffPnUk5jYtpAQCAbVFUAACAbVFUgozL5dKECRPkcrlMRwkJzNu/mLd/MW//Yt7nFtAX0wIAgODGGRUAAGBbFBUAAGBbFBUAAGBbFBUAAGBbFJUQ4Ha71bVrVzkcDn3//fem4wSlPXv26P7771fLli0VHR2t1q1ba8KECSotLTUdLWi89tpratmypaKiotStWzetXLnSdKSglJaWph49eiguLk4NGjTQ4MGDtXXrVtOxQkZaWpocDocee+wx01Fsg6ISAkaPHq3GjRubjhHUtmzZIp/PpzfeeEMbN27USy+9pNdff13jxo0zHS0ofPDBB3rsscc0fvx4rVu3Ttdcc40GDhyo7Oxs09GCzvLlyzVixAh9/fXXysjIUFlZmfr376/CwkLT0YLe6tWr9eabbyo1NdV0FFvh6clB7uOPP9bIkSM1e/ZsderUSevWrVPXrl1NxwoJ06ZN0/Tp07Vr1y7TUQJez549ddlll2n69Onlax06dNDgwYOVlpZmMFnwO3r0qBo0aKDly5erd+/epuMErVOnTumyyy7Ta6+9pmeffVZdu3bVyy+/bDqWLXBGJYgdPnxYDzzwgN577z3FxMSYjhNy8vLyVLduXdMxAl5paanWrl2r/v37n7Hev39/ffnll4ZShY68vDxJ4nu5ho0YMUKDBg3SDTfcYDqK7QT0mxLi/CzL0n333acHH3xQ3bt31549e0xHCik7d+7UX/7yF7344oumowS83Nxceb1eJSYmnrGemJioQ4cOGUoVGizL0siRI3X11VcrJSXFdJygNXPmTH333XdavXq16Si2xBmVADNx4kQ5HI4L/lmzZo3+8pe/KD8/X2PHjjUdOaBVdN4/lZOTowEDBuj222/Xb37zG0PJg4/D4TjjtmVZZ62hej388MPKzMzU+++/bzpK0Nq3b58effRR/fOf/1RUVJTpOLbENSoBJjc3V7m5uRc8pkWLFvrlL3+pBQsWnPGD3Ov1Kjw8XHfddZf+/ve/13TUoFDReZ/+AZOTk6O+ffuqZ8+eevfddxUWxv8FLlZpaaliYmI0a9YsDRkypHz90Ucf1ffff6/ly5cbTBe8HnnkEc2dO1crVqxQy5YtTccJWnPnztWQIUMUHh5evub1euVwOBQWFia3233Gx0IRRSVIZWdnKz8/v/x2Tk6ObrzxRqWnp6tnz55q2rSpwXTB6cCBA+rbt6+6deumf/7znyH/w6U69ezZU926ddNrr71WvtaxY0fddtttXExbzSzL0iOPPKIPP/xQy5YtU9u2bU1HCmoFBQXau3fvGWu//vWvlZycrDFjxrDlJq5RCVrNmjU743atWrUkSa1bt6ak1ICcnBz16dNHzZo10wsvvKCjR4+Wf6xhw4YGkwWHkSNH6p577lH37t3Vq1cvvfnmm8rOztaDDz5oOlrQGTFihGbMmKF58+YpLi6u/Dqg2rVrKzo62nC64BMXF3dWGYmNjVW9evUoKT+iqADVYPHixdqxY4d27NhxVhHkpOXFu+OOO3Ts2DE9/fTTOnjwoFJSUrRo0SI1b97cdLSgc/op4H369Dlj/Z133tF9993n/0AIeWz9AAAA2+JKPwAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQC2cfToUTVs2FCTJ08uX/vmm28UGRmpxYsXG0wGwBTe6weArSxatEiDBw/Wl19+qeTkZF166aUaNGiQXn75ZdPRABhAUQFgOyNGjNCSJUvUo0cPrV+/XqtXr1ZUVJTpWAAMoKgAsJ3i4mKlpKRo3759WrNmjVJTU01HAmAI16gAsJ1du3YpJydHPp9Pe/fuNR0HgEGcUQFgK6Wlpbr88svVtWtXJScn609/+pOysrKUmJhoOhoAAygqAGxl1KhRSk9P1/r161WrVi317dtXcXFx+uijj0xHA2AAWz8AbGPZsmV6+eWX9d577yk+Pl5hYWF677339MUXX2j69Omm4wEwgDMqAADAtjijAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbIuiAgAAbOv/AQ/bRugKbTnzAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.arange(-5,5,0.2), np.maximum(0,np.arange(-5,5,0.2)))\n",
    "plt.title('ReLU(x)')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('f(x)')\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "There are many other variations. See for example [PyTorch Non-linear Activations](https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Relating Back to _Another_ Earlier Lecture\n",
    "\n",
    "__Question__\n",
    "\n",
    "What does \n",
    "\n",
    "$$ \\mathrm{sigmoid}(\\sum_i w_i x_i + b) \\hspace{10pt} \\textrm{where} \\hspace{10pt} \\mathrm{sigmoid}(x) = \\frac{1}{1 + e^{-x}}$$\n",
    "\n",
    "remind you of?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Answer__\n",
    "\n",
    "How about the Logistic Regression model?\n",
    "\n",
    "$$\n",
    "\\begin{split}\n",
    "P(y=1\\mid x) & = \\frac{e^{\\alpha+\\beta x}}{1+e^{\\alpha+\\beta x}}\\\\\n",
    "             & = \\frac{e^{\\alpha+\\beta x}}{1+e^{\\alpha+\\beta x}} \\left( \\frac{e^{-(\\alpha+\\beta x)}}{e^{-(\\alpha+\\beta x)}} \\right) \\\\\n",
    "             & = \\frac{e^{(\\alpha+\\beta x)-(\\alpha+\\beta x)}}{e^{-(\\alpha+\\beta x)}+e^{(\\alpha+\\beta x)-(\\alpha+\\beta x)}} \\\\\n",
    "             & = \\frac{e^0}{e^{-(\\alpha+\\beta x)} + e^0} \\\\\n",
    "             & = \\frac{1}{1 + e^{-(\\alpha+\\beta x)}}\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "Which is the Sigmoid with $\\alpha = 0$ and $\\beta = 1$.\n",
    "\n",
    "In fact, just like in Logistic Regression, we use the sigmoid function on the last layer of a neural network that is\n",
    "doing binary classification to output the probabilities.\n",
    "\n",
    "> So Logistic Regression is similar to one neuron with a sigmoid activation function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "hide_input": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/IAAAItCAYAAACafUF2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAADke0lEQVR4nOzdd3hUZdrH8e9MMum9EAgk9E5oUsQKCljAyrIC6oq4ujYU665i13d1d+1tXdEFV7HhKqggFkAWpLfQA4QaAqmQXiYz5/1jMgOBBNInk/w+18U1k3PmnHMPkzL3PPdzPybDMAxERERERERExCOY3R2AiIiIiIiIiFSfEnkRERERERERD6JEXkRERERERMSDKJEXERERERER8SBK5EVEREREREQ8iBJ5EREREREREQ+iRF5ERERERETEgyiRFxEREREREfEgSuRFREREREREPIgSeRGRSvz666+YTCaGDx/e4NeaPHkyJpOJWbNmNfi1WpJnnnkGk8nEM8884+5QREREROqVEnkRcZsOHTo0+wR206ZNPPPMM8ydO9fdoWAymc76rynEWR379+/nmWeeadbfOyIiIiJV8XZ3ACIiTVFAQADdu3cnPj6+TufZtGkTzz77LLfccgvXXnttpY9p06YN3bt3JzQ0tE7Xqq4+ffpUea3IyMhGiaGu9u/fz7PPPsvFF1/M5MmTK31MVFQU3bt3JyoqqnGDExEREWlgSuRFRCoxZMgQdu7c2SjXevHFF3nxxRcb5VoAb731VqNMGXC3e++9l3vvvdfdYYiIiIjUO5XWi4iIiIiIiHgQJfIi4lEOHjzIXXfdRceOHfH19SUqKoorrriCH374ocpjDMPgvffeo1+/fvj7+xMTE8OkSZPYu3cvs2bNwmQynVaefaZmd1u3buXGG28kLi4OHx8fwsLC6Nq1K5MmTWLhwoWux3Xo0IFbb70VgI8++qjCXPSTz3u2ZndJSUnccccddOnSBX9/fyIjIznnnHN4+umnOXLkSLX/72pi+PDhmEwmfv3110r3VxXzydtTU1OZMmUKbdq0wc/Pj969e/POO++c8bpr167lpptuIj4+Hl9fX2JiYjjvvPP4+9//Tk5Ojiu2ESNGALB06dIK/68dOnRwnetsze5WrFjB9ddfT0xMDD4+PrRr144//OEP7Nix46z/Jzt37mT8+PFERUXh7+/POeecw5dfflnpcQUFBTz33HP07duXwMBA/Pz8iIuLY/jw4bz00ktYrdYz/p+IiIiInEql9SLiMVavXs3ll1/O8ePHCQwMJCEhgbS0NBYuXMjChQt58sknee6550477o9//CP//ve/AejYsSMRERF88803LFy4kHvuuadGMaxZs4bhw4dTVFREaGgovXr1wmazcejQIT777DMKCwu5/PLLARg8eDA+Pj7s3r2bVq1a0bVrV9d5EhISqnW92bNnM2XKFEpLS/H396dXr14UFhaybds2NmzYQMeOHaucI+5OBw4c4JxzzuH48eP06tULs9nM9u3buffeezl+/DjTp08/7Zi///3v/OUvf8EwDEJCQkhISOD48eOsW7eOlStXMmTIEIYPH05CQgJZWVls3brV9TinNm3aVCu+f/7zn9xzzz0YhkGrVq3o168fe/bs4eOPP2bOnDl89dVXjBkzptJj169f7/qQoFu3bhw8eJANGzZwww03UFpayk033eR6bFlZGSNHjmTVqlWYzWa6du1KcHAwqampLFu2jKVLl3LnnXcSFhZWs/9gERERadkMERE3ad++vQEYM2fOPOtjCwoKjPj4eAMwfv/73xu5ubmufbNmzTK8vLwMwFiwYEGF4+bMmWMAhq+vrzF37lzX9qysLGP06NGGxWIxAOOWW26pcNySJUsMwLj44osrbB87dqwBGI8//rhRUlJSYd/atWuN2bNnV9g2c+bMSs9/sltuuaXS/4e1a9e64nv00UeN/Px8177S0lLjs88+M5YtW1bleU8FGICxZMmSsz724osvPuNjq4rZud1isRi/+93vjGPHjrn2vfvuuwZg+Pn5VdhuGIYxd+5cAzC8vLyMV155xSgtLXXtKygoMN5//31j+/btrm1VvT4ne/rppw3AePrppyts37hxo+Ht7W0Axt///nfDZrMZhmEYxcXFxt13320ARmhoqJGamlrp/4nFYjHuvfdeo6ioyDAMw7Db7caf//xnAzBiY2ONsrIy1zFfffWVARj9+vUzDh06VOF86enpxuuvv24UFBRU+RxEREREKqPSehHxCJ9++ikHDx4kJiaGjz76iODgYNe+W265hT/96U8ApzWNe+WVVwB48sknueaaa1zbIyIi+PzzzwkMDKxRHLt37wbgz3/+Mz4+PhX2DRo0iEmTJtXofGfy9NNPY7VamTJlCn/7298qxGqxWJgwYQIXXHBBjc87YsSISpeeq8+R/cjISGbNmlVhpPmuu+5i4MCBFBcXs2TJkgqPf/zxxwHHc37wwQexWCyufQEBAdx+++307NmzXmJ7+eWXKSsr45prruGRRx7BbHb8KfT19eXtt9+md+/e5OTk8M9//rPS43v16sUbb7yBn58f4FjW7/nnn6d169akpqayefNm12Od3y9TpkyhXbt2Fc4THR3N/fffT0BAQL08LxEREWk5lMiLiEf46aefALj99ttdCdTJ7r//fsAx77mgoACAvLw8Vq9eDeCaq36y8PDwKpeEq0pcXBxAlfOh60tRURE///wzAI8++mi9nrtPnz6cf/75p/3r1q1bvV1j4sSJlX5IMnjwYAD27t3r2rZnzx62b9+Oj48P06ZNq7cYquL8Xpo6depp+0wmE/fdd1+Fx51qypQpruTfyWKx0K9fP6Dic3N+v8yfP5/CwsK6By8iIiKC5siLiIfYtWsX4BgNrUzXrl3x8fGhtLSU5ORk+vbty549ezAMg8jISGJjYys9rm/fvjWKY9q0afzyyy/cfvvtvPLKK1x22WVccMEFjBgxol7XYN+zZw9Wq5WwsDC6d+9eb+eFxll+rnPnzpVub9WqFQD5+fmubc7mcr169apQadEQjh8/TkZGhut6lenduzdw4nvuVDV5btdeey0dOnTgp59+IjY2lssvv5wLL7yQ4cOHu64jIiIiUlMakRcRj+BMjpzJ0qlMJhPR0dGAYyQecI3Mnyk5rGniOGbMGObPn895553Hrl27eOONNxg/fjytW7fm97//PYcPH67R+aqSm5sL4LFN0KqasuAcyTYMw7WtMZ/ryUl2Vd9LMTExwInvo1PV5LkFBgaybNkybr31Vux2O1988QX33nsvffr0oXfv3nz//fe1eh4iIiLSsimRFxGPEBQUBEB6enql+w3DcI20OpNzZ8J1cvJ2qqqStTO58sor+e2338jIyGDu3LlMnTqVsLAw5syZw1VXXVUvy4k5n8Px48frfK7aMJlMQMWk9GTOD0nqQ2M+V+f3EVT9vZSWllYhrrpq164d//73v8nOzmbVqlW89NJLDBo0iO3bt3Pttde6pn+IiIiIVJcSeRHxCM7529u3b690/+7duyktLcXLy8tV+tylSxfMZjOZmZlVrre+ZcuWWscUERHBNddcw5tvvsnWrVsJDQ1l48aNrFu3zvUYZ0JcU86pAsePHycpKanWMdaW80MQ54cjp9qzZ0+9XctZYr59+/Zqf7BS2//XsLAwV+VGVd9L27ZtA6jXngEA3t7eDB06lD//+c+sXbuWCRMmYLPZXEsjioiIiFSXEnkR8QiXXXYZADNmzKC4uPi0/W+++SYA559/visJDQ4OZujQoQDMmjXrtGNycnKYO3duvcQXExNDx44dAUhNTXVt9/f3BxzN62rC39+f0aNHA44u642tU6dOAKxdu/a0fevWrSMxMbHertW5c2f69OlDaWmp63U8m9r+v8KJ76W33nrrtH2GYbi2Ox/XUM4991yg4veLiIiISHUokRcRjzBx4kTi4+NJS0tj8uTJFcrlP/nkE/71r38B8Je//KXCcQ8++CAAzz//PN99951r+7Fjx5gwYcIZy+4rM2HCBObPn09paWmF7V999RVbtmzBZDIxYMAA1/aTE+Kadi1/+umnsVgsfPDBBzz++OMVjrdarXzxxRcsX768RuesriuuuAJwfHCyZs0a1/bdu3dzyy234O1dv71SX3jhBQCeeeYZ3nzzzQrTEwoLC/nggw9cTfEA14cm27dvr7JqoCoPPfQQ3t7ezJs3j1deeQW73Q5AaWkp999/v6u64q677qrr0+K1117j9ddfd5XrOx08eJAPPvgAgIEDB9b5OiIiItKyKJEXEbebOnUqUVFRVf7bunUrAQEBfPnll4SGhvLFF1/QunVrBg8eTHx8PDfffDNlZWU88cQTrgTU6Xe/+x1TpkyhqKiIq6++ms6dOzN48GDatm3L6tWrXUu7eXl5VSvWhQsXMnbsWEJCQkhISGDIkCHExsYyfvx4bDYbTzzxhCt5B0eS1rVrV/bt20d8fDznnXcew4cPr9Yya4MGDeLf//43FouFF198kejoaM455xx69epFSEgIEyZMqNcS95NdfvnljBw5kpycHIYNG0aPHj1ISEigR48eREVFMX78+Hq93jXXXMOLL76IzWbj/vvvJzo6msGDB9OtWzfCwsK4/fbbKyTD0dHRXHLJJeTn59O5c2fOPfdchg8fzoQJE856rf79+/Pmm29iMpl4+OGHiY2NZciQIcTExPDWW2/h6+vL7Nmzad26dZ2f14EDB3jggQdo3bo1HTt2ZOjQofTs2ZNOnTqxdetW+vTp4/qwSURERKS6lMiLiNvl5+eTlZVV5b+ysjIAhg4dSmJiIn/605+Iiopi8+bN5OfnM3r0aObPn8/zzz9f6flnzJjBu+++S0JCAocPH+bAgQNcddVVrFmzhjZt2gDVb2z20Ucfcccdd9C1a1dSU1PZvHkzAQEBXHfddSxdupTnnnuuwuPNZjPz58/nd7/7HV5eXqxZs4alS5eyadOmal3vpptuYtOmTdx6662uDzUyMjLo3bs3zzzzDJdffnm1zlNTJpOJb775hgcffJDY2Fj27dtHQUEBjz32GD/99BMWi6Xer/mXv/yFFStW8Pvf/56AgAASExPJzc1l8ODB/OMf/zht5PrTTz9l8uTJhISEsH79epYuXcqqVauqda277rqLZcuWce2112K329m0aRMBAQHcdNNNbNiwgTFjxtTLc7rzzjt55plnuOiii7BarWzatIljx44xePBg3nrrLdasWUNoaGi9XEtERERaDpNRVUtiEZEWYOrUqbz99tu89tpr1RolFxERERFxN43Ii0iLlZ+fz1dffQU4muSJiIiIiHgCJfIi0uy9/vrrp5WyHz58mHHjxnH06FEGDhzI4MGD3ROciIiIiEgNqbReRJq94cOHs3TpUkJDQ+nUqRMlJSXs3LkTu91OVFQUS5YsoU+fPu4OU0RERESkWup3/SARkSbo/vvvJzw8nA0bNrBz504Mw6Bz585cccUVPProo7Rt29bdIYqIiIiIVJtG5EVEREREREQ8iObIi4iIiIiIiHgQldZXk91uJzU1leDgYEwmk7vDEREREZFTGIZBXl4esbGxmM0arxKR5kuJfDWlpqYSFxfn7jBERERE5CwOHTpEu3bt3B2GiEiDUSJfTcHBwYDjD0NISIiboxERERGRU+Xm5hIXF+d63yYi0lwpka8mZzl9SEiIEnkRERGRJkzTIEWkudPkIREREREREREPokReRERERERExIMokRcRERERERHxIErkRURERERERDyIEnkRERERERERD6JEXkRERERERMSDKJEXERERERER8SBK5EVEREREREQ8iBJ5EREREREREQ+iRF5ERERERETEgyiRFxEREREREfEgSuRFREREREREPEiTT+Tz8vJ49NFHGT16NNHR0ZhMJp555plqH5+ens7kyZOJiooiICCAYcOGsWjRooYLWERERERERKQBNflEPisri/fff5+SkhKuvfbaGh1bUlLCpZdeyqJFi3jjjTeYN28eMTExXH755SxdurRhAhYRERERERFpQN7uDuBs2rdvz7FjxzCZTGRmZvLBBx9U+9gPP/yQrVu3smLFCoYNGwbAiBEj6NevH48++iirV69uqLBFREREREREGkSTH5E3mUyYTKZaHfvNN9/QvXt3VxIP4O3tzU033cSaNWs4fPhwfYUpIiIiIiIi0iia/Ih8XWzdupULL7zwtO19+/YFYNu2bbRt27axwxKRJspuN7Db7Nhtxin/TtpmNzAMAwwwDAPD7rgFMOwGhnO7ARgnvsbxEOeN63hO+rri/pP2nf7l6c62v9JDanGQSDNl2A2w26DMhmG3YdhsYLODzfm1HXtZGdjt5b8H7I6fb7sdw25gNwywO7bZ7XYw7I7fBTbHT5phtzuu47xW+U+g6/fAKb8vjJN/Qg04+SfWcfwp8RunPv60u6f8/rE7ro8dDAO746LYsTu2Gs4YjfJzl//u46TYy+M+OTrDOHm/Y1t4fDRX3Hh1DV4NERE5m2adyGdlZREREXHadue2rKysKo8tKSmhpKTE9XVubm79BygidWYYBoW5peRkFFGcZ6W0pAxrsY3SYuetDWtxGaUl5bfFNqwlJ/bbyuyuBF15rYg0nCZfBNlg8pIS4UZ3RyEi0rw060QeOGNZ/pn2vfjiizz77LMNEZKI1JDdZif/WAk56UXkZBaRk15ITkYRuZlF5GQUUVZqb7Brm80mzF4mTF6OW7PZhMlsKp/2A5hw3TeZHPsAx9fm8sdgKn/cifOe/Punwq8i53krcfp201n2i9QvA7DZDWx2gzK7vfw+2Jz3DcO1/+R/dgPshmOU11ZeuWI/a5nJCd52Gz52Kz62MnxsVnzsJ269jIb7+XeMKjt+sAyTqcLXzhvjpJ9Dw7nD5PjKADAZFfZS1TbnV83g5/i0p+CT6Y4wRESatWadyEdGRlY66p6dnQ1Q6Wi902OPPcaDDz7o+jo3N5e4uLj6D1JEKrCW2NiXmMHR5JzypL2IvKxix4h5FUwmCI70IyDEB4ufNz5+Xo5bXy8sfl74+Hlj8fVybbf4eeHj67j1tpgxlSfrXl7m05N2ZcfSzBWWlpGVX0pmfgmZ+aVk5Ze47mfml5CVX0pWgePrY4WlZ5/m4WSmWoPQJhP4W7wI8PEiwAv6p+2i19FdROemE5WTQXhOBpay0jOeoyQ0gpLwKOz+ARj+Adj9A8A/ACMgAAICMAUEQEAg5sBATAGBmIMC8QoMxOznh9nijZe3BbOPN2aLBS+LpXybN97eXpjNJrxMJsxm8DKZwGQnqziT9MIjpBUeJb3wKEcLjzj+FRzhSMERisqKqvmfdNJ/l8lMsE8wIT4hBFmC8Pf2x8/bD18vX/y8/E7cP+nWz8sPX2/Hfh8vHyxmC95mb7zN3q77rm0mb9e+kx9jKivBqzAbc1E2XgXZmAuzMBdkYC7MgoIMyE+HgkwoSIfCLKjNByd9xtX8GBEROaNmncgnJCSwZcuW07Y7t/Xp06fKY319ffH19W2w2ETkBMNukLrnODtXHSV5fTrWEttpjzF7mwiN8ic02p/Q6ABCov0JbeVPaJQ/wZF+eHm33LJVkaoYhsHxQiuHjhVyMLuQQ9lFHDpWyKFsx7/0vBIKS0//eTsbP4uZYD8LwX7eBPtZCPHzJtjPm5CTtjlvg3y9CfT1wt/ihV95wu7vc+JrX28zJbt3kzN3HjnffYsto5LRW7MZS9u2+MTFYWkfj09cPD7t47HExeETF4fZ378e/rdOV2orZWf2TjZnbGZz5ma2ZW4jNT+VMqPsrMdG+kUSGxRL68DWhPuGE+IbQohP+b9K7gdaAjGbGuD3WNFxOH4Aju2CY/vL7x9w3M87CqV5NT+nly/4hYBvCPiFnnQ/BHzLv/YLPbEtvEP9PicREWneifx1113H3XffzerVqxk6dCgAZWVlfPLJJwwdOpTY2Fg3RyjSsh1PLyRp9VGSVh0lL6vYtT0kyo9O/aMJbx3oSNij/QkK83WVrYvICYZhcDC7kOSMfEeinl2etB9z3M8vOXvS6eNtJjrIl6ggHyIr3JbfD/QlKthxG+pvwacePjgry84m96vvOTJ3HsXbt7u2e4WFETx6NL7du+ETH49PfDyW2FhMFkudr3kmhmGQkp/ClowtbM7czJaMLezI3oHVbj3tsd4mb2ICY4gNiqVNYBvaBLZx3Xcm775ejTQYYBiQkwKZSY7k3JmkOxP24uNnP4fZAkGtIDAKAltBYDQERTtuA8u3B5Vv9w8Hbw10iIi4m0ck8j/88AMFBQXk5Tk+Nd6+fTtfffUVAFdeeSUBAQHcdtttfPTRRyQnJ9O+fXsApkyZwjvvvMP48eN56aWXaNWqFe+++y5JSUn88ssvbns+Ii1ZSVEZe9alkbTqKEeSc1zbLX5edDmnFT3ObUObzqFK2kUqYbXZ2ZOez7bUXLal5rAtNZcdqbnknSVZbxXsS1xEAHHh/o7biADiwgNoE+pHZJAPQb7ejTKNxF5aSv6vv5Izdx75//sflJXHbbEQdPFFhF17LUEXXYTJx6fBYymwFrA5YzNbMre4brOLs097XLhvOH2j+5IQlUBCdAKdQjsR7R+Nl9mrwWM8jd0GWXvgyGY4mlh+uxmKjp35uIAox6h4eHsIa++4De8AIW0dyblfqJpsiIh4GI9I5O+66y4OHDjg+nrOnDnMmTMHgH379tGhQwdsNhs2m63Cck6+vr4sWrSIRx99lKlTp1JYWEj//v354YcfuPjiixv9eYi0VHabnUM7j5G08gh7EzOxWR1zLE0miOsZQfdhrenYLxqLjxveGIs0UUWlNnYczWVbai7by5P2nUfzKC07fY6yj7eZztFBxEf4ExceQHykI1GPi/CnXXgAfhb3/WwZhkHxli3kzJ1H7vz52HJOfIDn16cPoddeS8iYK/EOD2/wWOyGnXVH1/H1nq/55cAvlNhKKuz3NnvTM6KnK3HvG92XdkHt3NMro6wE0refSNaPJELaNrAWnv5YszdEdXMk5ycn6mHtISwefIMaO3oREWlgJsOoQdvYFiw3N5fQ0FBycnIICQlxdzgiHmPHiiOsmpdMYc6JZlURsYF0P7c13Ye0JjBMJZoiAHnFVlYkZ7F0VwZr9mWzNyOfyno8Bvt60zM2hD6xofSODaF32xA6Rwdh8Wp6fSIK167lyLPPUron2bXNu1UrQq+5mtBrrsG3S5dGiSOtII1vk7/l691fk5Kf4toeGxhLv+h+jsQ9OoEeET0aryT+VKWFcOA32PML7P8NMnaAvZJKC0sAtE6A1n2hTV/HbaueKncvp/drItJSeMSIvIh4pk2/HOS3r/YA4BdooeuQGHqc25ro+GB1g5cWzzAMdhzJY+muDH5NSmf9gWOUnZK5Rwf7OpL12BB6lyfuceEBmD1g6kn+suWk3HsvRkkJJj8/gkeNIvSaawgcdi4mr4avELDarfzv0P/4es/XLD+8HHt5t/VASyBXdLyC67tcT5+oPu77XWQYkLnbkbjv+cWRxJcVV3yMf8SJZL1NP8dtZGdwR1m/iIg0KUrkRaRBrPthP6vn7QVg4GXxDLmqkzrLS4t3vLCU5XsyWZqUwdJdGaTnVSzt7hgVyMXdormgSxR924XSKsTPTZHWTd7ixRy+fxqG1UrQ8OHEvvwPvIIap7x7X84+vtn9DfOS51WY8z6w1UCu73o9o9qPIsAS0CixnKYkD/b970Tyfvxgxf0h7aDrSOh8CbQ9xzGHXR96iohIJZTIi0i9MgyDNd/tY92C/QAMuaojg67soBF4aZEMw2BzSo5r1H3ToeMVyuX9LV6c1zmS4d2juahbNO0jA90XbD3JXbiQww8/AmVlBI8eTduX/9HgzesKrYX8dOAnvt79NRvTN7q2R/pFcnWXq7muy3V0DO3YoDFUyjAc89x3/+xI3A+ugpO74Hv5QPvzoctI6DrKMc9dvytFRKQalMiLSL0xDIMV/93Dpl8OATDs+s4MHN3ezVGJNL6cQitfbUhh9uoD7M0oqLCvW0wQF3eLZnj3VgzqEI6vd/Mpk86ZN4/Uxx4Hu52Qq64i9sW/YvJu2Lcavx3+jadWPEV6YToAZpOZi9pexHVdr+PCdhdiMTfssnWVKi2ALXNg7QdwdEvFfRGdHIl7l5HQ4QLw8fwPb0REpPEpkReRemHYDZZ9sYstSw8DcOEN3eg7op2boxJpXFtScvh41X6+TUyluHx1hgAfLy7qGs3F5aPubcP83Rxlwzj25ZccffoZMAxCfzeONs8+26Bz4Quthbyy7hW+3PUl4GhcN777eK7ufDWtAlo12HXPKCMJ1n4IiZ9BSa5jm5cvdBpenrxf6pjjLiIiUkdK5EWkzux2g19n72THb0fABCNu7EGvC2LdHZZIoyi22vguMZVPVh8k8dBx1/YerYO56dz2XDugLUG+zfvPbfYns0l74QUAwidNIuaJ6ZjMDdcTY0PaBp747QkO5Tmqfyb2mMi0gdPcM/e9rBR2fg/r/g37l53YHtEJBk2B/jdCQETjxyUiIs1a835nISINzm6zs+ijHexak4bJBJdO7kX3oa3dHZZIg9uXWcDsVQeYsz6FnCLHvGeLl4krE9pw07ntGdQ+vEX0hsj68EPS//EyABFTptDqkYcb7HmX2Ep4Z+M7zNo2CwOD1oGtef785zm3zbkNcr0zykmB9bNgw38gP82xzWSG7lc6EvhOI6ABP8wQEZGWTYm8iNSarczOzx9uI3ljBmaziVG39abLOW4qaRVpBGU2O7/sSGf26gMs253p2t42zJ8bz43n94PiiApqGet5G4ZB5rvvkvnW2wBE3X0XUVOnNlgSvz1rO9OXT2fPcceSltd0voY/D/kzwT7BDXK9StntsHeJo3x+1w9QvqQdQTEw8BY45xYI1ZQiERFpeErkRaRWyqw2fnx/K/u3ZGH2NnH57X3o2C/a3WGJNAjDMPhm42Fe/jGJ1BzHWt8mEwzvFs3Nw9pzcbdWeHnA2u71xTAMMl59jawZMwCInjaNqDv/1CDXKrOX8cGWD/hX4r8oM8qI8IvgmWHPMCJ+RINcr1KGAUkL4OenIGvPie0dLoTBt0GPseDlhqZ6IiLSYimRF5Eas5ba+OGfmzm04xheFjNX3plAfO9Id4cl0iC2pebw9LxtrDtwDICIQB9uGBzHpCHxxEW4aT1yNzIMg7QXX+TYfz4GoNVf/kzk5MkNcq29x/cyffl0tmZtBWBU+1E8ee6ThPuFN8j1KpWVDD/8Gfb87PjaNwT6TXSUz7fq0XhxiIiInESJvIjUSGlxGfPf2Uzq7uN4+3ox9u6+tO3eiG+qRRrJ8cJSXvlpF7NXH8BuOLrPT72kK7ee3wE/S/NZMq4mDLudo88+x/EvvgCg9dNPET5xYr1fx27Y+WT7J7y58U1KbCUE+wQzfeh0rux4ZeP1HSgtgGWvwIq3wFYKZgucNxUufAh8gxonBhERkSookReRaispKuP7tzZxdG8uPn5ejJ3anzadQ90dlki9stsNvlx3iL//mER2QSkAY/u2YfqYnrQJbZ5Lx1WHYbNx5IknyfnmGzCZaPPCC4SNu77er3Mk/wiPL3+cdWnrADg/9nyePe9ZYgJj6v1alTIM2D4XfpwOuY7lNOkyEi7/G0R1aZwYREREzkKJvIhU2+q5yRzdm4tvgDdX39+fVu1D3B2SSL1KPHScp+ZtJTElB4CurYJ49prenNc5ys2RuV/O3LmOJN7Li9i//Y3QsWPq/RrZxdnc+uOtHM4/jL+3Pw8Pepjx3cY33ih8RhIseAT2LXV8HRYPl7/k6ETfAlYgEBERz6FEXkSqpaSojB2rjgIw+o+9lcRLs5JdUMrfF+7ki3WHMAwI8vVm2siu3HJeByxeWkLMMAyyy+fER993X4Mk8SW2Eu5ffD+H8w/TLqgd7496n7iQuHq/TuUXz4Olf4NV/wR7GXj5wgUPwAXTwNJyqzBERKTpUiIvItWyc8URykpsRMQGEtczwt3hiNQLm93g09UHePmnXa614K8f0Ja/XNmDVsF+bo6u6Shav56SpCRMfn6E3/D7ej+/YRg8+duTbMrYRLBPMO+MfKdxknjDgC1z4KcnId/xQSXdr4TL/goRHRv++iIiIrWkRF5EzsqwG2z5NQWAhOHtGq/MVaQBrT+QzZNzt7H9SC4APduE8Nw1vRncQR9UnSr7k9kAhF51FV5hYfV+/ncT3+WHfT/gbfLmteGv0Sm0U71f4zRp22D+w3BwhePriE6OefDdRjf8tUVEROpIibyInNXB7dnkZBTh4+9NtyGN1HBKpIEYhsGHy/fxfwt2YBgQ4ufNI5d1Z9LQ9i1qLfjqsh49St7PjqXXwm+6sd7P/13yd7yX+B4ATw17iqFthtb7NU6T+Dl8O9XRjd7bHy562NGR3tu34a8tIiJSD5TIi8hZbV7iGI3veV4bfPz0a0M8l9Vm56l52/hszUHAUUY/fUxPIoOUwFXl2Oefg81GwKBB+HXvXq/nXp+2nqdXPA3AbX1u47qu19Xr+U9jt8OS/4NlLzu+7nY5XPkyhDXSXHwREZF6onfkInJGx9MKObgtC0yQMLytu8MRqbWcQit3f7qe3/ZkYTLBE2N6MeX8Dpoqcgb20lKOfzkHgPCbbqrXcx/MPcj9S+7Harcyqv0o7ht4X72e/zTWIvjmTsfScgAXPAiXPAlmNTMUERHPo0ReRM5o61LHOsrt+0QSGh3g5mhEaudAVgFTZq0lOaOAAB8v3po4gEt7aprI2eT98AO27Gy8W7cmeOSl9XbenJIc7ll0DzklOSREJfB/F/wfZlMDJtR5afD5RDi8HswWuOoNGFD/0wREREQaixJ5EalSaXEZO1akAo4mdyKeaM2+bP708TqOFVppE+rHh7cMpleslk+sDmeTu/AJEzB5189bBqvNyrQl09ifu582gW1485I38fduwCXejm6FzyZAziHwD4cbPoEOFzTc9URERBqBEnkRqdKu1UcpLbYR2sqfeC05Jx7ov+tT+MvXm7HaDPq2C+WDPwyiVYiWlauOosREirdswWSxEDb+d/VyTsMweGblM6xLW0egJZB3Ln2HKP+oejl3pXb9BF/dCqX5ENkFJn0JkZ0b7noiIiKNRIm8iFTKMAw2/+ooq08Y3g6TunmLB7HbDV79eRdvL9kDwBV9WvPq7/vj7+Pl5sg8h3M0PuTKK/GOjKyXc36w5QO+Tf4WL5MXr1z8Cl3Du9bLeU9jGLD6X/DjY2DYocOF8Pv/QIA+kBQRkeZBibyIVCol6RjHjhRg8fWi57A27g5HpNqKrTYe+jKR+VuOAHD38M48PLo7Zn0YVW1lmZnkLlwI1F+Tu4X7F/LmxjcBeGzIY5zf9vx6Oe9pbGWw8M+w9gPH1wNuhjGvgrdPw1xPRETEDZTIi0iltpQvOdf93Nb4+OtXhXiG9Lxibv/PehIPHcfiZeL/rkvg94O0tFhNHfvyS7Ba8e/XD/+EPnU+X2JGItOXTQfgpp43cUOPG+p8zkoV58CcyZC8GDDBqGfhvPtAKxOIiEgzo3fnInKa3Mwi9m/OBNTkTjzHjiO5/PGjdRw+XkRYgIX3bjqHczvVT0l4S2JYrRz//AugfkbjU/JSuG/xfZTaSxnebjgPD3q4zues1LH98OkNkLETLAFw/QzoObZhriUiIuJmSuRF5DRb/3cYw4B2PcKJaBPo7nBEzmrJznTu/XQDBaU2OkYF8u/Jg+kYpe/d2sj7+WfK0tPxiooi5LLRdTtXaR73LrqX7OJsekT04G8X/Q0vcwP0KTi0Bj6bCIWZENwGJn4Osf3r/zoiIiJNhBJ5EamgrNTG9t8cS871HaHReGn6Vu3N4o6P12G1GZzbKYL3bjqHsADNh66t7NmfAhD++99j8qnb/+Nr618jOSeZVv6teOuStwiwBNRHiBWlbYOPr3N0pm/dFyZ9ASGx9X8dERGRJkSJvIhUsGttGiUFZQRH+tE+oQGXhRKpB/syC7jzk/VYbQaX927NmxMH4ONtdndYHqt4xw6K1q8Hb2/CbqjbPPZDuYf4Zvc3ALx00Uu0DmxdHyFWlJfmKKcvzYf2FziSeN+g+r+OiIhIE6N3OyLiYhgGm8ub3PW5uK26fEuTdrywlNtmreV4oZV+cWG8PqG/kvg6yv7kEwBCRo/GEtOqTud6N/Fdyowyzm97PoNbD66P8CqyFsHnkyDnEER0hhs+VhIvIiItht7xiIjLkeQcslLy8baY6XW+SlOl6bLa7Nz1yQb2ZhYQG+rHjD+cg59Fa8TXRdmxY+R+Px+oe5O7Pcf2MH+v41xTB0ytc2ynsdth7l1weB34hcGNc7RGvIiItChK5EXExbnkXLchMfgFWtwcjUjlDMPgyblbWbk3i0AfLz6cPJhWwX7uDstjGIZR6fbjX32FUVKCb6+e+A/oX6drvLPpHQwMRrUfRe/I3nU6V6V+fRG2fQNmC9zwCUR2rv9riIiINGFK5EUEgPxjJSRvzAAgQU3upAmbsWwvn689hNkEb00aQM82Ie4OyeMZNhvHPvsMgIgbb8JUh3XXt2Vu45eDv2DCxD3976mvEE9I/AL+93fH/ateh44X1v81REREmjgl8iICwLZlhzHsBm26hBLVLtjd4YhU6sdtR3nxh50APDGmF5f0iHFzRJ6nsiQ9f8kSylKP4BUWRsiYK+t0/rc2vgXA2E5j6RxWzyPlB1bCt/c67p8/DQbUfZ17ERERT6REXkSwWe1sW3YYgL4j4twcjUjlth7OYdrnmzAMuOnceG49v4O7Q2o2sj+ZDUDY+PGY/Wo/TWF92np+S/0Nb5M3d/W/q77Cc8jeB1/cCLZS6DEWLn26fs8vIiLiQZTIiwh7NqRTlGclMMyXjv215Jw0PUdzivnjR+sostq4sGsUT1/Vu07l33JCye7dFK5aBWYz4RMn1Po8hmHw5oY3Abiu63XEBdfjh4JFxx3LzBVmQZt+cP37YNZbGBERabn0V1BETiw5d1FbvLz0a0GalsLSMv74n7UczS2mS6sg3p40EIu+T2vt1GZ32Z9+CkDwpZdgia39ahUrUlewIX0DPmYf7uh7R51irMBmhTmTITMJgmNh4hfgE1h/5xcREfFAeick0sKl7cslfX8uZm8TvS7QknPStNjtBg98sYmth3OJCPTh37cMJtRfKyrUF1teHjnzvgUg/Mbazzc3DMM1N/6GHjfQOrB1vcSHYcAPj8LeJWAJgEmfQ0ib+jm3iIiIB1MiL9LCbf71EABdz4khIMTHzdGIVPT3H5P4cVsaPl5m3r/5HOIjA9wdksc7eUpCztdfYxQW4tu1KwFDh9T6nIsPLmZb1jb8vf25rc9t9RGmw+r3YN2/AROM+8BRVi8iIiJK5EVassLcUvasSwe05Jw0PV+uO8R7S5MB+Pvv+jKoQ4SbI2peDLvdVVYffuONte45YLPbeHvT2wDc1PMmIv0j6yfAXT/Cj4877o96DnqMqZ/zioiINANK5EVasO3LD2O3GcR0DCGmg9bilqZjZXIWj3+9BYD7LunCtQPaujmi5qdg2TKsBw5iDg4m9Oqran2eH/b/wJ7jewj2CWZyn8n1E9zRrfDVFDDsMPAPcN7U+jmviIhIM6FEXqSFstnsbF3qWHIuYbhG46Xp2JdZwF2z11NmNxjbtw0PjOrm7pCajZMb3WXPLl9y7vrrMQfUbsqC1W7l3U3vAnBr71sJ8amHDwTz0hwd6kvzoeNFMOZV0AoFIiIiFSiRF2mhDm7NoiCnFP9gC10GtnJ3OCIAWG127vpkPccLrfSPC+Pl8f20zFwDsB45QsH/loHJRPiNk2p9nnl75nEo7xARfhHc2PPGugdmGPDf2yA3BSK7wO//A15qbigiInIqJfIiLdSR5BwAOvaPxsuiXwXSNMxYtpedR/OICPTh/T+cg5/Fy90hNSvOD0WKNm0CwK93b3zi42t1rhJbCe8lvgfAHxP+SIClHhoRJn4G+5eBt79jmTn/8LqfU0REpBnSu3eRFir9QB4AMe01N16ahoNZhbzxy24AnhjTk1bBfm6OqPkq2roVAL+EPrU+x5dJX5JWmEZMQAy/7/77ugdVkAU/TnfcH/4XiOpS93OKiIg0U0rkRVogw26QcdCRyEe3D3ZzNCKOudtPzNtKSZmd8zpHcp2a2zWo4q3bAPDvU7tEvtBayAdbPgDgT/3+hK+Xb92D+vlJKMqGVr1h2D11P5+IiEgzpkRepAXKySiitKgML28zEbGB7g5HhO82H+F/uzLw8TbzwrV9NC++ATgb3Rl2O8XbHIm8Xy0T+dk7ZpNdnE1ccBzXdrm27sHtWwabZgMmuOp1zYsXERE5CyXyIi1Q+sFcAKLigvDy0q8Bca+cQivPfbcdgHtHdKFTdJCbI2reSg8cwJ6fj8nXF9/OnWt8fG5pLjO3zQTg7v53YzHXMekuK4HvpznuD5oCcUPqdj4REZEWQO/gRVog5/z4VvEqqxf3+9uPO8nML6FzdCB/uriTu8NptpxVDs6yer+ePTF5e9f4PLO2ziKvNI8uYV24osMVdQ9s+WuQtQeCYuDSp+p+PhERkRZAibxIC5RxwDk/Xo3uxL3WH8jm09UHAfjrdQn4eqtLfUMrdja6q0VZfVZRFp/s+ASAe/vfi5e5jq9X5m5Y9orj/uUvgX9Y3c4nIiLSQiiRF2lh7HaD9PJGd63U6E7cyGqz8/jXjqTy94PaMbRTpJsjahlcHev79K7xsR9u/ZCisiJ6R/bmkvhL6haIYcD3D4CtFLqMgt7X1e18IiIiLUjNa+pExKMdP1pIWYkNbx8z4W3U6E7cZ8ayvSSlOdaMf+yKnu4Op9kzDAPsdoq3O/oR+Cck1Oj4tII0vtj5BQBTB0yte0PCk9eMH/MyqMGhiIhItWlEXqSFcTa6i44PxmzWG2dxj5PXjH9ybE/CA33cHFHLULp3L0ZREeaAAHw6dKjRsd/s+YZSeykDWg3gvNjz6hbIyWvGj3gMwmsWi4iISEunRF6khTnR6E7z48U9DMNg+twtlJTZOb9LJNf215rxjcFkMlHkbHTXqxcmr5rNb/9x/48AXNflurqPxv/0hGPN+Jg+cO7ddTuXiIhIC6REXqSFyThQPiKv+fHiJt8mprJsd2b5mvEJWjO+EdW20d2eY3vYc3wP3mbvus+N3/c/SPwUMMHY17VmvIiISC0okRdpQew2OxmH8gE1uhP3yCm08vz3jjnaU0d0oWOU+jQ0BsMwgNon8gv3LwTg/NjzCfUNrX0g1mJHgzuAwbdB3ODan0tERKQFUyIv0oJkHynEZrVj8fMirFWAu8ORFuilhTvJzC+lS6sg7tCa8Y3KsFop3rkTAP8adKw3DMNVVn9Zh8vqFoTWjBcREakXSuRFWpD08rL6VvHBmNToThrZuv3ZfLbGsWb8/13bR2vGN7KSPXswSkowBwdjiY+v9nFJx5LYn7sfH7MPI+JG1D6AjF2w/FXH/Sv+Bn51GNkXERFp4ZTIi7QgrkZ37dXoThpXaZmdx7/ZAsANg+K0ZnwjczS6O7F+vMlc/T//C/c5yuovbHchQT5BtQvg5DXju46GXtfW7jwiIiICKJEXaVHU6E7cZcayvexKyycy0IfHruzh7nBapOLyjvX+NZgfbxiGa3785R0vr/3FN30KB5Y71oy/UmvGi4iI1JUSeZEWwma1k5nibHSnEXlpPAeyCnhzkWPN+CfG9iQsQGvGNzbDME40uutd/UR+a+ZWDucfxt/bn4vaXlS7ixdkOZabg/I149vX7jwiIiLiokRepIXISs3HbjPwDfAmJMrP3eFIC2EYBk/M3ao1493IMAxHo7tdu4Cadax3jsZf3O5iAiy1bJCpNeNFRETqnRJ5kRbixPz4YK3bLY3mu81HtGa8m5lMJkqSdoHVildYGJa2sdU6zm7YXd3qL+9Qy7L6fcu0ZryIiEgDUCIv0kKcmB+vsnppHGU2O6/8lATAPcO1Zrw7FW87sX58dT9MScxIJK0wjUBLIBe0u6DmFzUMWPy84/6gW7VmvIiISD1SIi/SQqSdNCIv0hjmbznCgaxCwgMs3H5RR3eH06IVbXGsGOBXg/Xjnd3qR8SNwNfLt+YX3b8cDq0GL1+4+M81P15ERESqpERepAUoK7WRnVoAqNGdNA673eCdJXsAmHJ+RwJ8vN0cUcvlaHRXs471NruNnw78BNShrH7Zy47bgTdDcOvanUNEREQqpURepAXITMnHsBv4B1sICq/FyJpIDf28I41dafkE+3rzh/M6uDucFs0oLqZkj+NDFb+EhGodsyF9A5lFmQT7BHNe7Hk1v2jKetj7K5i94fz7a368iIiInJESeZEW4ESjuxA1G5MGZxgnRuNvHtaeUH81OHOn4p07wWbDKzoK71atqnWMs6z+0vhLsdSmQZ1zNL7vDRAWX/PjRURE5IyUyIu0ACca3Wl+vDS8Zbsz2ZySg5/FzG0XaG68u7nK6ntXr9Fdmb2Mnw/8DMAVHa6o+QXTtkHSAsAEFzxQ8+NFRETkrJTIizRzhmGQfvDEiLxIQ3u7fDR+4pB4IoM0lcPdiree6FhfHWuOrOFYyTHCfcMZ0mZIzS+47BXHbe9rIaprzY8XERGRs1IiL9LMlZXaOXakvNFdvEbkpWGt3Z/Nmn3ZWLxM3HFRJ3eH0+IZhkGRa+m56nWsX7jfUVY/sv1IvM01bFKYlQzbvnHcv/Chmh0rIiIi1eYRiXx+fj7Tpk0jNjYWPz8/+vfvz+eff16tY5csWcKoUaNo1aoVQUFB9O3blzfffBObzdbAUYs0DRkH8zAMCAz1ITBMo6PSsN5e7BiN/9057WgT6u/maMReUEhp8l4A/HufPZG32qz8cvAXoJbd6pe/CoYdul0OravXWE9ERERqziPWA7r++utZu3YtL730Et26dePTTz9l4sSJ2O12Jk2aVOVxv/zyC5dddhkXXXQRM2bMIDAwkG+//Zb777+f5ORk3njjjUZ8FiLuke6aH6+yemlYW1JyWLorA7MJ7ry4s7vDEaBkx3YwDLxbt8Y7Ovqsj195ZCV5pXlE+UdxTsw5NbvY8UOQWP4h+4UP1yJaERERqa4mn8gvWLCAn3/+2ZW8A4wYMYIDBw7wyCOPcMMNN+Dl5VXpsbNmzcJisfD9998TGBgIwMiRI0lKSmLWrFlK5KVFcHasj+mgsnppWO/+6hiNv7pfLO0jA90cjQAUORvdJVRvfryzW/2o9qPwMlf+t7VKK94Eexl0vAjiBtfsWBEREamRJl9a/8033xAUFMT48eMrbL/11ltJTU1l9erVVR5rsVjw8fHB379ieWdYWBh+fn4NEq9IU5NR3uhOI/LSkPak57Fw21EA7h7Rxc3RiJOr0V3vsyfyJbYSFh9aDNSirD4/HTb8x3Ffc+NFREQaXJNP5Ldu3UrPnj3x9q5YPNC3b1/X/qrceeedlJaWct9995Gamsrx48f5+OOP+eabb3j00UcbNG6RpqCkqIzjaYWAGt1Jw3p3STKGAZf1jqFbjL7XmoqirVuA6nWsX354OQXWAloFtKJ/q/41u9DKt6GsGNoOgo4X1yJSERERqYkmX1qflZVFp06ndz6OiIhw7a/K0KFDWbx4MePHj+edd94BwMvLixdffJGHHjrziEFJSQklJSWur3Nzc2sTvohbOUfjgyP88A/2cXM00lwdzCpkXmIqAPeO0HJjTYUtNxfrgYMA+PXuddbH/7jvR8AxGm821eBz/sJsWPuh4/5FD0M11qoXERGRumnyiTyA6QxvCs60b/369Vx33XUMHTqUf/3rXwQGBrJ48WKeeOIJiouLefLJJ6s89sUXX+TZZ5+tU9wi7uZsdNeqvUZIpeG8979kbHaDi7pFk9Au1N3hSLni7dsBsLRrh3d4+BkfW2gt5NeUX4FalNWveR9K8yGmj6NbvYiIiDS4Jp/IR0ZGVjrqnp2dDZwYma/MPffcQ0xMDN98842rId6IESMwm80888wz3HjjjZWO9gM89thjPPjgg66vc3NziYuLq8tTEWl0GQec8+OVyEvDOJpTzFfrUgC4V3Pjm5SiLdUvq//f4f9RVFZE26C29ImqXmM8AEryYNU/HfcvfFCj8SIiIo2kyc+RT0hIYMeOHZSVlVXYvqX8DUqfM7xB2bRpE+ecc85pXe0HDx6M3W5nx44dVR7r6+tLSEhIhX8inubEiLy+f6VhzFi2l1KbnSEdIhjSseoPVqXxFW9x9JDx73P29eOdZfWXdbjsjJVup1n3byg+DpFdoNe1tYhSREREaqPJJ/LXXXcd+fn5/Pe//62w/aOPPiI2NpahQ4dWeWxsbCzr1q3DZrNV2L5y5UoA2rVrV/8BizQRxflWcjOLAYhWoztpANkFpXy62jEH+55LNBrf1Lg61vdJOOPjCqwFLDu8DKhhWb21CFa87bh/wQNQ0+XqREREpNaafGn9FVdcwahRo7jrrrvIzc2lS5cufPbZZyxcuJBPPvnENdp+22238dFHH5GcnEz79u0BeOCBB7jvvvu46qqr+NOf/kRAQACLFi3ilVdeYeTIkfTr18+dT02kQaUfdIzGh0b74xdocXM00hz9e/k+iqw2EtqGclHXKHeHIycpO3YMa6qjAeHZGt0tObSEElsJ7UPa0yOiR/UvsvETKEiH0Djoe0NdwhUREZEaavKJPMDXX3/N9OnTeeqpp8jOzqZHjx589tlnTJgwwfUYm82GzWbDMAzXtqlTp9K2bVtee+01/vjHP1JUVESHDh14+umneeCBB9zxVEQaTXr5/Hg1upOGkFts5aOV+wG4Z0SXmpVjS4Mr3roNAJ8OHfAKPvPvgFqV1dus8Nsbjvvn3w9e+rBQRESkMXlEIh8UFMQbb7zBG2+8UeVjZs2axaxZs07bfv3113P99dc3YHQiTZNz6blozY+XBvDxygPkFZfRLSaI0b1i3B2OnKJ4m7Os/syN63JLc1meuhyoYVn95i8g5xAEtoIBN9U6ThEREamdJj9HXkRqJ32/lp6ThlFYWsaHy/cBcPfwLpjNGo1vaoq2OBP5Mze6W3xwMWX2MjqHdqZreNfqndxug2WvOu6fdy9Y/OsSqoiIiNSCEnmRZqgwt5T8YyVggug4JfJSvz5bc4jsglLiIwIY27eNu8ORShSXr+zif5YR+YX7FgJweccajMZvnwvZyeAXBoOm1DJCERERqQsl8iLNkHPZufCYAHz8PWIGjXiIkjIb7/8vGYC7hnfG20t/Rpoaa3o6ZenpYDbj17NnlY87VnyMVUdWATUoqzeME6Px594FvvqgUERExB30DkykGTrR6E7z46V+/Xf9YdJyS2gd4sf1A9u6OxypRPE2R6M7386dMAcGVvm4Xw7+gs2w0SOiBx1CO1Tv5LsWQtpW8AmCIXfUQ7QiIiJSG0rkRZqhjPIR+WjNj5d6VGaz895Sx2j8HRd1wtdb64Y3Rc6O9X69z1xWf3K3+moxDPjfy477g2+DgIhaxygiIiJ1o0RepJkxDEMj8tIg5m85wsHsQiIDfZg4JN7d4UgVireevWN9VlEWa9PWAjVI5Pcvh8PrwNsPht1b5zhFRESk9pTIizQzBcdLKcwtxWSCqLggd4cjzchnaw4C8IdhHfD30Wh8U2QYBkXlpfX+Z+hYv/boWuyGnW7h3YgLjqveydfPctz2mwhBreoYqYiIiNSFEnmRZsbZ6C4iNhCLki2pJ/szC1i1NxuzCX4/uJ27w5EqlKWlYcvMBC8vfHv0qPJxa486RuOHtB5SvRMXZsOO7xz3z7mlrmGKiIhIHSmRF2lm0l3z41VWL/Xny3WHALioWzRtQrVueFNVVL7snG/Xrpj9/Kp83Lq0dQAMihlUvRNvmQO2EohJgDb96xqmiIiI1JESeZFmJsM5Pz5eje6kfpTZ7Hy1PgWACYOrWYYtbuFqdHeGsvqsoiz25uwF4JyYc85+UsOADf9x3B/4BzCZ6hyniIiI1I0SeZFmpEKjuw4akZf68WtSBul5JUQG+nBJjxh3hyNn4Gx0598nocrHOEfju4Z3Jcwv7OwnTd3oWHLOyxf6jq+PMEVERKSOlMiLNCN5WcUUF1gxe5mIaqtGd1I/Pl/rKKsfd047fLz1Z6OpMgyjWh3rnfPjB8cMrt6JN37suO11NfiH1ylGERERqR96RybSjDhH4yPbBuFl0Y+31F16bjFLktIB+P0gldU3ZdbDh7Hl5IDFgm+3rlU+bt1Rx4j84NbVSORLC2HLV477A26ujzBFRESkHuidvkgzknHQ2ehO8+Olfvx3w2FsdoNz2ofTpZWqPJoy12h89+6YfXwqfUx2cTbJOclANefHb58HJbkQ3gE6XFhfoYqIiEgdKZEXaUbS1ehO6pFhGK5u9TeoyV2Td6KsvupGd+vT1gPQJawL4X7VKJN3NrkbcBOY9ZZBRESkqdBfZZFmwrCf1OhOS89JPVizL5t9mQUE+ngxJqGNu8ORsyja4mx0d/b58dVadi5zDxxcASYz9L+xXmIUERGR+qFEXqSZyMkoorSoDC9vMxFtA90djjQDX5Q3ubuqXyyBvt5ujkbOxLDbKd7mXHqu6kTe2bG+WvPjnU3uuoyCkNg6xygiIiL1R4m8SDORXj4/PiouCC8v/WhL3eQUWVmw9QigsnpPUHrgAPb8fEy+vvh26VLpY44VH2P3sd1ANebH26yw6VPH/YFqciciItLU6N2+SDOhsnqpT98mplJstdMtJoj+cWHuDkfOonhr+Wh8z56YvCuvntiQtgGAzqGdifSPPPMJd/8EBekQGA3dLq/XWEVERKTuVCsp0kxklCfy0Wp0J/Xgy7XOJnfxmEwmN0cjZ+NqdNe76kZ3a9PK58e3rsb8eGeTu34TwctS5/hEPEFZWRllZWXuDkNEWiBvb2+8q/ggvspjGigWEWlEdrtBxkHniLwSeambbak5bDmcg8XLxHUD2ro7HKmGom3OjvXVaHR3tkQ+94hjRB5g4B/qJT6RpqywsJDMzEwKCgrcHYqItGCBgYFERUUREBBQrccrkRdpBnLSC7GW2PD2MRPeuno//CJVcY7Gj+7VmojAytcjl6bDMAxKtu8AwL+KpedySnLYdWwXUI2O9YmfgmGH+GEQ1bVeYxVpakpLSzl06BAWi4U2bdrg6+urKiQRaVSGYVBSUkJ2djaHDh2iY8eO+Pic/f2XEnmRZiAnowiAsJgAzGp0J3VQbLXxzcbDgJrceQpbZib2wkIwm/Fp377Sxzi71XcM7UiUf1TVJ7PbYUN5t/oBanInzV96ejpeXl60b98eLy8vd4cjIi2Uv78/wcHB7Nu3j/T0dNq1a3fWY/SOX6QZyM10JPIhUf5ujkQ83Y/bjpJbXEbbMH8u6HKGhE+ajNJDKQBYWrfGVMUn+OuOli87F3OWZecO/AbH9oFPMPS+tj7DFGlyDMOgsLCQ0NBQJfEi4nZeXl6EhoZSWFiIYRhnfbwSeZFmIDejGICQSD83RyKezrl2/PhB7TCbVV7qCawpjtfMcoZP750j8medH+9scpcwDnwC6yU+kabKarVis9nw99eH4CLSNPj7+2Oz2bBarWd9rBJ5kWYgRyPyUg8OZBWwIjkLkwnGD1JZvacoPVSeyMdVnsjnlOSQlJ0EnGV+fNFx2PGt476a3EkLYLfbATQaLyJNhvP3kfP305kokRdpBvKyyhP5aCXyUntz1jlKtC/oEkXbMH0veQpriqOngU9c5R++bEjbgIFBh5AORAdEV32iLXOgrBha9YbYgQ0RqkiTpOZ2ItJU1OT3kRJ5EQ9nGAY5mY7S+lCNyEstldnsfLXekchPGBzv5mikJqzOEfl2lSfyNS6rH3gzKLERERFp0pTIi3i4ojwrZSU2MEFwhObIS+38b3cGR3OLCQ+wMLJXK3eHIzVQmuL4AManitJ61/rxZyqrT90ERzeDlw/0vaG+QxQREZF6pkRexMM5O9YHhfniZdGPtNSOs8nd9QPb4eut+aKewl5SQllaGlB5s7vc0lySjlVjfvzG8iXneoyFgIh6j1NERETql971i3g4LT0ndZWRV8KiHemA1o73NNbDqWAYmAIC8Io4PQHfmLYRu2GnfUh7YgJjqjhJEWye47ivJnciIiIeQYm8iIfLLZ8fHxKlsnqpna83pFBmN+gfF0a3mGB3hyM1YD1cXlbfrl2lDXJc8+PPNBq/4zsoyYGweOh4cYPEKSIiIvVLibyIh9OIvNSFYRh8sc5RVj9Bo/Ee58TSc5W/dq758WdqdOdscjfgZjDrbYGIiEB6ejoPPvggXbt2xdfXl+DgYIYNG8YXX3zh7tCknP5ii3g4JfJSF+sOHGNvRgEBPl6M7Rfr7nCkhqyHTozInyqvNI8d2TuAM4zIZ++F/csAE/Sf1FBhioi0CE888QQmkwmTycQLL7zg7nBqbc+ePQwYMIDXXnuNlJQUevToQWhoKKtWrWLChAl8+eWX7g4RgAULFjBy5EgiIiIIDAxk4MCBvPXWW9Vag706mvrrqURexMPlKJGXOnA2uRvbtw1Bvt5ujkZqypriXHru9ER+Y7pjfny7oHa0Dmxd+Qk2fuK47XIphFbe9V5ERM5ux44d/OMf/3B3GHVmt9uZOHEiqampTJo0ibS0NBITE0lJSeHBBx8EYObMmW6OEl566SXGjBnDokWLCA8Pp0uXLiQmJnLfffdx3XXX1TmZ94TXU4m8iAezldnJP1YCaI681FxesZX5m48AanLnqUrLR+QtlSw9t+6oY3784NaDKz/YVgYbZzvuq8mdiEitGYbBn/70JywWC5dccom7w6mTH3/8kXXr1tGzZ09mzpxJSEiIa9+NN94IQGZmprvCA2DlypU8/vjjmM1mPv30U5KTk0lMTGTDhg3ExMTw7bff8uqrr9b6/J7yeiqRF/FgednFYIC3xUxAiI+7wxEP813iEYqsNjpHBzIwPtzd4UgNGYaB1bWG/OkfxDgb3VWZyO/5BfKPQkAUdLuiweIUEWnuPvzwQ5YtW8ZTTz1FXBU9SzzF3LlzAbj11lvx8an43jI93bHCTZs2bRo7rApeeOEFDMPgj3/8IxMnTnRt79evnyuBf+mll7BarbU6v6e8nkrkRTyYa358tH+lHatFzuSLtQcBmDA4Xt8/Hsh2/Dj2/HwALG3bVthXYC1ge9Z24Azz451N7vpNAG99ECgiZ5eSksLUqVPp0KED/v7+dO/enVdffRXDMAAYN24cvr6+pJR/yNgSZGRk8Oc//5levXrxwAMPuDucOluxYgUAo0ePPm3ft99+C8B5553XqDGdLDc3l19++QWA22677bT948ePJyQkhKysLJYsWVLj83vS66lEXsSDnVh6TvPjpWaSjuaRmJKDt9nEdQPbnv0AaXKco/HerVph9qs4tWZj+kZsho22QW1pE1TJyEl+Ouxa6LivsnoRqYb58+fTu3dv3n77bUwmE506dWL37t089NBD/Oc//yEpKYm5c+dy66230q6Svh3N1QMPPEB2djbvvvsuFovF3eHUSUlJCTt37sTHx4devXpV2Ldz504+/vhjzGYz48aNc1OEsHHjRkpLS/Hz82PgwIGn7bdYLAwe7KhEW716dY3P70mvpzobiXiw3IzyEflIzY+XmvkuMRWAET1aERXk6+ZopDash6pudOdadq6q0fjt88CwQdtzILp7g8Uo4skMw6DIanN3GLXib/Gq10qrtWvXMm7cOGw2G7Nnz2bSJMcqF++88w733nsvn376KUuXLsVsNvOXv/ylyvP89a9/ZcGCBTW+/ltvvcWAAQNqHX9DWbRoEbNnz+amm27i4osvdnc4dbZ9+3bKysoYMGBAhSR23rx5TJs2jfz8fO644w66du161nM11Gu9e/duAOLj4/H2rjyV7dSpE4sWLXI9tro87fVUIi/iwbT0nNSGYRgs2OJocje2r3vnuUntORvd+VTW6O5s8+O3z3Pc9r6uQWITaQ6KrDZ6PfWju8Oole3PXUaAT/29zb/nnnsoKSnh4YcfdiXxAHfccQfTpk1j06ZNLFmyhJtvvpkOHTpUeZ5du3bx22+/1fj6OTk5tQm7QRUXF3PnnXcSGhrKyy+/7O5w6kViYiKAK5G+9NJLWbt2LXl5eQDccMMNvPHGG9U6V0O91seOHQMgPLzq3j7Ofc7HVocnvp4qrRfxYLlZ5aX10Urkpfp2HMljb2YBPt5mLu0Z4+5wpJacpfWWdhUb8RRaC9mWuQ2AQa0rGZHPS4P9yx33e13ToDGKiOdbs2YNa9euJTw8nCeffLLCPovFQnR0NOnp6djtdh577LEznmvWrFkYhlHjf8OHD2/AZ1g7L7zwAnv27OH//u//iIlpHn9LN2/eDED//v1JT09n8eLFriQe4NChQ+zcubNa52qo17q42PHe99RGfCfz9XVUGhYVFVUrVvDM11Mj8iIe7MSIvErrpfqco/HDu0Vr7XgPVupcQ/6UEflN6ZuwGTZiA2NpG1RJ/4Md3wIGtB0EYfGNEKmIZ/K3eLH9ucvcHUat+Fu86u1czi7mV111VYWlyE41YcKEapVcN6ZHH33U1aCtJmbOnMmwYcOq3O9cY3zgwIHcdddddQmxSXGOyPfv359WrVpRXFxMWloaq1ev5pVXXmHFihWMHDmSXbt2ERER4ZYY/cp7wpSWllb5mJISx9LM/v7VG+jy1NdT7+BEPFRxgZWSwjIAQiI1Ii/VYxgG88sT+TEqq/doVmdp/Slz5Nemlc+Pr2w0HmDbXMetyupFzshkMtVrebqncnYxv+KKypepNAwDs9nM9OnTGzOsaklNTSUpKanGxxUUFJxx/913301ZWRn//Oc/MZubT4Hz5s2bMZlM9OvXD3CMbMfHxxMfH88111xDr169SE5O5scff6yw7Ftjqk7ZfHXK70/mqa+nfjuJeCjnaLx/iA8W3/r75F2atx1H8tinsnqPZ5SVYT3i+EDGcsoat2dsdJeXBgfK5yyqrF5EqsGZCPfs2bPSfRkZGfTs2bPS/adq7GZ3n3zyCZ988kmNjzubjRs3YjKZuPrqq0/b55zj/be//Y23336buLg41q5dW+8x1LfU1FQyMzPp1KlTpZUXPj4+9O/fn+TkZOx2+1nP11CvtbPq4+DBg5SVlVXa8G7v3r0VHns2nvp6KpEX8VDOpedCVVYvNTB/S3m3+u4qq/dk1qNHwWbD5OODd3S0a/vJ8+MrbXTnLKtvNxjC4k7fLyJyiuzsbACCgoJO2/fYY49hs9mqvUxXc2p2Z7PZSEtLq3J/fn4++fn5rlLwpu7ksvrKGIbBmjVrAFwj9mfSUK+1s6N+cXExGzZsYMiQIRX2W61WV6I9dOjQal/XE19Pz6kdEJEK1LFeasrRrf4oAFcmqKzek5289JzppDLATRmbKDPKaB3YuvL58c6y+l7XNnyQItIsBAYGApCcnFxh+08//cQ333wDUO2l7ppLs7vjx49XGestt9wCwPPPP49hGOzfv9+9wVaTM5G32SpfcvG7777j0KFD9OzZkz59+pz1fA31WoeEhDBy5EgAPvzww9P2z5kzh9zcXCIjI6v9feOpr6cSeREPpUReamr7kVz2ZRbgq7J6j1d6qPJGd+uOli87FzP49DfWeUdVVi8iNXbOOecAjkTGOXd806ZN3Hzzzfj7+xMYGMju3bvJzMx0Z5geY9asWZhMpjMu0+cOzo718+bN4+WXX66Q0C9YsIApU6YAju8Dd5s+fTomk4kPPviAzz77zLU9MTGRBx98EHA0Ojy1s/3DDz9Mhw4dePjhhxs13oaiRF7EQ6ljvdSUq1u9yuo9nqvRXdtTEvny9eMrbXS3XWX1IlJzjzzyCCaTieXLlxMfH8+AAQMYPHgwWVlZzJw5k0GDBpGfn8/AgQN56qmn3B2u1JJzRB4cr3mbNm0YNGgQrVu3ZsyYMWRnZ/Pcc88xbtw4N0bpcP755/P8889jt9uZNGkSnTt3pl+/fgwcOJC0tDTGjBnDQw89dNpxmZmZHDhwoNl86KREXsRD5ZTPkdeIvFSHYRjM3+zsVh/r5mikrk4sPXciIS8qK2JL5hbAMSJ/mu1zHbfqVi8iNTB69GjmzZvHsGHDKCkpISkpid69e/P9999zww038N577zFw4EBSUlLcHapHOHrUMcWtqrno7lBSUsKuXbsIDQ1l5syZ9OvXj7y8PLZs2YKvry+TJk1ixYoVPPnkk+4O1WX69Ol89913XHLJJWRlZbFnzx4SEhJ4/fXXmTdvHl5ezb8RtIZkRDyQ3W6Qn6VEXqpv+5Fc9mcVOsrqe7RydzhSR9aUwwD4nFRan5iRSJm9jJiAGNoFVxypd5TVO5aQUlm9iNTUVVddxVVXXVXpvh49erB+/fpGjqjpmjVrFrNmzapyv3M5P2cJeFOwbds2ysrK6NOnD5MnT2by5MnuDqlaxo4dy9ixY6v9+LO9NvV1TGPRiLyIB8o/VozdbmD2NhEY5uvucMQDOEfjR3RvRaDK6j2eq9ndSSPyzvnxg1oPOn1+vKusfgiEnpLki4hIo1m5ciWDBg3ioosucncoLs6y+oSEBDdHIjWhd3MiHsi59FxIpD9mc/U6xUrL5ehW70jkr+yrbvWezpaXh+34cQAsJ82RP+P68dscnaXpfW0DRyciIlVJSkoiMzOTN998092hVOBM5KvTjV6aDiXyIh7I1eguUo3u5Oy2paqsvjmxls9D9QoPxyvIsSxUcVnxifnxp64fn3sEDq503FdZvYiI23Tv3h3DMNwdxmmcHes1Iu9ZVFov4oFyM7T0nFSfczReZfXNQ2klZfWbMzZjtVuJ9o8mPji+4gE7VFYvIiJVcybyGpH3LHpHJ+KBctXoTqrJMAzmb3F2q1dZfXPganTX7qSy+rTysvrK5sdvm+u4Vbd6ERGpRHNZjq2l0Yi8iAdyldZHq7Rezmxbai4HysvqL1FZfbNgrWTpOWeju9PL6lNVVi8iItIMKZEX8UCuRF4j8nIWztH4S3qorL65KD3kmCPvXHrOarOyOcNRFnlOzDkVH+zsVh83FELbNmaYIiIi0oCUyIt4mNLiMoryrIASeTkzwzBcy85dmaCy+ubCtfRceWn9ruO7KLWXEuITQseQjhUfvH2u47bXtY0XoIiIiDQ4JfIiHsa59JxvoDe+/hphlaptS83lYHYhfhaV1TcXht2O9bBjjrylnaO0fmvGVgD6RPWpOD9eZfUiIiLNlhJ5EQ/jLKsP1Wi8nMX3m9WtvrkpS0/HsFrB2xtL6xgA17JzfaJO6Ta8/VvHrcrqRUREmh0l8iIeRvPjpToMw3AtO6du9c2Hq6w+NhaTt+PDmW1Z2wDoE3lKIr/tG8etutWLiIg0O0rkRTyMs7ReibycydbDKqtvjlyN7srnxxdYC0g+ngycMiKfmwqHVjnu97y6UWMUERGRhqdEXsTDnBiR19JzUrWTu9UH+KisvrlwLT1Xnshvz9qOgUHrwNZEB0SfeOD2eY7buHNVVi8iItIMKZEX8TAqrZezMQyD+VtSARiTEOvmaKQ+laY4RuQt5UvPbc0sb3R3Wln9XMetyupFRESaJSXyIh7EsBvkZqm0Xs5s6+FcDmUX4WcxM6JH9NkPEI9hda0h7+hYX2mju5zDJ8rqe6msXkREpDlSIi/iQQpzS7FZ7ZjMJoIifN0djjRR35ePxl/aI0Zl9c1Mqau03pHIb8ssb3R3ciK/o7xbffwwCFFFhoiISHOkRF7Eg+SUl9UHhfvi5aUfXzndyd3qr0xQt/rmxF5UhC0jEwCfdm3JKsoitSAVEyZ6RfY68UBnt/pe1zZ+kCIiItIolAmIeBDNj5ez2XI4h0PZRfhbvFRW38xYy+fHm4ODMYeGupad6xjakWCfYMeDclLg0GrApLJ6ERGRZkyJvIgHcS49F6qO9VIFdatvvk5udGcymSqfH7/dWVZ/rsrqRUREmjEl8iIexDUiH60ReTmdYRjM3+xI5Mf0VVl9c+NqdNfuDI3unGX16lYvIiJ1kJ6ezoMPPkjXrl3x9fUlODiYYcOG8cUXX7g7NCnnEYl8fn4+06ZNIzY2Fj8/P/r378/nn39e7ePnzZvHxRdfTEhICIGBgfTu3Zv333+/ASMWaRgqrZcz2XI4h5Rj5WX13Vu5OxypZ65Gd3HtMAzD1eguISrB8YCcFEhZA5igp8rqRUQaWnp6Og8//DC9e/cmICAAPz8/OnfuzB133MGePXvcHV6t7dmzhwEDBvDaa6+RkpJCjx49CA0NZdWqVUyYMIEvv/zS3SECsGDBAkaOHElERASBgYEMHDiQt956C7vdXi/nf+KJJzCZTJhMJl544YV6OWd98ohE/vrrr+ejjz7i6aef5ocffmDw4MFMnDiRTz/99KzHvvTSS1x//fX06dOHL7/8km+//Za7776b0tLSRohcpH7lZpQn8pFK5OV0ztH4S3q2wt/Hy83RSH07MSLfjpT8FI6XHMfb7E238G6OB2yf57iNHwYhqsgQEWlISUlJJCQk8Morr7B7927i4+Pp2rUrqampzJgxg379+rF06VJ3h1ljdrudiRMnkpqayqRJk0hLSyMxMZGUlBQefPBBAGbOnOnmKB053pgxY1i0aBHh4eF06dKFxMRE7rvvPq677ro6J/M7duzgH//4Rz1F2zCafCK/YMECfv75Z959913+9Kc/MWLECGbMmMGoUaN45JFHsNlsVR67fv16pk+fzosvvsg777zD5ZdfzqWXXso999zDvffe24jPQqTuykptFOQ4PoAKidYceanIMAzX/Pgx6lbfLFlPWnrOORrfI7wHPl4+jgdsm+u47X1t4wcnItLC3HPPPaSnp3P++eezd+9edu7cyZYtW0hJSeHqq6+msLCQW2+9FcMw3B1qjfz444+sW7eOnj17MnPmTEJCQlz7brzxRgAyMzPdFR4AK1eu5PHHH8dsNvPpp5+SnJxMYmIiGzZsICYmhm+//ZZXX3211uc3DIM//elPWCwWLrnkknqMvH41+UT+m2++ISgoiPHjx1fYfuutt5Kamsrq1aurPPbtt9/G19eXqVOnNnSYIg0uL9vR6M7i54VfoMXN0UhTszlFZfXNmWEYlKYcBsAnrp1rfnzvqN6OB6isXkSk0RQWFrJkyRIA/vnPf9KuXTvXvsjISGbNmoXJZGLfvn3s3LnTXWHWyty5cwFHruXj41NhX3p6OgBt2rh3wOCFF17AMAz++Mc/MnHiRNf2fv36uRL4l156CavVWqvzf/jhhyxbtoynnnqKuLi4eom5ITT5RH7r1q307NkTb++K3Zf79u3r2l+V//3vf/Ts2ZP//ve/dO/eHS8vL9q1a8df/vIXldaLx8nJODE/3mQyuTkaaWqca8errL55smVlYRQVgcmEJTaWrZmOv32u+fEqqxeRRpCSksLUqVPp0KED/v7+dO/enVdffdU16jxu3Dh8fX1JKV9lo7kqLS11lW536tTptP3h4eFEREQAUFZW1qix1dWKFSsAGD169Gn7vv3WsTLKeeed16gxnSw3N5dffvkFgNtuu+20/ePHjyckJISsrCzXhy01kZGRwZ///Gd69erFAw88UOd4G1KTT+SzsrJcPwgnc27Lysqq8tjDhw+ze/du7rvvPu677z5++eUXJk+ezMsvv8ytt956xuuWlJSQm5tb4Z+IO51Yek7z46UiwzD4vnx+/FiV1TdLpYccZfXebVpj8zazI3sHcFIir271ItLA5s+fT+/evXn77bcxmUx06tSJ3bt389BDD/Gf//yHpKQk5s6dy6233lphhLo5CgsLc43UOhPfkyUlJZGVlUVYWBhdu3Zt7PBqraSkhJ07d+Lj40OvXr0q7Nu5cycff/wxZrOZcePGuSlC2LhxI6Wlpfj5+TFw4MDT9lssFgYPHgxwxsrtqjzwwANkZ2fz7rvvYrE07QpYj1hk+Eyjj2faZ7fbycvL47PPPmPChAkAjBgxgoKCAl5//XWeffZZunTpUumxL774Is8++2zdAhepR86O9cFaQ15OsS01l8PHHWX1w1VW3yxZy0e3fNq2I/l4MkVlRQRaAukQ2gFyj0DKWhxl9Ve5NU6RZsUwwFro7ihqxxIA9Vi9t3btWsaNG4fNZmP27NlMmjQJgHfeeYd7772XTz/9lKVLl2I2m/nLX/5S5Xn++te/smDBghpf/6233mLAgAG1jr8hvPDCC9xyyy1MmTKF119/neHDh+Pt7c2qVauYNm0aJpOJv//97/j5ec77tu3bt1NWVsaAAQMqJLHz5s1j2rRp5Ofnc8cdd1Trw4mGeq13794NQHx8/GkV206dOnVi0aJFrsdW16JFi5g9ezY33XQTF198cY2OdYcmn8hHRkZWOuqenZ0NUOlo/cnHHj16lMsuu6zC9iuuuILXX3+dDRs2VJnIP/bYY67OjOAo42jKcySk+XMm8hqRl1P9tD0NgIu6Ramsvplyjshb4uJYn+VodNc7sjdmkxmSyt8otRuksnqR+mQthL/GujuK2nk8FXwC6+1099xzDyUlJTz88MOuJB7gjjvuYNq0aWzatIklS5Zw880306FDhyrPs2vXLn777bcaXz8nJ6c2YTeoP/zhDwQFBfH888/zu9/9rsK+vn37smDBAi6//HI3RVc7iYmJAK5E+tJLL2Xt2rXk5eUBcMMNN/DGG29U61wN9VofO3YMcExfqIpzn/Ox1VFcXMydd95JaGgoL7/8crWPc6cmX1qfkJDAjh07TptfsmWLo9FPnz59qjzWOY/+VM55PGZz1U/f19eXkJCQCv9E3MlZWq815OVUP5cn8qN6tXZzJNJQrGdqdOdM5Ltf6Y7QRKSZW7NmDWvXriU8PJwnn3yywj6LxUJ0dDTp6enY7XYee+yxM55r1qxZGIZR43/Dhw9vwGdYO4ZhsHfvXrKysvDy8qJLly706tULHx8ftm7dyvvvv+8aePQUmzdvBqB///6kp6ezePFiVxIPcOjQoWo372uo17q42PF++NRGfCfz9fUFoKioqFqxgqPCYs+ePfzf//0fMTEx1T7OnZr8iPx1113HjBkz+O9//8sNN9zg2v7RRx8RGxvL0KFDqzx23Lhx/PTTT/zwww8VPj1csGABZrPZNX9CpKkzDMM1Ih+i0no5yaHsQnYcycVsgkt6qKy+ubIeOrH03NbMxUD5/PjiXNhbvk5xjzHuCk+kebIEOEa2PZEloN5O5exiftVVV51xYGvChAlNbj74o48+6mrQVhMzZ85k2LBhZ3zMnXfeyfvvv895553H//73P1clQnp6OrfddhvffPMNycnJbNiwAS8vz6iWc47I9+/fn1atWlFcXExaWhqrV6/mlVdeYcWKFYwcOZJdu3adsSq6ITmnKpypcXlJSQkA/v7VG/xyrhk/cOBA7rrrrroH2UiafCJ/xRVXMGrUKO666y5yc3Pp0qULn332GQsXLuSTTz5x/WDcdtttfPTRRyQnJ9O+fXvAsWzCv/71L+6++24yMzPp1asXv/zyC++88w53332363EiTV1xvhVriQ1MEBypRF5OcI7GD+oQQURg1Z9Oi2crLZ8jb8S2Yvc2x5y/hKgE2PML2K0Q0RmiurkzRJHmx2Sq1/J0T+Vs5nbFFVdUut8wDMxmM9OnT2/MsKolNTWVpKSkGh9XUFBwxv2JiYnMmDEDi8XC559/XmH6batWrZg9ezadO3dm8+bNfPnllxWWSGvKNm/ejMlkol+/foBjZDs+Pp74+HiuueYaevXqRXJyMj/++KPbnlN1yuarU35/srvvvpuysjL++c9/nrFiu6lp8ok8wNdff8306dN56qmnyM7OpkePHhUa2AHYbDZsNpurbB4c5T4///wzjz/+OH/961/Jzs6mY8eOvPTSSxXmv4s0dTnlo/GBob54WzzjU11pHM5EfnQvzygDk5qzl5ZSdvQoAPsDC7EZNiL9IokJiDlRVt/jynptbCUi4uRMhHv27FnpvoyMDHr27Fnp/lM1drO7Tz75hE8++aTGx53Nb7/9hmEYdOvWrdIeWiEhIQwZMoQFCxawbt06j0jkU1NTyczMpFOnTpVWXvj4+NC/f3+Sk5NdS++dSUO91s6qj4MHD1JWVlZpw7u9e/dWeOzZbNy4EZPJxNVXX33aPuec/b/97W+8/fbbxMXFsXbt2mqdt6F5RCIfFBTEG2+8ccbmCrNmzWLWrFmnbY+IiOC9997jvffea8AIRRqWyuqlMscLS1mz3zH/bpQS+WbLevgwGAYmf3+22hwl9glRCZjsZbD7J8eDuqusXkQahnOed1BQ0Gn7HnvsMWw2W7WX6Wouze5OnjdeFefgonNOd1N3cll9ZQzDYM2aNQCuEfszaajX2tlRv7i4mA0bNjBkyJAK+61WqyvRPtMU7FPZbDbS0tKq3J+fn09+fn6TWoXAc2oHRFowrSEvlVmSlI7NbtAtJoj2kSr/bK5cje7atWNL1lYA+kT1gQO/QXEOBERB3JAznUJEpNYCAx1/X5KTkyts/+mnn/jmm2+AMy8HfbLm0uzOOdK7a9cuDpX3MDlZbm6uK5ns1s0zpj05E3mbzVbp/u+++45Dhw7Rs2fPMzYbd2qo1zokJISRI0cC8OGHH562f86cOeTm5hIZGVnt75vjx49XGc8tt9wCwPPPP49hGOzfv79a52wMSuRFPIBrRD5aibyccKKsXt3qmzNryoml57ZmnpTI73R2q78czJpyIyIN45xzzgEciYxz7vimTZu4+eab8ff3JzAwkN27d5OZmenOMBvV6NGjiYqKwmq1MmHChArJXXp6OjfeeCOZmZn4+fmdtjTdrFmzMJlMZ1ymzx2cHevnzZvHyy+/XCGhX7BgAVOmTAEc3wfuNn36dEwmEx988AGfffaZa3tiYqJr+vSjjz56Wmf7hx9+mA4dOvDwww83arwNRYm8iAc4UVqvRF4cSspsLE3KAFRW39yVHjrR6O5g3kEA+kT2PmnZOZXVi0jDeeSRRzCZTCxfvpz4+HgGDBjA4MGDycrKYubMmQwaNIj8/HwGDhzIU0895e5wG0VQUBD/+c9/8PPzY8WKFXTp0oVu3brRu3dv4uLi+P777/H29ua9996jbdu27g63Wpwj8uB4zdu0acOgQYNo3bo1Y8aMITs7m+eee45x48a5MUqH888/n+effx673c6kSZPo3Lkz/fr1Y+DAgaSlpTFmzBgeeuih047LzMzkwIEDzeZDJyXyIh4gN6N8DXl1rJdyK5KzKCi1ERPiS0LbUHeHIw3IufRcRqijdDUuOI7QYwch5xB4+0On4W6MTkSau9GjRzNv3jyGDRtGSUkJSUlJ9O7dm++//54bbriB9957j4EDB5JSvrpGS3HFFVeQmJjIHXfcQceOHTl48CB79uyhTZs23HzzzaxevdpVln2yo+XNS6uai+4OJSUl7Nq1i9DQUGbOnEm/fv3Iy8tjy5Yt+Pr6MmnSJFasWMGTTz7p7lBdpk+fznfffccll1xCVlYWe/bsISEhgddff5158+Z5zJJ/deERze5EWjKbzU7+sfJEXqX1Us5ZVj+yZwxms7qVN2elhx1vjvcFFQLOsvr5jp2dLwGf+lsvWkSkMldddRVXXXVVpft69OjB+vXrGzmipqFbt27861//qtExzuX8mtIKWtu2baOsrIw+ffowefJkJk+e7O6QqmXs2LGMHTu22o+vqjl6fR/TWDQiL9LE5WcXYxjgZTETEKJ1wgXsdsOVyKusvnkzDAPrQceI/GYfx2ueEJUASeWJfI8r3RWaiIjUwsqVKxk0aBAXXXSRu0NxcZbVJyQkuDkSqQmNyIs0ca6y+ij/aneFleYtMeU4GXklBPl6M6xzpLvDkQZkz8nBnp8PwCpjHwB9fKPh6BYwmaHb5e4MT0REaiApKYnMzEzefPNNd4dSgTORr043emk6lMiLNHE5WkNeTuEcjb+4WzS+3s1/DlhL5mx0Z4qM4IgtCy+TFz3S9zh2xg2FwCg3RiciIjXRvXt31/ryTYmzY71G5D2LSutFmjh1rJdTqay+5XAuPVcc42ho2CWsC/67fnLs7KFu9SIiUnfORF4j8p5FibxIE5eb6SitD1UiL8D+zAJ2p+fjbTYxonsrd4cjDay0vAt0Zrij8qJPeDc48JtjZ3fNjxcRkbrLzMzEMAwiIiLcHYrUgBJ5kSYuV6X1chLnaPzQThGEBljcHI00NGt5af3BoBIAEqwG2MsgugdEdnZnaCIiIuJGSuRFmjiV1svJXGX1PVVW3xI4S+u3+WUB0Ccj2bFDo/EiIiItmhJ5kSasuMBKSWEZAMGRGpFv6bILSll3IBuAkZof3yI4m90dCirBz8uXzsmO9Yc1P15ERKRlUyIv0oTlZTnmx/sHW/Dx0yITLd2iHWnYDejVJoR24QHuDkcamFFWhjU1FYC0MOgZEIt3aR4EtYbYge4NTkRERNxKibxIE6ayejnZT+pW36JYj6aBzYbN28yxYOhjtTl2dL8czPrzLSIi0pLpnYBIE5ajRF7KFZXaWLY7A1Ai31I458cfC7dgmEwkpO917OiusnoREZGWTom8SBPmWnouWol8S7d8TybFVjttw/zpHRvi7nCkEZQeciTyKSFWAPrkpINPEHS8yJ1hiYiISBOgRF6kCXOW1qvRnfy8/SjgGI03mUxujkYag3PpubRQg1CzD+3KyqDLpWDR7wMREZGWrk7dsxYtWsTixYtZsWIFKSkpZGZmEhAQQHR0NAkJCVx88cWMHTuW1q1b11e8Ii1KboYjkQ9VaX2LZrMbLNqRDqisviVxltanhZnoU2rDBCqrFxEREaAWiXx+fj5vvvkmM2bM4ODBgxiGAYCfnx8REREUFRWxdetWNm/ezOzZs/H29ubqq6/mgQce4Pzzz6/3JyDSXNntBnnZjtL6EJXWt2gbDx4jq6CUED9vhnSMcHc40khKUw4Djo71A/OyweQFXUe5NygRERFpEmpUWv/ee+/RpUsXnnjiCcLCwnjhhRdYvHgxubm5FBYWkpKSQlZWFlarlZ07d/LRRx9xww038NNPP3HRRRdx/fXXs2/fvoZ6LiLNSsHxEuw2A7OXicAwX3eHI270c3m3+hE9WmHx0oyolsJaPkc+PcxEQkkJtD8PAvRBjoiIiNQwkZ86dSqXX345W7ZsYePGjTz22GMMHz6coKCgCo8zmUx069aNm2++mY8//pi0tDRmzJjBli1b+Pjjj+v1CYg0V86y+uBIP8xmzYluyX7WsnMtji0/H9uxYwCkh0HvklLoobJ6ERERcahRaf3OnTvp3LlzjS/i7+/PlClTuOWWW0hJSanx8SItkZaeE4A96fnszSzA4mXi4m7R7g5HGom1/G9lrj+EedmIstuh+5VujkpERESaihqNyNcmiT+Zl5cX7du3r9M5RFqKXCXyAvxU3q1+WOcogv0sbo5GGkupq6we+pSUQkwfCNffTxEREXGo02TLZcuW1VccInIK5xryIVFaaqolU1l9y2R1Nboz0aekRGX1IiLiMdLT03nwwQfp2rUrvr6+BAcHM2zYML744gt3h9as1CmRv+yyy5g7d249hSIiJ3OOyGvpuZYrPa+YTYeOAzCqpxL5lsR60oh8QkmpyupFRJqYffv2MWPGDG6//Xb69euHt7c3JpOJF154ocpjDMNg+fLlPPLII5x77rmEhYXh4+NDbGws48aNY8mSJY34DBrGnj17GDBgAK+99hopKSn06NGD0NBQVq1axYQJE/jyyy/dHSILFixg5MiRREREEBgYyMCBA3nrrbew2+21PqfNZmPGjBlcfPHFREVF4efnR/v27bn22muZN29ePUZ/Qp0S+Y4dOzJ+/Hjef//9Kh+Tn5/P008/XZfLiLRIKq2XRTvSMQzo1y6U1qGqzGhJCg7sBRyJfC+faGjTz70BiYhIBW+88QZ33HEHH3zwAZs3b8Zms531mMWLF3PhhRfy8ssvs3btWmJiYujTpw95eXl8/fXXXHLJJTz55JONEH3DsNvtTJw4kdTUVCZNmkRaWhqJiYmkpKTw4IMPAjBz5ky3xvjSSy8xZswYFi1aRHh4OF26dCExMZH77ruP6667rlbJ/LFjx7jgggu44447WLZsGVFRUfTp0wer1cq8efMarNl7nRL53377jXPPPZe77rqL5557rsK+0tJSXnvtNTp16nTGT6ZE5HSlxWUU5VkBlda3ZCqrb7kKDzqWavUKtBHY/QowaeUKEZGmJCoqirFjx/Lcc8/xww8/MG7cuLMeYxgGXbp04d133yUzM5OkpCQ2bNhAVlYWjz32GAAvvPAC33//fUOH3yB+/PFH1q1bR8+ePZk5cyYhISGufTfeeCMAmZmZ7gqPlStX8vjjj2M2m/n0009JTk4mMTGRDRs2EBMTw7fffsurr75ao3Pa7XauvvpqVq1axfXXX8/BgwfZuXMn69atIzU1lUOHDnHfffc1yPOpUdf6U4WFhfHLL78wadIknnnmGY4ePcqbb77JJ598wjPPPMOhQ4dc682LSPXlZTnmx/sGeOMboAZnLVFBSRnL9zj+2I3q1drN0UhjMux2TEcdr31UgBV6qKxeRKSpeeKJJyp8/fnnn5/1mCFDhrBjxw68vSumYD4+Pvz1r39l06ZN/PDDD8yYMYOxY8fWa7yNwTnl+tZbb8XHx6fCvvT0dADatGnT2GG5vPDCCxiGwe23387EiRNd2/v168err77KjTfeyEsvvcT999+PxVK999/vv/8+y5cvZ8SIEcyZMwezueI4ebt27WjXrl29Pg+nOo3IA/j6+vLVV19xzz338K9//YtWrVpx2223cezYMaZPn86+fftcnzCJSPWorF6W7c6gtMxOfEQA3WKC3B2ONKKyjAy8rDZsJujob4L2F7g7JBERAFJSUpg6dSodOnTA39+f7t278+qrr2IYBgDjxo3D19dXy01XISQk5LQk/mSjRo0CYNeuXY0VUr1asWIFAKNHjz5t37fffgvAeeed16gxOeXm5vLLL78AcNttt522f/z48YSEhJCVlVWjXgVvvPEGAM8///xpSXxDq9OIvNPq1avZuXMnhmFw/PhxWrduzYYNG2jdWqNIIrVxomO9EvmW6qeTyupNKqtuUUoPHgQgIxQSYgeBt89ZjhARaXjz589n0qRJ5Obm0qFDBzp16sSOHTt46KGHiIyM5Nxzz2Xu3LncfvvtDTYC2dwVFzve//n7e977v5KSEnbu3ImPjw+9evWqsG/nzp18/PHHmM3mak1BaAgbN26ktLQUPz8/Bg4ceNp+i8XC4MGDWbRoEatXr670w4hT7d69m507dxIREcF5553HvHnzmDNnDkeOHCE6OpqRI0dy88034+vr2xBPqW6J/Pr163nqqadYuHAh4Jj7EBUVxRtvvMHkyZP573//S2BgYL0EKtKS5LhG5DU/viUqs9lZvNNRgqb58S1P2u7NAGSGwgU9rndzNCItl2EYFJUVuTuMWvH39q/XD4HXrl3LuHHjsNlszJ49m0mTJgHwzjvvcO+99/Lpp5+ydOlSzGYzf/nLX6o8z1//+lcWLFhQ4+u/9dZbDBgwoNbxewLDMJgzZw4A559/vpujqbnt27dTVlbGgAEDKpSlz5s3j2nTppGfn88dd9xB165dz3iehvoe2b17NwDx8fFVVkV06tSJRYsWuR57NuvXrwegR48e3HzzzcyePbvC/i+++IJXXnmFhQsX0r59+2qdsybqlMgPHjwYgCuuuIIXX3yRvn37AtChQwcefPBBhg8fzoIFC4iOjq57pCItiErrW7a1+49xvNBKWICFQe3D3R2ONLK07SsIA0pDDCzdL3d3OCItVlFZEUM/HeruMGpl9aTVBFgC6u1899xzDyUlJTz88MOuJB7gjjvuYNq0aWzatIklS5Zw880306FDhyrPs2vXLn777bcaXz8nJ6c2YXuUGTNmsHHjRnx8fJg2bZq7w6mxxMREAFcyfemll7J27Vry8vIAuOGGG1xl6GfSUN8jx44dAyA8vOr3Vc59zseezZEjRwDHB10rVqzgj3/8I0888QStW7dm+fLl3HHHHezcuZNx48axZs2aei+9r9PZzj33XH799Vfmz5/vSuIB7r//fmbPns2WLVs4//zz2bdvX50DFWlJnKX1WkO+ZXJ2q7+kRyu8vRp3vpW4X17yDgB8IwPBL9TN0YhIS7dmzRrWrl1LeHj4aUujWSwWoqOjSU9Px263n7Uv1qxZszAMo8b/hg8f3oDP0P02bNjA/fffDzgasnXu3NnNEdXc5s2OarL+/fuTnp7O4sWLXUk8wKFDh9i5c+dZz9NQ3yPOaQunNuE7mbMEvqioepU4BQUFAFitVi688EJmzJhB+/bt8fX15dJLL+Xrr7/GZDKxfv165s+fX61z1kSdRuSdDQ0qM2HCBKKiorj++us577zzXJ9YiMiZGYZxYkQ+WqX1LY1hGPyyw5HIj1a3+pYpwzGqENGhm5sDEWnZ/L39WT1ptbvDqBV/7/obCHB2Ir/qqqsqLCd2qgkTJpy1bLqxPfroo64mazUxc+ZMhg0b1gARnW7fvn2MHTuW4uJiJk2axMMPP9wo161vzhH5/v3706pVK4qLi0lLS2P16tW88sorrFixgpEjR7Jr1y4iIiIaPT4/P8d76tLS0iofU1JSAlS/R4HznIDrg5iT9evXjxEjRrB48WIWLlzIVVddVZOQz6pemt1VZeTIkfz666+MGTOmIS8j0qwU5pZis9oxmSAoQol8S5OUlsfB7EJ8vc1c1C3K3eFIIyvLPULQcTsA8YOvdnM0Ii2byWSq1/J0T+UcuLviiisq3W8YBmazmenTpzdmWNWSmppKUlJSjY9zjrQ2tKNHjzJq1CiOHDnCmDFjmDVrlsc2uN28eTMmk4l+/foBjtHt+Ph44uPjueaaa+jVqxfJycn8+OOPFZZ+ayzVKZuvTvl9ZecExzz5yvTs2ZPFixezf//+akZafQ2ayAMMHDiwVvMcRFqq3AzHaHxQuB9eKqtucX7e5hiNv6BLFAE+Df4rWpqYPes+Ijzfcb/9gMvcG4yICLgS4Z49e1a6LyMjg549e1a6/1SN3ezuk08+4ZNPPqnxcY0hOzubUaNGkZyczMUXX8ycOXOqvXZ5U5OamkpmZiadOnWqtGrDx8eH/v37k5ycjN1uP+O5Gup7xFktcvDgQcrKyipteLd3794Kjz2b7t27u+5X1Zneud1ms1XrnDVRo3eJY8eO5dlnn+Wcc86p0UU6depEUVER77zzDoGBgdx11101Ol6kJVFZfcv2844Ty85Jy7M78Se6ACW+JryrOSIgItKQsrOzAQgKCjpt32OPPYbNZqt2Aqpmdw75+flceeWVbN26lcGDB/Pdd9955JJzTieX1VfGMAzWrFkD4Bqxr0pDfY84u+kXFxezYcMGhgwZUmG/1Wpl7dq1AAwdWr0mlwMGDMDPz4/i4mL27t1Lly5dTnuM88OBtm3bVuucNVGj4b5Dhw4xZMgQLr30UmbNmkVubu5Zj1m3bh3Tpk2jffv2PPXUU0RFqVRU5Exys7SGfEt1NKeYzSk5mExwaU8l8i1OaQGphw8BUNIqxGPLK0WkeXEuJZ2cnFxh+08//cQ333wDUO3fV2p255iHfc0117B69Wp69+7NwoULCQ4OdndYdeJM5Ksadf7uu+84dOgQPXv2pE+fPmc8V0N9j4SEhDBy5EgAPvzww9P2z5kzh9zcXCIjI6v9/RYYGMiVV14JwEcffXTa/qNHj/Ljjz8CcMkll1TrnDVRo0R+06ZNzJgxg+TkZKZMmUJERAR9+vThD3/4A4888gh//etfefLJJ7nnnnsYPXo0ERERDB06lHfeeYdRo0axfft2xo8fX+9PQqQ5cZbWK5FveZyj8QPiwogOrrxES5qx5MXkFjj+LFviO7g3FhGRcs5K3Oeff941d3zTpk3cfPPN+Pv7ExgYyO7du8nMzHRnmB7BZrMxYcIEFi9eTOfOnfn555+r3fjNOX/+TMv7uYuzY/28efN4+eWXKyT0CxYsYMqUKYDje8idpk+fjslk4oMPPuCzzz5zbU9MTOTBBx8EHA0ST+1s//DDD9OhQ4dKGxE+9dRTeHl58fnnn1dI5o8fP87kyZMpKiqiU6dODZID16i03mQyMWXKFCZPnsz8+fOZNWsWS5curXTuidlspm/fvlx77bX88Y9/JDY2tt6CFmnOcspL67X0XMvz07ajAIxSt/oWqWjHd5jzvACDsE6VN80REWlsjzzyCIsWLWL58uWu5mVbt27FMAxmz57NP//5T5YuXcrAgQOZPHkyzz33nLtDbhS//fYb11xzjevr/HxHg5MXX3yR119/3bV948aNxMXFAfDll1+6VgEwm81VJndt2rRhzpw5DRN4A3COyIPj++Xvf/878fHxpKSkkJaWhslk4rnnnmPcuHFujBLOP/98nn/+eZ544gkmTZrEE088QVBQEFu3bsVutzNmzBgeeuih047LzMzkwIEDlX5Y1a9fP95++23uvvtuJk+ezFNPPUWrVq3Yvn07hYWFREVF8d///veMy97VVq06KZnNZq666ipXC/0dO3aQkpJCVlYW/v7+REdH07t3b0JDtf6tSE0515APjtIc+ZYkt9jKqr1ZgObHt0i2MnbuX0R0jqMSI0xLz4lIEzF69GjmzZvHiy++yObNm0lKSqJ379689NJLXH755fTr148bb7yRjRs3ujvURmW1WsnKyjpte2FhIYWFha6vTx6ddi5vBrB79252795d6bnbt29/2rajRx0f9lc1D91dSkpK2LVrF6Ghobz++uu8/vrrJCUlsWXLFlq3bs2kSZOYOnUq5557rrtDBRyj8v369eO1115j/fr1HD16lISEBG699VbuvfdevLy8anzOO++8k969e/OPf/yDlStXsnnzZmJjYxkzZgyPPfZYg8yPBzAZhmE0yJmbmdzcXEJDQ8nJyTnjGpoidVFmtfGvqUsBmPKPC/APrv9P76Rp+i4xlamfbaRTdCCLHxru7nCkse3/jY+/voFW3wQTnwFxM94n6MIL3R2ViMepyfu14uJi9u3bR8eOHSusBy3SFF199dV89913LF26lIsuusjd4bhs2LCBc845h/PPP5/ly5e7OxyPV5PfS/WyttGqVavYuXMnVquVuLg4LrzwQldjDBGpvrzyRncWXy/8gjxzCRKpnZ+3q1t9i5a0gC0+Pkw87vjS0q6dW8MREZGmZeXKlQwaNKhJJfFwoqw+ISHBzZG0PHVK5FNTUxk/fjyrVq0CHEsLmEwmfHx8uPnmm3n22Wdp06ZNvQQq0hI4y+pDovzVsboFsdrsLElKB2C0EvmWxzBg53z244OfFQyTCUsDleGJiIjnSUpKIjMzkzfffNPdoZzGmcifrRu91L86JfJTp05l5cqV9O3blxtvvJHIyEgOHjzIDz/8wAcffMC3337LnDlzuFDlgSLV4lpDXvPjW5TVe7PJKy4jKsiH/nFaO7zFydhJTs4ByrzbAja8Y1phboCmOCIi4pm6d+9OU50N7exYrxH5xlenRP7nn38mISGBdevW4e194lTPPPMMS5cu5ZZbbmHMmDGsW7eObt3UuEfkbHIytfRcS/TzdkcDm0t7xOBlViVGi7NzPlt9fWiV5XiT5tsuzs0BiYiIVI8zkdeIfOOr0Tryp7Lb7YwePbpCEu908cUX8+uvv2IymXj22WfrchmRFiPvpNJ6aRkMw9D8+JYuaQFbfH1oddzxpSVOibyIiHiGzMxMDMMgIiLC3aG0OHVK5Hv06EFaWlqV+zt06MD111/PTz/9VJfLiLQYOSqtb3G2peaSmlOMv8WLC7pGuTscaWy5R+Dwerb5+BJz3DEib4lTozsRERE5szol8rfeeitff/01e/furfIxUVFRFBUV1eUyIi2CYRiuOfKh0RqRbyl+Kh+Nv7BrFH6Wmq9dKh4uaQEGsCUg0DUi76MReRERETmLOiXyRUVFxMXFMWLECH777bfT9ttsNn7++WcGDRpUl8uItAjFBVasxTYAgiM1It9SOMvqR/du7eZIxC2SFpDm5UWWyX5iRF5Lz4mIiMhZ1KnZ3aOPPorJZMIwDC666CLOPfdcRo8eTXx8PDk5OXz22WekpqYyc+bM+opXpNnKzXDMjw8M9cFbI7MtwqHsQnYcycVsgkt6tHJ3ONLYSvJg3//Y4uuDl80gMs+x2UeJvIiIiJxFnRL5b7/9lk2bNrFx40Y2btzIypUrWblyZYX1r8877zy+/vprkpOT6d+/P126dKlz0CLNUW5W+fx4ldW3GL/scIzGD+oQQUSglhtrcfb8ArZStoR1ICrHhtkAk58fXlHqlSAiIiJnVqdEfuzYsYwdO9b1dU5ODhs3bqyQ3K9evZrffvvNldwHBgbSt29fli9fXrfIRZqZXC091+K4yurVrb5l2rkAgG3BEcQcTgfAJ65dhQ/DRURERCpTp0T+VKGhoQwfPpzhw4e7tpWUlLBlyxZXcr9hwwY2bdpUn5cVaRZyM5TItyQ5hVZW78sGtOxci2Szwu4fsQPbbPmcd9yx2aI15EVERKQa6jWRr4yvry+DBg2q0PDOMIyGvqyIx8lxrSGvRnctwZKkdGx2g24xQbSPDHR3ONLYDqyA4hz2B0dTYCsmNtcLsKvRnYiIiFRLnbrW15bKBkVOp9L6luWn7UcBjca3WEmOsvot8f0A6FwQBDhK60VERETOxi2JvIhUZLPZyT9WAkCoEvlmr6TMxtKkDABG99Kycy2OYbjmx28JigSgda7jz7FK60VERKQ6lMiLNAH52SUYdgMvi5mAEHUvb+5WJGdRUGojJsSXhLah7g5HGlvaVsg5CN7+bCvLASA4sxDQiLyIiIhUjxJ5kSbAVVYf6YfJrKknzZ2zW/3InjGY9Xq3PDvnA1DaaTg7j+8isMjAK9/xO0Bz5EVERKQ6lMiLNAGaH99y2O0Gv5Qn8pof30KVJ/K72p9Dmb2MzoXBAHhFRWH21+8AEREROTsl8iJNgBL5lmPz4RzS80oI8vVmWOdId4cjje34ITi6GUxmtgSFATCwLBYAH43Gi4iISDUpkRdpAnK19FyL8XN5t/qLu0Xj6+3l5mik0SX94LiNG8rWvP0AdCsOA8ASp0Z3IiLi+dLT03nwwQfp2rUrvr6+BAcHM2zYML744gt3h9asKJEXaQI0It9y/LRNZfUtWpKjrJ7uV7I1cysAbXO9ATW6ExHxFIZhsHz5ch555BHOPfdcwsLC8PHxITY2lnHjxrFkyZKznmPBggWMHDmSiIgIAgMDGThwIG+99RZ2u70RnkHD2bNnDwMGDOC1114jJSWFHj16EBoayqpVq5gwYQJffvml22Lbt28fM2bM4Pbbb6dfv354e3tjMpl44YUX6nxud7yeSuRFmoCc8kQ+NFqJfHO2P7OA3en5eJlNjOjeyt3hSGMrOg77lwOQ33kE+3L2ARCS6Wx0pxF5ERFPsHjxYi688EJefvll1q5dS0xMDH369CEvL4+vv/6aSy65hCeffLLK41966SXGjBnDokWLCA8Pp0uXLiQmJnLfffdx3XXXeWwyb7fbmThxIqmpqUyaNIm0tDQSExNJSUnhwQcfBGDmzJlui++NN97gjjvu4IMPPmDz5s3YbLZ6Oa+7Xk8l8iJuVlJopaSgDIDgSJXWN2fObvXndoogNMDi5mik0e35BexlEN2D7UYRBgaxgbGQ6vi+sLRr6+YARUSkOgzDoEuXLrz77rtkZmaSlJTEhg0byMrK4rHHHgPghRde4Pvvvz/t2JUrV/L4449jNpv59NNPSU5OJjExkQ0bNhATE8O3337Lq6++2thPqV78+OOPrFu3jp49ezJz5kxCQkJc+2688UYAMjMz3RUeUVFRjB07lueee44ffviBcePG1fmc7nw9lciLuJlzfrx/sAUfP283RyMNyZnIj+qpsvoWaeeJsvotmVsA+P/27js8qjL9//h7Jpl0kpBA6AHpvTc7KrIiKpZlKa67C+6666pr96ti11V3f67KWndtqAiWVVAUe6+A0nuHQID0TNpMppzfH5MZEkhnSibzeXnNNTPnnDlz40lg7nnu536GpA3CkZ0NQIzmyIuIhIWxY8eyefNmrrzyStq2bevbHhMTw4MPPsjkyZMBeO6554557QMPPIBhGPzxj39k5syZvu3Dhg3zJXwPP/wwDocjwH8K/1uyZAkAs2fPJiYmpsa+nJwcADp16hTssHzuuOMOli5dyp133sk555xDUlLScZ8zlNdTibxIiFnzNT8+EhSUVfLz3gIAJmp+fORxVnpG5AH6T/HNjx9hygSnE5PFQnSGpluISMu2f/9+rrnmGnr06EF8fDz9+vXj0UcfxTAMAC655BJiY2PZv39/iCMNrOTkZKKj6x58OfvsswHYtm1bje1Wq5XPPvP8W3D55Zcf87pp06aRnJxMfn5+o+bZtzQ//PADAJMmTTpm33vvvQfASSedFNSYAinU11OJvEiIWXO9HeuVyLdmn28+jNuAgZ2S6do2IdThSLDt+RbsVkjqCJ1HsiHfk8gPsHlGcixdumCK0ioGItJyffDBBwwaNIgnn3wSk8lEz5492b59OzfeeCOvvPIKW7duZcmSJcyePZuuEb6cps1WVW0ZX/Oz3erVq6msrCQuLo6RI0ce8zqLxcKYMWMAWL58eeAD9SO73c6WLVuIiYlh4MCBNfZt2bKFV199FbPZ7Jdy9pYi1NdTdbwiIXakY73mx7dmvrJ6jcZHJl9Z/Tnk2Qs4VHYIs8lMF6uFArT0nIi0bCtXruSSSy7B5XLx2muvMWvWLACeeuoprr76ahYuXMjXX3+N2Wzm1ltvrfM8Dz74IMuWLWvy+z/xxBOMGDGi2fEHk2EYvPXWWwCcfPLJNfZt374dgMzMzDpH9Hv27Mnnn3/uOzZcbNq0CafTyYgRI7BYjvQBevfdd7nuuusoLS3liiuuoE+fPvWeJ5x+RkJ9PZXIi4SYlp5r/SoqXXyzPRdQIh+RDOPI+vH9jpTV90zpiWmXZ86gGt2JtDyGYWBUVIQ6jGYxxcdjMpn8dr6rrroKu93OTTfd5EviAa644gquu+461qxZw5dffslll11Gjx496jzPtm3b+P7775v8/sXFxc0JOySee+45Vq9eTUxMDNddd12NfYWFhQA15tUfzbvPe2y4WLt2LYAvmT7rrLNYuXIlJSUlAEyfPp158+Y1eJ5w+hkJ9fUMi0S+tLSUO+64gzfffJOCggL69+/PrbfeyowZM5p0njvuuIO///3vDBo0iA0bNgQoWpGmseartL61+25HHjaHmy6p8QzqnNzwC6R1yV4NJdlgSYQTTmP9Bk/zo8HtBuP4xjOPNEZLz4m0OEZFBVtHjgp1GM3Sb9UvmBL8M41rxYoVrFy5krZt2x6zpJrFYqF9+/YcPHiQqKgoX8f2usyfP5/58+f7Ja6WaNWqVVx77bWApwlar169auz3ltwf3QiuutjYWAAqwuxLpHXr1gEwfPhwcnJy+OKLL2rsz8rKYsuWLQwfPrze84TTz0ior2dYJPIXX3wxK1eu5OGHH6Zv374sXLiQmTNn4na7a3wrWJ81a9bwyCOP0KGDRsOk5XC7jWrN7lRa31p9uukQABMHZPh1hETCxNaqEsHeZ4ElzjciP6TdECr3vw2ApVtkzycVkZbL24n8/PPPr7Gc2NFmzJjRYNl0sN1yyy2+JmtN8dJLL3HiiSc26TW7d+/mvPPOw2azMWvWLG666aZjjomL83zWq6ysrPM8drsdOHZ+fUvnHZEfPnw4GRkZ2Gw2Dh8+zPLly/nXv/7FDz/8wMSJE9m2bRtpaWkhjtY/Qn09W3wiv2zZMj799FNf8g5wxhlnsHfvXm6++WamT59OVAMNgpxOJ7Nnz+bPf/4za9euDen6hSLVlRXZcTsNzGYTSW2VyLdGLrfB55s95dOTBnUMcTQSEluqEvn+52EYhi+RH9RuEI4sT5mhlp4TaXlM8fH0W/VLqMNoFpMfkwZvJ3LvkmpHMwwDs9nM3Llz/fae/pKdnc3WrVub/LqysrImHX/o0CHOPvtsDh48yJQpU5g/f36tX9w3psy6MeXaLdG6deswmUwMGzYM8IxEZ2ZmkpmZydSpUxk4cCA7d+7k448/rrFMWzgL9fVs8Yn84sWLSUpKYtq0aTW2z549m1mzZrF8+fIGlzF4+OGHKSgo4O9//zvnnXdeIMMVaRLv/Pik9DjMZo3Utkar9xWSX1ZJclw0Y09oHd9ASxMU7oGcjWCKgj5nk1WShbXSSow5ht6WLuwq8CxJaInwDs8iLZHJZPJbeXo48ybCAwYMqHVfbm4uAwYMqHX/0YLdyGzBggUsWLCgya9rioKCAs4++2x27tzJ6aefzltvvVWj2Vt13oqFffv24XQ6a22QtmvXrhrHhoPs7Gzy8vLo2bNnrVUbMTExDB8+nJ07d+J2u+s9Vzg1uwv19WzxifyGDRsYMGDAMf9jhg4d6ttfXyK/adMmHnjgAd555x2SkpICGqtIU3kT+RSV1bda3m71Z/TPwBKlFT8jjnc0vvtJkJDGhl0/AdA/rT/GQc/PRlRKClFt2oQqQhGRehVUfeFY2+fo2267DZfLVWfierRwamTWGKWlpZx77rls2LCBMWPGsHTp0npLqL0d3W02G6tWrWLs2LE19jscDlauXAnAuHHjAhq7P1Uvq6+NYRisWLECwDdiX5dw+hkJ9fVs8Z8q8/Pza51H4d2Wn59f52vdbjdz5szh4osv5txzz23S+9rtdqxWa42biL9Z89TorjUzDIMPN3jmx6tbfYTaXDU3s/8UANbnrQeqGt3t9zS609JzItKSJSYmArBz584a2z/55BMWL14M0Oj+L/Pnz/esBtDE24QJE/z6Z/IHu93O1KlTWb58OYMGDeKjjz6iTQNfyiYnJzNx4kQAXnjhhWP2v/XWW1itVtLT01vkn7ku3kTe5XLVun/p0qVkZWUxYMAABg8eXO+5wulnJNTXs8Un8lD/Xw717Xv00UfZvn07jz/+eJPf86GHHiIlJcV366YPWhIAWnquddtwwMq+gnLiLGbO7J8R6nAk2KzZsO9Hz+OBUwF88+MHtxtMZVYWoEZ3ItKyjRrl6dx///33++aOr1mzhssuu4z4+HgSExPZvn17RPWgcrlczJgxgy+++IJevXrx6aefNrqB29y5czGZTDz//PMsWrTIt33t2rXccMMNgKdJ39Gd0L3z7utb3i9UvB3r3333XR555JEaCf2yZcuYM2cO4PkZCkc33XQTPXr0qLWBYXOvpz+0+EQ+PT291lF3b5lPXb80+/bt46677uLuu+8mJiaGoqIiioqKcDqduN1uioqK6l0G4LbbbqO4uNh3y6r6wCXiT0rkW7cP1h8E4Mz+GSTEtPiZTOJvm6pG47uNh+TOONwOthRsAapG5LO09JyItHw333wzJpOJ7777jszMTEaMGMGYMWPIz8/npZdeYvTo0ZSWljJy5EjuuuuuUIcbFG+++aavm7/ZbGbatGmccsopx9yO7vEFcPLJJ3P//ff7Vt/q1asXw4YNY+TIkRw+fJgpU6Zw4403BvlPdHy8I/Lg+Xnp1KkTo0ePpmPHjkyZMoWCggLuu+8+LrnkkhBGCd9//z3t2rXz3V5//XXAM4BbffvReV9eXh579+6t9cuqUF7PFv/JcsiQISxatOiYBgLr11eVJ9ZRnrFr1y4qKiq49tprfes5Vte2bVuuvfbaOkfrY2Njfev+iQRKsa+0XnPkWxvDMPhgfTYAU4Z0DnE0EhIbPSWnDLoIgJ1FO7G5bLSxtKF7cnf2768akVejOxFpwSZNmsS7777LQw89xLp169i6dSuDBg3i4Ycf5pxzzmHYsGFceumlrF69OtShBo13STGA7du3s3379lqP6969e63b586dy7Bhw3jsscf45ZdfOHToEEOGDGH27NlcffXVta7IdeiQZ6peQ+uwB5vdbmfbtm2kpKTw+OOP8/jjj7N161bWr19Px44dmTVrFtdccw3jx48Pdag4HI5aB4jLy8spLy/3Pa9rikBdmnM9/cFkGIYRkDP7yYcffsi5557L66+/zvTp033bJ0+ezLp169i3b1+t/3OKiopYs2bNMduvu+46iouLeemll+jatSu9e/duVBxWq5WUlBSKi4vrXUNTpLEcdhf/vfZrAC7/16nEJTauUYyEh/X7izn/ye+Is5hZdefZGpGPNNZseLSqg/MNmyG5M//b9j/u/fFexnUax/OTnmfnuVOo3LWLzBdfILGB1VdEpHGa8nnNZrOxe/duTjjhBN960CIt1QUXXMDSpUv5+uuvOe2000Idjs+qVasYNWoUJ598Mt99912owwl7Tfl7qcV/spw8eTJnn302V155JVarld69e7No0SI++ugjFixY4EviL7/8cl5++WV27txJ9+7dSU1NrbWpQGpqKk6nM6waSEjrZM33lNXHJkQriW+F3q8ajT+rfwcl8ZFo07ue+8wTIdlTkeGdHz+k3RAMtxvHgQOAmt2JiEjDfvzxR0aPHt2ikng4UlY/ZMiQEEcSecLi0+U777zD3LlzueuuuygoKKB///4sWrSIGTNm+I5xuVy4XC5aeIGBiI861rdehmGwrGp+/LlDOoU4GgkJb1n9wAt9m6p3rHfm5mHY7RAVhaVjxxAEKCIi4WLr1q3k5eXx73//O9ShHMObyDfUjV78LywS+aSkJObNm8e8efPqPGb+/PnMnz+/wXN99dVX/gtM5DhYc72N7lTO19qsP1BMVkEF8ZYozujfPtThSLAVH4Cs5YAJBl4AQLmjnJ1FnqWbBqcPxrGlan58p06YGrn+soiIRKZ+/fq12MFKb8d6jcgHX4vvWi/SWvk61qdrRL61Ubf6COcrqx/vK6vfUrAFl+EiIz6DDokdjiw9p0Z3IiISxryJvEbkg0+fMEVCxJfIt1ci35oYhsEH6zyJ/JShKquPSJuWeO6rutXDkfnxg9oNAjiy9JzWkBcRkTBW25JsEhwakRcJEWu+lp5rjdYfKGZ/YVVZfb+MUIcjwVa8/0hZ/YALfJurN7oDcOz3JPIWrSEvIiIizaBEXiQEDMOoNkdeI/KtiXc0/swBGcTHBGbdUGnBNr3nuc88EZKPVGRUb3QHULlfI/IiIiLSfErkRUKg3FqJ0+HGZII2aRqRby0Mw/DNj5+ibvWRydutftCFvk1FtiL2l3oS9yOl9VVz5LX0nIiIiDSDEnmREPAuPZfYNpaoaP0athbr9qusPqIV74f9KzimrD7fU1bfI7kHyTHJuG02nDk5gJrdiYiISPMogxAJAW+juxSV1bcqvm71KquPTL5u9fWX1TsOHADAnJhIVGpqUEMUERGR1kGJvEgI+DrWK5FvNap3qz9PZfWRaeMSz321bvUAG/M2AtUSeW+ju27dMJlMQQtPREREWg8l8iIhoES+9Vm7v5gDRRUkxEQxQWX1kaco60hZ/cAjZfWGYRzb6E5Lz4mIiMhxUiIvEgLeOfLJ7dXorrVY5i2r76+y+ojkLavvfhK06ejbfKjsEAW2AqJN0fRP6w9Ua3SnpedERESkmZTIi4SAb0Q+XSPyrUGNsvqhKquPSJuWeO6PKqv3jsb3aduH2KhY4MjSc5auXYIWnoiIiLQuSuRFgszlcFNaZAdUWt9aqKw+whVlwf6VHN2tHo4k8kPaDfFtc/jWkNeIvIiIiDSPEnmRICspsIEB0bFRxLexhDoc8YMP1mUDcNaADsRZVFYfcXxl9SdDmw41dq3KWQXAsIxhgKd6Q6X1IiIicryUyIsEWbFv6bk4daxuBQzDYNn6QwBMGdKxgaOlVdq42HM/6MIam8sd5WzK2wTA6A6jAXAVFuIuLweTCUuXzsGMUkRERFoRJfIiQWbN9STybTQ/vlVYk1WksvpIVrQPDvxMbWX1a3LW4DScdEnqQuckT9LuHY2PzsjAHBsb7GhFRESklVAiLxJkVt+IvBL51sDbrV5l9RGqnrL6lYdXAjCqwyjfNu/ScxYtPSciIiLHQYm8SJBZ87X0XGtRs6xe3eoj0sYlnvujyuoBVh7yJPJjOo7xbfM1utP8eBERaaKcnBxuuOEG+vTpQ2xsLG3atOHEE0/kjTfeCHVoEgJK5EWCzLf0nEbkw563rD4xJooJ/dqHOhwJtsK9dZbVlzvK2Zi3EaiZyFfur2p0pxF5EZGAWLZsGRMnTiQtLY3ExERGjhzJE088gdvtDnVox2XHjh2MGDGCxx57jP3799O/f39SUlL46aefmDFjBm+++WZI49u9ezfPPfccf/rTnxg2bBjR0dGYTCYeeOCB4z53a72mx0uJvEgQGYbhmyOvRD78edeOV1l9hPKW1fc45Ziy+jW5nvnxnRI70SXpyHrxjiwtPSciEigPP/wwU6ZM4fPPP6dt27b07t2btWvX8re//Y2LLroobBM/t9vNzJkzyc7OZtasWRw+fJi1a9eyf/9+brjhBgBeeumlkMY4b948rrjiCp5//nnWrVuHy+Xyy3lb6zX1ByXyIkFkL3NSafP8xZacrtL6cOYpq/ck8ueqrD4ybVriuR849ZhdPx/6Gag5Gg9UW3pOI/IiIv70448/cvvtt2M2m1m4cCE7d+5k7dq1rFq1ig4dOvDee+/x6KOPhjrMZvn444/5+eefGTBgAC+99BLJycm+fZdeeikAeXl5oQoPgHbt2nHeeedx33338eGHH3LJJZcc9zlb8zX1ByXyIkHkXXouISWG6BiN4Iaz1VlFZBfbVFYfqQr3woFfwGQ+pqwe4OfDnkTeu+wcgOFw4Djk6amgRF5ExL8eeOABDMPgj3/8IzNnzvRtHzZsmC/Ze/jhh3E4HKEKsdmWLFkCwOzZs4mJiamxLycnB4BOnUI7qHDHHXewdOlS7rzzTs455xySkpKO+5yt+Zr6gxJ5kSBSx/rWY5nK6iNbPd3qK5wVrM9bD8DojkcSecfBg+B2Y4qNJbq9vvwRkfCzf/9+rrnmGnr06EF8fDz9+vXj0UcfxTAMAC655BJiY2PZX9XYM1isViufffYZAJdffvkx+6dNm0ZycjL5+fl8+eWXQY3NH3744QcAJk2adMy+9957D4CTTjopqDEFWmu/pv6gRF4kiNTornVwu4+U1U8ZqrL6iLRxsee+lm71a3PX4nQ76ZDQga5JR0beK7OONLozmUzBiFJExG8++OADBg0axJNPPonJZKJnz55s376dG2+8kVdeeYWtW7eyZMkSZs+eTdcgVx2tXr2ayspK4uLiGDly5DH7LRYLY8Z4pjotX748qLEdL7vdzpYtW4iJiWHgwIE19m3ZsoVXX30Vs9nsl1L2lqQ1X1N/iQ51ACKRxJpXtfRcO82PD2dr9h8pqz+9r0ZWI07hXsheVWdZffVl56on7L5Gd1p6TkTCzMqVK7nkkktwuVy89tprzJo1C4CnnnqKq6++moULF/L1119jNpu59dZb6zzPgw8+yLJly5r8/k888QQjRoyoc//27dsByMzMJDq69vSmZ8+efP75575jw8WmTZtwOp2MGDECi8Xi2/7uu+9y3XXXUVpayhVXXEGfPn0aPFeg/v8HQmu+pv6iRF4kiDQi3zp4u9VPHKiy+ojkbXLX/WRIyjhmt7fRXfX58QCO/Wp0JxJODMPAWRmeHbGjY8x+rfy56qqrsNvt3HTTTb4kHuCKK67guuuuY82aNXz55Zdcdtll9OjRo87zbNu2je+//77J719cXFzv/sLCQgDatm1b5zHefd5jw8XatWsBfIn0WWedxcqVKykpKQFg+vTpzJs3r1HnCtT//0BozdfUX5TIiwSREvnwV72sXt3qI9TGJZ77WsrqbU6bb3780R3rK31LzymRFwkHzko3/73261CH0SxXzDsdS6x/vmhesWIFK1eupG3bttx555019lksFtq3b8/BgweJioritttuq/dc8+fPZ/78+X6JqzqbzVPxeHQjuOpiY2MBqKio8Pv7B9K6desAGD58ODk5OXzxxRc19mdlZbFlyxaGDx/e4LkC9f8/EFrzNfUXzZEXCRK3y01JgR1QIh/OVmcVcVBl9ZGrcE+9ZfXrctfhcDvIiM+gW5uaJfSOquZPFq0hLyJhxNsx/fzzz6+x7NnRZsyY0ajy7kCIi/NMWaysrKzzGLvd8xksPj68PoN5R+SHDx9ORkYGNpuNvXv38uabbzJu3Dh++OEHJk6cSEFBQYgj9a/WfE39RSPyIkFSWmjHcBtERZtJTKn720Vp2VRWH+Gqd6uvpax+5WHP/PjRHUcfU9Za6U3kVVovEhaiY8xcMe/0UIfRLNEx/hur83ZMnzx5cq37DcPAbDYzd+5cv71nUzWmxLoxpdot0bp16zCZTAwbNgzwjEJnZmaSmZnJ1KlTGThwIDt37uTjjz+usURbuGvN19RflMiLBEmxr6w+DpNZHavDkdtt8OGGqm71KquPTL5u9RfVuts3P75jzfnxruJi3FVzDGOUyIuEBZPJ5Lfy9HC2detWAAYMGFDrvtzcXAYMGFDr/qMFqtmatxJg3759OJ3OWpuj7dq1q8ax4SA7O5u8vDx69uxZazVETEwMw4cPZ+fOnbjdDfdzCKdmd631mvqTEnmRILHmehL5NumRWf7TGqzOKuRgsY2k2GhOU1l95CnYDdmr6yyrt7vsrMv1zGUc0+Go+fFVo/FR6emYExICH6uIiJ94S7aTkpKO2XfbbbfhcrlqdFOvT6CarXk7uttsNlatWsXYsWNr7Hc4HKxc6amYGjduXJPfP1Sql9XXxjAMVqxYAeAbsa9PODW7a63X1J80R14kSKz5nqYdKVp6Lmy9uyYbgLMGZKisPhJt+J/nvvvJkHTsFznrctdR6a6kfXx7uid3r7HPsf8AoNF4EQk/iYmJAOzcubPG9k8++YTFiz1VSo3tkD9//nwMw2jybcKECfWeNzk5mYkTJwLwwgsvHLP/rbfewmq1kp6e3uC5WhJvIu9yuWrdv3TpUrKyshgwYACDBw9u8HyB+v8fCK31mvqTEnmRIPF1rG+vEflwZHO4WLLak4xdMlLJWMRxu2H1As/j4bNqPaT6snNHf6j1LT2nRnciEmZGjRoFwP33309ZWRkAa9as4bLLLiM+Pp7ExES2b99OXl5eKMNk7ty5mEwmnn/+eRYtWuTbvnbtWm644QYAbrnlllq7oM+fPx+TyVTv0nmh4O1Y/+677/LII4/USOiXLVvGnDlzAM+1CVc33XQTPXr04Kabbjpm3/Fc00igRF4kSLyl9epYH54+2nAIq81Jl9R4TundLtThSLDt+dbTsT6mDQycWush1RvdHa0yy5vI60sgEQkvN998MyaTie+++47MzExGjBjBmDFjyM/P56WXXmL06NGUlpYycuRI7rrrrpDFefLJJ3P//ffjdruZNWsWvXr1YtiwYYwcOZLDhw8zZcoUbrzxxpDF1xzeEXnwXIdOnToxevRoOnbsyJQpUygoKOC+++7jkksuCWGUHt9//z3t2rXz3V5//XUAHnrooRrbs6r+PfTKy8tj7969tX4R1BqvqT8pkRcJEmuep7ReiXx4emOl5x+eaaO7Ylazwsiz+lXP/ZBfQ0ziMburz4+vLZF3eNeQ76oReREJL5MmTeLdd9/lxBNPxG63s3XrVgYNGsT777/P9OnTefbZZxk5ciT7q3qBhNLcuXNZunQpZ555Jvn5+ezYsYMhQ4bw+OOP8+677xIVVfu0uEOHDgF1z0UPBbvdzrZt20hJSeGll15i2LBhlJSUsH79emJjY5k1axY//PADd955Z6hDBTxz1vPz830379Jw5eXlNbbXNU2gLs29ppFAze5EgqCywomtzAF4utZLeNmbX8aPu/IxmWDaaCViEaeiEDa953k88rJaD1mfux67y056XDonJJ9wzP5Kb2m95siLSBg6//zzOf/882vd179/f3755ZcgR1S38847j/POO69Jr/Eusect124JNm7ciNPpZPDgwfzhD3/gD3/4Q6hDqteECRMwDKPJr5s/fz7z58+v95jmXNNIoBF5kSCw5nvK6uOSLMTE6fuzcPPmz54k7NQ+7emSqoqKiLPuLXDZocNg6Dyy1kN+Pnxk2bmj58cbLheObM+yhTEqrRcRaXF+/PFHRo8ezWmnnRbqUHy8ZfVDhgwJcSTSUimjEAkCa67K6sOV0+Xmf794ygWnazQ+8hgGrHrF83jEZVBHZ2Zvo7ujl50DcB4+DA4HWCxEd+gQsFBFRKTptm7dSl5eHv/+979DHUoN3kS+Md3oJTIpkRcJguKqjvVaei78fL0tl8NWO2mJMUwcmBHqcCTYDq6Bw+shKgaG/qbWQypdlazN9Xzgqr3RXdX8+M6dMUXwXD4RkZaoX79+zSoJDzRvx3qNyEtdVFovEgTepefaaEQ+7Hib3F00ogux0UrCIs6qqiZ3A86HhLRaD9mQtwGby0ZaXBo9U3oes9+h+fEiItJE3kReI/JSF43IiwSB1Tcir0Q+nOSU2PhiSw4A08eorD7iVJbD+v95Ho+ovckdHJkfP6rDqGPmx4OWnhMRkaarbTk2keo0Ii8SBEeWnlNpfTh5Z9UBnG6DEZmp9O3QJtThSLBtfg/sxZCaCSecXudhKw951o8f0/HY+fEAjv0HAIjppi+DRERExD+UyIsEmOE2fF3r1ewufBiGwZtVZfVqchehvGX1Iy4Dc+3/XDpcDt/8+Noa3QE4vCPyWkNeRERE/ESJvEiAlRXbcTsNzGYTSW1jQx2ONNLKPYXsyisjISaK84Z1DnU4Emz5O2Hvd4AJhs+q87CN+RupcFbQNrYtvVJ71XpM5f6qZncqrRcRERE/USIvEmDe+fFJabGYo/QrFy68Te7OG9qJpFi1E4k4q6tG43tPhJS6E3BvWX1t68cDuMvKcOXnA2p2JyIiIv6jrEIkwIq1hnzYsdocLFt/EIDpYzJDHI0EncsJaxZ6Ho+su8kd1Gx0V5vKqvnx5pQUopKT/RejiIiIRDQl8iIB5psf316JfLhYujabCoeL3hlJjMxMDXU4EmzbP4HSw5DQDvpOrvMwh9vB6pzVQD2N7g5UldVrNF6kxWqJa4iLSGRqyt9HSuRFAkxLz4Wf6k3uaiuXllbOW1Y/bAZEx9R52Kb8TVQ4K0iJTaF3au9aj/E1ulPHepEWJyoqCgCHwxHiSEREPLx/H3n/fqqPEnmRACvO8STybdK19Fw42HzQytr9xViiTFw0skuow5FgKzkE2z72PB75u3oP9c2P7zAas6n2f04r9+4DIKarfpZEWhqLxUJsbCzFxcUalReRkDMMg+LiYmJjY7FYLA0erw5OIgHkdhvk7y8FIL1LUoijkcbwNrmbOKAD7ZK0ykDEWbMQDBd0Gwft+9V7qHd+/OgOo+s8xrZ5MwCxffv6L0YR8Zt27dpx4MAB9u/fT0pKChaLRZVYIhJUhmHgcDgoLi6mtLSULl0a9+W/EnmRACo8WIbT4cYSG0Vqh4RQhyMNsDlcLFnjaU42fYxKoSOOYRwpqx9Rf5M7p9vJ6sP1z483nE5fIh83eLD/4hQRv0muakKZl5fHgQMHQhyNiESy2NhYunTp4vt7qSFK5EUCKHdfCQDtM9tgNusb/pbuk02HKSp30DkljlP7tA91OBJse7+Hgl0QkwSDLqr30M35myl3lpMck0yftn1qPca+cxeGzYY5IYGYHj0CELCI+ENycjLJyck4HA5cLleowxGRCBQVFdWocvrqlMiLBFDOHisA7bu3CXEk0hjeJne/Ht2NKH3xEnlWVY3GD74YYuufCrPysGd+/KgOo+qcH2/bsAGAuEGDMJnVkkakpbNYLE3+IC0iEir6ZCESQDlVI/IZSuRbvKyCcr7bkYfJBNNGaamwiFNRBJve9TweUX+TO4CfDzVifvzGqkReZfUiIiLiZ0rkRQLE5XKTl+VpdJeR2bi5LhI6b/3sGY0/uVc7uqWpn0HE2fA/cFZA+wHQte7kHDzz41flrALqnh8PULFhIwDxQ5TIi4iIiH8pkRcJkILsMlxONzHx0aRkaA35lszlNnjrl/2AmtxFLG9Z/cjLoIGO1VsLtlLmKKNNTBv6tq29G71RWYl9yxZAI/IiIiLif0rkRQIkd++RsnotZdOyfbM9l4PFNlITLEwa1CHU4UiwHVwHB9eA2QJDZzR4uHf9+FEZo4gyR9V6jG37dozKSszJyVi66cshERER8S8l8iIBcnivp9Gd5se3fN4mdxeN6EJsdO2JmbRi3iXn+k+BxPQGD/c2uhvdsZ758d6y+sGD9EWeiIiI+J0SeZEA8Y7It9f8+BYtr9TOZ5sPAyqrj0iOClj3hufxyPrXjgdwuV2sOuyZH19/Iu/tWK+yehEREfE/JfIiAeByuMk/UNXoTiPyLdriVQdwuAyGdUulf0d96RJxNr8PtmJI6QY9z2jw8C2FWyh1lJJkSaJ/2/51HlehjvUiIiISQErkRQIg70ApbpdBXKKFNulxoQ5H6mAYBm9UdaufPlqj8RFp9Sue++GXQh3z3avzLjs3ssPIOufHu+127Nt3AJ7SehERERF/UyIvEgC51ebHa35sy7VqXxE7ckqJt0Rx/rBOoQ5Hgq1gN+z+BjDBiEsb9RJvIj+mQ93Lztm3bAGHg6i0NKI7d/ZHpCIiIiI1KJEXCYAcb8f6HirVbsneWLkPgClDO9EmzhLiaCToVi/w3Pc6A1IzGzzc5XbxS84vQP3z4yu88+PV6E5EREQCRIm8SADk+BrdaX58S1Vqd/L+uoOAmtxFJJcT1iz0PB7RcJM7gG2F2yipLCHRkkj/tLrnxx/pWK/58SIiIhIYSuRF/MxR6aLgYBkAGd01It9S/e/nLMorXfRsn8jo7m1DHY4E25b3oSQb4tM8y841wvfZ3wMwImME0eboOo/zdaxXIi8iIiIBokRexM/y95diuA0SkmNITI0JdThSi0qnm/98swuAy085QeXPkcYw4Nt/eR6P/RNExzbqZR/v+RiAMzPPrPMYd3k59p07AS09JyIiIoGjRF7Ez3LU6K7FW7x6PweLbXRIjuXXo7qGOhwJtu2fwqF1YEmEcX9p1Ev2FO9hS8EWokxRTMycWOdxti1bwO0mOiMDS4cMf0UsIiIiUoMSeRE/y9lTNT9eZfUtktPl5pmvPCOmfzq1J7HRDS85Jq2IYcC3j3gej5kDCWmNetlHez4CYHzn8bSNq3sqhsrqRUREJBiUyIv4WfUReWl5Plh/kD355aQlxjBrXMOdyqWV2fMdZC2HqFg48epGv8xbVn9Oj3PqPa56x3oRERGRQFEiL+JHlTYnhYfLATW6a4ncboOnv/SMxs85uQcJMXU3LJNWyjsaP/IyaNOxUS/ZUbiDHUU7sJgt9c6PB7Ct9yTy6lgvIiIigaREXsSP8rJKwICktrEkJKvRXUvz2ebDbD1cQpvYaC47sUeow5Fg2/8L7PoKTFFw0t8a/TJvWf3JnU8mOabuL+hcpaVU7t4NQNwgjciLiIhI4CiRF/Ej7/rxGo1veQzD4KkvdwBw2YndSYm3hDgiCTrvaPzQ6dC2e6NeYhiGr6z+Vyf8qt5jbRs3ARDduRPR6enNj1NERESkAWGRyJeWlnLdddfRuXNn4uLiGD58OK+//nqDr3vnnXeYOXMmvXv3Jj4+nh49enDppZeyffv2IEQtkcibyLfP1Pz4lua7HXms3V9MnMXM5aecEOpwJNgOb4StywATnHpDo1+2tXAre6x7iI2K5YxuZ9R7rLfRXbyWnRMREZEAC4sJohdffDErV67k4Ycfpm/fvixcuJCZM2fidruZNWtWna/7xz/+QceOHZk7dy49e/YkKyuLBx98kJEjR/LTTz8xSKWP4mc5e6oa3fVQIt/SPPmFZzR+5thM0pMat264tCLedeMHToV2fRr9so92e8rqT+1yKomWxHqPtW2sanQ3ZEjzYhQRERFppBafyC9btoxPP/3Ul7wDnHHGGezdu5ebb76Z6dOnExVV+/JRS5cuJSOj5jq+Z555Jj169OCxxx7j+eefD3j8EjlsZQ6KcysAyMhUaX1L8vOeApbvLsASZeKK03qGOhwJtvydsHGx5/GpNzb6ZYZh+ObHN1RWD1CxYSMA8epYLyIiIgHW4kvrFy9eTFJSEtOmTauxffbs2WRnZ7N8+fI6X3t0Eg/QuXNnunbtSlZWlt9jlciWm+Upq09uF0dckuZftyRPVs2N//WornRKiQ9xNBJ03z0Khhv6/Ao6DW30yzbkbeBA6QHio+M5rctp9R7rKi7GsW8foEZ3IiIiEngtPpHfsGEDAwYMIDq6ZvHA0KFDffubYteuXezdu1dl9eJ3uWp01yJtOFDMV1tzMZvgL6f3CnU4EmxFWbC2qqfKaTc16aXe0fgJXSeQYEmo91jv+vGWzEyiUlKaHqeIiIhIE7T40vr8/Hx69jy2FDYtLc23v7GcTieXX345SUlJXH/99fUea7fbsdvtvudWq7XR7yORKWev52dEje5aFm+n+guGdaZ7ev1znKUV+uHf4HZCj1Oh29hGv8xtuBvdrR7AprJ6ERERCaIWPyIPYDKZmrWvOsMwuPzyy/n222955ZVX6NatW73HP/TQQ6SkpPhuDR0vcmTpOSXyLcWOnBI+2ngIgL+e0TvE0UjQlebAqlc8j5s4Gr82dy2Hyw+TZEnilC6nNHi8t2N9nDrWi4iISBC0+EQ+PT291lH3goIC4MjIfH0Mw+CPf/wjCxYsYP78+UydOrXB19x2220UFxf7bppTL/WpKKmkJN8GQHuV1rcYT3+5E8OAXw3qQN8O+oIl4vz4JDht0GU0nHB6k17q7VZ/RrcziI1qeJWDCl/HeiXyIiIiEngtPpEfMmQImzdvxul01ti+fv16AAYPrv9DkzeJf+mll3j++ef57W9/26j3jY2NJTk5ucZNpC45+zyj8akdEoiNb/EzViLCvvxy3l2bDcDVZzR+uTFpJcoLYOULnsen3QSNrN4CcLldfLL3EwDOOeGcBo935ufjzD4IJhNxA1VaLyIiIoHX4hP5iy66iNLSUt5+++0a219++WU6d+7MuHHj6nytYRj86U9/4qWXXuI///kPs2fPDnS4EqFyNT++xXn2m5243Aan9W3PkK5qPhZxVvwXKkuhw2Do23AyXt2qnFXkVeSRHJPMiZ1ObPB420bP/PiYE04gKkl9GERERCTwWvzQ4eTJkzn77LO58sorsVqt9O7dm0WLFvHRRx+xYMEC3xryl19+OS+//DI7d+6ke/fuAPztb3/jhRdeYM6cOQwZMoSffvrJd97Y2FhGjBgRkj+TtD6aH9+yHCq28b+f9wNwtebGRx57Cfz0jOfxqTc0aTQejpTVn5V5FpaohpeS9Hasj1OjOxEREQmSFp/IA7zzzjvMnTuXu+66i4KCAvr378+iRYuYMWOG7xiXy4XL5cIwDN+2pUuXAvDiiy/y4osv1jhn9+7d2bNnT1Dil9YvR0vPtSjPfbuLSpebsT3SGHtCw300pJX5+UWwFUF6bxh4YZNe6nQ7+XTvpwCc06NxI/m29Z5EPr6BqV4iIiIi/hIWiXxSUhLz5s1j3rx5dR4zf/585s+fX2ObEnUJhrJiO2VFdkwmaNctKdThRLyCskoWLt8HwFVnajQ+4jgq4IcnPY9PuR7MUU16+YqDKyi0F9I2ti1jOzVuuTpfx/rBQ5r0XiIiIiLN1eLnyIu0dLlVo/FtOyUSExcW3421ai9+t5sKh4shXVI4rU+7UIcjwbZ6AZTlQEo3GDq9yS//aI+nrP7s7mcTbW7499lxOAdnbi6YzcQN6N/k9xMRERFpDiXyIsfpcFWju4zMNjWmdkjwWW0OXv5xDwBXndEbUxPnRkuYczng+6rKrZOvhUbMb6/O4XLw2b7PgMZ1qwewVS07F9u7N+b4+Ca9n4iIiEhzKZEXOU7eEXmtHx96r/64lxKbk74dkpg0sEOow5FgW/cmFGdBYgaMaNxSo9X9ePBHSipLaBffjpEZIxv1miNl9ZofLyIiIsGjRF7kOBiGQY53RF4d60OqvNLJC9/tBuCvE3pjNms0PqK4XfDdo57HJ10NlqaPjnu71U/qPomoRs6tV8d6ERERCQUl8iLHobTQTkWJA7PZRLuuSSrlDqFFK7IoKKskMy2B84Z2CnU4Emyb3oX8HRCXCqPnNPnldpedL7K+ABpfVm8YBrYNnjXk1bFeREREgkmJvMhx8JbVp3VJJDqmad2xxX9sDhfPfbMLgCsn9CI6Sn+1RRS3C779l+fx+CshtunVMd8d+I4yRxkdEjowrP2wRr3GmZ2Nq6AAoqOJ7devye8pIiIi0lz6tCtyHHLU6K5FeOKL7Ryy2uicEsfFI7uEOhwJtp9fhMMbIDYZxl7RrFN8vPtjAH7V41eYTY37p7GiajQ+tm8fzLGxzXpfERERkeZQIi9yHLyJvBrdhc62wyX852vPaPzdFwwiNlqVERHFehA+u9fz+Ky7ICGtyacod5Tz1f6vAJh8wuRGv87b6C5+kMrqRUREJLiUyIs0k6fRnae0Xo3uQsPtNrj9nfU43QZnD+zArwZ1DHVIEmwf/R9UlkCX0c2aGw/wzYFvqHBW0DWpK4PSG9+0zrv0XNwQJfIiIiISXErkRZrJmmfDXu7EHG0ivbMa3YXCGz9n8fPeQhJiorj3AnUNjzhbP/I0uTNFwfmPQyM7zR+tell9Y3+PDcPwldar0Z2IiIgEmxJ5kWbyltW365JElEW/SsGWW2LnoWWbAbhxUj86pzZ9uTEJY5VlsOwmz+MTr4KOQ5p1mjJHGd8e+BZofLd6AEdWFm6rFVNMDLG9ezfrvUVERESaS9mHSDN5O9a3756sRnch8MAHm7DanAzukszvT+we6nAk2L56CIqzICUTJtza7NN8mfUldpedHsk96Ne28Z3nvfPjY/v3xxQT0+z3FxEREWkOJfIizZSzr6pjvebHB90323J5d002ZhM8dNFQLTcXaQ6ugx+f9jye8i+ISWz2qZpTVg9Qsb6q0d1gTekQERGR4NOnX5FmMNyGb0Q+o3uy5scHkc3h4o4lniTq9yf1YEjXlBBHJEHldsH714HhgoEXQt9JzT6VtdLKd9nfAXBOj8aX1cOREfm4QYNVkSMiIiJBp0RepBmKcsqptLmIsphJ65QQ6nAiyhNfbGdfQTkdk+O4cVLjS6GllVj5Ahz4xbNm/OR/HNepvtj3BU63k96pvendtvHz3A23G9tGT6O7uCGD9UWeiIiIBJ0SeZFm8C47175bEmaVdQdN9TXj7506iKTY6BBHJEFlzYbP7/M8nng3tDm+5QY/2vMR0PTR+Mo9e3CXl2OKjye2Z8/jikFERESkOZSBiDRD9UZ3gEprg0BrxgsfVlszflTz1oz3KrAV8FP2T0DTutVDtbL6AQMwRevLJBEREQk+JfIizaBGd8GnNeMj3NYPYfN7VWvGzwPz8f3zNX/jfFyGi8Hpg+me3LRVDyq8ibwa3YmIiEiIKJEXaSK32yB3X1Wju0zPiLzmyAaW1oyPcPZSWHaz5/FJV0PHwcd1utzyXBZtXgTAX4b9pcmvt23wzI+PH3x8cYiIiIg0lxJ5kSYqPFSGs9KNJTaK1I5qdBcMWjM+wnnXjE/NhNP/77hP99z657C5bAxtP5TTup7WpNcaTie2TZsAiFMiLyIiIiGiRF6kiXL2VM2Pz2yD2ayR+EDTmvER7uBa+OkZz+Mpjx7XmvEA2aXZvLXtLQCuHXFtk6tp7Dt3YdhsmBMTienR47hiEREREWkufSIWaaLcvZ758e2r5ser0V3gaM34COd2wdLrPGvGD7oI+px93Kd8du2zON1OxnUax9hOY5v8+iPrxw/CdJzz9EVERESaS59CRJooxzs/Xo3uAs67ZnynFK0ZH5FWvgDZqzxrxp/z8HGfbk/xHt7b+R4Afxvxt2adw7bR2+hOZfUiIiISOkrkRZrA5XKTl1UKqNFdoG09VG3N+Au0ZnzE8fOa8QBPr3kal+FiQtcJDG0/tFnnqPA1ulPHehEREQkdJfIiTVCQXYbL6SYmPpqU9uqcHihut8Htiz1rxk8a2IFJWjM+8nx4i2fN+K5jjnvNeICtBVv5cM+HAFw94upmncOorMS+ZQugEXkREREJLSXyIk2Qu/dIozuTGt0FzOsrs/hlbyGJMVHcozXjI8+WZbB5KZij/bJmPMCTa54E4Jwe59AvrXnTNOw7dmBUVmJOTsbSrdtxxyQiIiLSXErkRZpg/9ZC4Mj8eDW687/NB638/QPP8l5aMz4CFe2DpVXz10+8Gjoc/xc563LX8VXWV5hNZv46/K/NPk/Z8hWAp6xeU2pEREQklJTIizRSubWSnatzAOg5on2Io2mdckpsXD5/JWWVLk7qlc7vtGZ8ZLFZYeF0KMuFDkP8smY8wL9X/xuAC3pdwAkpJzTrHIbbTdHrrwOQNHGiX+ISERERaS4l8iKNtOm7bNxOg4zubeh4gmcZNI3K+Y/N4eJPr/xCdrGNnu0TeebSUVozPpK4nPC/OZCzCZI6wqzXISbhuE+74uAKlh9cTrQ5mr8M+0uzz1P2/fdU7t2LOSmJ1KlTjzsuERERkeOhT8kijeByudnwzQEAhp7RNcTRtD5ut8GNb61lbVYRqQkWXvz9GFISLKEOS4Lpk7mw41OIjoeZiyDl+H/PDMPwjcb/us+v6ZLUpdnnKliwAICUiy/CnJh43LGJiIiIHA8l8iKNsHtNHmVFduLbWOg9qkOow2l1HvtsGx+sO4glysR/fjuKHu2UKEWUFc/B8mc9jy/+D3QZ6ZfTfnvgW9bmriUuKo4rhl7R7PNU7t1L2TffApA2a5ZfYhMRERE5HkrkRRph/Vf7ARh0aheiLEd+bdTs7vi9s2o/T3yxA4CHLh7KuJ7pIY5IgmrHZ/Bh1Vz4s+6Cgf4pW3cbbp5Y/QQAM/vPpH1C8/taFC5cBIZB4mmnEtOjh1/iExERETkeSuRFGpC3v5Ts7UWYzCYGndr80lw51so9Bdz69noA/jqhF78epWkLESVnM7w1GwwXDJsFp9zgt1N/uvdTthRsIdGSyJzBzV+H3l1WRtE77wCQ9tvf+is8ERERkeOiRF6kAeu/zAKg5/D2JLWNrbFPze6ab19+OX9+9RcqXW4mD+7ITZOat7a3hKnSXFj4G7BbofvJnvXi/fT75HK7eGrNUwD8buDvSI1Lbfa5ipcuxV1SgqV7JomnnOKX+ERERESOlxJ5kXrYyhxsW3EYUJM7fyqucDB7/goKyioZ2jWFR38zHLNZX4pEDIcN3rjUs2Z82xNg+gKIjvHb6d/f9T67i3eTEpvC7wb+rtnnMQyDwtdeAzxz401m/ZMpIiIiLYM+lYjUY/P3B3E63KR3TaJT75RQh9MqOFxurnptFTtzy+iUEsfzvxtNfExUqMOSYDEMeO9qyFoOcSlw6VuQkOa30ztcDp5Z+wwAcwbPISkmqdnnKl++Avv2HZgSEki5+GJ/hSgiIiJy3JTIi9TB7TZY/7Wnyd3QM7qqjN4PDMPg7vc28t2OPBJionj+96PJSI4LdVgSTF//E9a/BeZo+M2r0K6PX0//zvZ3OFB6gHbx7ZjZf+Zxnavwtaol56ZeQFSbNv4IT0RERMQvlMiL1GHv+jxK8m3EJkbTd4yWnPOHF7/fw8Ll+zCZ4N8zRjCos6ocIsr6/8FXD3oeT3kUep7u19PbnDb+u+6/APxpyJ+Ij45v9rkcBw5Q8vkXAKRdeqlf4hMRERHxFyXyInVY96VnNH7gSZ2JVun3cft882Ee+GATAHPPHcDEgfpyJKJkrYQlf/U8PukaGPV7v7/FG1vfIKcih06Jnfh1318f17kKX38d3G4Sxo8ntndvP0UoIiIi4h9K5EVqUXCwjP1bCjGZYPDpWnLueG3KtnLNotUYBswcm8nlp5wQ6pAkmAr3wuszwWWHfufCxHv9/hZljjJeWP8CAFcOu5KYqOY3z3PbbBS99T8A0n6r0XgRERFpeZTIi9Riw1ee0fgeQ9uR3K755bkCOVYbf3x5JeWVLk7unc59Uwep30AksVlh0Qwoy4WOQ+Di58Ds/wqXVze9SqG9kB7JPTi/1/nHdS7rB8twFRVh6dyZpDPO8FOEIiIiIv6jRF7kKJUVTrb8dAiAIVpy7rjszS9j5nM/kV1so2f7RJ6eNQpLlP7aiRglh+HViyBnE7TpBDPfgNjmd5Gvy47CHby04SUArhp+FdHm6GafyzAMCqqa3LWdNRNTlKbViIiISMvT/E87Iq3U5h8P4rC7aNsxga792oY6nLC1YncBf371ZwrLHXRKiePF348hJcES6rAkWA5vhIXToTgL4tvCzEWQ4v9pKnkVeVz1+VWUO8sZ3WE0k3pMOq7zVaxejX3TZkyxsaRccomfohQRERHxLyXyItUYboP1VWX1QyZoybnmemfVfm59ez2VLjdDu6bw/O+0zFxE2fYJ/G82VJZCem+Y9Sak9/L729icNq794lqyy7LpntydxyY8htl0fBUfhQs8o/HJ500huq2+yBMREZGWSYm8SDVZmwsozqkgJi6KfuM7hjqcsON2Gzz66Tae/HIHAOcM6shj04cTr67/kWP5f+CjW8FwQ49T4TevQEKa39/GbbiZ+91c1uWtIyU2hafOeorUuNTjOqfjcA7WTz4FIO23v/VDlCIiIiKBoURepJp1VaPx/U/qREycfj2awuZwceNba/lg3UEArpzQi5sn9cNsVlVDRHA5PQn8yuc8z0dc5lkrPrr53ePr8+TqJ/lk7ydEm6N5bMJjdE/uftznLHrjDXA6iR81irgBA/wQpYiIiEhgKFMRqVKcW87eDfkADDldTe6aIqfExp9e+YW1WUVYokz8/aIh/GZ0t1CHJcFiK4a3ZsPOzwETnH0vnPQ3CNDUlMXbF/Pces8XBveedC9jOo457nMalZUUvvkmoCXnREREpOVTIi9SZf1XB8CAzEHppHZICHU4YWPLISuXz/+ZA0UVpCZYePa3oxjfMz3UYUmwFO7xNLXL3QKWBM/ycgPOC9jbrTi4gvt+vA+AK4ZewQW9LvDLea0ff4wrL4/ojAzaTJzol3OKiIiIBIoSeRHAYXex+QdPSfiQCf7vrN1afbklh6sXrqKs0kXPdom88IcxnNAuMdRhSbBkrYBFM6E8r2p5udeh8/CAvd3u4t1c/9X1OA0n5/Q4h6uGX+W3cxcueA2A1BnTMVm0uoKIiIi0bErkRYCtyw9RWeEkpX083QdpNLkx5n+/m/ve34TbgBN7pvPMb0eSmhCY+dDSAq3/Hyz5K7js0HEozHoDkjsH7O0KbYVc9flVWCutDGs/jAdOeeC4O9R7VazfQMXatZgsFtr+5jd+OaeIiIhIICmRl4hnGEctOddAczbDMCJ6WTqny81972/ilR/3AjB9dDfuv3AwMdH+SaqkhTMM+Pof8NVDnuf9psDF/4XYpIC9ZaWrkuu+vI6skiy6JHVh3hnziI2K9dv5vUvOtZl8DtHt2vntvCIiIiKBokReIt6BbUUUZJcRHRtF/5M6hTqcFq243MHfXl/N19tyMZng1nP6c8VpPSP6i42IUlkGS6+F9W95np90DUy8F8yBW17QMAzu/uFuVuWsoo2lDU+d9RTp8f6rmnHm52NdtgzQknMiIiISPpTIS8Rb/2XVknPjOhIb3/CvRCQmrW63wf9+2c8/PtpCflkl8ZYoHp8xnF8N6hjq0CQYDAM2vwcf3Q7W/WCO9iwtN+r3AX/rZ9c9y/u73ifKFMW/JvyLXqm9/Hr+orfewnA4iBs6lPihQ/16bhEREZFAUSIvEa2kwMbutbkADFaTu1qt21/Ene9uZG1WEQB9MpJ4bPpwBndJCW1gEhy5W+HDW2DXV57nKZlw4VNwwmkBf+sPdn3A02ueBuCO8XdwYucT/Xp+w+mkcNHrAKRdOsuv5xYREREJJCXyEtE2fH0Aw4Au/dqS3jlwc3zDUUFZJf/v4628vnIfhgFJsdFcN7EPvz+pB5YozYdv9ewl8PU/4aenwe2EqFg45To4+TqICfzyjKtzVnPn93cC8IdBf+DXfX/t9/co+exznIcPE5WeTpvJk/1+fhEREZFAUSIvEctZ6WLTd9kADD2ja6Nf19qb3bncBotW7OORT7ZSVO4A4KIRXbhtcn8ykuNCHJ0EnGHAhrfhkzugxLMkI30nwzkPQlrPoISQZc3i2i+uxeF2cGa3M7l+1PUBeR9vk7vU30zDHKMVF0RERCR8KJGXiGQYBj8s3omtzEGbtDh6DG1cp2rDMAIcWWj9sreQu9/bwIYDVgD6d2zDfVMHM/aEtBBHJkFxeCMsuxn2fu953vYEmPwP6PuroIXwddbX3PPjPRTaCxmYPpCHTn3Ib8vMVVf0zmLKf/4ZoqNpO2OG388vIiIiEkhK5CXiGG6DrxdtZeO3ntH4cVN7Ym5gyTmv1joSn1ti5x8fbeF/v3ga/7WJi+amSf24dFwm0Sqjb/0qiuCrh2HFf8FwQXQ8nHqjpyu9JThVGKWVpfxz5T9ZvGMxAL1SevHkmU+SYPF/GX/h669z6J57AUi77DIsHTr4/T1EREREAkmJvEQUt9vgy1c3s+XHQ2CCM37bn37jIrfzutPl5tWf9vLop9sosTkB+M3ortxyTn/aJflvnW5podxuWPc6fHoXlHmaPjLgAvjV3yE1M2hhrDi4gju/v5PssmxMmPjdwN9x9YiriYv2/5cIBS+/zOGHHgag7WWXkXHLzX5/DxEREZFAUyIvEcPlcvP5S5vY/nMOJrOJiX8YQN+xkZnE2xwu3l93kOe+2cXWwyUADOmSwn1TBzEis22Io5OAczlgy/vww5Nw4GfPtvQ+njL63mcFLQyb08a8VfNYsNkzV71LUhceOPkBRnccHZD3y/vPf8l97DEA0v/0R9rfcEOrrbIRERGR1k2JvEQEl8PNJy9sZNeaXMxRJiZdPoheIzNCHVbQ7ckr47Xle3nrl/2+RnapCRZu+VV/po/pRlQjpxhImCo+AL/Mh1WvQOkhzzZLIpx+C4z/K0QHr+Hbutx1zP1uLnusewCY1ncaN42+KSCl9IZhkPfEE+Q9/QwA7a6+mnZX/VVJvIiIiIQtJfLS6jkrXXz4nw3s25iPOdrE5CuGNLq5XWvgdLn5YksOr/60l2+35/m2d0mN59Lxmcwam0lqgjp2t1puN+z+Cla+AFs/9MyBB0jMgFG/h9GXQ3KnoIXjcDl4Zu0zvLDhBdyGm4z4DO49+V5O6XJKQN7PMAxy/t8jFLz4IgDtb7yBdn/6U0DeS0RERCRYlMhLq+awu/jg6XUc2FpItMXMuVcOpdvAyOjAnlNi440VWSxcsY+DxTYATCaY0Lc9l53YndP7ZmgEvjUrL4A1C+HnF6Bg15HtPU6F0XOg/3lBHYEH2Fa4jbnfzWVLwRYApvScwm1jbyMlNiUg72e43Rx+4O8ULlwIQIfbbyftd5cF5L1EREREgkmJvLRalRVO3n9qLQd3FGOJjeK8q4fSuU/rnv9tGAY/7SpgwU97+XjjIZxuz3J5aYkx/GZ0Ny4dl0m3NP+XLksLYRhwYJUned/wNjg9X+AQmwzDZnoS+Iz+QQ/L5XYxf+N8nlrzFA63g9TYVO4cfyeTekwK2HsaLhcH776b4v+9DSYTHe+5h7bTfxOw9xMREREJJiXy0irZyhwsfWItOXusxMRHc/41w+jYMzCjfqFmGAZ78sv5fPNhXl+ZxY6cUt++Ud3bctn47kwe0pHY6KgQRikBVbQPtn8Cq16Fg2uObO84xFM6P2QaxCYFPSzDMFiXt45HVj7CmlxPXBO6TeDuE++mXXzgprcYTifZt92OdelSMJvp9ODfSb3wwoC9n4iIiEiwKZGXVqeipJL3/r2GvKxSYhOjmXrtCNpntgl1WH5VZnfy4858vt6Wy9fbctlXUO7blxATxYUjuvDbcd0Z2Dk5hFFKwDhssPd72PE57PgM8rYe2RcVA4MuhjGXQ9cxnvkUQVZgK2DpzqUs3r6YncU7AUiyJHHr2Fu5oNcFAW0yZ1RWcuDmWyj5+GOIiqLL//snyeeeG7D3ExEREQkFJfLSqpQV23n38TUUHiwjvo2FqdeNIL1L8Eci/c0wDLYdLuXrbTl8vS2XlbsLqXS5ffstUSbG9EjjnMEduWhEF9rEWUIYrQRE/k5P0r7jM9j9LTgrjuwzmaHrWOg/BYZfConpQQ/P5XbxQ/YPLN6xmC+zvsTpdgIQFxXHpB6TuHr41XRKCmxTPbfdzoHrrqf0yy8xWSx0efwx2pwVvOX0RERERIJFiby0GqWFNt59fA1Fh8tJTIlh6vUjaNsxMdRhNVtxhYMfduT5Rt29Deu8uraNZ0K/9kzom8GJvdJJjNWvc6tSWeZJ2L3Je+HumvvbdPKs+d77bOh5OsSHpv/D/pL9LNmxhCU7lnC4/LBv++D0wVzU5yImnzCZNjGBr4hxV1Sw/6qrKfvhB0yxsXR98gmSTj014O8rIiIiEgr65C9hr6zIztYVh1j/5X5KC+0kpcVy4fUjSGkfPk3dbA4Xmw9a2ZjtuW3KLmZDthVXVbM6gNhoM+N7pjOhX3tO79ueE9olah3s1sJph5zNcGgdHFznuc9eDa7KI8eYLZA5HvqcDb0nQsbAkJTNA9hddr7Y9wVvb3+b5QeX+7anxKZwXs/zuKj3RfRL6xeUWFylpZR89BEFry3Evnkzpvh4uj3zNInjxwfl/UVERERCQYm8hCVnpYtda3PZ+uMhsjYXYFTluynt45l6/QjapMWFNsB6FJc72HiwmE1VSfuGA8XszC2lWs7u07N9Iqf3bc+EfhmMOyGNOIsa1oU9ewkc2lCVtK/1JO65W8DtOPbYlEzoM9Ez6n7CqRAbul4POeU5rM9dz/JDy/lg1wdYK62+feM7jeeSPpdwRuYZxEbFBjwWw+Wi7IcfKX73XUo++wzD5qlWMScl0e0/z5IwalTAYxAREREJJSXyEjYMw+DgjmK2/nSQHb/kUGlz+fZ16pVCv/Ed6TOmAzFxof+xNgyD4goHWQUVZBWWsyOnlI3ZxWzMtrK/sKLW16QnxjCoSwqDOiczqHMyw7qmaqm4cGYrhsK9ULQX8rYfGW0v2AXU8q1NXCp0Ggodh0KnYdB5JKT3Csmou81pY3PBZtblrvPc8tZxqOxQjWM6Jnbkwt4XcmHvC+mS1CUocdl37KB4yRKK31uKMyfHtz2mZ09SLryQlAunYsnICEosIiIiIqEU+oynEUpLS7njjjt48803KSgooH///tx6663MmDGjwdfm5ORwyy238P7771NeXs6wYcN44IEHOEsNkMKGNa+CLT8dYutPB7HmHZkn3iYtjn7jO9JvfEdSM4Kf8NocLvYXlvuS9X355WR5nxeUU2J31vnarm3jqxL2FN99h+RYlcqHE2clFGd55q57E/bCPUceVxTW/do2nasl7VWJe0q3kCTthmGw17qXdXnrfIn79sLtOI2aP79mk5neqb0Z0m4IE7tP5MROJxJlDnyFiLOwEOsHyyhesgTbhg2+7VEpKSRPmULKhVOJGzJEvzsiIiISUcIikb/44otZuXIlDz/8MH379mXhwoXMnDkTt9vNrFmz6nyd3W7nrLPOoqioiHnz5pGRkcFTTz3FOeecw2effcbpp58exD+FNJZhGNhKHexel8eWHw9ycEexb58lNopeozLoP74jnXunYjL798O7YRiU2J3kl1aSV2onv9RObmkl+aX2queV5JTYySooJ6fE3uD52reJpVvbeLqnJzKwUzKDuiQzqFMKKQnqKt8iGQZUlkJZLpTmeu7LcqAsr2pbjudWtBes2dQ6sl5dQjqkdoe0EzxruntH2xMDt4Z6bRxuBznlOWSXZnOo7BDZpdkcLDvIgdIDbC7YTLG9+JjXtItvx9B2QxnSfgjD2g9jUPogEizB+cLMXVZG2Y8/UrRkCaVffwOOqmkH0dEknXYaKRdOJWnCBMwxMUGJR0RERKSlMRmG0cAn0dBatmwZU6ZM8SXvXpMmTWLjxo3s27ePqKjaR4WefvpprrrqKn744QdOPPFEAJxOJ8OGDSMpKYnly5fX+rraWK1WUlJSKC4uJjlZa3MfL8NtUFZciTWvnKKcCqy5FRRXu1VWVBsNNEG3/m3pN74TPYe3xxLb8CigzeGixOakxOagxObEWnV/5LnnsbXCSUGZnfyySvJK7OSVVVLpdDd4fq+k2Gi6pSXQrW287z4zPYFubRPo2jaB+BjNaQ8Jpx1sVrBbPSXudmu159W226xQnlczcXfWPvWhVpYET6Letge07V71uOp5ambA57QbhkG5sxyr3Yq10kpuRa4vSfcl7WXZ5JTn4Dbq/rmOjYplQNoAhravStzbDaNjYseAjXIbhoGrqAjHvn1UVt08j7OozMrClZdX4/i4gQNJufBCks+bQnRaWkBiEpHWQZ/XRCRStPgR+cWLF5OUlMS0adNqbJ89ezazZs1i+fLlnHTSSXW+tl+/fr4kHiA6Oprf/va33H777Rw4cIAuXYIztzMSuJxuHDYXlTYnDruLSpsLh82JvcJJqdVOUY4nSS/Jq6A034bbWf93SFGpMUT1TMLVLZ6d0SY25hdS8VEeFZUuyh0ubJUuKhxVt6rHpTYnJTZnjTXWmyMpNpr0pBjaJcWSnhhDuzaxtPPeJ8XStW083domkJpgUUlvQwwDXA5w2T3l6C67J9Gud1vVzVEOjoqq+2qPK6tvr7bfXuJJzl0NV0vUy5IAie09t6QMzwh6YkbV8/aeJnRte3i2N/H6u9wu7C47NpcNu9NOhasCu9OO3WWnwlnh21fuOJKgWyutNR9XPS+pLDmmBL7OP5LZQqfETp5bUic6J3amU1In+qT2oW/bvliijq9KxHC5cJeVHXNzlZV5kvas/dUS9n24S0vrPV90x44kn3uup3S+b9/jik1ERESktWnxifyGDRsYMGAA0dE1Qx06dKhvf12J/IYNGzi1lnWEva/duHFji0zk8w4d5L1HnwGqFe4aR+4NTDX3+Z6YMKjab5iqNpp8z4+cy1RjO5gwDHO1e8/NMMxV+80YmDEZnnuIAsMCWDAZFiAGExZMTfxxMgw3bqy4KK66FVW7t0KhE37Bc6tiBpKqbo14B6LNJqKjTESbzVX3piP3Zs92S7SJmCgzMdFmYqLMWKLMRJmrTmGruuXVPK8V2FijmKW2LyUMqK3gxTCOOt6otr3aU4yj9hlVT2v8MFQ99z52Hzne9xqj2jE1X2P4jnNXe43nsXHMdgOMque4wO197j3GDYYLDHfVeaue1/J/p6Hn1f5UNZ+bqj+v+XvgO94Uj0GC57k5GqJicEdZMKIsGGYLRFkwoqIxzFXboiy4o2JwR8d4tkXH4MaMGzduuxvDZuDOq8AwduM2dmIYblyGG7fbhcuourmrbriqtruPbK+6d7odVLoduNxOjkn9a/kfYAJM1babDM+21KobVc8BokxRJETFkWRJom1MKqkxyaTGpJASk0yKxXNLNCdgKjUwrO6qa2eAkYfhPEyR8ysMpwOcTgyHE8NZdXM5ofpzpxPD6cAoK8dVXoa79EjC7u0c3xTRHToQk5mJJbMbMZndicnshiUzk5jMTKLahK5Dv4iIiEhL1+IT+fz8fHr27HnM9rSq8sr8/Px6X5tWSxlmY15rt9ux24+M6lmt1jqP9bfDe3dRYZ0QtPerzlTH46Ywux1EOW1EuWxEu+xEOW1YnOXEV+QSX5HnubflEWfLx1xPua+Eo6iqW0tiAJVVt9bKDTiAEuDgMXvLqm5BYbEQlZCAOTHxyC25DTFdu2LplklM90xiunXD0q0b5riWu0ykiIiISEvW4hN5oN7S5YbKmpv72oceeoh777234eACIKFNEnHl64BqyXQdo7+m2sYtDaPads/I6JGxS+/zI+OYJsMFhguT4cKEu47HLkzGkedmw4bZbcfsPvbexFHJefVRVDOUJ3puHmZajpZQIl9HDA2GZmrgMFMD56nrfet6ganWY0xHn8dUy1dDR52z3j9aLa8/9njTUac8+rhj99c4q8lUbZvn3vt3Q9WzqtdX22cy4fvPZMKMuWqbZ78JE2aT2bffhAmz2ezZZvLcm01RNf8stf2/rnov7x/IG+ux+0xVQ/gmTCYzmM1grnpsMoHZ7GkM6Xt+ZJ/JEg1RUZiiLZiioz3Po6MxRUX7npuiq7ZVHWNOiMeclFQzWffe1IBOREREJOBafCKfnp5e68h5QUEBQK0j7v547W233cYNN9zge261WunWrVuj4z4eJwwcxuWvDAvKe4mIiIiIiEh4aUnDobUaMmQImzdvxums2dBp/fr1AAwePLje13qPa+prY2NjSU5OrnETERERERERCbUWn8hfdNFFlJaW8vbbb9fY/vLLL9O5c2fGjRtX72u3bNlSY5k5p9PJggULGDduHJ07dw5Y3CIiIiIiIiKB0OJL6ydPnszZZ5/NlVdeidVqpXfv3ixatIiPPvqIBQsW+NaQv/zyy3n55ZfZuXMn3bt3B2DOnDk89dRTTJs2jYcffpiMjAyefvpptm7dymeffRbKP5aIiIiIiIhIs7T4RB7gnXfeYe7cudx1110UFBTQv39/Fi1axIwZM3zHuFwuXC5X1ZJZHrGxsXz++efccsstXHPNNZSXlzN8+HA+/PBDTj/99FD8UURERERERESOi8kwalvoWo5mtVpJSUmhuLhY8+VFREREWiB9XhORSNHi58iLiIiIiIiIyBFK5EVERERERETCiBJ5ERERERERkTCiRF5EREREREQkjCiRFxEREREREQkjSuRFREREREREwogSeREREREREZEwokReREREREREJIwokRcREREREREJI0rkRURERERERMKIEnkRERERERGRMKJEXkRERERERCSMKJEXERERERERCSPRoQ4gXBiGAYDVag1xJCIiIiJSG+/nNO/nNhGR1kqJfCOVlJQA0K1btxBHIiIiIiL1KSkpISUlJdRhiIgEjMnQV5aN4na7yc7Opk2bNphMpoC/n9VqpVu3bmRlZZGcnBzw95OWRdc/sun6RzZd/8im6398DMOgpKSEzp07YzZrBqmItF4akW8ks9lM165dg/6+ycnJ+oc8gun6RzZd/8im6x/ZdP2bTyPxIhIJ9FWliIiIiIiISBhRIi8iIiIiIiISRpTIt1CxsbHcfffdxMbGhjoUCQFd/8im6x/ZdP0jm66/iIg0hprdiYiIiIiIiIQRjciLiIiIiIiIhBEl8iIiIiIiIiJhRIm8iIiIiIiISBhRIt/ClJSUcMsttzBp0iTat2+PyWTinnvuqfP4VatWMXHiRJKSkkhNTeXiiy9m165dwQtY/Kq0tJTrrruOzp07ExcXx/Dhw3n99ddDHZb4WVN+z/U73vp88cUXzJkzh/79+5OYmEiXLl2YOnUqv/zyyzHH6vq3PmvWrGHKlClkZmYSHx9PWloaJ554IgsWLDjmWF1/ERGpixL5FiY/P5///ve/2O12LrzwwnqP3bJlCxMmTKCyspI333yTF198kW3btnHqqaeSm5sbnIDFry6++GJefvll7r77bj788EPGjBnDzJkzWbhwYahDEz9q7O+5fsdbp2eeeYY9e/Zw7bXXsmzZMubNm0dOTg7jx4/niy++8B2n6986FRUV0a1bNx588EGWLVvGK6+8Qo8ePbjssst44IEHfMfp+ouISL0MaVHcbrfhdrsNwzCM3NxcAzDuvvvuWo+dNm2a0a5dO6O4uNi3bc+ePYbFYjFuueWWYIQrfvTBBx8YgLFw4cIa288++2yjc+fOhtPpDFFk4m+N/T3X73jrdPjw4WO2lZSUGB06dDDOOuss3zZd/8gybtw4o1u3br7nuv4iIlIfjci3MCaTCZPJ1OBxTqeT999/n0suuYTk5GTf9u7du3PGGWewePHiQIYpAbB48WKSkpKYNm1aje2zZ88mOzub5cuXhygy8bfG/J7rd7z1ysjIOGZbUlISAwcOJCsrC9D1j0Tt2rUjOjoa0PUXEZGGKZEPUzt37qSiooKhQ4ces2/o0KHs2LEDm80WgsikuTZs2MCAAQN8H+S8vNd4w4YNoQhLQkS/45GluLiYVatWMWjQIEDXPxK43W6cTie5ubk8/fTTfPzxx/zf//0foOsvIiINUyIfpvLz8wFIS0s7Zl9aWhqGYVBYWBjssOQ45Ofn13k9vfslcuh3PLJcddVVlJWVMXfuXEDXPxL89a9/xWKxkJGRwfXXX8+///1v/vznPwO6/iIi0jAl8gH01Vdf+UpoG7qtWbOmWe9RX3luY0r0pWXR9ZSj6Wei9bvzzjt57bXXeOyxxxg1alSNfbr+rdftt9/OypUr+eCDD5gzZw5XX301jzzySI1jdP1FRKQu0Q0fIs3Vr18/nnvuuUYdm5mZ2aRzp6enA7WP0hYUFGAymUhNTW3SOSW00tPT67yeUPvIjLRe+h2PDPfeey8PPPAAf//737n66qt923X9W7/MzEzfv/3nnnsuALfddhu///3vdf1FRKRBSuQDqFOnTvzxj38MyLl79epFfHw869evP2bf+vXr6d27N3FxcQF5bwmMIUOGsGjRIpxOZ4158t5rPHjw4FCFJiGg3/HW79577+Wee+7hnnvu4fbbb6+xT9c/8owdO5Znn32WXbt2MWrUKF1/ERGpl0rrw1R0dDTnn38+77zzDiUlJb7t+/bt48svv+Tiiy8OYXTSHBdddBGlpaW8/fbbNba//PLLdO7cmXHjxoUoMgkF/Y63bvfffz/33HMPd9xxB3ffffcx+3X9I8+XX36J2WymZ8+euv4iItIgk2EYRqiDkJo+/PBDysrKKCkpYc6cOUybNo3f/OY3gKf8LiEhAYAtW7YwZswYRo4cya233orNZuOuu+6ioKCANWvW0L59+1D+MaQZJk2axM8//8w//vEPevfuzaJFi3juuedYsGABl156aajDEz9qzO+5fsdbp3/961/cdNNNnHPOObUm8ePHjwf0d3xrdcUVV5CcnMzYsWPp0KEDeXl5vPXWW7zxxhvcfPPN/POf/wR0/UVEpAGhXMReate9e3cDqPW2e/fuGsf+/PPPxllnnWUkJCQYycnJxoUXXmjs2LEjNIHLcSspKTH+9re/GR07djRiYmKMoUOHGosWLQp1WBIAjf091+9463P66afXee2P/mdZ17/1efHFF41TTz3VaNeunREdHW2kpqYap59+uvHqq68ec6yuv4iI1EUj8iIiIiIiIiJhRHPkRURERERERMKIEnkRERERERGRMKJEXkRERERERCSMKJEXERERERERCSNK5EVERERERETCiBJ5ERERERERkTCiRF5EREREREQkjCiRFxEREREREQkjSuRFREREREREwogSeREREREREZEwokReREREREREJIwokRcREREREREJI0rkRUQihGEYTJo0CZPJxNtvv11jn9vt5swzz8RkMrFkyZLQBCgiIiIijWIyDMMIdRAiIhIchw4dYujQoTidTtauXUu3bt0AePjhh7ntttv485//zLPPPhviKEVERESkPkrkRUQizPvvv8/555/PqaeeyldffcWqVas46aST6N27N7/88gvx8fGhDlFERERE6qFEXkQkAl1zzTU8+eST/N///R/vvPMOe/fuZfny5QwfPjzUoYmIiIhIA5TIi4hEIJvNxtixY1m/fj0Ajz76KNdff32IoxIRERGRxlCzOxGRCBQXF8fkyZMBSEhIYM6cOSGOSEREREQaS4m8iEgE+vHHH3n00UdJT0+nvLycq666KtQhiYiIiEgjKZEXEYkwVquVSy+9FIvFwjfffMPUqVN57bXXWLBgQahDExEREZFG0Bx5EZEIc+mll7Jw4UKeeeYZ/vKXv5Cfn8/QoUMpKSlhzZo19OzZM9QhioiIiEg9NCIvIhJBXn31VRYuXMgFF1zAX/7yFwDS09N5+eWXKS0t5dJLL8XpdIY4ShERERGpjxJ5EZEIsWvXLq666io6derECy+8UGPfxIkTufHGG/npp5+45557QhOgiIiIiDSKSutFREREREREwohG5EVERERERETCiBJ5ERERERERkTCiRF5EREREREQkjCiRFxEREREREQkjSuRFREREREREwogSeREREREREZEwokReREREREREJIwokRcREREREREJI0rkRURERERERMKIEnkRERERERGRMKJEXkRERERERCSMKJEXERERERERCSP/HzzJM7UxOcW4AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "alphas = [-4, -8,-12,-20, 0]\n",
    "betas = [0.4,0.4,0.6,1, 1]\n",
    "x = np.arange(-10,35)\n",
    "fig = plt.figure(figsize=(8, 6)) \n",
    "ax = plt.subplot(111)\n",
    "\n",
    "for i in range(len(alphas)):\n",
    "    a = alphas[i]\n",
    "    b = betas[i]\n",
    "    y = np.exp(a+b*x)/(1+np.exp(a+b*x))\n",
    "    ax.plot(x,y,label=r\"$\\alpha=%d,$    $\\beta=%3.1f$\" % (a,b))\n",
    "\n",
    "ax.tick_params(labelsize=12)\n",
    "ax.set_xlabel('x', fontsize = 14)\n",
    "ax.set_ylabel('$p(x)$', fontsize = 14)\n",
    "ax.legend(loc='center left', bbox_to_anchor=(1, 0.5), prop={'size': 16})\n",
    "ax.set_title('Logistic Functions', fontsize = 16);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Multi-Layer Perceptron (MLP) or Fully Connected Network (FCN)\n",
    "\n",
    "<center>\n",
    "    \n",
    "<img src=\"figs/NN-figs/neural_net2.jpeg\" width=\"75%\">\n",
    "    \n",
    "</center>\n",
    "\n",
    "\n",
    "From [cs231n](https://cs231n.github.io/convolutional-networks/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiple artificial neurons can be acting on the same inputs, in what we call\n",
    "a _layer_, and we can have more than one _layer_ until we produce one or more\n",
    "outputs.\n",
    "\n",
    "The example above shows a network with _3 inputs_, two layers of neurons, each\n",
    "with 4 neurons, followed by one layer that produces a single value output.\n",
    "\n",
    "E.g. a binary classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Computation Graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way we are going to differentiate more complex functions is to first build a \"computation graph\" to apply our operations on. We'll see that it breaks down the process into simple steps that easily scale to large networks.\n",
    "\n",
    "It's the concept employed by TensorFlow and PyTorch, and in fact we'll follow PyTorch interface definition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Building the `Value` Class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do that we will build a data wrapper as a `class` called `Value` and gradually build in on all the functionality we need to define a Multi-Layer Neural Network (a.k.a. Multi-Layer Perceptron) and train it.\n",
    "\n",
    "First, the class has only a simple initialization method and a representation method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Value version 1\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data):\n",
    "        self.data = data\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=4.0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(4.0)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are not familiar with [python classes](https://docs.python.org/3/tutorial/classes.html), there are a few things to note here.\n",
    "1. The property `self` is just a pointer to the object itself.\n",
    "2. The `__init__` method is called when you initialize a class object\n",
    "3. The `__repr__` method is how you represent the class object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Implementing Addition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the Value object doesn't do much yet except for taking a value and printing it. We'd also like to do things like addition and other operations with them, but..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uh oh! unsupported operand type(s) for +: 'Value' and 'Value'\n"
     ]
    }
   ],
   "source": [
    "a = Value(4.0)\n",
    "b = Value(-3.0)\n",
    "\n",
    "try:\n",
    "    a+b \n",
    "except Exception as e:\n",
    "    print(\"Uh oh!\", e)\n",
    "else:\n",
    "    print(\"It worked!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "When python tries to add two objects `a` and `b`, internally it will call\n",
    "`a.__add__(b)`. So we have to add the `__add__()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Value version 2\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data):\n",
    "        self.data = data\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\"\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data + other.data)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "It worked!\n"
     ]
    }
   ],
   "source": [
    "a = Value(4.0)\n",
    "b = Value(-3.0)\n",
    "\n",
    "try:\n",
    "    a+b\n",
    "except Exception as e:\n",
    "    print(\"Uh oh!\", e)\n",
    "else:\n",
    "    print(\"It worked!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which, as mentioned is equivalent to calling the `__add__` method on `a`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.__add__(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Implementing More Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly we can support multiplication and implement a ReLU function as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Value version 3\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data):\n",
    "        self.data = data\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\"\n",
    "\n",
    "    def __add__(self, other): # self + other\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data + other.data)\n",
    "        return out\n",
    "    \n",
    "    def __mul__(self, other): # self * other\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data * other.data)\n",
    "        return out\n",
    "    \n",
    "    def relu(self):\n",
    "        out = Value(np.maximum(0, self.data))\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=-4.0)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(4.0)\n",
    "b = Value(-3.0)\n",
    "c = Value(8.0)\n",
    "\n",
    "d = a*b+c\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=0.0)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.relu()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "By the way, internally, python will call `__mul__` on `a`, then `__add__` on the temporary product object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=-4.0)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(a.__mul__(b)).__add__(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Child Nodes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to calculate the gradients, we will need to capture the computation graphs. To do that, we'll need to store pointers to the operands of each operation.\n",
    "\n",
    "To start with, we'll accept a tuple of child nodes in the initializer and store that as a set in the object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Value version 4\n",
    "class Value:\n",
    "                        #    vvvvvvvvvvvv\n",
    "    def __init__(self, data, _children=()):\n",
    "        self.data = data\n",
    "        self._prev = set(_children)\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\"\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data + other.data, (self, other)) # store tuple of children\n",
    "        return out                        # ^^^^^^^^^^^^^\n",
    "    \n",
    "    def __mul__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data * other.data, (self, other)) # store tuple of children\n",
    "        return out                        # ^^^^^^^^^^^^^\n",
    "    \n",
    "    def relu(self):\n",
    "        out = Value(np.maximum(0, self.data), (self,))\n",
    "        return out                         #  ^^^^^^^"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "a = Value(4.0)\n",
    "b = Value(-3.0)\n",
    "c = Value(8.0)\n",
    "\n",
    "d = a*b\n",
    "e = d + c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "We can now see the children of the operands that produced the output value by printing the `_prev` value. The name `_prev` might not be intuitive yet, but it will make more sense when we view these operations as a graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{Value(data=-3.0), Value(data=4.0)}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d._prev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{Value(data=-12.0), Value(data=8.0)}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e._prev"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Child Operations\n",
    "\n",
    "Now we've recorded pointers to the child nodes. It would be helpful to also record the operator used.\n",
    "\n",
    "We'll also add labels for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Value version 5\n",
    "class Value:\n",
    "                                    #     vvvvvvv  vvvvvvvv\n",
    "    def __init__(self, data, _children=(), _op='', label=''):\n",
    "        self.data = data\n",
    "        self._prev = set(_children)\n",
    "        self._op = _op # store the operation that created this node\n",
    "        self.label = label # label for the node\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\"\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)   \n",
    "        out = Value(self.data + other.data, (self, other), '+') # store tuple of children\n",
    "        return out                                      #  ^^^\n",
    "    \n",
    "    def __mul__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data * other.data, (self, other), '*') # store tuple of children\n",
    "        return out                                      #  ^^^\n",
    "    \n",
    "    def relu(self):                                 #  vvvvvv\n",
    "        out = Value(np.maximum(0, self.data), (self,), 'ReLU')\n",
    "        # out = Value(0 if self.data < 0 else self.data, (self,), 'ReLU')\n",
    "\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = Value(4.0, label='a')\n",
    "b = Value(-3.0, label='b')\n",
    "c = Value(8.0, label='c')\n",
    "\n",
    "d = a*b ; d.label = 'd'\n",
    "e = d + c ; e.label = 'e'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({Value(data=-3.0), Value(data=4.0)}, '*', 'd')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d._prev, d._op, d.label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({Value(data=-12.0), Value(data=8.0)}, '+', 'e')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e._prev, e._op, e.label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U8ab3WOOP391",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### The Compute Graph\n",
    "\n",
    "We now have enough information stored about the compute graph to visualize it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are two functions to walk the graph and build sets of all nodes and edges (`trace`) and then draw them as a\n",
    "directed graph (`draw_dot`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# draw_dot version 1\n",
    "from graphviz import Digraph\n",
    "\n",
    "def trace(root):\n",
    "  # builds a set of all nodes and set of all edges in a graph\n",
    "  nodes, edges = set(), set()\n",
    "  def build(v):\n",
    "    if v not in nodes:\n",
    "      nodes.add(v)\n",
    "      for child in v._prev:\n",
    "        edges.add((child, v))\n",
    "        build(child)\n",
    "  build(root)\n",
    "  return nodes, edges\n",
    "\n",
    "def draw_dot(root):\n",
    "  dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n",
    "\n",
    "  nodes, edges = trace(root)\n",
    "  for n in nodes:\n",
    "    uid = str(id(n))\n",
    "    # for any value in the graph, create a rectangular ('record') node for it\n",
    "    dot.node(name = uid, label = \"{ %s | data %.4f }\" % (n.label, n.data), shape='record')\n",
    "    if n._op:\n",
    "      # if this value is a result of some operation, create an op node for it\n",
    "      dot.node(name = uid + n._op, label = n._op)\n",
    "      # and connect this no de to it\n",
    "      dot.edge(uid + n._op, uid)\n",
    "\n",
    "  for n1, n2 in edges:\n",
    "    # connect n1 to the op node of n2\n",
    "    dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n",
    "\n",
    "  return dot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "a = Value(4.0, label='a')\n",
    "b = Value(-3.0, label='b')\n",
    "c = Value(8.0, label='c')\n",
    "\n",
    "d = a*b ; d.label = 'd'\n",
    "e = d + c ; e.label = 'e'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "j3s1BCjMP391",
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"587pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 587.00 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 583,-124 583,4 -4,4\"/>\n",
       "<!-- 4895757840 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895757840</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-83.5 0,-119.5 107,-119.5 107,-83.5 0,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-84 22.75,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "</g>\n",
       "<!-- 4895385968* -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895385968*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"170\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"170\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895757840&#45;&gt;4895385968* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895757840&#45;&gt;4895385968*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M107.39,-88.57C116.15,-86.42 125.05,-84.25 133.26,-82.24\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.97,-85.67 142.85,-79.89 132.3,-78.87 133.97,-85.67\"/>\n",
       "</g>\n",
       "<!-- 4895386832 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895386832</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"472.75,-27.5 472.75,-63.5 579,-63.5 579,-27.5 472.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"483.75\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"494.75,-28 494.75,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"536.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895386832+ -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895386832+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"409.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"409.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895386832+&#45;&gt;4895386832 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895386832+&#45;&gt;4895386832</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M437.22,-45.5C444.46,-45.5 452.64,-45.5 461.01,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"460.98,-49 470.98,-45.5 460.98,-42 460.98,-49\"/>\n",
       "</g>\n",
       "<!-- 4895385968 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895385968</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"233,-55.5 233,-91.5 346.75,-91.5 346.75,-55.5 233,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"244.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"255.75,-56 255.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"301.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "</g>\n",
       "<!-- 4895385968&#45;&gt;4895386832+ -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895385968&#45;&gt;4895386832+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M347.05,-60.16C355.84,-58.07 364.7,-55.96 372.87,-54.02\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"373.48,-57.48 382.4,-51.76 371.86,-50.67 373.48,-57.48\"/>\n",
       "</g>\n",
       "<!-- 4895385968*&#45;&gt;4895385968 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895385968*&#45;&gt;4895385968</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M197.45,-73.5C204.64,-73.5 212.76,-73.5 221.13,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"221.11,-77 231.11,-73.5 221.11,-70 221.11,-77\"/>\n",
       "</g>\n",
       "<!-- 4954283408 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4954283408</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"239,-0.5 239,-36.5 340.75,-36.5 340.75,-0.5 239,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"250\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"261,-1 261,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"300.88\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "</g>\n",
       "<!-- 4954283408&#45;&gt;4895386832+ -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4954283408&#45;&gt;4895386832+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M341.21,-30.03C351.82,-32.46 362.82,-34.98 372.77,-37.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"371.9,-40.65 382.43,-39.47 373.46,-33.83 371.9,-40.65\"/>\n",
       "</g>\n",
       "<!-- 4895758752 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895758752</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.62,-28.5 2.62,-64.5 104.38,-64.5 104.38,-28.5 2.62,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"13.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"24.62,-29 24.62,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "</g>\n",
       "<!-- 4895758752&#45;&gt;4895385968* -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895758752&#45;&gt;4895385968*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M104.72,-58.34C114.16,-60.57 123.86,-62.86 132.78,-64.96\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"131.94,-68.36 142.48,-67.25 133.55,-61.54 131.94,-68.36\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123cf5a90>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oaMxcCKSP391",
    "tags": []
   },
   "source": [
    "Note that every value object becomes a node in the graph. The operators are also represented as a kind of fake node so they can be visualized too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "lGXCjYq-P391",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Nodes:  {Value(data=-3.0), Value(data=-4.0), Value(data=-12.0), Value(data=8.0), Value(data=4.0)}\n",
      "Edges:  {(Value(data=4.0), Value(data=-12.0)), (Value(data=-12.0), Value(data=-4.0)), (Value(data=8.0), Value(data=-4.0)), (Value(data=-3.0), Value(data=-12.0))}\n"
     ]
    }
   ],
   "source": [
    "nodes, edges = trace(e)\n",
    "print(\"Nodes: \", nodes)\n",
    "print(\"Edges: \", edges)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "44GcUYSvP392",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "Lets add one more operation, or stage in the compute graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "uxv577Z_P392",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"822pt\" height=\"156pt\"\n",
       " viewBox=\"0.00 0.00 821.50 156.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 152)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-152 817.5,-152 817.5,4 -4,4\"/>\n",
       "<!-- 4895003024 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895003024</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-111.5 0,-147.5 107,-147.5 107,-111.5 0,-111.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-124.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-112 22.75,-147.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-124.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "</g>\n",
       "<!-- 4895773840* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895773840*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"170\" cy=\"-101.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"170\" y=\"-96.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895003024&#45;&gt;4895773840* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895003024&#45;&gt;4895773840*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M107.39,-116.57C116.15,-114.42 125.05,-112.25 133.26,-110.24\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.97,-113.67 142.85,-107.89 132.3,-106.87 133.97,-113.67\"/>\n",
       "</g>\n",
       "<!-- 4895773840 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895773840</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"233,-83.5 233,-119.5 346.75,-119.5 346.75,-83.5 233,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"244.38\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"255.75,-84 255.75,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"301.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "</g>\n",
       "<!-- 4895774416+ -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895774416+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"409.75\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"409.75\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895773840&#45;&gt;4895774416+ -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895773840&#45;&gt;4895774416+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M347.05,-88.16C355.84,-86.07 364.7,-83.96 372.87,-82.02\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"373.48,-85.48 382.4,-79.76 371.86,-78.67 373.48,-85.48\"/>\n",
       "</g>\n",
       "<!-- 4895773840*&#45;&gt;4895773840 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895773840*&#45;&gt;4895773840</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M197.45,-101.5C204.64,-101.5 212.76,-101.5 221.13,-101.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"221.11,-105 231.11,-101.5 221.11,-98 221.11,-105\"/>\n",
       "</g>\n",
       "<!-- 4895774416 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895774416</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"472.75,-55.5 472.75,-91.5 579,-91.5 579,-55.5 472.75,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"483.75\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"494.75,-56 494.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"536.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895774176* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895774176*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"642\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"642\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895774416&#45;&gt;4895774176* -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895774416&#45;&gt;4895774176*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M579.26,-60.65C588.02,-58.5 596.93,-56.31 605.16,-54.29\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"605.89,-57.72 614.77,-51.94 604.23,-50.92 605.89,-57.72\"/>\n",
       "</g>\n",
       "<!-- 4895774416+&#45;&gt;4895774416 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895774416+&#45;&gt;4895774416</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M437.22,-73.5C444.46,-73.5 452.64,-73.5 461.01,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"460.98,-77 470.98,-73.5 460.98,-70 460.98,-77\"/>\n",
       "</g>\n",
       "<!-- 4895004416 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895004416</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.62,-56.5 2.62,-92.5 104.38,-92.5 104.38,-56.5 2.62,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"13.62\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"24.62,-57 24.62,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.5\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "</g>\n",
       "<!-- 4895004416&#45;&gt;4895773840* -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895004416&#45;&gt;4895773840*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M104.72,-86.34C114.16,-88.57 123.86,-90.86 132.78,-92.96\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"131.94,-96.36 142.48,-95.25 133.55,-89.54 131.94,-96.36\"/>\n",
       "</g>\n",
       "<!-- 4895774608 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895774608</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"475.75,-0.5 475.75,-36.5 576,-36.5 576,-0.5 475.75,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"486\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"496.25,-1 496.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"536.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895774608&#45;&gt;4895774176* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895774608&#45;&gt;4895774176*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M576.27,-30.19C585.93,-32.47 595.88,-34.83 605,-36.98\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"604.01,-40.35 614.55,-39.24 605.62,-33.53 604.01,-40.35\"/>\n",
       "</g>\n",
       "<!-- 4895004608 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895004608</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"239,-28.5 239,-64.5 340.75,-64.5 340.75,-28.5 239,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"250\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"261,-29 261,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"300.88\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "</g>\n",
       "<!-- 4895004608&#45;&gt;4895774416+ -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895004608&#45;&gt;4895774416+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M341.21,-58.03C351.82,-60.46 362.82,-62.98 372.77,-65.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"371.9,-68.65 382.43,-67.47 373.46,-61.83 371.9,-68.65\"/>\n",
       "</g>\n",
       "<!-- 4895774176 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895774176</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"705,-27.5 705,-63.5 813.5,-63.5 813.5,-27.5 705,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"717.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"729.25,-28 729.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"771.38\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "</g>\n",
       "<!-- 4895774176*&#45;&gt;4895774176 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895774176*&#45;&gt;4895774176</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M669.44,-45.5C676.66,-45.5 684.81,-45.5 693.18,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"693.14,-49 703.14,-45.5 693.14,-42 693.14,-49\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123cf97f0>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(4.0, label='a')\n",
    "b = Value(-3.0, label='b')\n",
    "c = Value(8.0, label='c')\n",
    "\n",
    "d = a*b; d.label = 'd'\n",
    "e = d + c; e.label = 'e'\n",
    "f = Value(2.0, label='f')\n",
    "\n",
    "L = e*f; L.label = 'L'\n",
    "\n",
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V1cWIyciP392",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Recap\n",
    "\n",
    "So far we've built a Value class and associated data structures to capture a computational graph and calculate the output based on the inputs and operations. We'll call this the __forward pass__.\n",
    "\n",
    "But now, we're interested in calculating the gradients with respect to some of the parameters with respect to $L$. \n",
    "\n",
    "So next we'll update our Value class to capture the partial derivative at each node relative to L."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "80fEbziTP392",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Calculating Gradients\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5ZDgQ7NZP392",
    "tags": []
   },
   "source": [
    "Add a gradient member variable, `grad`, to our class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Value version 6\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data, _children=(), _op='', label=''):\n",
    "        self.data = data\n",
    "        self.grad = 0.0 # default to 0  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
    "        self._prev = set(_children)\n",
    "        self._op = _op # store the operation that created this node\n",
    "        self.label = label # label for the node\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data})\"\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)   \n",
    "        out = Value(self.data + other.data, (self, other), '+') # store tuple of children\n",
    "        return out\n",
    "    \n",
    "    def __mul__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)   \n",
    "        out = Value(self.data * other.data, (self, other), '*') # store tuple of children\n",
    "        return out\n",
    "    \n",
    "    def relu(self):         \n",
    "        out = Value(np.maximum(0, self.data), (self,), 'ReLU')\n",
    "        # out = Value(0 if self.data < 0 else self.data, (self,), 'ReLU')\n",
    "\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HBZR6rldP392",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "And update `draw_dot()` to show `grad` in the node info."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "Ahr9SeMZP392",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# draw_dot version 2\n",
    "from graphviz import Digraph\n",
    "\n",
    "def trace(root):\n",
    "  # builds a set of all nodes and set of all edges in a graph\n",
    "  nodes, edges = set(), set()\n",
    "  def build(v):\n",
    "    if v not in nodes:\n",
    "      nodes.add(v)\n",
    "      for child in v._prev:\n",
    "        edges.add((child, v))\n",
    "        build(child)\n",
    "  build(root)\n",
    "  return nodes, edges\n",
    "\n",
    "def draw_dot(root):\n",
    "  dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n",
    "\n",
    "  nodes, edges = trace(root)\n",
    "  for n in nodes:\n",
    "    uid = str(id(n))\n",
    "    # for any value in the graph, create a rectangular ('record') node for it\n",
    "    dot.node(name = uid, label = \"{ %s | data %.4f | grad %.4f }\" % (n.label, n.data, n.grad), shape='record')\n",
    "    if n._op:\n",
    "      # if this value is a result of some operation, create an op node for it\n",
    "      dot.node(name = uid + n._op, label = n._op)\n",
    "      # and connect this node to it\n",
    "      dot.edge(uid + n._op, uid)\n",
    "\n",
    "  for n1, n2 in edges:\n",
    "    # connect n1 to the op node of n2\n",
    "    dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n",
    "\n",
    "  return dot        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "And reinitialize and redraw..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "9wV7JoeRP393",
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1147pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 1146.50 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 1142.5,-124 1142.5,4 -4,4\"/>\n",
       "<!-- 4895599728 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895599728</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"320.25,-83.5 320.25,-119.5 503.25,-119.5 503.25,-83.5 320.25,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"342.25,-84 342.25,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"422,-84 422,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"462.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440+ -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895599440+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"572.25\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"572.25\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895599728&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895599728&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M503.57,-85.46C514.47,-83.53 525.06,-81.66 534.51,-79.99\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"535.08,-83.44 544.32,-78.26 533.86,-76.55 535.08,-83.44\"/>\n",
       "</g>\n",
       "<!-- 4895600304 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895600304</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"948.75,-27.5 948.75,-63.5 1138.5,-63.5 1138.5,-27.5 948.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"960.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"973,-28 973,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1015.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1057.25,-28 1057.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1097.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600304* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895600304*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"885.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"885.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895600304*&#45;&gt;4895600304 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895600304*&#45;&gt;4895600304</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M913.19,-45.5C920.34,-45.5 928.53,-45.5 937.24,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"937.01,-49 947.01,-45.5 937.01,-42 937.01,-49\"/>\n",
       "</g>\n",
       "<!-- 4895599392 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895599392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.62,-56.5 2.62,-92.5 185.62,-92.5 185.62,-56.5 2.62,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"13.62\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"24.62,-57 24.62,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.5\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"104.38,-57 104.38,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"145\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895600544*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"251.25\" cy=\"-46.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"251.25\" y=\"-41.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895599392&#45;&gt;4895600544* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895599392&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.81,-58.13C195.66,-56.35 205.21,-54.63 213.82,-53.08\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"214.19,-56.57 223.41,-51.34 212.95,-49.68 214.19,-56.57\"/>\n",
       "</g>\n",
       "<!-- 4895599440 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895599440</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"635.25,-55.5 635.25,-91.5 822.75,-91.5 822.75,-55.5 635.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"657.25,-56 657.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"699.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"741.5,-56 741.5,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"782.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440&#45;&gt;4895600304* -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895599440&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M823.14,-56.65C831.9,-55.06 840.38,-53.53 848.09,-52.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"848.46,-55.62 857.68,-50.4 847.21,-48.73 848.46,-55.62\"/>\n",
       "</g>\n",
       "<!-- 4895599440+&#45;&gt;4895599440 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895599440+&#45;&gt;4895599440</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M599.5,-73.5C606.66,-73.5 614.87,-73.5 623.6,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"623.39,-77 633.39,-73.5 623.39,-70 623.39,-77\"/>\n",
       "</g>\n",
       "<!-- 4895596896 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895596896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-1.5 0,-37.5 188.25,-37.5 188.25,-1.5 0,-1.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-2 22.75,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"107,-2 107,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"147.62\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895596896&#45;&gt;4895600544* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895596896&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.49,-35.75C197.27,-37.28 205.77,-38.76 213.51,-40.1\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"212.66,-43.51 223.11,-41.78 213.86,-36.61 212.66,-43.51\"/>\n",
       "</g>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"638.25,-0.5 638.25,-36.5 819.75,-36.5 819.75,-0.5 638.25,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"648.5\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"658.75,-1 658.75,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"698.62\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"738.5,-1 738.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"779.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600016&#45;&gt;4895600304* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895600016&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M820.02,-34.21C829.89,-35.93 839.48,-37.6 848.13,-39.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"847.32,-42.52 857.77,-40.79 848.52,-35.63 847.32,-42.52\"/>\n",
       "</g>\n",
       "<!-- 4895600544 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895600544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"314.25,-28.5 314.25,-64.5 509.25,-64.5 509.25,-28.5 314.25,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"325.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-29 337,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"428,-29 428,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"468.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895600544&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M509.52,-62.98C518.33,-64.49 526.85,-65.94 534.59,-67.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"533.74,-70.66 544.19,-68.89 534.92,-63.76 533.74,-70.66\"/>\n",
       "</g>\n",
       "<!-- 4895600544*&#45;&gt;4895600544 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895600544*&#45;&gt;4895600544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M278.42,-46.5C285.59,-46.5 293.84,-46.5 302.63,-46.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"302.51,-50 312.51,-46.5 302.51,-43 302.51,-50\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123c3d7c0>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(4.0, label='a')\n",
    "b = Value(-3.0, label='b')\n",
    "c = Value(8.0, label='c')\n",
    "\n",
    "d = a*b; d.label = 'd'\n",
    "e = d + c; e.label = 'e'\n",
    "f = Value(2.0, label='f')\n",
    "\n",
    "L = e*f; L.label = 'L'\n",
    "\n",
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PhEm4yKIP393",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Manual Gradient Calculation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GY5YV3K6P393",
    "tags": []
   },
   "source": [
    "Before we start implementing backpropagation, it is helpful to manually calculate some gradients to better understand the procedure.\n",
    "\n",
    "For the node $L$, we trivially calculate $\\frac{dL}{dL} = 1$. \n",
    "\n",
    "From limit ratio perspective, \n",
    "\n",
    "$$ \\frac{\\partial L}{\\partial L} = \\lim_{h \\rightarrow 0} \\frac{ (L+h) - L }{h} = \\frac{h}{h} = 1$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "GsGq4xDzP393",
    "tags": []
   },
   "outputs": [],
   "source": [
    "L.grad = 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W71k54M8P393",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "If we go backwards a step in the graph, we see that $L=e*f$, so we calculate\n",
    "\n",
    "$$\\frac{\\partial{L}}{\\partial{e}} = \\frac{\\partial}{\\partial{e}} (e\\times f) = f$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\\frac{\\partial{L}}{\\partial{f}} = \\frac{\\partial}{\\partial{f}} (e\\times f) = e.$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "lD7btWQXP393",
    "tags": []
   },
   "outputs": [],
   "source": [
    "e.grad = f.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "id": "uvyZcb3wP393",
    "tags": []
   },
   "outputs": [],
   "source": [
    "f.grad = e.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6-rlYnptP393",
    "tags": []
   },
   "source": [
    "> To summarize, the partial derivative w.r.t. to one operand of a simple product is simply the other operand."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "And we can redraw the graph above again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "Siw6YrqsP393",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1147pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 1146.50 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 1142.5,-124 1142.5,4 -4,4\"/>\n",
       "<!-- 4895599728 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895599728</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"320.25,-83.5 320.25,-119.5 503.25,-119.5 503.25,-83.5 320.25,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"342.25,-84 342.25,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"422,-84 422,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"462.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440+ -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895599440+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"572.25\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"572.25\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895599728&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895599728&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M503.57,-85.46C514.47,-83.53 525.06,-81.66 534.51,-79.99\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"535.08,-83.44 544.32,-78.26 533.86,-76.55 535.08,-83.44\"/>\n",
       "</g>\n",
       "<!-- 4895600304 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895600304</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"948.75,-27.5 948.75,-63.5 1138.5,-63.5 1138.5,-27.5 948.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"960.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"973,-28 973,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1015.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1057.25,-28 1057.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1097.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4895600304* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895600304*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"885.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"885.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895600304*&#45;&gt;4895600304 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895600304*&#45;&gt;4895600304</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M913.19,-45.5C920.34,-45.5 928.53,-45.5 937.24,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"937.01,-49 947.01,-45.5 937.01,-42 937.01,-49\"/>\n",
       "</g>\n",
       "<!-- 4895599392 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895599392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.62,-56.5 2.62,-92.5 185.62,-92.5 185.62,-56.5 2.62,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"13.62\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"24.62,-57 24.62,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.5\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"104.38,-57 104.38,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"145\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895600544*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"251.25\" cy=\"-46.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"251.25\" y=\"-41.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895599392&#45;&gt;4895600544* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895599392&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.81,-58.13C195.66,-56.35 205.21,-54.63 213.82,-53.08\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"214.19,-56.57 223.41,-51.34 212.95,-49.68 214.19,-56.57\"/>\n",
       "</g>\n",
       "<!-- 4895599440 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895599440</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"635.25,-55.5 635.25,-91.5 822.75,-91.5 822.75,-55.5 635.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"657.25,-56 657.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"699.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"741.5,-56 741.5,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"782.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440&#45;&gt;4895600304* -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895599440&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M823.14,-56.65C831.9,-55.06 840.38,-53.53 848.09,-52.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"848.46,-55.62 857.68,-50.4 847.21,-48.73 848.46,-55.62\"/>\n",
       "</g>\n",
       "<!-- 4895599440+&#45;&gt;4895599440 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895599440+&#45;&gt;4895599440</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M599.5,-73.5C606.66,-73.5 614.87,-73.5 623.6,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"623.39,-77 633.39,-73.5 623.39,-70 623.39,-77\"/>\n",
       "</g>\n",
       "<!-- 4895596896 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895596896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-1.5 0,-37.5 188.25,-37.5 188.25,-1.5 0,-1.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-2 22.75,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"107,-2 107,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"147.62\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895596896&#45;&gt;4895600544* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895596896&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.49,-35.75C197.27,-37.28 205.77,-38.76 213.51,-40.1\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"212.66,-43.51 223.11,-41.78 213.86,-36.61 212.66,-43.51\"/>\n",
       "</g>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"636,-0.5 636,-36.5 822,-36.5 822,-0.5 636,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"656.5,-1 656.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"696.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"736.25,-1 736.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"779.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895600016&#45;&gt;4895600304* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895600016&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M822.25,-34.6C831.39,-36.19 840.23,-37.73 848.25,-39.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"847.41,-42.54 857.87,-40.81 848.62,-35.64 847.41,-42.54\"/>\n",
       "</g>\n",
       "<!-- 4895600544 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895600544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"314.25,-28.5 314.25,-64.5 509.25,-64.5 509.25,-28.5 314.25,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"325.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-29 337,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"428,-29 428,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"468.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895600544&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M509.52,-62.98C518.33,-64.49 526.85,-65.94 534.59,-67.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"533.74,-70.66 544.19,-68.89 534.92,-63.76 533.74,-70.66\"/>\n",
       "</g>\n",
       "<!-- 4895600544*&#45;&gt;4895600544 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895600544*&#45;&gt;4895600544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M278.42,-46.5C285.59,-46.5 293.84,-46.5 302.63,-46.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"302.51,-50 312.51,-46.5 302.51,-43 302.51,-50\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123c9a610>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How do the parameters $e$ and $f$ influence $L$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Try uncommenting `e += h` or `f += h` and calling `wiggle()` then `wiggle(1.0)`\n",
    "# to see the influence of e or f on L\n",
    "def wiggle(h = 0.0):\n",
    "    a = Value(4.0, label='a')\n",
    "    b = Value(-3.0, label='b')\n",
    "    c = Value(8.0, label='c')\n",
    "\n",
    "    d = a*b; d.label = 'd'\n",
    "    e = d + c; e.label = 'e'\n",
    "    # e += h\n",
    "    f = Value(2.0, label='f')\n",
    "    f += h\n",
    "\n",
    "    L = e*f; L.label = 'L'\n",
    "    print(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=-8.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=-12.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle(1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5OfZey4aP394",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Propagating Back"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Xes9Hn2gP394",
    "tags": []
   },
   "source": [
    "Now we want to calculate\n",
    "\n",
    "$$\\frac{\\partial{L}}{\\partial{c}}$$\n",
    "\n",
    "or put another way, we want to know how much $L$ wiggles if we wiggle $c$, or how $c$ influences $L$.\n",
    "\n",
    "Looking at the graph again we see that $c$ influences $e$ and $e$ influences $L$, so we should be able see the ripple effect of $c$ on $L$.\n",
    "\n",
    "$$ c \\rightarrow e \\rightarrow L $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B9tNqWrKP394",
    "tags": []
   },
   "source": [
    "So $e = c + d$, and so we calculate\n",
    "\n",
    "$$ \\frac{\\partial{e}}{\\partial{c}} = \\frac{\\partial{}}{\\partial{c}} (d + c) = 1$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-M8dDtrAP394",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Question\n",
    "\n",
    "$$ c \\rightarrow e \\rightarrow L $$\n",
    "\n",
    "So now we know $\\partial{L}/\\partial{e}$ and we also know $\\partial{e}/\\partial{c}$,\n",
    "\n",
    "How do we get $\\partial{L}/\\partial{c}$?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w_K-_xnpP394",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### The Chain Rule\n",
    "\n",
    "To paraphrase from the Wikipedia page on [Chain rule](https://en.wikipedia.org/wiki/Chain_rule), if a variable $L$ depends on the variable $e$, which itself depends on the variable $c$ (that is, $e$ and $L$ are dependent variables), then $L$ depends on $c$ as well, via the intermediate variable $e$. In this case, the chain rule is expressed as\n",
    "\n",
    "$$\\frac{\\partial L}{\\partial c} = \\frac{\\partial L}{\\partial e} \\cdot \\frac{\\partial e}{\\partial c},$$\n",
    "\n",
    "and\n",
    "\n",
    "$$ \\left.\\frac{\\partial L}{\\partial c}\\right|_{c} = \\left.\\frac{\\partial L}{\\partial e}\\right|_{e(c)}\\cdot \\left. \\frac{\\partial e}{\\partial c}\\right|_{c} ,$$\n",
    "\n",
    "for indicating at which points the derivatives have to be evaluated."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NTXkw1pAP394",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "Now since we've established that\n",
    "\n",
    "$$ \\frac{\\partial{e}}{\\partial{c}} = 1$$\n",
    "\n",
    "then\n",
    "\n",
    "$$\\frac{dL}{dc} = \\frac{dL}{de} \\cdot 1.$$\n",
    "\n",
    "So in the case of an operand in an addition operation, we just copy the gradient of the parent node.\n",
    "\n",
    "Or put another way, \n",
    "\n",
    "> in the addition operator, we just route the parent gradient to the child."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "aH3JwlM2P394",
    "tags": []
   },
   "outputs": [],
   "source": [
    "d.grad = e.grad\n",
    "c.grad = e.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1147pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 1146.50 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 1142.5,-124 1142.5,4 -4,4\"/>\n",
       "<!-- 4895599728 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895599728</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"320.25,-83.5 320.25,-119.5 503.25,-119.5 503.25,-83.5 320.25,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"342.25,-84 342.25,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"422,-84 422,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"462.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440+ -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895599440+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"572.25\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"572.25\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895599728&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895599728&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M503.57,-85.46C514.47,-83.53 525.06,-81.66 534.51,-79.99\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"535.08,-83.44 544.32,-78.26 533.86,-76.55 535.08,-83.44\"/>\n",
       "</g>\n",
       "<!-- 4895600304 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895600304</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"948.75,-27.5 948.75,-63.5 1138.5,-63.5 1138.5,-27.5 948.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"960.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"973,-28 973,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1015.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1057.25,-28 1057.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1097.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4895600304* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895600304*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"885.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"885.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895600304*&#45;&gt;4895600304 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895600304*&#45;&gt;4895600304</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M913.19,-45.5C920.34,-45.5 928.53,-45.5 937.24,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"937.01,-49 947.01,-45.5 937.01,-42 937.01,-49\"/>\n",
       "</g>\n",
       "<!-- 4895599392 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895599392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.62,-56.5 2.62,-92.5 185.62,-92.5 185.62,-56.5 2.62,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"13.62\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"24.62,-57 24.62,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.5\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"104.38,-57 104.38,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"145\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895600544*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"251.25\" cy=\"-46.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"251.25\" y=\"-41.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895599392&#45;&gt;4895600544* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895599392&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.81,-58.13C195.66,-56.35 205.21,-54.63 213.82,-53.08\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"214.19,-56.57 223.41,-51.34 212.95,-49.68 214.19,-56.57\"/>\n",
       "</g>\n",
       "<!-- 4895599440 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895599440</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"635.25,-55.5 635.25,-91.5 822.75,-91.5 822.75,-55.5 635.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"657.25,-56 657.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"699.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"741.5,-56 741.5,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"782.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440&#45;&gt;4895600304* -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895599440&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M823.14,-56.65C831.9,-55.06 840.38,-53.53 848.09,-52.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"848.46,-55.62 857.68,-50.4 847.21,-48.73 848.46,-55.62\"/>\n",
       "</g>\n",
       "<!-- 4895599440+&#45;&gt;4895599440 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895599440+&#45;&gt;4895599440</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M599.5,-73.5C606.66,-73.5 614.87,-73.5 623.6,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"623.39,-77 633.39,-73.5 623.39,-70 623.39,-77\"/>\n",
       "</g>\n",
       "<!-- 4895596896 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895596896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-1.5 0,-37.5 188.25,-37.5 188.25,-1.5 0,-1.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-2 22.75,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"107,-2 107,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"147.62\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895596896&#45;&gt;4895600544* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895596896&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.49,-35.75C197.27,-37.28 205.77,-38.76 213.51,-40.1\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"212.66,-43.51 223.11,-41.78 213.86,-36.61 212.66,-43.51\"/>\n",
       "</g>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"636,-0.5 636,-36.5 822,-36.5 822,-0.5 636,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"656.5,-1 656.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"696.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"736.25,-1 736.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"779.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895600016&#45;&gt;4895600304* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895600016&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M822.25,-34.6C831.39,-36.19 840.23,-37.73 848.25,-39.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"847.41,-42.54 857.87,-40.81 848.62,-35.64 847.41,-42.54\"/>\n",
       "</g>\n",
       "<!-- 4895600544 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895600544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"314.25,-28.5 314.25,-64.5 509.25,-64.5 509.25,-28.5 314.25,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"325.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-29 337,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"428,-29 428,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"468.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895600544&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M509.52,-62.98C518.33,-64.49 526.85,-65.94 534.59,-67.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"533.74,-70.66 544.19,-68.89 534.92,-63.76 533.74,-70.66\"/>\n",
       "</g>\n",
       "<!-- 4895600544*&#45;&gt;4895600544 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895600544*&#45;&gt;4895600544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M278.42,-46.5C285.59,-46.5 293.84,-46.5 302.63,-46.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"302.51,-50 312.51,-46.5 302.51,-43 302.51,-50\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123c43d60>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How does $c$ and $d$ influence $L$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Try uncommenting `c += h` or `d += h` and calling `wiggle()` then `wiggle(1.0)`\n",
    "# to see the influence of c or d on L\n",
    "def wiggle(h = 0.0):\n",
    "    a = Value(4.0, label='a')\n",
    "    b = Value(-3.0, label='b')\n",
    "    c = Value(8.0, label='c')\n",
    "    # c += h\n",
    "\n",
    "    d = a*b; d.label = 'd'\n",
    "    # d += h\n",
    "\n",
    "    e = d + c; e.label = 'e'\n",
    "    f = Value(2.0, label='f')\n",
    "\n",
    "    L = e*f; L.label = 'L'\n",
    "    print(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=-8.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=-8.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle(1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bEerwcjPP394",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Propagating Back Again\n",
    "\n",
    "Now we want to calculate\n",
    "\n",
    "$$ \\frac{\\partial{L}}{\\partial{b}} \\hspace{10pt} \\textrm{and} \\hspace{10pt} \\frac{\\partial{L}}{\\partial{a}}$$\n",
    "\n",
    "But we have\n",
    "\n",
    "$$ \\frac{\\partial{L}}{\\partial{d}}$$\n",
    "\n",
    "and we know that\n",
    "\n",
    "$$ \\frac{\\partial{d}}{\\partial{b}} = \\frac{\\partial{}}{\\partial{b}}(a\\cdot b) = a$$\n",
    "\n",
    "so again from the chain rule\n",
    "\n",
    "$$\\frac{\\partial{L}}{\\partial{b}} \n",
    "  = \\frac{\\partial{L}}{\\partial{d}} \\cdot \\frac{\\partial{d}}{\\partial{b}}\n",
    "  = \\frac{\\partial{L}}{\\partial{d}} \\cdot a$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "JsZ69n-6P395",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1147pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 1146.50 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 1142.5,-124 1142.5,4 -4,4\"/>\n",
       "<!-- 4895599728 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895599728</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"320.25,-83.5 320.25,-119.5 503.25,-119.5 503.25,-83.5 320.25,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"342.25,-84 342.25,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"422,-84 422,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"462.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440+ -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895599440+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"572.25\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"572.25\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895599728&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895599728&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M503.57,-85.46C514.47,-83.53 525.06,-81.66 534.51,-79.99\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"535.08,-83.44 544.32,-78.26 533.86,-76.55 535.08,-83.44\"/>\n",
       "</g>\n",
       "<!-- 4895600304 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895600304</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"948.75,-27.5 948.75,-63.5 1138.5,-63.5 1138.5,-27.5 948.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"960.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"973,-28 973,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1015.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1057.25,-28 1057.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1097.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4895600304* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895600304*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"885.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"885.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895600304*&#45;&gt;4895600304 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895600304*&#45;&gt;4895600304</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M913.19,-45.5C920.34,-45.5 928.53,-45.5 937.24,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"937.01,-49 947.01,-45.5 937.01,-42 937.01,-49\"/>\n",
       "</g>\n",
       "<!-- 4895599392 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895599392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0.38,-56.5 0.38,-92.5 187.88,-92.5 187.88,-56.5 0.38,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.38,-57 22.38,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"62.25\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"102.12,-57 102.12,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"145\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;6.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895600544*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"251.25\" cy=\"-46.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"251.25\" y=\"-41.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895599392&#45;&gt;4895600544* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895599392&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.05,-57.73C196.96,-56.12 205.59,-54.56 213.44,-53.14\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"213.99,-56.6 223.21,-51.38 212.75,-49.71 213.99,-56.6\"/>\n",
       "</g>\n",
       "<!-- 4895599440 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895599440</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"635.25,-55.5 635.25,-91.5 822.75,-91.5 822.75,-55.5 635.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"657.25,-56 657.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"699.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"741.5,-56 741.5,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"782.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440&#45;&gt;4895600304* -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895599440&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M823.14,-56.65C831.9,-55.06 840.38,-53.53 848.09,-52.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"848.46,-55.62 857.68,-50.4 847.21,-48.73 848.46,-55.62\"/>\n",
       "</g>\n",
       "<!-- 4895599440+&#45;&gt;4895599440 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895599440+&#45;&gt;4895599440</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M599.5,-73.5C606.66,-73.5 614.87,-73.5 623.6,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"623.39,-77 633.39,-73.5 623.39,-70 623.39,-77\"/>\n",
       "</g>\n",
       "<!-- 4895596896 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895596896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-1.5 0,-37.5 188.25,-37.5 188.25,-1.5 0,-1.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-2 22.75,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"107,-2 107,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"147.62\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 8.0000</text>\n",
       "</g>\n",
       "<!-- 4895596896&#45;&gt;4895600544* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895596896&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.49,-35.75C197.27,-37.28 205.77,-38.76 213.51,-40.1\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"212.66,-43.51 223.11,-41.78 213.86,-36.61 212.66,-43.51\"/>\n",
       "</g>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"636,-0.5 636,-36.5 822,-36.5 822,-0.5 636,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"656.5,-1 656.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"696.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"736.25,-1 736.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"779.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895600016&#45;&gt;4895600304* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895600016&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M822.25,-34.6C831.39,-36.19 840.23,-37.73 848.25,-39.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"847.41,-42.54 857.87,-40.81 848.62,-35.64 847.41,-42.54\"/>\n",
       "</g>\n",
       "<!-- 4895600544 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895600544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"314.25,-28.5 314.25,-64.5 509.25,-64.5 509.25,-28.5 314.25,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"325.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-29 337,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"428,-29 428,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"468.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895600544&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M509.52,-62.98C518.33,-64.49 526.85,-65.94 534.59,-67.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"533.74,-70.66 544.19,-68.89 534.92,-63.76 533.74,-70.66\"/>\n",
       "</g>\n",
       "<!-- 4895600544*&#45;&gt;4895600544 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895600544*&#45;&gt;4895600544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M278.42,-46.5C285.59,-46.5 293.84,-46.5 302.63,-46.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"302.51,-50 312.51,-46.5 302.51,-43 302.51,-50\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123ce9850>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b.grad = a.data * d.grad\n",
    "a.grad = b.data * d.grad\n",
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Try uncommenting `a += h` or `b += h` and calling `wiggle()` then `wiggle(1.0)`\n",
    "# to see the influence of a or b on L\n",
    "def wiggle(h = 0.0):\n",
    "    a = Value(4.0, label='a')\n",
    "    # a += h\n",
    "    b = Value(-3.0, label='b')\n",
    "    b += h\n",
    "    c = Value(8.0, label='c')\n",
    "\n",
    "    d = a*b; d.label = 'd'\n",
    "\n",
    "    e = d + c; e.label = 'e'\n",
    "    f = Value(2.0, label='f')\n",
    "\n",
    "    L = e*f; L.label = 'L'\n",
    "    print(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=-8.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value(data=0.0)\n"
     ]
    }
   ],
   "source": [
    "wiggle(1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tcq9FkAGP395",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Recap\n",
    "\n",
    "As you saw, we recursively went backwards through the computation graph and applied the local gradients to the gradients calculated so far to get the partial gradients. Put another we propagated this calculations backwards through the graph.\n",
    "\n",
    "Of course, in practice, we will only need the gradients on the parameters, not the inputs, so we won't bother calculating them on inputs.\n",
    "\n",
    "_That is the essence of Back Propagation._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8WZ1VSdKP395",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## A Step in Optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the graph again. Assume we want the value of L to _decrease_. We are free to change the values of the leaf nodes -- all the other nodes are derived from children and leaf nodes.\n",
    "\n",
    "The leaf nodes are $a, b, c$ and $f$.\n",
    "\n",
    "> Again, in practice we would only update the parameter leaf nodes, not the input leaf node, but we'll ignore that distinction temporarily for this exmaple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "id": "KIPB3tFwP395",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1147pt\" height=\"128pt\"\n",
       " viewBox=\"0.00 0.00 1146.50 128.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 124)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-124 1142.5,-124 1142.5,4 -4,4\"/>\n",
       "<!-- 4895599728 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895599728</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"320.25,-83.5 320.25,-119.5 503.25,-119.5 503.25,-83.5 320.25,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.25\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">c</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"342.25,-84 342.25,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"422,-84 422,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"462.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440+ -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895599440+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"572.25\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"572.25\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895599728&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895599728&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M503.57,-85.46C514.47,-83.53 525.06,-81.66 534.51,-79.99\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"535.08,-83.44 544.32,-78.26 533.86,-76.55 535.08,-83.44\"/>\n",
       "</g>\n",
       "<!-- 4895600304 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895600304</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"948.75,-27.5 948.75,-63.5 1138.5,-63.5 1138.5,-27.5 948.75,-27.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"960.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">L</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"973,-28 973,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1015.12\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;8.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1057.25,-28 1057.25,-63.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1097.88\" y=\"-40.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4895600304* -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895600304*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"885.75\" cy=\"-45.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"885.75\" y=\"-40.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895600304*&#45;&gt;4895600304 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895600304*&#45;&gt;4895600304</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M913.19,-45.5C920.34,-45.5 928.53,-45.5 937.24,-45.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"937.01,-49 947.01,-45.5 937.01,-42 937.01,-49\"/>\n",
       "</g>\n",
       "<!-- 4895599392 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895599392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0.38,-56.5 0.38,-92.5 187.88,-92.5 187.88,-56.5 0.38,-56.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">a</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.38,-57 22.38,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"62.25\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"102.12,-57 102.12,-92.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"145\" y=\"-69.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;6.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544* -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895600544*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"251.25\" cy=\"-46.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"251.25\" y=\"-41.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895599392&#45;&gt;4895600544* -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895599392&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.05,-57.73C196.96,-56.12 205.59,-54.56 213.44,-53.14\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"213.99,-56.6 223.21,-51.38 212.75,-49.71 213.99,-56.6\"/>\n",
       "</g>\n",
       "<!-- 4895599440 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895599440</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"635.25,-55.5 635.25,-91.5 822.75,-91.5 822.75,-55.5 635.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">e</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"657.25,-56 657.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"699.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;4.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"741.5,-56 741.5,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"782.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895599440&#45;&gt;4895600304* -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895599440&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M823.14,-56.65C831.9,-55.06 840.38,-53.53 848.09,-52.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"848.46,-55.62 857.68,-50.4 847.21,-48.73 848.46,-55.62\"/>\n",
       "</g>\n",
       "<!-- 4895599440+&#45;&gt;4895599440 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895599440+&#45;&gt;4895599440</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M599.5,-73.5C606.66,-73.5 614.87,-73.5 623.6,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"623.39,-77 633.39,-73.5 623.39,-70 623.39,-77\"/>\n",
       "</g>\n",
       "<!-- 4895596896 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895596896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-1.5 0,-37.5 188.25,-37.5 188.25,-1.5 0,-1.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"11.38\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22.75,-2 22.75,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"64.88\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"107,-2 107,-37.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"147.62\" y=\"-14.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 8.0000</text>\n",
       "</g>\n",
       "<!-- 4895596896&#45;&gt;4895600544* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895596896&#45;&gt;4895600544*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M188.49,-35.75C197.27,-37.28 205.77,-38.76 213.51,-40.1\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"212.66,-43.51 223.11,-41.78 213.86,-36.61 212.66,-43.51\"/>\n",
       "</g>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"636,-0.5 636,-36.5 822,-36.5 822,-0.5 636,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"646.25\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"656.5,-1 656.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"696.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"736.25,-1 736.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"779.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;4.0000</text>\n",
       "</g>\n",
       "<!-- 4895600016&#45;&gt;4895600304* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4895600016&#45;&gt;4895600304*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M822.25,-34.6C831.39,-36.19 840.23,-37.73 848.25,-39.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"847.41,-42.54 857.87,-40.81 848.62,-35.64 847.41,-42.54\"/>\n",
       "</g>\n",
       "<!-- 4895600544 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4895600544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"314.25,-28.5 314.25,-64.5 509.25,-64.5 509.25,-28.5 314.25,-28.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"325.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">d</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-29 337,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"382.5\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;12.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"428,-29 428,-64.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"468.62\" y=\"-41.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4895600544&#45;&gt;4895599440+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895600544&#45;&gt;4895599440+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M509.52,-62.98C518.33,-64.49 526.85,-65.94 534.59,-67.25\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"533.74,-70.66 544.19,-68.89 534.92,-63.76 533.74,-70.66\"/>\n",
       "</g>\n",
       "<!-- 4895600544*&#45;&gt;4895600544 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895600544*&#45;&gt;4895600544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M278.42,-46.5C285.59,-46.5 293.84,-46.5 302.63,-46.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"302.51,-50 312.51,-46.5 302.51,-43 302.51,-50\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123ce2670>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the current value of L."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "id": "EDOS_8h4P395",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-8.0\n"
     ]
    }
   ],
   "source": [
    "# remind ourselves what L is\n",
    "print(L.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "As we showed before, we want to nudge each of those leaf nodes by the negative\n",
    "of the gradient, multiplied by a step size, $\\eta$.\n",
    "\n",
    "$$ w_{n+1} = w_n - \\eta * \\frac{\\partial{L}}{\\partial{w_n}} $$\n",
    "\n",
    "where $n$ is the iteration number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "id": "YToAWL-5P395",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-9.230591999999998\n"
     ]
    }
   ],
   "source": [
    "# nudge all the leaf nodes along the negative direction of the gradient\n",
    "step_size = 0.01    # also called eta above\n",
    "\n",
    "a.data -= step_size * a.grad\n",
    "b.data -= step_size * b.grad\n",
    "c.data -= step_size * c.grad\n",
    "f.data -= step_size * f.grad\n",
    "\n",
    "# recompute the forward pass\n",
    "d = a*b\n",
    "e = d + c\n",
    "L = e*f\n",
    "\n",
    "print(L.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R1cBugc3P396",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## A Single Neuron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now programmatically define a single neuron with\n",
    "* two inputs\n",
    "* two weights (1 for each input)\n",
    "* a bias\n",
    "* the ReLU activation function\n",
    "\n",
    "Recall the neuron figure above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "id": "jzqdgbSbP396",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# inputs x0, x1\n",
    "x1 = Value(2.0, label='x1')\n",
    "x2 = Value(0.0, label='x2')\n",
    "\n",
    "# weights w1, w2\n",
    "w1 = Value(-3.0, label='w1')\n",
    "w2 = Value(1.0, label='w2')\n",
    "\n",
    "# bias of the neuron\n",
    "b = Value(6.8813735870195432, label='b')\n",
    "\n",
    "x1w1 = x1*w1; x1w1.label = 'x1*w1'\n",
    "x2w2 = x2*w2; x2w2.label = 'x2*w2'\n",
    "\n",
    "x1w1x2w2 = x1w1 + x2w2; x1w1x2w2.label = 'x1w1 + x2w2'\n",
    "n = x1w1x2w2 + b; n.label = 'n'\n",
    "o = n.relu(); o.label = 'o'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "-2AFiyyrP396",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1566pt\" height=\"210pt\"\n",
       " viewBox=\"0.00 0.00 1565.88 210.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 206)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-206 1561.88,-206 1561.88,4 -4,4\"/>\n",
       "<!-- 4895680016 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895680016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"326.25,-55.5 326.25,-91.5 540,-91.5 540,-55.5 326.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"352.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2*w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"379,-56 379,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"458.75,-56 458.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"499.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895681120+ -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4895681120+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"605.25\" cy=\"-100.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"605.25\" y=\"-95.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895680016&#45;&gt;4895681120+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4895680016&#45;&gt;4895681120+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M540.42,-90.37C549.89,-91.87 559,-93.32 567.21,-94.62\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"566.63,-98.07 577.05,-96.18 567.72,-91.16 566.63,-98.07\"/>\n",
       "</g>\n",
       "<!-- 4895680016* -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4895680016*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895680016*&#45;&gt;4895680016 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4895680016*&#45;&gt;4895680016</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-73.5C296,-73.5 305.08,-73.5 314.82,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"314.55,-77 324.55,-73.5 314.55,-70 314.55,-77\"/>\n",
       "</g>\n",
       "<!-- 4895707632 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4895707632</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-165.5 0,-201.5 198,-201.5 198,-165.5 0,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"16.25\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"32.5,-166 32.5,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"116.75,-166 116.75,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895679056* -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4895679056*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4895707632&#45;&gt;4895679056* -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>4895707632&#45;&gt;4895679056*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M172.12,-165.01C180.9,-162.35 189.7,-159.5 198,-156.5 208.09,-152.86 218.82,-148.27 228.47,-143.88\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"229.86,-147.09 237.45,-139.69 226.9,-140.75 229.86,-147.09\"/>\n",
       "</g>\n",
       "<!-- 4895679056 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4895679056</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"324,-110.5 324,-146.5 542.25,-146.5 542.25,-110.5 324,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"350.38\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1*w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"376.75,-111 376.75,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"461,-111 461,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"501.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895679056&#45;&gt;4895681120+ -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>4895679056&#45;&gt;4895681120+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M542.35,-110.69C551.12,-109.24 559.54,-107.86 567.19,-106.6\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"567.72,-110.06 577.02,-104.98 566.58,-103.15 567.72,-110.06\"/>\n",
       "</g>\n",
       "<!-- 4895679056*&#45;&gt;4895679056 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4895679056*&#45;&gt;4895679056</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-128.5C295.29,-128.5 303.43,-128.5 312.17,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"312.01,-132 322.01,-128.5 312.01,-125 312.01,-132\"/>\n",
       "</g>\n",
       "<!-- 4895681120 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4895681120</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"668.25,-82.5 668.25,-118.5 925.5,-118.5 925.5,-82.5 668.25,-82.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"714.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1w1 + x2w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"760,-83 760,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"802.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"844.25,-83 844.25,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"884.88\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895681936+ -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>4895681936+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"988.5\" cy=\"-127.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"988.5\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4895681120&#45;&gt;4895681936+ -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>4895681120&#45;&gt;4895681936+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M925.93,-118.73C934.56,-119.96 942.77,-121.13 950.21,-122.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"949.71,-125.65 960.1,-123.6 950.69,-118.72 949.71,-125.65\"/>\n",
       "</g>\n",
       "<!-- 4895681120+&#45;&gt;4895681120 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4895681120+&#45;&gt;4895681120</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M632.62,-100.5C639.63,-100.5 647.71,-100.5 656.45,-100.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"656.32,-104 666.32,-100.5 656.32,-97 656.32,-104\"/>\n",
       "</g>\n",
       "<!-- 4895709360 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4895709360</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"705,-137.5 705,-173.5 888.75,-173.5 888.75,-137.5 705,-137.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"716.38\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"727.75,-138 727.75,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.62\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 6.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"807.5,-138 807.5,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"848.12\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895709360&#45;&gt;4895681936+ -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4895709360&#45;&gt;4895681936+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M889.08,-142.03C910.92,-138.81 933.02,-135.54 950.73,-132.93\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"951.08,-136.42 960.46,-131.49 950.06,-129.49 951.08,-136.42\"/>\n",
       "</g>\n",
       "<!-- 4896400144 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4896400144</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3.75,-55.5 3.75,-91.5 194.25,-91.5 194.25,-55.5 3.75,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"33.25,-56 33.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"113,-56 113,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4896400144&#45;&gt;4895680016* -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>4896400144&#45;&gt;4895680016*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M194.46,-73.5C204.21,-73.5 213.66,-73.5 222.21,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.12,-77 232.12,-73.5 222.12,-70 222.12,-77\"/>\n",
       "</g>\n",
       "<!-- 4895681936 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4895681936</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1051.5,-109.5 1051.5,-145.5 1235.25,-145.5 1235.25,-109.5 1051.5,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1062.88\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">n</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1074.25,-110 1074.25,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1114.12\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1154,-110 1154,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1194.62\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895680496ReLU -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>4895680496ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1304.69\" cy=\"-127.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1304.69\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 4895681936&#45;&gt;4895680496ReLU -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>4895681936&#45;&gt;4895680496ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1235.66,-127.5C1243.88,-127.5 1251.92,-127.5 1259.44,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1259.25,-131 1269.25,-127.5 1259.25,-124 1259.25,-131\"/>\n",
       "</g>\n",
       "<!-- 4895681936+&#45;&gt;4895681936 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4895681936+&#45;&gt;4895681936</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1015.78,-127.5C1022.92,-127.5 1031.11,-127.5 1039.82,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1039.56,-131 1049.56,-127.5 1039.56,-124 1039.56,-131\"/>\n",
       "</g>\n",
       "<!-- 4895708592 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>4895708592</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.25,-0.5 2.25,-36.5 195.75,-36.5 195.75,-0.5 2.25,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"34.75,-1 34.75,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"114.5,-1 114.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"155.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895708592&#45;&gt;4895680016* -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4895708592&#45;&gt;4895680016*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M168.91,-36.94C178.74,-39.93 188.67,-43.15 198,-46.5 207.96,-50.07 218.58,-54.47 228.18,-58.68\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"226.54,-61.78 237.1,-62.67 229.4,-55.39 226.54,-61.78\"/>\n",
       "</g>\n",
       "<!-- 4896402400 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>4896402400</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3.75,-110.5 3.75,-146.5 194.25,-146.5 194.25,-110.5 3.75,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"33.25,-111 33.25,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.12\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"113,-111 113,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4896402400&#45;&gt;4895679056* -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4896402400&#45;&gt;4895679056*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M194.46,-128.5C204.21,-128.5 213.66,-128.5 222.21,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.12,-132 232.12,-128.5 222.12,-125 222.12,-132\"/>\n",
       "</g>\n",
       "<!-- 4895680496 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>4895680496</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1374.13,-109.5 1374.13,-145.5 1557.88,-145.5 1557.88,-109.5 1374.13,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1385.5\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">o</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1396.88,-110 1396.88,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1436.75\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1476.63,-110 1476.63,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1517.25\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4895680496ReLU&#45;&gt;4895680496 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4895680496ReLU&#45;&gt;4895680496</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1338.31,-127.5C1345.65,-127.5 1353.83,-127.5 1362.39,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1362.31,-131 1372.31,-127.5 1362.31,-124 1362.31,-131\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123cdcf40>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(o)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zZXrUumbP396",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "The only new operation we've added is the ReLU, so let's take a quick look at how we \n",
    "differentiate the ReLU. \n",
    "\n",
    "Like before, for the output node, o:\n",
    "\n",
    "$$ \\frac{\\partial o}{\\partial o} = 1 $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "id": "ERCeRfigP396",
    "tags": []
   },
   "outputs": [],
   "source": [
    "o.grad = 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<!-- Credit: _Understanding Deep Learning_, Figure 7.6 -->\n",
    "ReLU is technically not differentiable at 0, but practically we implement the\n",
    "derivative as 0 when $ \\le 0$ and 1 when $ 1 > 0 $\n",
    "\n",
    "<center>\n",
    "\n",
    "<img src=\"figs/NN-figs/Train2ReLUDeriv.svg\" width=\"50%\">\n",
    "\n",
    "</center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "id": "UaDJKbOjP396",
    "tags": []
   },
   "outputs": [],
   "source": [
    "n.grad = (o.data > 0) * o.grad  # = 0 when o.data <= 0; = o.grad when o.data > 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JjQqsriiP396",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Coding Backpropagation\n",
    "\n",
    "Now we'll update our `Value` class once more to support the backward pass.\n",
    "\n",
    "There's a\n",
    "* private `_backward()` function _in each operator_ that implements the local\n",
    "step of the chain rule, and\n",
    "* a `backward()` function in the class that topologically sorts the graph and calls the operator `_backward()` function starting at the end of the graph and going _backward_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "id": "51yG2gFAP396",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# version 9\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data, _children=(), _op='', label=''):\n",
    "        self.data = data\n",
    "        self.grad = 0.0 # default to 0, no impact on the output\n",
    "        self._backward = lambda: None  # by default backward doesn't do anything\n",
    "        self._prev = set(_children)\n",
    "        self._op = _op\n",
    "        self.label = label\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data}, grad={self.grad})\"\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data + other.data, (self, other), '+')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += out.grad\n",
    "            other.grad += out.grad\n",
    "        out._backward = _backward\n",
    "\n",
    "        return out\n",
    "\n",
    "    def __mul__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data * other.data, (self, other), '*')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += other.data * out.grad\n",
    "            other.grad += self.data * out.grad\n",
    "        out._backward = _backward\n",
    "        return out\n",
    "\n",
    "    def relu(self):\n",
    "        out = Value(np.maximum(0, self.data), (self,), 'ReLU')\n",
    "        # out = Value(0 if self.data < 0 else self.data, (self,), 'ReLU')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += (out.data > 0) * out.grad\n",
    "        out._backward = _backward\n",
    "\n",
    "        return out\n",
    "\n",
    "    def backward(self):\n",
    "\n",
    "        # topological order all of the children in the graph\n",
    "        topo = []\n",
    "        visited = set()\n",
    "        def build_topo(v):\n",
    "            if v not in visited:\n",
    "                visited.add(v)\n",
    "                for child in v._prev:\n",
    "                    build_topo(child)\n",
    "                topo.append(v)\n",
    "        build_topo(self)\n",
    "\n",
    "        # go one variable at a time and apply the chain rule to get its gradient\n",
    "        self.grad = 1\n",
    "        for v in reversed(topo):\n",
    "            v._backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We redefined the class so we have to reinitialize the objects and run the operations again.\n",
    "\n",
    "This constitutes the _forward pass_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "id": "lb5q0OtZP397",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# inputs x0, x1\n",
    "x1 = Value(2.0, label='x1')\n",
    "x2 = Value(0.0, label='x2')\n",
    "\n",
    "# weights w1, w2\n",
    "w1 = Value(-3.0, label='w1')\n",
    "w2 = Value(1.0, label='w2')\n",
    "\n",
    "# bias of the neuron\n",
    "#b = Value(6.7, label='b')\n",
    "b = Value(6.8813735870195432, label='b')\n",
    "\n",
    "x1w1 = x1*w1; x1w1.label = 'x1*w1'\n",
    "x2w2 = x2*w2; x2w2.label = 'x2*w2'\n",
    "\n",
    "x1w1x2w2 = x1w1 + x2w2; x1w1x2w2.label = 'x1w1 + x2w2'\n",
    "n = x1w1x2w2 + b; n.label = 'n'\n",
    "o = n.relu(); o.label = 'o'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we've filled the data values for all the nodes, but haven't calculated the gradients."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "m0xjnpuVP397",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1566pt\" height=\"210pt\"\n",
       " viewBox=\"0.00 0.00 1565.88 210.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 206)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-206 1561.88,-206 1561.88,4 -4,4\"/>\n",
       "<!-- 4407747152 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4407747152</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"668.25,-82.5 668.25,-118.5 925.5,-118.5 925.5,-82.5 668.25,-82.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"714.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1w1 + x2w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"760,-83 760,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"802.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"844.25,-83 844.25,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"884.88\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407748544+ -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>4407748544+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"988.5\" cy=\"-127.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"988.5\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4407747152&#45;&gt;4407748544+ -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4407747152&#45;&gt;4407748544+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M925.93,-118.73C934.56,-119.96 942.77,-121.13 950.21,-122.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"949.71,-125.65 960.1,-123.6 950.69,-118.72 949.71,-125.65\"/>\n",
       "</g>\n",
       "<!-- 4407747152+ -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4407747152+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"605.25\" cy=\"-100.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"605.25\" y=\"-95.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4407747152+&#45;&gt;4407747152 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4407747152+&#45;&gt;4407747152</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M632.62,-100.5C639.63,-100.5 647.71,-100.5 656.45,-100.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"656.32,-104 666.32,-100.5 656.32,-97 656.32,-104\"/>\n",
       "</g>\n",
       "<!-- 4407748736 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4407748736</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"324,-110.5 324,-146.5 542.25,-146.5 542.25,-110.5 324,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"350.38\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1*w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"376.75,-111 376.75,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"461,-111 461,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"501.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407748736&#45;&gt;4407747152+ -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>4407748736&#45;&gt;4407747152+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M542.35,-110.69C551.12,-109.24 559.54,-107.86 567.19,-106.6\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"567.72,-110.06 577.02,-104.98 566.58,-103.15 567.72,-110.06\"/>\n",
       "</g>\n",
       "<!-- 4407748736* -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4407748736*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4407748736*&#45;&gt;4407748736 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4407748736*&#45;&gt;4407748736</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-128.5C295.29,-128.5 303.43,-128.5 312.17,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"312.01,-132 322.01,-128.5 312.01,-125 312.01,-132\"/>\n",
       "</g>\n",
       "<!-- 4893764240 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4893764240</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3.75,-55.5 3.75,-91.5 194.25,-91.5 194.25,-55.5 3.75,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"33.25,-56 33.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"113,-56 113,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407750128* -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>4407750128*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4893764240&#45;&gt;4407750128* -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4893764240&#45;&gt;4407750128*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M194.46,-73.5C204.21,-73.5 213.66,-73.5 222.21,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.12,-77 232.12,-73.5 222.12,-70 222.12,-77\"/>\n",
       "</g>\n",
       "<!-- 4407749792 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4407749792</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"705,-137.5 705,-173.5 888.75,-173.5 888.75,-137.5 705,-137.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"716.38\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"727.75,-138 727.75,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.62\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 6.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"807.5,-138 807.5,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"848.12\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407749792&#45;&gt;4407748544+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4407749792&#45;&gt;4407748544+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M889.08,-142.03C910.92,-138.81 933.02,-135.54 950.73,-132.93\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"951.08,-136.42 960.46,-131.49 950.06,-129.49 951.08,-136.42\"/>\n",
       "</g>\n",
       "<!-- 4407750320 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4407750320</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1374.13,-109.5 1374.13,-145.5 1557.88,-145.5 1557.88,-109.5 1374.13,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1385.5\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">o</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1396.88,-110 1396.88,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1436.75\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1476.63,-110 1476.63,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1517.25\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407750320ReLU -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4407750320ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1304.69\" cy=\"-127.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1304.69\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 4407750320ReLU&#45;&gt;4407750320 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4407750320ReLU&#45;&gt;4407750320</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1338.31,-127.5C1345.65,-127.5 1353.83,-127.5 1362.39,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1362.31,-131 1372.31,-127.5 1362.31,-124 1362.31,-131\"/>\n",
       "</g>\n",
       "<!-- 4407749840 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4407749840</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-165.5 0,-201.5 198,-201.5 198,-165.5 0,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"16.25\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"32.5,-166 32.5,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"116.75,-166 116.75,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407749840&#45;&gt;4407748736* -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>4407749840&#45;&gt;4407748736*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M172.12,-165.01C180.9,-162.35 189.7,-159.5 198,-156.5 208.09,-152.86 218.82,-148.27 228.47,-143.88\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"229.86,-147.09 237.45,-139.69 226.9,-140.75 229.86,-147.09\"/>\n",
       "</g>\n",
       "<!-- 4894209264 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4894209264</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3.75,-110.5 3.75,-146.5 194.25,-146.5 194.25,-110.5 3.75,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"33.25,-111 33.25,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.12\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"113,-111 113,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4894209264&#45;&gt;4407748736* -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>4894209264&#45;&gt;4407748736*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M194.46,-128.5C204.21,-128.5 213.66,-128.5 222.21,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.12,-132 232.12,-128.5 222.12,-125 222.12,-132\"/>\n",
       "</g>\n",
       "<!-- 4894208256 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>4894208256</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.25,-0.5 2.25,-36.5 195.75,-36.5 195.75,-0.5 2.25,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"34.75,-1 34.75,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"114.5,-1 114.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"155.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4894208256&#45;&gt;4407750128* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4894208256&#45;&gt;4407750128*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M168.91,-36.94C178.74,-39.93 188.67,-43.15 198,-46.5 207.96,-50.07 218.58,-54.47 228.18,-58.68\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"226.54,-61.78 237.1,-62.67 229.4,-55.39 226.54,-61.78\"/>\n",
       "</g>\n",
       "<!-- 4407748544 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>4407748544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1051.5,-109.5 1051.5,-145.5 1235.25,-145.5 1235.25,-109.5 1051.5,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1062.88\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">n</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1074.25,-110 1074.25,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1114.12\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1154,-110 1154,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1194.62\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407748544&#45;&gt;4407750320ReLU -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>4407748544&#45;&gt;4407750320ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1235.66,-127.5C1243.88,-127.5 1251.92,-127.5 1259.44,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1259.25,-131 1269.25,-127.5 1259.25,-124 1259.25,-131\"/>\n",
       "</g>\n",
       "<!-- 4407748544+&#45;&gt;4407748544 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4407748544+&#45;&gt;4407748544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1015.78,-127.5C1022.92,-127.5 1031.11,-127.5 1039.82,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1039.56,-131 1049.56,-127.5 1039.56,-124 1039.56,-131\"/>\n",
       "</g>\n",
       "<!-- 4407750128 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>4407750128</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"326.25,-55.5 326.25,-91.5 540,-91.5 540,-55.5 326.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"352.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2*w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"379,-56 379,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"458.75,-56 458.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"499.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4407750128&#45;&gt;4407747152+ -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>4407750128&#45;&gt;4407747152+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M540.42,-90.37C549.89,-91.87 559,-93.32 567.21,-94.62\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"566.63,-98.07 577.05,-96.18 567.72,-91.16 566.63,-98.07\"/>\n",
       "</g>\n",
       "<!-- 4407750128*&#45;&gt;4407750128 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4407750128*&#45;&gt;4407750128</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-73.5C296,-73.5 305.08,-73.5 314.82,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"314.55,-77 324.55,-73.5 314.55,-70 314.55,-77\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123b7bee0>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(o)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now, all we have to do is call the `backward()` method of the last node..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "o.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And voila! We have all the gradients!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"1566pt\" height=\"210pt\"\n",
       " viewBox=\"0.00 0.00 1565.88 210.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 206)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-206 1561.88,-206 1561.88,4 -4,4\"/>\n",
       "<!-- 4407747152 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4407747152</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"668.25,-82.5 668.25,-118.5 925.5,-118.5 925.5,-82.5 668.25,-82.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"714.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1w1 + x2w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"760,-83 760,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"802.12\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"844.25,-83 844.25,-118.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"884.88\" y=\"-95.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407748544+ -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>4407748544+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"988.5\" cy=\"-127.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"988.5\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4407747152&#45;&gt;4407748544+ -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4407747152&#45;&gt;4407748544+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M925.93,-118.73C934.56,-119.96 942.77,-121.13 950.21,-122.19\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"949.71,-125.65 960.1,-123.6 950.69,-118.72 949.71,-125.65\"/>\n",
       "</g>\n",
       "<!-- 4407747152+ -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>4407747152+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"605.25\" cy=\"-100.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"605.25\" y=\"-95.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 4407747152+&#45;&gt;4407747152 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>4407747152+&#45;&gt;4407747152</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M632.62,-100.5C639.63,-100.5 647.71,-100.5 656.45,-100.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"656.32,-104 666.32,-100.5 656.32,-97 656.32,-104\"/>\n",
       "</g>\n",
       "<!-- 4407748736 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>4407748736</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"324,-110.5 324,-146.5 542.25,-146.5 542.25,-110.5 324,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"350.38\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1*w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"376.75,-111 376.75,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;6.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"461,-111 461,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"501.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407748736&#45;&gt;4407747152+ -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>4407748736&#45;&gt;4407747152+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M542.35,-110.69C551.12,-109.24 559.54,-107.86 567.19,-106.6\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"567.72,-110.06 577.02,-104.98 566.58,-103.15 567.72,-110.06\"/>\n",
       "</g>\n",
       "<!-- 4407748736* -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4407748736*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4407748736*&#45;&gt;4407748736 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>4407748736*&#45;&gt;4407748736</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-128.5C295.29,-128.5 303.43,-128.5 312.17,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"312.01,-132 322.01,-128.5 312.01,-125 312.01,-132\"/>\n",
       "</g>\n",
       "<!-- 4893764240 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4893764240</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3.75,-55.5 3.75,-91.5 194.25,-91.5 194.25,-55.5 3.75,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"33.25,-56 33.25,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.12\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"113,-56 113,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407750128* -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>4407750128*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 4893764240&#45;&gt;4407750128* -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>4893764240&#45;&gt;4407750128*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M194.46,-73.5C204.21,-73.5 213.66,-73.5 222.21,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.12,-77 232.12,-73.5 222.12,-70 222.12,-77\"/>\n",
       "</g>\n",
       "<!-- 4407749792 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>4407749792</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"705,-137.5 705,-173.5 888.75,-173.5 888.75,-137.5 705,-137.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"716.38\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">b</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"727.75,-138 727.75,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.62\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 6.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"807.5,-138 807.5,-173.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"848.12\" y=\"-150.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407749792&#45;&gt;4407748544+ -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>4407749792&#45;&gt;4407748544+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M889.08,-142.03C910.92,-138.81 933.02,-135.54 950.73,-132.93\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"951.08,-136.42 960.46,-131.49 950.06,-129.49 951.08,-136.42\"/>\n",
       "</g>\n",
       "<!-- 4407750320 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>4407750320</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1374.13,-109.5 1374.13,-145.5 1557.88,-145.5 1557.88,-109.5 1374.13,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1385.5\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">o</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1396.88,-110 1396.88,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1436.75\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1476.63,-110 1476.63,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1517.25\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407750320ReLU -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>4407750320ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1304.69\" cy=\"-127.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1304.69\" y=\"-122.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 4407750320ReLU&#45;&gt;4407750320 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>4407750320ReLU&#45;&gt;4407750320</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1338.31,-127.5C1345.65,-127.5 1353.83,-127.5 1362.39,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1362.31,-131 1372.31,-127.5 1362.31,-124 1362.31,-131\"/>\n",
       "</g>\n",
       "<!-- 4407749840 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>4407749840</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-165.5 0,-201.5 198,-201.5 198,-165.5 0,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"16.25\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">w1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"32.5,-166 32.5,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"116.75,-166 116.75,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"157.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 2.0000</text>\n",
       "</g>\n",
       "<!-- 4407749840&#45;&gt;4407748736* -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>4407749840&#45;&gt;4407748736*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M172.12,-165.01C180.9,-162.35 189.7,-159.5 198,-156.5 208.09,-152.86 218.82,-148.27 228.47,-143.88\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"229.86,-147.09 237.45,-139.69 226.9,-140.75 229.86,-147.09\"/>\n",
       "</g>\n",
       "<!-- 4894209264 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>4894209264</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1.5,-110.5 1.5,-146.5 196.5,-146.5 196.5,-110.5 1.5,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"16.25\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">x1</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"31,-111 31,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"70.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"110.75,-111 110.75,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"153.62\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;3.0000</text>\n",
       "</g>\n",
       "<!-- 4894209264&#45;&gt;4407748736* -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>4894209264&#45;&gt;4407748736*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M196.76,-128.5C205.77,-128.5 214.47,-128.5 222.4,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"222.25,-132 232.25,-128.5 222.25,-125 222.25,-132\"/>\n",
       "</g>\n",
       "<!-- 4894208256 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>4894208256</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.25,-0.5 2.25,-36.5 195.75,-36.5 195.75,-0.5 2.25,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"18.5\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"34.75,-1 34.75,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.62\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"114.5,-1 114.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"155.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 4894208256&#45;&gt;4407750128* -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4894208256&#45;&gt;4407750128*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M168.91,-36.94C178.74,-39.93 188.67,-43.15 198,-46.5 207.96,-50.07 218.58,-54.47 228.18,-58.68\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"226.54,-61.78 237.1,-62.67 229.4,-55.39 226.54,-61.78\"/>\n",
       "</g>\n",
       "<!-- 4407748544 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>4407748544</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1051.5,-109.5 1051.5,-145.5 1235.25,-145.5 1235.25,-109.5 1051.5,-109.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1062.88\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">n</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1074.25,-110 1074.25,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1114.12\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8814</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1154,-110 1154,-145.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1194.62\" y=\"-122.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407748544&#45;&gt;4407750320ReLU -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>4407748544&#45;&gt;4407750320ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1235.66,-127.5C1243.88,-127.5 1251.92,-127.5 1259.44,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1259.25,-131 1269.25,-127.5 1259.25,-124 1259.25,-131\"/>\n",
       "</g>\n",
       "<!-- 4407748544+&#45;&gt;4407748544 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>4407748544+&#45;&gt;4407748544</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1015.78,-127.5C1022.92,-127.5 1031.11,-127.5 1039.82,-127.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1039.56,-131 1049.56,-127.5 1039.56,-124 1039.56,-131\"/>\n",
       "</g>\n",
       "<!-- 4407750128 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>4407750128</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"326.25,-55.5 326.25,-91.5 540,-91.5 540,-55.5 326.25,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"352.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">x2*w2</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"379,-56 379,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"418.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"458.75,-56 458.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"499.38\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 1.0000</text>\n",
       "</g>\n",
       "<!-- 4407750128&#45;&gt;4407747152+ -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>4407750128&#45;&gt;4407747152+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M540.42,-90.37C549.89,-91.87 559,-93.32 567.21,-94.62\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"566.63,-98.07 577.05,-96.18 567.72,-91.16 566.63,-98.07\"/>\n",
       "</g>\n",
       "<!-- 4407750128*&#45;&gt;4407750128 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4407750128*&#45;&gt;4407750128</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M288.21,-73.5C296,-73.5 305.08,-73.5 314.82,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"314.55,-77 324.55,-73.5 314.55,-70 314.55,-77\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x106b8efd0>"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(o)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eS7x_XhbP398",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Accumulating the Gradients\n",
    "\n",
    "The observant viewer will notice that we are accumulating the gradients.\n",
    "\n",
    "That is to handle cases like where a `Value` object is on both sides of the operand like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "10ZTwAWSP398",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=3.0, grad=2.0)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(3.0, label='a')\n",
    "b = a + a ; b.label = 'b'\n",
    "b.backward()\n",
    "\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we didn't have the accumulation, then `a.grad = 1` instead."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5CBZ9-vsP398",
    "tags": []
   },
   "source": [
    "Or the other case where a node goes to different operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "QvRJ9LFZP399",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"194pt\" height=\"45pt\"\n",
       " viewBox=\"0.00 0.00 194.00 45.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 41)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-41 190,-41 190,4 -4,4\"/>\n",
       "<!-- 4895600016 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>4895600016</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-0.5 0,-36.5 186,-36.5 186,-0.5 0,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"10.25\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">f</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"20.5,-1 20.5,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"60.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0400</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"100.25,-1 100.25,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"143.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad &#45;4.0000</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x123b34490>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = Value(-2.0, label='a')\n",
    "b = Value(3.0, label='b')\n",
    "d = a * b  ; d.label = 'd'\n",
    "e = a + b   ; e.label = 'e'\n",
    "  \n",
    "draw_dot( f) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see analytical verification of the above result in the note in the online book.\n",
    "\n",
    "The risk now is that if you don't zero the gradients for the next update iteration, you will have incorrect gradients. \n",
    "\n",
    "> Always remember to zero the gradient in each iteration of the training loop!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oUi8yJM3P399",
    "slideshow": {
     "slide_type": "skip"
    },
    "tags": []
   },
   "source": [
    "```{note}\n",
    "\n",
    "We can verify that the gradients are correct analytically.\n",
    "\n",
    "To find the partial derivative $\\frac{\\partial f}{\\partial a}$, we first need to define $f$ in terms of $a$ and $b$.\n",
    "\n",
    "Given:\n",
    "$$\\begin{aligned}\n",
    "d &= a \\times b \\\\\n",
    "e &= a + b      \\\\\n",
    "f &= d \\times e\n",
    "\\end{aligned}$$\n",
    "\n",
    "Then $f$ can be expanded as:\n",
    "$$\\begin{aligned}\n",
    "f &= (a \\times b) \\times (a + b) \\\\\n",
    "f &= a^2 \\times b + a \\times b^2\n",
    "\\end{aligned}$$\n",
    "\n",
    "Next, we find the partial derivative of $f$ with respect to $a$:\n",
    "$$ \\frac{\\partial f}{\\partial a} = 2a \\times b + b^2 $$\n",
    "\n",
    "Finally, we plug in the given values $a = -2.0$ and $b = 3.0$:\n",
    "$$\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial a} &= 2(-2.0) \\times 3.0 + 3.0^2 \\\\\n",
    "\\frac{\\partial f}{\\partial a} &= -12.0 + 9.0                 \\\\\n",
    "\\frac{\\partial f}{\\partial a} &= -3.0\n",
    "\\end{aligned}$$\n",
    "\n",
    "So the partial derivative $\\frac{\\partial f}{\\partial a}$ for the value $a = -2.0$ is $-3.0$.\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sl3dShRGP399",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Enhancements to `Value` Class\n",
    "\n",
    "There are still some useful operations that `Value` doesn't support, so to be more\n",
    "complete we have the final version of the `Value` class below.\n",
    "\n",
    "We added:\n",
    "* `__radd__` for when the `Value` object is the right operand of an add\n",
    "* `__rmul__` for when the `Value` object is the right operand of a product \n",
    "* `__pow__` to support the ** operator\n",
    "* plus some others you can see below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# version 9\n",
    "class Value:\n",
    "\n",
    "    def __init__(self, data, _children=(), _op='', label=''):\n",
    "        self.data = data\n",
    "        self.grad = 0.0 # default to 0, no impact on the output\n",
    "        self._backward = lambda: None  # by default backward doesn't do anything\n",
    "        self._prev = set(_children)\n",
    "        self._op = _op\n",
    "        self.label = label\n",
    "\n",
    "    def __add__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data + other.data, (self, other), '+')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += out.grad\n",
    "            other.grad += out.grad\n",
    "        out._backward = _backward\n",
    "\n",
    "        return out\n",
    "    \n",
    "    def __mul__(self, other):\n",
    "        other = other if isinstance(other, Value) else Value(other)\n",
    "        out = Value(self.data * other.data, (self, other), '*')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += other.data * out.grad\n",
    "            other.grad += self.data * out.grad\n",
    "        out._backward = _backward\n",
    "        \n",
    "        return out\n",
    "\n",
    "    def __pow__(self, other):\n",
    "        \"\"\"Adding support for ** operator, which we'll need for the \n",
    "        squared loss function\"\"\"\n",
    "        assert isinstance(other, (int, float)), \"only supporting int/float powers for now\"\n",
    "        out = Value(self.data**other, (self,), f'**{other}')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += (other * self.data**(other-1)) * out.grad\n",
    "        out._backward = _backward\n",
    "\n",
    "        return out\n",
    "\n",
    "    def relu(self):\n",
    "        out = Value(np.maximum(0, self.data), (self,), 'ReLU')\n",
    "        # out = Value(0 if self.data < 0 else self.data, (self,), 'ReLU')\n",
    "\n",
    "        def _backward():\n",
    "            self.grad += (out.data > 0) * out.grad\n",
    "        out._backward = _backward\n",
    "\n",
    "        return out\n",
    "\n",
    "    def __neg__(self): # -self\n",
    "        return self * -1\n",
    "\n",
    "    def __radd__(self, other): # other + self\n",
    "        return self + other\n",
    "\n",
    "    def __sub__(self, other): # self - other\n",
    "        return self + (-other)\n",
    "\n",
    "    def __rsub__(self, other): # other - self\n",
    "        return other + (-self)\n",
    "\n",
    "    def __rmul__(self, other): # other * self\n",
    "        return self * other\n",
    "\n",
    "    def __truediv__(self, other): # self / other\n",
    "        return self * other**-1\n",
    "\n",
    "    def __rtruediv__(self, other): # other / self\n",
    "        return other * self**-1\n",
    "    \n",
    "    def backward(self):\n",
    "\n",
    "        # topological order all of the children in the graph\n",
    "        topo = []\n",
    "        visited = set()\n",
    "        def build_topo(v):\n",
    "            if v not in visited:\n",
    "                visited.add(v)\n",
    "                for child in v._prev:\n",
    "                    build_topo(child)\n",
    "                topo.append(v)\n",
    "        build_topo(self)\n",
    "\n",
    "        # go one variable at a time and apply the chain rule to get its gradient\n",
    "        self.grad = 1\n",
    "        for v in reversed(topo):\n",
    "            v._backward()\n",
    "\n",
    "    def __repr__(self):\n",
    "        \"\"\"Return a string representation of the object for display\"\"\"\n",
    "        return f\"Value(data={self.data}, grad={self.grad})\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mU4SFgwAP39_",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Comparing to PyTorch\n",
    "\n",
    "We're using a class implementation that resembles the PyTorch implementation, and in fact we can compare our implementation with PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "id": "OSFutF8zP39_",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "id": "51dJ5c74P39_",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.881373405456543\n",
      "---\n",
      "x2.grad 1.0\n",
      "w2.grad 0.0\n",
      "x1.grad -3.0\n",
      "w1.grad 2.0\n"
     ]
    }
   ],
   "source": [
    "x1 = torch.Tensor([2.0]).double()                ; x1.requires_grad = True\n",
    "x2 = torch.Tensor([0.0]).double()                ; x2.requires_grad = True\n",
    "w1 = torch.Tensor([-3.0]).double()               ; w1.requires_grad = True\n",
    "w2 = torch.Tensor([1.0]).double()                ; w2.requires_grad = True\n",
    "b = torch.Tensor([6.8813735870195432]).double()  ; b.requires_grad = True\n",
    "n = x1*w1 + x2*w2 + b\n",
    "o = torch.relu(n)\n",
    "\n",
    "print(o.data.item())\n",
    "o.backward()\n",
    "\n",
    "print('---')\n",
    "print('x2.grad', x2.grad.item())\n",
    "print('w2.grad', w2.grad.item())\n",
    "print('x1.grad', x1.grad.item())\n",
    "print('w1.grad', w1.grad.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RlcUXqEmP39_",
    "tags": []
   },
   "source": [
    "By default, tensors don't store gradients and so won't support backprop, so we explicitly set `requires_grad = True`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w2sRT-ZZP39_",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Neural Network Modules\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll define some classes which help us build out a small neural network.\n",
    "\n",
    "__Module__ -- A base class\n",
    "\n",
    "__Neuron__ -- Implement a single linear or nonlinear neuron with `nin` inputs.\n",
    "\n",
    "__Layer__ -- Implement a layer of network consisting of `nout` neurons, each taking `nin` inputs\n",
    "\n",
    "__MLP__ -- A _Multi-Layer Perceptron_ that implements `len(nouts)` layers of neurons.\n",
    "\n",
    "Each class can calculate a forward pass and enumerate all its parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "id": "uo2Z3-vsP39_",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import random\n",
    "# we assume that Value class is already defined\n",
    "\n",
    "class Module:\n",
    "    \"\"\"Define a Neural Network Module base class \"\"\"\n",
    "\n",
    "    def zero_grad(self):\n",
    "        \"\"\"When we run in a training loop, we'll need to zero out all the gradients\n",
    "        since they are defined to accumulate in the backwards passes.\"\"\"\n",
    "        for p in self.parameters():\n",
    "            p.grad = 0\n",
    "\n",
    "    def parameters(self):\n",
    "        return []\n",
    "\n",
    "class Neuron(Module):\n",
    "    \"\"\"Define a Neuron as a subclass of Module\"\"\"\n",
    "\n",
    "    def __init__(self, nin, nonlin=True):\n",
    "        \"\"\"Randomly initialize a set of weights, one for each input, and initialize the bias to zero.\"\"\"\n",
    "        self.w = [Value(random.uniform(-1,1)) for _ in range(nin)]\n",
    "        self.b = Value(0.0)\n",
    "        self.nonlin = nonlin\n",
    "\n",
    "    def __call__(self, x):\n",
    "        \"\"\"Implement the forward pass of the neuron\"\"\"\n",
    "        act = sum((wi*xi for wi,xi in zip(self.w, x)), self.b)\n",
    "        return act.relu() if self.nonlin else act\n",
    "\n",
    "    def parameters(self):\n",
    "        return self.w + [self.b]\n",
    "\n",
    "    def __repr__(self):\n",
    "        return f\"{'ReLU' if self.nonlin else 'Linear'}Neuron({len(self.w)})\"\n",
    "\n",
    "class Layer(Module):\n",
    "    \"\"\"Define a Layer of Network as a subclass of Module\"\"\"\n",
    "\n",
    "    def __init__(self, nin, nout, **kwargs):\n",
    "        \"\"\"Initialize nout Neurons, each with nin inputs\"\"\"\n",
    "        self.neurons = [Neuron(nin, **kwargs) for _ in range(nout)]\n",
    "\n",
    "    def __call__(self, x):\n",
    "        \"\"\"Forward pass each neuron in the layer\"\"\"\n",
    "        out = [n(x) for n in self.neurons]\n",
    "        return out[0] if len(out) == 1 else out\n",
    "\n",
    "    def parameters(self):\n",
    "        return [p for n in self.neurons for p in n.parameters()]\n",
    "\n",
    "    def __repr__(self):\n",
    "        return f\"Layer of [{', '.join(str(n) for n in self.neurons)}]\"\n",
    "\n",
    "class MLP(Module):\n",
    "    \"\"\"Define a Multi-Layer Perceptron\"\"\"\n",
    "\n",
    "    def __init__(self, nin: int, nouts: list):\n",
    "        \"\"\"\n",
    "        Initialize the Multi-Layer Perceptron, by initializing each layer\n",
    "        then initializing each neuron of each layer.\n",
    "\n",
    "        Parameters:\n",
    "            nin: Number of inputs (int)\n",
    "            nouts: A list of the number of neurons in each layer\n",
    "        \"\"\"\n",
    "        sz = [nin] + nouts\n",
    "        # self.layers = [Layer(sz[i], sz[i+1]) for i in range(len(nouts))]\n",
    "\n",
    "        # Create a list of layer objects for this MLP. All but the last layer\n",
    "        # have ReLU activations. The last layer is linear.\n",
    "        self.layers = [Layer(sz[i], sz[i+1], nonlin=i!=len(nouts)-1) for i in range(len(nouts))]\n",
    "\n",
    "    def __call__(self, x):\n",
    "        \"\"\"Forward pass through the MLP\"\"\"\n",
    "        for layer in self.layers:\n",
    "            x = layer(x)\n",
    "        return x\n",
    "\n",
    "    def parameters(self):\n",
    "        \"\"\"Recursively retrieve the parameters of the MLP\"\"\"\n",
    "        return [p for layer in self.layers for p in layer.parameters()]\n",
    "\n",
    "    def __repr__(self):\n",
    "        return f\"MLP of [{', '.join(str(layer) for layer in self.layers)}]\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "id": "eS8PQ2nAP39_",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# help(Module)\n",
    "# Module.__doc__\n",
    "# help(Neuron)\n",
    "# help(Layer)\n",
    "# help(MLP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ulC3bIaiP3-A",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Initialize and Evaluate a Neuron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "id": "TQrmA_KnP3-A",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=2.351314740082261, grad=0.0)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2 inputs\n",
    "x = [2.0, 3.0]\n",
    "\n",
    "# initialize neuron with 2 inputs\n",
    "n = Neuron(2, nonlin=False)\n",
    "\n",
    "# evaluate our neuron with our 2 inputs\n",
    "n(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearNeuron(2)"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "id": "kjcN7DwsP3-A",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Value(data=0.5226748811016748, grad=0.0),\n",
       " Value(data=0.4353216592929705, grad=0.0),\n",
       " Value(data=0.0, grad=0.0)]"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# list the 2 weights and the bias\n",
    "n.parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rRWp7vmGP3-A",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Initialize and Evaluate a Layer of Neurons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "id": "qMtq4bOBP3-A",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Value(data=0.7525061164928648, grad=0.0),\n",
       " Value(data=-3.245347683871597, grad=0.0),\n",
       " Value(data=0.4281765105075186, grad=0.0)]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# same 2 inputs again\n",
    "x = [2.0, 3.0]\n",
    "\n",
    "# Now initialize a layer of 3 neurons, each with 2 inputs\n",
    "l = Layer(2, 3, nonlin=False)\n",
    "\n",
    "# Evaluate our layer of neurons with the 2 inputs\n",
    "l(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Layer of [LinearNeuron(2), LinearNeuron(2), LinearNeuron(2)]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "id": "pe8aqee5P3-A",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Value(data=0.34599955314939446, grad=0.0),\n",
       " Value(data=0.020169003398025298, grad=0.0),\n",
       " Value(data=0.0, grad=0.0),\n",
       " Value(data=-0.15415728894929637, grad=0.0),\n",
       " Value(data=-0.9790110353243346, grad=0.0),\n",
       " Value(data=0.0, grad=0.0),\n",
       " Value(data=-0.06169429508650448, grad=0.0),\n",
       " Value(data=0.18385503356017585, grad=0.0),\n",
       " Value(data=0.0, grad=0.0)]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l.parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GUjyoQG2P3-A",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Initialize and Evaluate an MLP\n",
    "\n",
    "We'll instantiate an MLP like the picture below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kL5ihr2uP3-A",
    "tags": []
   },
   "source": [
    "<center>\n",
    "<img src=\"figs/NN-figs/neural_net2.jpeg\" width=\"75%\">\n",
    "</center>\n",
    "\n",
    "\n",
    "From [cs231n](https://cs231n.github.io/convolutional-networks/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "cRsVNlx9P3-A",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Value(data=-0.018605982977735535, grad=0.0)"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [2.0, 3.0, -1.0]\n",
    "m = MLP(3, [4, 4, 1])\n",
    "m(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLP of [Layer of [ReLUNeuron(3), ReLUNeuron(3), ReLUNeuron(3), ReLUNeuron(3)], Layer of [ReLUNeuron(4), ReLUNeuron(4), ReLUNeuron(4), ReLUNeuron(4)], Layer of [LinearNeuron(4)]]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "# m.parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "id": "NtrC1vh0P3-A",
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"5355pt\" height=\"1035pt\"\n",
       " viewBox=\"0.00 0.00 5355.38 1035.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 1031)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-1031 5351.38,-1031 5351.38,4 -4,4\"/>\n",
       "<!-- 5497405504 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>5497405504</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-440.5 933.75,-476.5 1119,-476.5 1119,-440.5 933.75,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-441 953.5,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0965</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-441 1037.75,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497405936+ -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>5497405936+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-458.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-453.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497405504&#45;&gt;5497405936+ -->\n",
       "<g id=\"edge90\" class=\"edge\">\n",
       "<title>5497405504&#45;&gt;5497405936+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1119.4,-458.5C1127.77,-458.5 1135.89,-458.5 1143.34,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.28,-462 1153.28,-458.5 1143.28,-455 1143.28,-462\"/>\n",
       "</g>\n",
       "<!-- 5497405504+ -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>5497405504+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"870.75\" cy=\"-458.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"870.75\" y=\"-453.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497405504+&#45;&gt;5497405504 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>5497405504+&#45;&gt;5497405504</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M898.16,-458.5C905.2,-458.5 913.25,-458.5 921.8,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"921.79,-462 931.79,-458.5 921.79,-455 921.79,-462\"/>\n",
       "</g>\n",
       "<!-- 5497421936 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>5497421936</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-550.5 1895.5,-586.5 2076.25,-586.5 2076.25,-550.5 1895.5,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-551 1915.25,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-551 1995,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497422128+ -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>5497422128+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-458.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-453.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497421936&#45;&gt;5497422128+ -->\n",
       "<g id=\"edge81\" class=\"edge\">\n",
       "<title>5497421936&#45;&gt;5497422128+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2063.01,-550.03C2068.43,-547.53 2073.66,-544.7 2078.5,-541.5 2100.46,-526.99 2118.98,-503.31 2131.19,-484.95\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2134.05,-486.97 2136.5,-476.66 2128.16,-483.2 2134.05,-486.97\"/>\n",
       "</g>\n",
       "<!-- 5497421936* -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>5497421936*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-568.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-563.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497421936*&#45;&gt;5497421936 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>5497421936*&#45;&gt;5497421936</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1851.25,-568.5C1860.6,-568.5 1871.76,-568.5 1883.59,-568.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1883.58,-572 1893.58,-568.5 1883.58,-565 1883.58,-572\"/>\n",
       "</g>\n",
       "<!-- 5497405696 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>5497405696</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"624.75,-385.5 624.75,-421.5 805.5,-421.5 805.5,-385.5 624.75,-385.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"634.62\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"644.5,-386 644.5,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"724.25,-386 724.25,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"764.88\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497405744* -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>5497405744*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"870.75\" cy=\"-403.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"870.75\" y=\"-398.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497405696&#45;&gt;5497405744* -->\n",
       "<g id=\"edge203\" class=\"edge\">\n",
       "<title>5497405696&#45;&gt;5497405744*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M805.93,-403.5C815.11,-403.5 824.03,-403.5 832.15,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"831.9,-407 841.9,-403.5 831.9,-400 831.9,-407\"/>\n",
       "</g>\n",
       "<!-- 5497422128 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5497422128</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-440.5 2219.63,-476.5 2400.38,-476.5 2400.38,-440.5 2219.63,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-441 2239.38,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-441 2319.13,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497422512+ -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>5497422512+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-403.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-398.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497422128&#45;&gt;5497422512+ -->\n",
       "<g id=\"edge106\" class=\"edge\">\n",
       "<title>5497422128&#45;&gt;5497422512+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2376.33,-440.14C2385.22,-437.38 2394.18,-434.46 2402.63,-431.5 2414.47,-427.34 2427.25,-422.28 2438.52,-417.61\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2439.53,-420.98 2447.39,-413.87 2436.82,-414.53 2439.53,-420.98\"/>\n",
       "</g>\n",
       "<!-- 5497422128+&#45;&gt;5497422128 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>5497422128+&#45;&gt;5497422128</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-458.5C2184.72,-458.5 2195.89,-458.5 2207.71,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2207.71,-462 2217.71,-458.5 2207.71,-455 2207.71,-462\"/>\n",
       "</g>\n",
       "<!-- 5497405744 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>5497405744</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-385.5 933.75,-421.5 1119,-421.5 1119,-385.5 933.75,-385.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-386 953.5,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0325</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-386 1037.75,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497405744&#45;&gt;5497405936+ -->\n",
       "<g id=\"edge195\" class=\"edge\">\n",
       "<title>5497405744&#45;&gt;5497405936+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1090.78,-421.84C1100.3,-424.89 1109.95,-428.16 1119,-431.5 1128.84,-435.13 1139.37,-439.52 1148.9,-443.69\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1147.21,-446.77 1157.77,-447.64 1150.06,-440.38 1147.21,-446.77\"/>\n",
       "</g>\n",
       "<!-- 5497405744*&#45;&gt;5497405744 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>5497405744*&#45;&gt;5497405744</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M898.16,-403.5C905.2,-403.5 913.25,-403.5 921.8,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"921.79,-407 931.79,-403.5 921.79,-400 921.79,-407\"/>\n",
       "</g>\n",
       "<!-- 5497422320 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>5497422320</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2217.38,-385.5 2217.38,-421.5 2402.63,-421.5 2402.63,-385.5 2217.38,-385.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2227.25\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2237.13,-386 2237.13,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2321.38,-386 2321.38,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2362\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497422320&#45;&gt;5497422512+ -->\n",
       "<g id=\"edge183\" class=\"edge\">\n",
       "<title>5497422320&#45;&gt;5497422512+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2402.72,-403.5C2413.53,-403.5 2424.04,-403.5 2433.45,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.21,-407 2443.21,-403.5 2433.21,-400 2433.21,-407\"/>\n",
       "</g>\n",
       "<!-- 5497422320* -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>5497422320*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-403.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-398.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497422320*&#45;&gt;5497422320 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>5497422320*&#45;&gt;5497422320</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-403.5C2184.23,-403.5 2194.71,-403.5 2205.84,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2205.64,-407 2215.64,-403.5 2205.64,-400 2205.64,-407\"/>\n",
       "</g>\n",
       "<!-- 5497405936 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>5497405936</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1245,-440.5 1245,-476.5 1430.25,-476.5 1430.25,-440.5 1245,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1254.88\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1264.75,-441 1264.75,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.1290</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1349,-441 1349,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1389.62\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406368+ -->\n",
       "<g id=\"node22\" class=\"node\">\n",
       "<title>5497406368+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-513.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-508.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497405936&#45;&gt;5497406368+ -->\n",
       "<g id=\"edge185\" class=\"edge\">\n",
       "<title>5497405936&#45;&gt;5497406368+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1401.43,-476.97C1411.12,-480.03 1420.99,-483.26 1430.25,-486.5 1441.92,-490.58 1454.54,-495.43 1465.73,-499.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1464.37,-503.11 1474.95,-503.61 1466.99,-496.62 1464.37,-503.11\"/>\n",
       "</g>\n",
       "<!-- 5497405936+&#45;&gt;5497405936 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5497405936+&#45;&gt;5497405936</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1209.41,-458.5C1216.45,-458.5 1224.5,-458.5 1233.05,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1233.04,-462 1243.04,-458.5 1233.04,-455 1233.04,-462\"/>\n",
       "</g>\n",
       "<!-- 5497422512 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>5497422512</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-385.5 2543.76,-421.5 2724.51,-421.5 2724.51,-385.5 2543.76,-385.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-386 2563.51,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-386 2643.26,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497422896+ -->\n",
       "<g id=\"node26\" class=\"node\">\n",
       "<title>5497422896+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-623.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-618.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497422512&#45;&gt;5497422896+ -->\n",
       "<g id=\"edge73\" class=\"edge\">\n",
       "<title>5497422512&#45;&gt;5497422896+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2715.36,-421.86C2719.52,-424.64 2723.37,-427.83 2726.76,-431.5 2777.36,-486.35 2728.15,-529.39 2762.76,-595.5 2763.64,-597.2 2764.66,-598.87 2765.77,-600.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2762.87,-602.48 2771.87,-608.07 2768.32,-598.09 2762.87,-602.48\"/>\n",
       "</g>\n",
       "<!-- 5497422512+&#45;&gt;5497422512 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>5497422512+&#45;&gt;5497422512</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-403.5C2508.85,-403.5 2520.01,-403.5 2531.84,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2531.84,-407 2541.84,-403.5 2531.84,-400 2531.84,-407\"/>\n",
       "</g>\n",
       "<!-- 5497406128 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>5497406128</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-550.5 933.75,-586.5 1119,-586.5 1119,-550.5 933.75,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-551 953.5,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-551 1037.75,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406176* -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>5497406176*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-513.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-508.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497406128&#45;&gt;5497406176* -->\n",
       "<g id=\"edge162\" class=\"edge\">\n",
       "<title>5497406128&#45;&gt;5497406176*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1093.79,-550.11C1102.33,-547.4 1110.92,-544.5 1119,-541.5 1128.98,-537.8 1139.61,-533.23 1149.2,-528.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1150.53,-532.11 1158.13,-524.73 1147.58,-525.77 1150.53,-532.11\"/>\n",
       "</g>\n",
       "<!-- 5497406176 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>5497406176</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-495.5 1247.25,-531.5 1428,-531.5 1428,-495.5 1247.25,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-496 1267,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.4152</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-496 1346.75,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406176&#45;&gt;5497406368+ -->\n",
       "<g id=\"edge97\" class=\"edge\">\n",
       "<title>5497406176&#45;&gt;5497406368+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1428.48,-513.5C1439.83,-513.5 1450.91,-513.5 1460.79,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1460.68,-517 1470.68,-513.5 1460.68,-510 1460.68,-517\"/>\n",
       "</g>\n",
       "<!-- 5497406176*&#45;&gt;5497406176 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>5497406176*&#45;&gt;5497406176</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1209.41,-513.5C1217.12,-513.5 1226.04,-513.5 1235.51,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1235.32,-517 1245.32,-513.5 1235.32,-510 1235.32,-517\"/>\n",
       "</g>\n",
       "<!-- 5497422704 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>5497422704</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-605.5 2543.76,-641.5 2724.51,-641.5 2724.51,-605.5 2543.76,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-606 2563.51,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0449</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-606 2643.26,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497422704&#45;&gt;5497422896+ -->\n",
       "<g id=\"edge152\" class=\"edge\">\n",
       "<title>5497422704&#45;&gt;5497422896+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2724.94,-623.5C2734.11,-623.5 2743.04,-623.5 2751.16,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2750.91,-627 2760.91,-623.5 2750.91,-620 2750.91,-627\"/>\n",
       "</g>\n",
       "<!-- 5497422704* -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>5497422704*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-623.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-618.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497422704*&#45;&gt;5497422704 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>5497422704*&#45;&gt;5497422704</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-623.5C2508.85,-623.5 2520.01,-623.5 2531.84,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2531.84,-627 2541.84,-623.5 2531.84,-620 2531.84,-627\"/>\n",
       "</g>\n",
       "<!-- 5497406368 -->\n",
       "<g id=\"node21\" class=\"node\">\n",
       "<title>5497406368</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-495.5 1569.13,-531.5 1754.38,-531.5 1754.38,-495.5 1569.13,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-496 1588.88,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.7138</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-496 1673.13,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406416ReLU -->\n",
       "<g id=\"node24\" class=\"node\">\n",
       "<title>5497406416ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-513.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-508.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497406368&#45;&gt;5497406416ReLU -->\n",
       "<g id=\"edge115\" class=\"edge\">\n",
       "<title>5497406368&#45;&gt;5497406416ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1754.47,-513.5C1762.81,-513.5 1770.98,-513.5 1778.6,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1778.57,-517 1788.57,-513.5 1778.57,-510 1778.57,-517\"/>\n",
       "</g>\n",
       "<!-- 5497406368+&#45;&gt;5497406368 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>5497406368+&#45;&gt;5497406368</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1527.12,-513.5C1535.98,-513.5 1546.45,-513.5 1557.59,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1557.39,-517 1567.39,-513.5 1557.39,-510 1557.39,-517\"/>\n",
       "</g>\n",
       "<!-- 5497406416 -->\n",
       "<g id=\"node23\" class=\"node\">\n",
       "<title>5497406416</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-495.5 1895.5,-531.5 2076.25,-531.5 2076.25,-495.5 1895.5,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-496 1915.25,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-496 1995,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406416&#45;&gt;5497422320* -->\n",
       "<g id=\"edge202\" class=\"edge\">\n",
       "<title>5497406416&#45;&gt;5497422320*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2064.33,-495.16C2069.33,-492.63 2074.11,-489.76 2078.5,-486.5 2101.96,-469.08 2095.26,-453.48 2114.5,-431.5 2116.62,-429.09 2118.94,-426.69 2121.34,-424.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2123.41,-427.22 2128.49,-417.92 2118.72,-422.03 2123.41,-427.22\"/>\n",
       "</g>\n",
       "<!-- 5497424000* -->\n",
       "<g id=\"node53\" class=\"node\">\n",
       "<title>5497424000*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-623.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-618.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497406416&#45;&gt;5497424000* -->\n",
       "<g id=\"edge189\" class=\"edge\">\n",
       "<title>5497406416&#45;&gt;5497424000*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2060.65,-531.81C2066.91,-534.61 2072.96,-537.82 2078.5,-541.5 2100.14,-555.86 2118.59,-579.05 2130.86,-597.12\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2127.79,-598.82 2136.19,-605.27 2133.65,-594.98 2127.79,-598.82\"/>\n",
       "</g>\n",
       "<!-- 5497425680* -->\n",
       "<g id=\"node99\" class=\"node\">\n",
       "<title>5497425680*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-733.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-728.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497406416&#45;&gt;5497425680* -->\n",
       "<g id=\"edge76\" class=\"edge\">\n",
       "<title>5497406416&#45;&gt;5497425680*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2067.11,-531.86C2071.27,-534.64 2075.12,-537.83 2078.5,-541.5 2129.1,-596.35 2076.3,-641.39 2114.5,-705.5 2116.14,-708.26 2118.15,-710.88 2120.36,-713.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2117.63,-715.57 2127.28,-719.94 2122.47,-710.5 2117.63,-715.57\"/>\n",
       "</g>\n",
       "<!-- 5497435616* -->\n",
       "<g id=\"node139\" class=\"node\">\n",
       "<title>5497435616*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-348.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-343.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497406416&#45;&gt;5497435616* -->\n",
       "<g id=\"edge131\" class=\"edge\">\n",
       "<title>5497406416&#45;&gt;5497435616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2067.53,-495.09C2071.47,-492.58 2075.17,-489.73 2078.5,-486.5 2115.41,-450.67 2085.99,-419.32 2114.5,-376.5 2116.28,-373.83 2118.38,-371.26 2120.64,-368.84\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2122.74,-371.67 2127.63,-362.27 2117.95,-366.58 2122.74,-371.67\"/>\n",
       "</g>\n",
       "<!-- 5497406416ReLU&#45;&gt;5497406416 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>5497406416ReLU&#45;&gt;5497406416</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1857.59,-513.5C1865.61,-513.5 1874.62,-513.5 1884.04,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1883.74,-517 1893.74,-513.5 1883.74,-510 1883.74,-517\"/>\n",
       "</g>\n",
       "<!-- 5497422896 -->\n",
       "<g id=\"node25\" class=\"node\">\n",
       "<title>5497422896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-605.5 2855.01,-641.5 3035.76,-641.5 3035.76,-605.5 2855.01,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-606 2874.76,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0449</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-606 2954.51,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497423280+ -->\n",
       "<g id=\"node34\" class=\"node\">\n",
       "<title>5497423280+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3101.01\" cy=\"-431.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3101.01\" y=\"-426.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497422896&#45;&gt;5497423280+ -->\n",
       "<g id=\"edge177\" class=\"edge\">\n",
       "<title>5497422896&#45;&gt;5497423280+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3026.44,-605.18C3030.58,-602.65 3034.48,-599.77 3038.01,-596.5 3048.39,-586.84 3076.24,-504.66 3090.82,-460.05\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3094.08,-461.35 3093.84,-450.76 3087.42,-459.19 3094.08,-461.35\"/>\n",
       "</g>\n",
       "<!-- 5497422896+&#45;&gt;5497422896 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>5497422896+&#45;&gt;5497422896</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-623.5C2824.87,-623.5 2833.79,-623.5 2843.26,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.08,-627 2853.08,-623.5 2843.08,-620 2843.08,-627\"/>\n",
       "</g>\n",
       "<!-- 5497406704 -->\n",
       "<g id=\"node27\" class=\"node\">\n",
       "<title>5497406704</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"624.75,-687.5 624.75,-723.5 805.5,-723.5 805.5,-687.5 624.75,-687.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"634.62\" y=\"-700.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"644.5,-688 644.5,-723.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-700.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"724.25,-688 724.25,-723.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"764.88\" y=\"-700.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406752* -->\n",
       "<g id=\"node31\" class=\"node\">\n",
       "<title>5497406752*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"870.75\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"870.75\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497406704&#45;&gt;5497406752* -->\n",
       "<g id=\"edge103\" class=\"edge\">\n",
       "<title>5497406704&#45;&gt;5497406752*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M805.93,-689.72C815.48,-688.04 824.76,-686.41 833.14,-684.93\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"833.61,-688.41 842.85,-683.23 832.39,-681.51 833.61,-688.41\"/>\n",
       "</g>\n",
       "<!-- 5497423088 -->\n",
       "<g id=\"node28\" class=\"node\">\n",
       "<title>5497423088</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-276.5 2855.01,-312.5 3035.76,-312.5 3035.76,-276.5 2855.01,-276.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-289.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-277 2874.76,-312.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-289.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-277 2954.51,-312.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-289.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497423088&#45;&gt;5497423280+ -->\n",
       "<g id=\"edge165\" class=\"edge\">\n",
       "<title>5497423088&#45;&gt;5497423280+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3024.98,-312.99C3029.59,-315.48 3033.98,-318.3 3038.01,-321.5 3064.49,-342.57 3081.57,-378.22 3090.97,-403.21\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3087.6,-404.18 3094.24,-412.43 3094.19,-401.83 3087.6,-404.18\"/>\n",
       "</g>\n",
       "<!-- 5497423088* -->\n",
       "<g id=\"node29\" class=\"node\">\n",
       "<title>5497423088*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-293.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-288.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497423088*&#45;&gt;5497423088 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>5497423088*&#45;&gt;5497423088</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-293.67C2824.87,-293.72 2833.79,-293.78 2843.26,-293.84\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.05,-297.34 2853.08,-293.91 2843.1,-290.34 2843.05,-297.34\"/>\n",
       "</g>\n",
       "<!-- 5497406752 -->\n",
       "<g id=\"node30\" class=\"node\">\n",
       "<title>5497406752</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-660.5 933.75,-696.5 1119,-696.5 1119,-660.5 933.75,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-661 953.5,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.6044</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-661 1037.75,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497406944+ -->\n",
       "<g id=\"node38\" class=\"node\">\n",
       "<title>5497406944+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497406752&#45;&gt;5497406944+ -->\n",
       "<g id=\"edge87\" class=\"edge\">\n",
       "<title>5497406752&#45;&gt;5497406944+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1119.4,-678.5C1127.77,-678.5 1135.89,-678.5 1143.34,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.28,-682 1153.28,-678.5 1143.28,-675 1143.28,-682\"/>\n",
       "</g>\n",
       "<!-- 5497406752*&#45;&gt;5497406752 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>5497406752*&#45;&gt;5497406752</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M898.16,-678.5C905.2,-678.5 913.25,-678.5 921.8,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"921.79,-682 931.79,-678.5 921.79,-675 921.79,-682\"/>\n",
       "</g>\n",
       "<!-- 5495006592 -->\n",
       "<g id=\"node32\" class=\"node\">\n",
       "<title>5495006592</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"936,-770.5 936,-806.5 1116.75,-806.5 1116.75,-770.5 936,-770.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"945.88\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"955.75,-771 955.75,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.5134</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1035.5,-771 1035.5,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1076.12\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407184* -->\n",
       "<g id=\"node42\" class=\"node\">\n",
       "<title>5497407184*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-733.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-728.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495006592&#45;&gt;5497407184* -->\n",
       "<g id=\"edge118\" class=\"edge\">\n",
       "<title>5495006592&#45;&gt;5497407184*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1093.79,-770.11C1102.33,-767.4 1110.92,-764.5 1119,-761.5 1128.98,-757.8 1139.61,-753.23 1149.2,-748.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1150.53,-752.11 1158.13,-744.73 1147.58,-745.77 1150.53,-752.11\"/>\n",
       "</g>\n",
       "<!-- 5497423280 -->\n",
       "<g id=\"node33\" class=\"node\">\n",
       "<title>5497423280</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3166.26,-413.5 3166.26,-449.5 3347.01,-449.5 3347.01,-413.5 3166.26,-413.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3176.13\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3186.01,-414 3186.01,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3225.88\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0449</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3265.76,-414 3265.76,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3306.38\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497423328ReLU -->\n",
       "<g id=\"node36\" class=\"node\">\n",
       "<title>5497423328ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3418.69\" cy=\"-431.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3418.69\" y=\"-426.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497423280&#45;&gt;5497423328ReLU -->\n",
       "<g id=\"edge184\" class=\"edge\">\n",
       "<title>5497423280&#45;&gt;5497423328ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3347.49,-431.5C3356.49,-431.5 3365.32,-431.5 3373.52,-431.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3373.51,-435 3383.51,-431.5 3373.51,-428 3373.51,-435\"/>\n",
       "</g>\n",
       "<!-- 5497423280+&#45;&gt;5497423280 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>5497423280+&#45;&gt;5497423280</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3128.41,-431.5C3136.12,-431.5 3145.04,-431.5 3154.51,-431.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3154.33,-435 3164.33,-431.5 3154.33,-428 3154.33,-435\"/>\n",
       "</g>\n",
       "<!-- 5497423328 -->\n",
       "<g id=\"node35\" class=\"node\">\n",
       "<title>5497423328</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-413.5 3490.38,-449.5 3671.13,-449.5 3671.13,-413.5 3490.38,-413.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-414 3510.13,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0449</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-414 3589.88,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436912* -->\n",
       "<g id=\"node162\" class=\"node\">\n",
       "<title>5497436912*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3736.38\" cy=\"-396.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3736.38\" y=\"-391.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497423328&#45;&gt;5497436912* -->\n",
       "<g id=\"edge161\" class=\"edge\">\n",
       "<title>5497423328&#45;&gt;5497436912*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3662.62,-413.08C3675.49,-410.14 3688.22,-407.25 3699.34,-404.71\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3699.83,-408.19 3708.81,-402.55 3698.28,-401.36 3699.83,-408.19\"/>\n",
       "</g>\n",
       "<!-- 5497423328ReLU&#45;&gt;5497423328 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>5497423328ReLU&#45;&gt;5497423328</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3452.47,-431.5C3460.49,-431.5 3469.5,-431.5 3478.92,-431.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3478.62,-435 3488.62,-431.5 3478.62,-428 3478.62,-435\"/>\n",
       "</g>\n",
       "<!-- 5497406944 -->\n",
       "<g id=\"node37\" class=\"node\">\n",
       "<title>5497406944</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1245,-660.5 1245,-696.5 1430.25,-696.5 1430.25,-660.5 1245,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1254.88\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1264.75,-661 1264.75,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.6044</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1349,-661 1349,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1389.62\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407376+ -->\n",
       "<g id=\"node46\" class=\"node\">\n",
       "<title>5497407376+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497406944&#45;&gt;5497407376+ -->\n",
       "<g id=\"edge74\" class=\"edge\">\n",
       "<title>5497406944&#45;&gt;5497407376+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1430.34,-678.5C1441.15,-678.5 1451.66,-678.5 1461.08,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1460.84,-682 1470.84,-678.5 1460.84,-675 1460.84,-682\"/>\n",
       "</g>\n",
       "<!-- 5497406944+&#45;&gt;5497406944 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>5497406944+&#45;&gt;5497406944</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1209.41,-678.5C1216.45,-678.5 1224.5,-678.5 1233.05,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1233.04,-682 1243.04,-678.5 1233.04,-675 1233.04,-682\"/>\n",
       "</g>\n",
       "<!-- 5495006736 -->\n",
       "<g id=\"node39\" class=\"node\">\n",
       "<title>5495006736</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-632.5 622.5,-668.5 807.75,-668.5 807.75,-632.5 622.5,-632.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-645.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-633 642.25,-668.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-645.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.3022</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-633 726.5,-668.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-645.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495006736&#45;&gt;5497406752* -->\n",
       "<g id=\"edge199\" class=\"edge\">\n",
       "<title>5495006736&#45;&gt;5497406752*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M808.15,-667.27C816.89,-668.87 825.36,-670.41 833.08,-671.82\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"832.19,-675.21 842.66,-673.56 833.45,-668.33 832.19,-675.21\"/>\n",
       "</g>\n",
       "<!-- 5497407136 -->\n",
       "<g id=\"node40\" class=\"node\">\n",
       "<title>5497407136</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"936,-715.5 936,-751.5 1116.75,-751.5 1116.75,-715.5 936,-715.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"945.88\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"955.75,-716 955.75,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1035.5,-716 1035.5,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1076.12\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407136&#45;&gt;5497407184* -->\n",
       "<g id=\"edge190\" class=\"edge\">\n",
       "<title>5497407136&#45;&gt;5497407184*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1117.18,-733.5C1126.36,-733.5 1135.28,-733.5 1143.4,-733.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.15,-737 1153.15,-733.5 1143.15,-730 1143.15,-737\"/>\n",
       "</g>\n",
       "<!-- 5497407184 -->\n",
       "<g id=\"node41\" class=\"node\">\n",
       "<title>5497407184</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-715.5 1247.25,-751.5 1428,-751.5 1428,-715.5 1247.25,-715.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-716 1267,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 1.5402</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-716 1346.75,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407184&#45;&gt;5497407376+ -->\n",
       "<g id=\"edge88\" class=\"edge\">\n",
       "<title>5497407184&#45;&gt;5497407376+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1403.96,-715.14C1412.84,-712.38 1421.8,-709.46 1430.25,-706.5 1442.09,-702.34 1454.87,-697.28 1466.14,-692.61\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1467.15,-695.98 1475.01,-688.87 1464.44,-689.53 1467.15,-695.98\"/>\n",
       "</g>\n",
       "<!-- 5497407184*&#45;&gt;5497407184 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>5497407184*&#45;&gt;5497407184</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1209.41,-733.5C1217.12,-733.5 1226.04,-733.5 1235.51,-733.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1235.32,-737 1245.32,-733.5 1235.32,-730 1235.32,-737\"/>\n",
       "</g>\n",
       "<!-- 5497423616 -->\n",
       "<g id=\"node43\" class=\"node\">\n",
       "<title>5497423616</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-825.5 1895.5,-861.5 2076.25,-861.5 2076.25,-825.5 1895.5,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-826 1915.25,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-826 1995,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497423808+ -->\n",
       "<g id=\"node48\" class=\"node\">\n",
       "<title>5497423808+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-843.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-838.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497423616&#45;&gt;5497423808+ -->\n",
       "<g id=\"edge172\" class=\"edge\">\n",
       "<title>5497423616&#45;&gt;5497423808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-843.5C2088.08,-843.5 2099.16,-843.5 2109.05,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2108.93,-847 2118.93,-843.5 2108.93,-840 2108.93,-847\"/>\n",
       "</g>\n",
       "<!-- 5497423616* -->\n",
       "<g id=\"node44\" class=\"node\">\n",
       "<title>5497423616*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-843.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-838.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497423616*&#45;&gt;5497423616 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>5497423616*&#45;&gt;5497423616</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1851.25,-843.5C1860.6,-843.5 1871.76,-843.5 1883.59,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1883.58,-847 1893.58,-843.5 1883.58,-840 1883.58,-847\"/>\n",
       "</g>\n",
       "<!-- 5497407376 -->\n",
       "<g id=\"node45\" class=\"node\">\n",
       "<title>5497407376</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1571.38,-660.5 1571.38,-696.5 1752.13,-696.5 1752.13,-660.5 1571.38,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1581.25\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1591.13,-661 1591.13,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.9358</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1670.88,-661 1670.88,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1711.5\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407808+ -->\n",
       "<g id=\"node55\" class=\"node\">\n",
       "<title>5497407808+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497407376&#45;&gt;5497407808+ -->\n",
       "<g id=\"edge79\" class=\"edge\">\n",
       "<title>5497407376&#45;&gt;5497407808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1752.61,-678.5C1763.96,-678.5 1775.04,-678.5 1784.92,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.8,-682 1794.8,-678.5 1784.8,-675 1784.8,-682\"/>\n",
       "</g>\n",
       "<!-- 5497407376+&#45;&gt;5497407376 -->\n",
       "<g id=\"edge20\" class=\"edge\">\n",
       "<title>5497407376+&#45;&gt;5497407376</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1527.12,-678.5C1536.47,-678.5 1547.63,-678.5 1559.46,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1559.46,-682 1569.46,-678.5 1559.46,-675 1559.46,-682\"/>\n",
       "</g>\n",
       "<!-- 5497423808 -->\n",
       "<g id=\"node47\" class=\"node\">\n",
       "<title>5497423808</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-825.5 2219.63,-861.5 2400.38,-861.5 2400.38,-825.5 2219.63,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-826 2239.38,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-826 2319.13,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424192+ -->\n",
       "<g id=\"node57\" class=\"node\">\n",
       "<title>5497424192+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-733.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-728.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497423808&#45;&gt;5497424192+ -->\n",
       "<g id=\"edge157\" class=\"edge\">\n",
       "<title>5497423808&#45;&gt;5497424192+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2388.45,-825.16C2393.45,-822.63 2398.24,-819.76 2402.63,-816.5 2426.08,-799.08 2419.38,-783.48 2438.63,-761.5 2440.74,-759.09 2443.06,-756.69 2445.46,-754.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2447.53,-757.22 2452.61,-747.92 2442.84,-752.03 2447.53,-757.22\"/>\n",
       "</g>\n",
       "<!-- 5497423808+&#45;&gt;5497423808 -->\n",
       "<g id=\"edge21\" class=\"edge\">\n",
       "<title>5497423808+&#45;&gt;5497423808</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-843.5C2184.72,-843.5 2195.89,-843.5 2207.71,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2207.71,-847 2217.71,-843.5 2207.71,-840 2207.71,-847\"/>\n",
       "</g>\n",
       "<!-- 5497407568 -->\n",
       "<g id=\"node49\" class=\"node\">\n",
       "<title>5497407568</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1245,-550.5 1245,-586.5 1430.25,-586.5 1430.25,-550.5 1245,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1254.88\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1264.75,-551 1264.75,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1349,-551 1349,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1389.62\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407616* -->\n",
       "<g id=\"node51\" class=\"node\">\n",
       "<title>5497407616*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-623.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-618.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497407568&#45;&gt;5497407616* -->\n",
       "<g id=\"edge193\" class=\"edge\">\n",
       "<title>5497407568&#45;&gt;5497407616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1401.43,-586.97C1411.12,-590.03 1420.99,-593.26 1430.25,-596.5 1441.92,-600.58 1454.54,-605.43 1465.73,-609.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1464.37,-613.11 1474.95,-613.61 1466.99,-606.62 1464.37,-613.11\"/>\n",
       "</g>\n",
       "<!-- 5497407616 -->\n",
       "<g id=\"node50\" class=\"node\">\n",
       "<title>5497407616</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-605.5 1569.13,-641.5 1754.38,-641.5 1754.38,-605.5 1569.13,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-606 1588.88,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.7947</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-606 1673.13,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407616&#45;&gt;5497407808+ -->\n",
       "<g id=\"edge174\" class=\"edge\">\n",
       "<title>5497407616&#45;&gt;5497407808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1725.56,-641.97C1735.25,-645.03 1745.11,-648.26 1754.38,-651.5 1766.04,-655.58 1778.67,-660.43 1789.86,-664.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1788.49,-668.11 1799.08,-668.61 1791.11,-661.62 1788.49,-668.11\"/>\n",
       "</g>\n",
       "<!-- 5497407616*&#45;&gt;5497407616 -->\n",
       "<g id=\"edge22\" class=\"edge\">\n",
       "<title>5497407616*&#45;&gt;5497407616</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1527.12,-623.5C1535.98,-623.5 1546.45,-623.5 1557.59,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1557.39,-627 1567.39,-623.5 1557.39,-620 1557.39,-627\"/>\n",
       "</g>\n",
       "<!-- 5497424000 -->\n",
       "<g id=\"node52\" class=\"node\">\n",
       "<title>5497424000</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-715.5 2219.63,-751.5 2400.38,-751.5 2400.38,-715.5 2219.63,-715.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-716 2239.38,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-716 2319.13,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424000&#45;&gt;5497424192+ -->\n",
       "<g id=\"edge119\" class=\"edge\">\n",
       "<title>5497424000&#45;&gt;5497424192+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2400.86,-733.5C2412.21,-733.5 2423.29,-733.5 2433.17,-733.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.05,-737 2443.05,-733.5 2433.05,-730 2433.05,-737\"/>\n",
       "</g>\n",
       "<!-- 5497424000*&#45;&gt;5497424000 -->\n",
       "<g id=\"edge23\" class=\"edge\">\n",
       "<title>5497424000*&#45;&gt;5497424000</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2166.27,-636.91C2171.41,-641.29 2176.87,-646.35 2181.38,-651.5 2200.62,-673.48 2193.93,-689.08 2217.38,-706.5 2218.75,-707.52 2220.16,-708.5 2221.61,-709.45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2219.83,-712.46 2230.24,-714.41 2223.31,-706.39 2219.83,-712.46\"/>\n",
       "</g>\n",
       "<!-- 5497407808 -->\n",
       "<g id=\"node54\" class=\"node\">\n",
       "<title>5497407808</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-660.5 1895.5,-696.5 2076.25,-696.5 2076.25,-660.5 1895.5,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-661 1915.25,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.1411</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-661 1995,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407856ReLU -->\n",
       "<g id=\"node59\" class=\"node\">\n",
       "<title>5497407856ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-678.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497407808&#45;&gt;5497407856ReLU -->\n",
       "<g id=\"edge98\" class=\"edge\">\n",
       "<title>5497407808&#45;&gt;5497407856ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-678.5C2085.74,-678.5 2094.57,-678.5 2102.77,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2102.76,-682 2112.76,-678.5 2102.76,-675 2102.76,-682\"/>\n",
       "</g>\n",
       "<!-- 5497407808+&#45;&gt;5497407808 -->\n",
       "<g id=\"edge24\" class=\"edge\">\n",
       "<title>5497407808+&#45;&gt;5497407808</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1851.25,-678.5C1860.6,-678.5 1871.76,-678.5 1883.59,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1883.58,-682 1893.58,-678.5 1883.58,-675 1883.58,-682\"/>\n",
       "</g>\n",
       "<!-- 5497424192 -->\n",
       "<g id=\"node56\" class=\"node\">\n",
       "<title>5497424192</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-715.5 2543.76,-751.5 2724.51,-751.5 2724.51,-715.5 2543.76,-715.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-716 2563.51,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-716 2643.26,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424576+ -->\n",
       "<g id=\"node70\" class=\"node\">\n",
       "<title>5497424576+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497424192&#45;&gt;5497424576+ -->\n",
       "<g id=\"edge140\" class=\"edge\">\n",
       "<title>5497424192&#45;&gt;5497424576+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2701.55,-715.11C2710.09,-712.4 2718.67,-709.5 2726.76,-706.5 2736.73,-702.8 2747.36,-698.23 2756.96,-693.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2758.28,-697.11 2765.88,-689.73 2755.34,-690.77 2758.28,-697.11\"/>\n",
       "</g>\n",
       "<!-- 5497424192+&#45;&gt;5497424192 -->\n",
       "<g id=\"edge25\" class=\"edge\">\n",
       "<title>5497424192+&#45;&gt;5497424192</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-733.5C2508.85,-733.5 2520.01,-733.5 2531.84,-733.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2531.84,-737 2541.84,-733.5 2531.84,-730 2531.84,-737\"/>\n",
       "</g>\n",
       "<!-- 5497407856 -->\n",
       "<g id=\"node58\" class=\"node\">\n",
       "<title>5497407856</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-605.5 2219.63,-641.5 2400.38,-641.5 2400.38,-605.5 2219.63,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-606 2239.38,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.1411</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-606 2319.13,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497407856&#45;&gt;5497422704* -->\n",
       "<g id=\"edge176\" class=\"edge\">\n",
       "<title>5497407856&#45;&gt;5497422704*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2400.86,-623.5C2412.21,-623.5 2423.29,-623.5 2433.17,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.05,-627 2443.05,-623.5 2433.05,-620 2433.05,-627\"/>\n",
       "</g>\n",
       "<!-- 5497424384* -->\n",
       "<g id=\"node64\" class=\"node\">\n",
       "<title>5497424384*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-678.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-673.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497407856&#45;&gt;5497424384* -->\n",
       "<g id=\"edge197\" class=\"edge\">\n",
       "<title>5497407856&#45;&gt;5497424384*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2373.81,-641.97C2383.5,-645.03 2393.36,-648.26 2402.63,-651.5 2414.29,-655.58 2426.92,-660.43 2438.11,-664.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2436.75,-668.11 2447.33,-668.61 2439.36,-661.62 2436.75,-668.11\"/>\n",
       "</g>\n",
       "<!-- 5497434320* -->\n",
       "<g id=\"node109\" class=\"node\">\n",
       "<title>5497434320*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-788.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-783.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497407856&#45;&gt;5497434320* -->\n",
       "<g id=\"edge122\" class=\"edge\">\n",
       "<title>5497407856&#45;&gt;5497434320*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2389.95,-641.75C2394.54,-644.55 2398.82,-647.78 2402.63,-651.5 2439.11,-687.17 2410.29,-718.07 2438.63,-760.5 2440.41,-763.17 2442.51,-765.73 2444.78,-768.15\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2442.08,-770.42 2451.76,-774.72 2446.88,-765.32 2442.08,-770.42\"/>\n",
       "</g>\n",
       "<!-- 5497436000* -->\n",
       "<g id=\"node145\" class=\"node\">\n",
       "<title>5497436000*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-513.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-508.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497407856&#45;&gt;5497436000* -->\n",
       "<g id=\"edge105\" class=\"edge\">\n",
       "<title>5497407856&#45;&gt;5497436000*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2387.13,-605.03C2392.56,-602.53 2397.79,-599.7 2402.63,-596.5 2424.58,-581.99 2443.1,-558.31 2455.32,-539.95\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2458.18,-541.97 2460.62,-531.66 2452.28,-538.2 2458.18,-541.97\"/>\n",
       "</g>\n",
       "<!-- 5497407856ReLU&#45;&gt;5497407856 -->\n",
       "<g id=\"edge26\" class=\"edge\">\n",
       "<title>5497407856ReLU&#45;&gt;5497407856</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.08,-667.62C2187.76,-662.49 2203.27,-656.43 2217.38,-651.5 2223.17,-649.48 2229.19,-647.46 2235.26,-645.48\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2236.3,-648.82 2244.75,-642.43 2234.16,-642.15 2236.3,-648.82\"/>\n",
       "</g>\n",
       "<!-- 5495007648 -->\n",
       "<g id=\"node60\" class=\"node\">\n",
       "<title>5495007648</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"311.25,-880.5 311.25,-916.5 496.5,-916.5 496.5,-880.5 311.25,-880.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"321.12\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"331,-881 331,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.8909</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"415.25,-881 415.25,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"455.88\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298896* -->\n",
       "<g id=\"node166\" class=\"node\">\n",
       "<title>5495298896*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"559.5\" cy=\"-898.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"559.5\" y=\"-893.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495007648&#45;&gt;5495298896* -->\n",
       "<g id=\"edge100\" class=\"edge\">\n",
       "<title>5495007648&#45;&gt;5495298896*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M496.9,-898.5C505.27,-898.5 513.39,-898.5 520.84,-898.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"520.78,-902 530.78,-898.5 520.78,-895 520.78,-902\"/>\n",
       "</g>\n",
       "<!-- 5495007696 -->\n",
       "<g id=\"node61\" class=\"node\">\n",
       "<title>5495007696</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-330.5 622.5,-366.5 807.75,-366.5 807.75,-330.5 622.5,-330.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-331 642.25,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0108</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-331 726.5,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495007696&#45;&gt;5497405744* -->\n",
       "<g id=\"edge187\" class=\"edge\">\n",
       "<title>5495007696&#45;&gt;5497405744*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M779.53,-366.84C789.05,-369.89 798.7,-373.16 807.75,-376.5 817.59,-380.13 828.12,-384.52 837.65,-388.69\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"835.96,-391.77 846.52,-392.64 838.81,-385.38 835.96,-391.77\"/>\n",
       "</g>\n",
       "<!-- 5495007744 -->\n",
       "<g id=\"node62\" class=\"node\">\n",
       "<title>5495007744</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-495.5 933.75,-531.5 1119,-531.5 1119,-495.5 933.75,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-496 953.5,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.4152</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-496 1037.75,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495007744&#45;&gt;5497406176* -->\n",
       "<g id=\"edge113\" class=\"edge\">\n",
       "<title>5495007744&#45;&gt;5497406176*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1119.4,-513.5C1127.77,-513.5 1135.89,-513.5 1143.34,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.28,-517 1153.28,-513.5 1143.28,-510 1143.28,-517\"/>\n",
       "</g>\n",
       "<!-- 5497424384 -->\n",
       "<g id=\"node63\" class=\"node\">\n",
       "<title>5497424384</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2541.51,-660.5 2541.51,-696.5 2726.76,-696.5 2726.76,-660.5 2541.51,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2551.38\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2561.26,-661 2561.26,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0280</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2645.51,-661 2645.51,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2686.13\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424384&#45;&gt;5497424576+ -->\n",
       "<g id=\"edge191\" class=\"edge\">\n",
       "<title>5497424384&#45;&gt;5497424576+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2727.16,-678.5C2735.53,-678.5 2743.64,-678.5 2751.09,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2751.04,-682 2761.04,-678.5 2751.04,-675 2751.04,-682\"/>\n",
       "</g>\n",
       "<!-- 5497424384*&#45;&gt;5497424384 -->\n",
       "<g id=\"edge27\" class=\"edge\">\n",
       "<title>5497424384*&#45;&gt;5497424384</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-678.5C2508.36,-678.5 2518.83,-678.5 2529.96,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2529.76,-682 2539.76,-678.5 2529.76,-675 2529.76,-682\"/>\n",
       "</g>\n",
       "<!-- 5495007792 -->\n",
       "<g id=\"node65\" class=\"node\">\n",
       "<title>5495007792</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"624.75,-495.5 624.75,-531.5 805.5,-531.5 805.5,-495.5 624.75,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"634.62\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"644.5,-496 644.5,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"724.25,-496 724.25,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"764.88\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495007792&#45;&gt;5497405504+ -->\n",
       "<g id=\"edge132\" class=\"edge\">\n",
       "<title>5495007792&#45;&gt;5497405504+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M782.54,-495.11C791.08,-492.4 799.67,-489.5 807.75,-486.5 817.73,-482.8 828.36,-478.23 837.95,-473.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"839.28,-477.11 846.88,-469.73 836.33,-470.77 839.28,-477.11\"/>\n",
       "</g>\n",
       "<!-- 5497408144 -->\n",
       "<g id=\"node66\" class=\"node\">\n",
       "<title>5497408144</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"936,-138.5 936,-174.5 1116.75,-174.5 1116.75,-138.5 936,-138.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"945.88\" y=\"-151.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"955.75,-139 955.75,-174.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-151.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1035.5,-139 1035.5,-174.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1076.12\" y=\"-151.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497408192* -->\n",
       "<g id=\"node68\" class=\"node\">\n",
       "<title>5497408192*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497408144&#45;&gt;5497408192* -->\n",
       "<g id=\"edge179\" class=\"edge\">\n",
       "<title>5497408144&#45;&gt;5497408192*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1117.18,-140.13C1126.73,-138.39 1136.01,-136.7 1144.39,-135.17\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1144.89,-138.64 1154.1,-133.4 1143.64,-131.75 1144.89,-138.64\"/>\n",
       "</g>\n",
       "<!-- 5497408192 -->\n",
       "<g id=\"node67\" class=\"node\">\n",
       "<title>5497408192</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-110.5 1247.25,-146.5 1428,-146.5 1428,-110.5 1247.25,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-111 1267,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.7867</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-111 1346.75,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497408384+ -->\n",
       "<g id=\"node72\" class=\"node\">\n",
       "<title>5497408384+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497408192&#45;&gt;5497408384+ -->\n",
       "<g id=\"edge112\" class=\"edge\">\n",
       "<title>5497408192&#45;&gt;5497408384+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1428.48,-128.5C1439.83,-128.5 1450.91,-128.5 1460.79,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1460.68,-132 1470.68,-128.5 1460.68,-125 1460.68,-132\"/>\n",
       "</g>\n",
       "<!-- 5497408192*&#45;&gt;5497408192 -->\n",
       "<g id=\"edge28\" class=\"edge\">\n",
       "<title>5497408192*&#45;&gt;5497408192</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1209.41,-128.5C1217.12,-128.5 1226.04,-128.5 1235.51,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1235.32,-132 1245.32,-128.5 1235.32,-125 1235.32,-132\"/>\n",
       "</g>\n",
       "<!-- 5497424576 -->\n",
       "<g id=\"node69\" class=\"node\">\n",
       "<title>5497424576</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2852.76,-660.5 2852.76,-696.5 3038.01,-696.5 3038.01,-660.5 2852.76,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2862.63\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2872.51,-661 2872.51,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0280</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2956.76,-661 2956.76,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2997.38\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424960+ -->\n",
       "<g id=\"node77\" class=\"node\">\n",
       "<title>5497424960+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3101.01\" cy=\"-541.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3101.01\" y=\"-536.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497424576&#45;&gt;5497424960+ -->\n",
       "<g id=\"edge84\" class=\"edge\">\n",
       "<title>5497424576&#45;&gt;5497424960+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3024.98,-660.01C3029.59,-657.52 3033.98,-654.7 3038.01,-651.5 3064.49,-630.43 3081.57,-594.78 3090.97,-569.79\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3094.19,-571.17 3094.24,-560.57 3087.6,-568.82 3094.19,-571.17\"/>\n",
       "</g>\n",
       "<!-- 5497424576+&#45;&gt;5497424576 -->\n",
       "<g id=\"edge29\" class=\"edge\">\n",
       "<title>5497424576+&#45;&gt;5497424576</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-678.5C2824.2,-678.5 2832.25,-678.5 2840.81,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2840.79,-682 2850.79,-678.5 2840.79,-675 2840.79,-682\"/>\n",
       "</g>\n",
       "<!-- 5497408384 -->\n",
       "<g id=\"node71\" class=\"node\">\n",
       "<title>5497408384</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1571.38,-110.5 1571.38,-146.5 1752.13,-146.5 1752.13,-110.5 1571.38,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1581.25\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1591.13,-111 1591.13,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.7867</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1670.88,-111 1670.88,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1711.5\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497408816+ -->\n",
       "<g id=\"node84\" class=\"node\">\n",
       "<title>5497408816+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-128.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-123.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497408384&#45;&gt;5497408816+ -->\n",
       "<g id=\"edge101\" class=\"edge\">\n",
       "<title>5497408384&#45;&gt;5497408816+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1752.61,-128.5C1763.96,-128.5 1775.04,-128.5 1784.92,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.8,-132 1794.8,-128.5 1784.8,-125 1784.8,-132\"/>\n",
       "</g>\n",
       "<!-- 5497408384+&#45;&gt;5497408384 -->\n",
       "<g id=\"edge30\" class=\"edge\">\n",
       "<title>5497408384+&#45;&gt;5497408384</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1527.12,-128.5C1536.47,-128.5 1547.63,-128.5 1559.46,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1559.46,-132 1569.46,-128.5 1559.46,-125 1559.46,-132\"/>\n",
       "</g>\n",
       "<!-- 5497424768 -->\n",
       "<g id=\"node73\" class=\"node\">\n",
       "<title>5497424768</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-495.5 2855.01,-531.5 3035.76,-531.5 3035.76,-495.5 2855.01,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-496 2874.76,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-496 2954.51,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497424768&#45;&gt;5497424960+ -->\n",
       "<g id=\"edge158\" class=\"edge\">\n",
       "<title>5497424768&#45;&gt;5497424960+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3036.19,-529.87C3045.74,-531.61 3055.01,-533.3 3063.4,-534.83\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3062.64,-538.25 3073.11,-536.6 3063.9,-531.36 3062.64,-538.25\"/>\n",
       "</g>\n",
       "<!-- 5497424768* -->\n",
       "<g id=\"node74\" class=\"node\">\n",
       "<title>5497424768*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-513.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-508.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497424768*&#45;&gt;5497424768 -->\n",
       "<g id=\"edge31\" class=\"edge\">\n",
       "<title>5497424768*&#45;&gt;5497424768</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-513.5C2824.87,-513.5 2833.79,-513.5 2843.26,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.08,-517 2853.08,-513.5 2843.08,-510 2843.08,-517\"/>\n",
       "</g>\n",
       "<!-- 5497408576 -->\n",
       "<g id=\"node75\" class=\"node\">\n",
       "<title>5497408576</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-0.5 1247.25,-36.5 1428,-36.5 1428,-0.5 1247.25,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-1 1267,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-1 1346.75,-36.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-13.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497408624* -->\n",
       "<g id=\"node80\" class=\"node\">\n",
       "<title>5497408624*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-73.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-68.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497408576&#45;&gt;5497408624* -->\n",
       "<g id=\"edge83\" class=\"edge\">\n",
       "<title>5497408576&#45;&gt;5497408624*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1403.96,-36.86C1412.84,-39.62 1421.8,-42.54 1430.25,-45.5 1442.09,-49.66 1454.87,-54.72 1466.14,-59.39\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1464.44,-62.47 1475.01,-63.13 1467.15,-56.02 1464.44,-62.47\"/>\n",
       "</g>\n",
       "<!-- 5497424960 -->\n",
       "<g id=\"node76\" class=\"node\">\n",
       "<title>5497424960</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3164.01,-523.5 3164.01,-559.5 3349.26,-559.5 3349.26,-523.5 3164.01,-523.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3173.88\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3183.76,-524 3183.76,-559.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3225.88\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0280</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3268.01,-524 3268.01,-559.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3308.63\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497425008ReLU -->\n",
       "<g id=\"node82\" class=\"node\">\n",
       "<title>5497425008ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3418.69\" cy=\"-541.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3418.69\" y=\"-536.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497424960&#45;&gt;5497425008ReLU -->\n",
       "<g id=\"edge126\" class=\"edge\">\n",
       "<title>5497424960&#45;&gt;5497425008ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3349.35,-541.5C3357.69,-541.5 3365.86,-541.5 3373.48,-541.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3373.45,-545 3383.45,-541.5 3373.45,-538 3373.45,-545\"/>\n",
       "</g>\n",
       "<!-- 5497424960+&#45;&gt;5497424960 -->\n",
       "<g id=\"edge32\" class=\"edge\">\n",
       "<title>5497424960+&#45;&gt;5497424960</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3128.41,-541.5C3135.45,-541.5 3143.5,-541.5 3152.06,-541.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3152.04,-545 3162.04,-541.5 3152.04,-538 3152.04,-545\"/>\n",
       "</g>\n",
       "<!-- 5495008368 -->\n",
       "<g id=\"node78\" class=\"node\">\n",
       "<title>5495008368</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-963.5 0,-999.5 185.25,-999.5 185.25,-963.5 0,-963.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"9.88\" y=\"-976.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"19.75,-964 19.75,-999.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"61.88\" y=\"-976.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.2985</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"104,-964 104,-999.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"144.62\" y=\"-976.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298464* -->\n",
       "<g id=\"node154\" class=\"node\">\n",
       "<title>5495298464*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"248.25\" cy=\"-953.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"248.25\" y=\"-948.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495008368&#45;&gt;5495298464* -->\n",
       "<g id=\"edge77\" class=\"edge\">\n",
       "<title>5495008368&#45;&gt;5495298464*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.65,-964.73C194.39,-963.13 202.86,-961.59 210.58,-960.18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"210.95,-963.67 220.16,-958.44 209.69,-956.79 210.95,-963.67\"/>\n",
       "</g>\n",
       "<!-- 5497408624 -->\n",
       "<g id=\"node79\" class=\"node\">\n",
       "<title>5497408624</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-55.5 1569.13,-91.5 1754.38,-91.5 1754.38,-55.5 1569.13,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-56 1588.88,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;2.6615</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-56 1673.13,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497408624&#45;&gt;5497408816+ -->\n",
       "<g id=\"edge206\" class=\"edge\">\n",
       "<title>5497408624&#45;&gt;5497408816+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1728.08,-91.86C1736.96,-94.62 1745.93,-97.54 1754.38,-100.5 1766.22,-104.66 1779,-109.72 1790.27,-114.39\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1788.56,-117.47 1799.14,-118.13 1791.28,-111.02 1788.56,-117.47\"/>\n",
       "</g>\n",
       "<!-- 5497408624*&#45;&gt;5497408624 -->\n",
       "<g id=\"edge33\" class=\"edge\">\n",
       "<title>5497408624*&#45;&gt;5497408624</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1527.12,-73.5C1535.98,-73.5 1546.45,-73.5 1557.59,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1557.39,-77 1567.39,-73.5 1557.39,-70 1557.39,-77\"/>\n",
       "</g>\n",
       "<!-- 5497425008 -->\n",
       "<g id=\"node81\" class=\"node\">\n",
       "<title>5497425008</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-523.5 3490.38,-559.5 3671.13,-559.5 3671.13,-523.5 3490.38,-523.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-524 3510.13,-559.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-524 3589.88,-559.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-536.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437296* -->\n",
       "<g id=\"node173\" class=\"node\">\n",
       "<title>5497437296*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3736.38\" cy=\"-502.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3736.38\" y=\"-497.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497425008&#45;&gt;5497437296* -->\n",
       "<g id=\"edge149\" class=\"edge\">\n",
       "<title>5497425008&#45;&gt;5497437296*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3654.52,-523.03C3670.12,-519.07 3685.93,-515.05 3699.39,-511.64\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3700.25,-515.03 3709.09,-509.18 3698.53,-508.24 3700.25,-515.03\"/>\n",
       "</g>\n",
       "<!-- 5497425008ReLU&#45;&gt;5497425008 -->\n",
       "<g id=\"edge34\" class=\"edge\">\n",
       "<title>5497425008ReLU&#45;&gt;5497425008</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3452.47,-541.5C3460.49,-541.5 3469.5,-541.5 3478.92,-541.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3478.62,-545 3488.62,-541.5 3478.62,-538 3478.62,-545\"/>\n",
       "</g>\n",
       "<!-- 5497408816 -->\n",
       "<g id=\"node83\" class=\"node\">\n",
       "<title>5497408816</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1893.25,-110.5 1893.25,-146.5 2078.5,-146.5 2078.5,-110.5 1893.25,-110.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1903.13\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1913,-111 1913,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.8749</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1997.25,-111 1997.25,-146.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2037.88\" y=\"-123.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497409248+ -->\n",
       "<g id=\"node95\" class=\"node\">\n",
       "<title>5497409248+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-183.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-178.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497408816&#45;&gt;5497409248+ -->\n",
       "<g id=\"edge154\" class=\"edge\">\n",
       "<title>5497408816&#45;&gt;5497409248+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2052.21,-146.86C2061.09,-149.62 2070.05,-152.54 2078.5,-155.5 2090.34,-159.66 2103.12,-164.72 2114.39,-169.39\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2112.69,-172.47 2123.26,-173.13 2115.4,-166.02 2112.69,-172.47\"/>\n",
       "</g>\n",
       "<!-- 5497408816+&#45;&gt;5497408816 -->\n",
       "<g id=\"edge35\" class=\"edge\">\n",
       "<title>5497408816+&#45;&gt;5497408816</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1851.25,-128.5C1860.1,-128.5 1870.58,-128.5 1881.71,-128.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1881.51,-132 1891.51,-128.5 1881.51,-125 1881.51,-132\"/>\n",
       "</g>\n",
       "<!-- 5497425296 -->\n",
       "<g id=\"node85\" class=\"node\">\n",
       "<title>5497425296</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1893.25,-935.5 1893.25,-971.5 2078.5,-971.5 2078.5,-935.5 1893.25,-935.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1903.13\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1913,-936 1913,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1997.25,-936 1997.25,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2037.88\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497425488+ -->\n",
       "<g id=\"node92\" class=\"node\">\n",
       "<title>5497425488+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-953.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-948.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497425296&#45;&gt;5497425488+ -->\n",
       "<g id=\"edge75\" class=\"edge\">\n",
       "<title>5497425296&#45;&gt;5497425488+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2078.6,-953.5C2089.4,-953.5 2099.91,-953.5 2109.33,-953.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2109.09,-957 2119.09,-953.5 2109.09,-950 2109.09,-957\"/>\n",
       "</g>\n",
       "<!-- 5497425296* -->\n",
       "<g id=\"node86\" class=\"node\">\n",
       "<title>5497425296*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-938.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-933.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497425296*&#45;&gt;5497425296 -->\n",
       "<g id=\"edge36\" class=\"edge\">\n",
       "<title>5497425296*&#45;&gt;5497425296</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1850.89,-940.94C1859.79,-941.78 1870.36,-942.77 1881.6,-943.82\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1881.24,-947.3 1891.52,-944.75 1881.89,-940.33 1881.24,-947.3\"/>\n",
       "</g>\n",
       "<!-- 5497409008 -->\n",
       "<g id=\"node87\" class=\"node\">\n",
       "<title>5497409008</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-220.5 1569.13,-256.5 1754.38,-256.5 1754.38,-220.5 1569.13,-220.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-221 1588.88,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-221 1673.13,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497409056* -->\n",
       "<g id=\"node90\" class=\"node\">\n",
       "<title>5497409056*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-183.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-178.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497409008&#45;&gt;5497409056* -->\n",
       "<g id=\"edge146\" class=\"edge\">\n",
       "<title>5497409008&#45;&gt;5497409056*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1725.56,-220.03C1735.25,-216.97 1745.11,-213.74 1754.38,-210.5 1766.04,-206.42 1778.67,-201.57 1789.86,-197.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1791.11,-200.38 1799.08,-193.39 1788.49,-193.89 1791.11,-200.38\"/>\n",
       "</g>\n",
       "<!-- 5495008800 -->\n",
       "<g id=\"node88\" class=\"node\">\n",
       "<title>5495008800</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"313.5,-990.5 313.5,-1026.5 494.25,-1026.5 494.25,-990.5 313.5,-990.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"323.38\" y=\"-1003.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"333.25,-991 333.25,-1026.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-1003.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"413,-991 413,-1026.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"453.62\" y=\"-1003.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298656+ -->\n",
       "<g id=\"node160\" class=\"node\">\n",
       "<title>5495298656+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"559.5\" cy=\"-953.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"559.5\" y=\"-948.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5495008800&#45;&gt;5495298656+ -->\n",
       "<g id=\"edge145\" class=\"edge\">\n",
       "<title>5495008800&#45;&gt;5495298656+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M471.29,-990.11C479.83,-987.4 488.42,-984.5 496.5,-981.5 506.48,-977.8 517.11,-973.23 526.7,-968.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"528.03,-972.11 535.63,-964.73 525.08,-965.77 528.03,-972.11\"/>\n",
       "</g>\n",
       "<!-- 5497409056 -->\n",
       "<g id=\"node89\" class=\"node\">\n",
       "<title>5497409056</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-165.5 1895.5,-201.5 2076.25,-201.5 2076.25,-165.5 1895.5,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-166 1915.25,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.7798</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-166 1995,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497409056&#45;&gt;5497409248+ -->\n",
       "<g id=\"edge198\" class=\"edge\">\n",
       "<title>5497409056&#45;&gt;5497409248+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-183.5C2088.08,-183.5 2099.16,-183.5 2109.05,-183.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2108.93,-187 2118.93,-183.5 2108.93,-180 2108.93,-187\"/>\n",
       "</g>\n",
       "<!-- 5497409056*&#45;&gt;5497409056 -->\n",
       "<g id=\"edge37\" class=\"edge\">\n",
       "<title>5497409056*&#45;&gt;5497409056</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1851.25,-183.5C1860.6,-183.5 1871.76,-183.5 1883.59,-183.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1883.58,-187 1893.58,-183.5 1883.58,-180 1883.58,-187\"/>\n",
       "</g>\n",
       "<!-- 5497425488 -->\n",
       "<g id=\"node91\" class=\"node\">\n",
       "<title>5497425488</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-935.5 2219.63,-971.5 2400.38,-971.5 2400.38,-935.5 2219.63,-935.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-936 2239.38,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-936 2319.13,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497425872+ -->\n",
       "<g id=\"node102\" class=\"node\">\n",
       "<title>5497425872+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-843.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-838.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497425488&#45;&gt;5497425872+ -->\n",
       "<g id=\"edge99\" class=\"edge\">\n",
       "<title>5497425488&#45;&gt;5497425872+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2387.13,-935.03C2392.56,-932.53 2397.79,-929.7 2402.63,-926.5 2424.58,-911.99 2443.1,-888.31 2455.32,-869.95\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2458.18,-871.97 2460.62,-861.66 2452.28,-868.2 2458.18,-871.97\"/>\n",
       "</g>\n",
       "<!-- 5497425488+&#45;&gt;5497425488 -->\n",
       "<g id=\"edge38\" class=\"edge\">\n",
       "<title>5497425488+&#45;&gt;5497425488</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-953.5C2184.72,-953.5 2195.89,-953.5 2207.71,-953.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2207.71,-957 2217.71,-953.5 2207.71,-950 2207.71,-957\"/>\n",
       "</g>\n",
       "<!-- 5495008992 -->\n",
       "<g id=\"node93\" class=\"node\">\n",
       "<title>5495008992</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"624.75,-825.5 624.75,-861.5 805.5,-861.5 805.5,-825.5 624.75,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"634.62\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"644.5,-826 644.5,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.6956</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"724.25,-826 724.25,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"764.88\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299328* -->\n",
       "<g id=\"node177\" class=\"node\">\n",
       "<title>5495299328*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"870.75\" cy=\"-843.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"870.75\" y=\"-838.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495008992&#45;&gt;5495299328* -->\n",
       "<g id=\"edge160\" class=\"edge\">\n",
       "<title>5495008992&#45;&gt;5495299328*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M805.93,-843.5C815.11,-843.5 824.03,-843.5 832.15,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"831.9,-847 841.9,-843.5 831.9,-840 831.9,-847\"/>\n",
       "</g>\n",
       "<!-- 5497409248 -->\n",
       "<g id=\"node94\" class=\"node\">\n",
       "<title>5497409248</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2217.38,-193.5 2217.38,-229.5 2402.63,-229.5 2402.63,-193.5 2217.38,-193.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2227.25\" y=\"-206.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2237.13,-194 2237.13,-229.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-206.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0951</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2321.38,-194 2321.38,-229.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2362\" y=\"-206.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497409296ReLU -->\n",
       "<g id=\"node97\" class=\"node\">\n",
       "<title>5497409296ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-252.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-247.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497409248&#45;&gt;5497409296ReLU -->\n",
       "<g id=\"edge125\" class=\"edge\">\n",
       "<title>5497409248&#45;&gt;5497409296ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2383.08,-229.96C2399.31,-234.12 2415.99,-238.39 2430.4,-242.08\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2429.4,-245.44 2439.96,-244.53 2431.14,-238.66 2429.4,-245.44\"/>\n",
       "</g>\n",
       "<!-- 5497409248+&#45;&gt;5497409248 -->\n",
       "<g id=\"edge39\" class=\"edge\">\n",
       "<title>5497409248+&#45;&gt;5497409248</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2174.3,-187.94C2183.39,-189.53 2194.27,-191.43 2205.87,-193.46\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2205.22,-196.9 2215.67,-195.17 2206.42,-190 2205.22,-196.9\"/>\n",
       "</g>\n",
       "<!-- 5497409296 -->\n",
       "<g id=\"node96\" class=\"node\">\n",
       "<title>5497409296</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-275.5 2543.76,-311.5 2724.51,-311.5 2724.51,-275.5 2543.76,-275.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-276 2563.51,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-276 2643.26,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497409296&#45;&gt;5497423088* -->\n",
       "<g id=\"edge151\" class=\"edge\">\n",
       "<title>5497409296&#45;&gt;5497423088*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2724.94,-293.5C2734.11,-293.5 2743.04,-293.5 2751.16,-293.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2750.91,-297 2760.91,-293.5 2750.91,-290 2750.91,-297\"/>\n",
       "</g>\n",
       "<!-- 5497409296&#45;&gt;5497424768* -->\n",
       "<g id=\"edge144\" class=\"edge\">\n",
       "<title>5497409296&#45;&gt;5497424768*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2715.8,-311.88C2719.74,-314.4 2723.43,-317.26 2726.76,-320.5 2750.32,-343.5 2772.56,-436.01 2782.83,-484.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2779.39,-484.76 2784.86,-493.84 2786.24,-483.33 2779.39,-484.76\"/>\n",
       "</g>\n",
       "<!-- 5497434704* -->\n",
       "<g id=\"node118\" class=\"node\">\n",
       "<title>5497434704*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-568.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-563.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497409296&#45;&gt;5497434704* -->\n",
       "<g id=\"edge186\" class=\"edge\">\n",
       "<title>5497409296&#45;&gt;5497434704*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2717.2,-311.89C2720.68,-314.41 2723.9,-317.27 2726.76,-320.5 2792.32,-394.78 2719.19,-451.51 2762.76,-540.5 2763.6,-542.22 2764.58,-543.91 2765.66,-545.55\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2762.74,-547.5 2771.69,-553.16 2768.22,-543.15 2762.74,-547.5\"/>\n",
       "</g>\n",
       "<!-- 5497436384* -->\n",
       "<g id=\"node149\" class=\"node\">\n",
       "<title>5497436384*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-239.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-234.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497409296&#45;&gt;5497436384* -->\n",
       "<g id=\"edge208\" class=\"edge\">\n",
       "<title>5497409296&#45;&gt;5497436384*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2698.49,-275.05C2708.02,-272.01 2717.68,-268.78 2726.76,-265.5 2736.38,-262.02 2746.69,-257.87 2756.08,-253.93\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2757.37,-257.18 2765.19,-250.03 2754.62,-250.74 2757.37,-257.18\"/>\n",
       "</g>\n",
       "<!-- 5497409296ReLU&#45;&gt;5497409296 -->\n",
       "<g id=\"edge40\" class=\"edge\">\n",
       "<title>5497409296ReLU&#45;&gt;5497409296</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2502.8,-260.12C2516.3,-263.57 2533,-267.85 2549.9,-272.18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2548.98,-275.56 2559.54,-274.65 2550.72,-268.78 2548.98,-275.56\"/>\n",
       "</g>\n",
       "<!-- 5497425680 -->\n",
       "<g id=\"node98\" class=\"node\">\n",
       "<title>5497425680</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-770.5 2219.63,-806.5 2400.38,-806.5 2400.38,-770.5 2219.63,-770.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-771 2239.38,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-771 2319.13,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497425680&#45;&gt;5497425872+ -->\n",
       "<g id=\"edge178\" class=\"edge\">\n",
       "<title>5497425680&#45;&gt;5497425872+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2373.81,-806.97C2383.5,-810.03 2393.36,-813.26 2402.63,-816.5 2414.29,-820.58 2426.92,-825.43 2438.11,-829.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2436.75,-833.11 2447.33,-833.61 2439.36,-826.62 2436.75,-833.11\"/>\n",
       "</g>\n",
       "<!-- 5497425680*&#45;&gt;5497425680 -->\n",
       "<g id=\"edge41\" class=\"edge\">\n",
       "<title>5497425680*&#45;&gt;5497425680</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2171.22,-743.29C2184.5,-748.95 2201.75,-756.02 2217.38,-761.5 2222.4,-763.26 2227.59,-765 2232.85,-766.71\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2231.64,-769.99 2242.23,-769.68 2233.75,-763.32 2231.64,-769.99\"/>\n",
       "</g>\n",
       "<!-- 5495009136 -->\n",
       "<g id=\"node100\" class=\"node\">\n",
       "<title>5495009136</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"311.25,-468.5 311.25,-504.5 496.5,-504.5 496.5,-468.5 311.25,-468.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"321.12\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"331,-469 331,-504.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.5483</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"415.25,-469 415.25,-504.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"455.88\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299904* -->\n",
       "<g id=\"node195\" class=\"node\">\n",
       "<title>5495299904*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"559.5\" cy=\"-458.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"559.5\" y=\"-453.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495009136&#45;&gt;5495299904* -->\n",
       "<g id=\"edge111\" class=\"edge\">\n",
       "<title>5495009136&#45;&gt;5495299904*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M496.9,-469.73C505.64,-468.13 514.11,-466.59 521.83,-465.18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"522.2,-468.67 531.41,-463.44 520.94,-461.79 522.2,-468.67\"/>\n",
       "</g>\n",
       "<!-- 5497425872 -->\n",
       "<g id=\"node101\" class=\"node\">\n",
       "<title>5497425872</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-825.5 2543.76,-861.5 2724.51,-861.5 2724.51,-825.5 2543.76,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-826 2563.51,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-826 2643.26,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497434512+ -->\n",
       "<g id=\"node114\" class=\"node\">\n",
       "<title>5497434512+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-788.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-783.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497425872&#45;&gt;5497434512+ -->\n",
       "<g id=\"edge102\" class=\"edge\">\n",
       "<title>5497425872&#45;&gt;5497434512+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2701.55,-825.11C2710.09,-822.4 2718.67,-819.5 2726.76,-816.5 2736.73,-812.8 2747.36,-808.23 2756.96,-803.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2758.28,-807.11 2765.88,-799.73 2755.34,-800.77 2758.28,-807.11\"/>\n",
       "</g>\n",
       "<!-- 5497425872+&#45;&gt;5497425872 -->\n",
       "<g id=\"edge42\" class=\"edge\">\n",
       "<title>5497425872+&#45;&gt;5497425872</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-843.5C2508.85,-843.5 2520.01,-843.5 2531.84,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2531.84,-847 2541.84,-843.5 2531.84,-840 2531.84,-847\"/>\n",
       "</g>\n",
       "<!-- 5495107648 -->\n",
       "<g id=\"node103\" class=\"node\">\n",
       "<title>5495107648</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1245,-55.5 1245,-91.5 1430.25,-91.5 1430.25,-55.5 1245,-55.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1254.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1264.75,-56 1264.75,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.8872</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1349,-56 1349,-91.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1389.62\" y=\"-68.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107648&#45;&gt;5497408624* -->\n",
       "<g id=\"edge207\" class=\"edge\">\n",
       "<title>5495107648&#45;&gt;5497408624*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1430.34,-73.5C1441.15,-73.5 1451.66,-73.5 1461.08,-73.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1460.84,-77 1470.84,-73.5 1460.84,-70 1460.84,-77\"/>\n",
       "</g>\n",
       "<!-- 5497450608 -->\n",
       "<g id=\"node104\" class=\"node\">\n",
       "<title>5497450608</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"5162.13,-382.5 5162.13,-418.5 5347.38,-418.5 5347.38,-382.5 5162.13,-382.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"5172.01\" y=\"-395.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"5181.88,-383 5181.88,-418.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"5224.01\" y=\"-395.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0186</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"5266.13,-383 5266.13,-418.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"5306.76\" y=\"-395.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497450608+ -->\n",
       "<g id=\"node105\" class=\"node\">\n",
       "<title>5497450608+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"5099.13\" cy=\"-400.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"5099.13\" y=\"-395.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497450608+&#45;&gt;5497450608 -->\n",
       "<g id=\"edge43\" class=\"edge\">\n",
       "<title>5497450608+&#45;&gt;5497450608</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M5126.54,-400.5C5133.58,-400.5 5141.63,-400.5 5150.18,-400.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"5150.17,-404 5160.17,-400.5 5150.17,-397 5150.17,-404\"/>\n",
       "</g>\n",
       "<!-- 5495107696 -->\n",
       "<g id=\"node106\" class=\"node\">\n",
       "<title>5495107696</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1893.25,-385.5 1893.25,-421.5 2078.5,-421.5 2078.5,-385.5 1893.25,-385.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1903.13\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1913,-386 1913,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.3066</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1997.25,-386 1997.25,-421.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2037.88\" y=\"-398.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107696&#45;&gt;5497422320* -->\n",
       "<g id=\"edge167\" class=\"edge\">\n",
       "<title>5495107696&#45;&gt;5497422320*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2078.6,-403.5C2089.4,-403.5 2099.91,-403.5 2109.33,-403.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2109.09,-407 2119.09,-403.5 2109.09,-400 2109.09,-407\"/>\n",
       "</g>\n",
       "<!-- 5495107792 -->\n",
       "<g id=\"node107\" class=\"node\">\n",
       "<title>5495107792</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1571.38,-550.5 1571.38,-586.5 1752.13,-586.5 1752.13,-550.5 1571.38,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1581.25\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1591.13,-551 1591.13,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.1540</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1670.88,-551 1670.88,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1711.5\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107792&#45;&gt;5497421936* -->\n",
       "<g id=\"edge82\" class=\"edge\">\n",
       "<title>5495107792&#45;&gt;5497421936*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1752.61,-568.5C1763.96,-568.5 1775.04,-568.5 1784.92,-568.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.8,-572 1794.8,-568.5 1784.8,-565 1784.8,-572\"/>\n",
       "</g>\n",
       "<!-- 5497434320 -->\n",
       "<g id=\"node108\" class=\"node\">\n",
       "<title>5497434320</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2541.51,-770.5 2541.51,-806.5 2726.76,-806.5 2726.76,-770.5 2541.51,-770.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2551.38\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2561.26,-771 2561.26,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0504</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2645.51,-771 2645.51,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2686.13\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497434320&#45;&gt;5497434512+ -->\n",
       "<g id=\"edge104\" class=\"edge\">\n",
       "<title>5497434320&#45;&gt;5497434512+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2727.16,-788.5C2735.53,-788.5 2743.64,-788.5 2751.09,-788.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2751.04,-792 2761.04,-788.5 2751.04,-785 2751.04,-792\"/>\n",
       "</g>\n",
       "<!-- 5497434320*&#45;&gt;5497434320 -->\n",
       "<g id=\"edge44\" class=\"edge\">\n",
       "<title>5497434320*&#45;&gt;5497434320</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-788.5C2508.36,-788.5 2518.83,-788.5 2529.96,-788.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2529.76,-792 2539.76,-788.5 2529.76,-785 2529.76,-792\"/>\n",
       "</g>\n",
       "<!-- 5495107888 -->\n",
       "<g id=\"node110\" class=\"node\">\n",
       "<title>5495107888</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-220.5 2543.76,-256.5 2724.51,-256.5 2724.51,-220.5 2543.76,-220.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-221 2563.51,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.5271</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-221 2643.26,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107888&#45;&gt;5497423088* -->\n",
       "<g id=\"edge164\" class=\"edge\">\n",
       "<title>5495107888&#45;&gt;5497423088*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2701.55,-256.89C2710.09,-259.6 2718.67,-262.5 2726.76,-265.5 2736.73,-269.2 2747.36,-273.77 2756.96,-278.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2755.34,-281.23 2765.88,-282.27 2758.28,-274.89 2755.34,-281.23\"/>\n",
       "</g>\n",
       "<!-- 5495107936 -->\n",
       "<g id=\"node111\" class=\"node\">\n",
       "<title>5495107936</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-550.5 2219.63,-586.5 2400.38,-586.5 2400.38,-550.5 2219.63,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-551 2239.38,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.3184</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-551 2319.13,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107936&#45;&gt;5497422704* -->\n",
       "<g id=\"edge127\" class=\"edge\">\n",
       "<title>5495107936&#45;&gt;5497422704*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2373.81,-586.97C2383.5,-590.03 2393.36,-593.26 2402.63,-596.5 2414.29,-600.58 2426.92,-605.43 2438.11,-609.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2436.75,-613.11 2447.33,-613.61 2439.36,-606.62 2436.75,-613.11\"/>\n",
       "</g>\n",
       "<!-- 5495107984 -->\n",
       "<g id=\"node112\" class=\"node\">\n",
       "<title>5495107984</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-495.5 2543.76,-531.5 2724.51,-531.5 2724.51,-495.5 2543.76,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-496 2563.51,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.8722</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-496 2643.26,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495107984&#45;&gt;5497424768* -->\n",
       "<g id=\"edge136\" class=\"edge\">\n",
       "<title>5495107984&#45;&gt;5497424768*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2724.94,-513.5C2734.11,-513.5 2743.04,-513.5 2751.16,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2750.91,-517 2760.91,-513.5 2750.91,-510 2750.91,-517\"/>\n",
       "</g>\n",
       "<!-- 5497434512 -->\n",
       "<g id=\"node113\" class=\"node\">\n",
       "<title>5497434512</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2852.76,-743.5 2852.76,-779.5 3038.01,-779.5 3038.01,-743.5 2852.76,-743.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2862.63\" y=\"-756.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2872.51,-744 2872.51,-779.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-756.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0504</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2956.76,-744 2956.76,-779.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2997.38\" y=\"-756.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497434896+ -->\n",
       "<g id=\"node123\" class=\"node\">\n",
       "<title>5497434896+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3101.01\" cy=\"-651.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3101.01\" y=\"-646.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497434512&#45;&gt;5497434896+ -->\n",
       "<g id=\"edge153\" class=\"edge\">\n",
       "<title>5497434512&#45;&gt;5497434896+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2980.41,-743.08C2998.09,-733.01 3019.76,-719.89 3038.01,-706.5 3051.37,-696.69 3065.26,-684.57 3076.5,-674.21\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3078.87,-676.78 3083.78,-667.39 3074.09,-671.67 3078.87,-676.78\"/>\n",
       "</g>\n",
       "<!-- 5497434512+&#45;&gt;5497434512 -->\n",
       "<g id=\"edge45\" class=\"edge\">\n",
       "<title>5497434512+&#45;&gt;5497434512</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2816.12,-784.04C2823.45,-782.75 2831.94,-781.26 2840.99,-779.67\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2841.53,-783.13 2850.78,-777.95 2840.32,-776.23 2841.53,-783.13\"/>\n",
       "</g>\n",
       "<!-- 5495108032 -->\n",
       "<g id=\"node115\" class=\"node\">\n",
       "<title>5495108032</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-770.5 1895.5,-806.5 2076.25,-806.5 2076.25,-770.5 1895.5,-770.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-771 1915.25,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-771 1995,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108032&#45;&gt;5497423808+ -->\n",
       "<g id=\"edge204\" class=\"edge\">\n",
       "<title>5495108032&#45;&gt;5497423808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2049.69,-806.97C2059.38,-810.03 2069.24,-813.26 2078.5,-816.5 2090.17,-820.58 2102.79,-825.43 2113.98,-829.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2112.62,-833.11 2123.2,-833.61 2115.24,-826.62 2112.62,-833.11\"/>\n",
       "</g>\n",
       "<!-- 5495108128 -->\n",
       "<g id=\"node116\" class=\"node\">\n",
       "<title>5495108128</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-920.5 1569.13,-956.5 1754.38,-956.5 1754.38,-920.5 1569.13,-920.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-933.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-921 1588.88,-956.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-933.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.6881</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-921 1673.13,-956.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-933.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108128&#45;&gt;5497425296* -->\n",
       "<g id=\"edge109\" class=\"edge\">\n",
       "<title>5495108128&#45;&gt;5497425296*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1754.47,-938.5C1765.27,-938.5 1775.79,-938.5 1785.2,-938.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.96,-942 1794.96,-938.5 1784.96,-935 1784.96,-942\"/>\n",
       "</g>\n",
       "<!-- 5497434704 -->\n",
       "<g id=\"node117\" class=\"node\">\n",
       "<title>5497434704</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-550.5 2855.01,-586.5 3035.76,-586.5 3035.76,-550.5 2855.01,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-551 2874.76,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-551 2954.51,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497434704&#45;&gt;5497434896+ -->\n",
       "<g id=\"edge114\" class=\"edge\">\n",
       "<title>5497434704&#45;&gt;5497434896+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3017.68,-586.94C3024.73,-589.74 3031.62,-592.92 3038.01,-596.5 3052.81,-604.8 3067.13,-617.13 3078.27,-628.01\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3075.55,-630.23 3085.06,-634.88 3080.52,-625.31 3075.55,-630.23\"/>\n",
       "</g>\n",
       "<!-- 5497434704*&#45;&gt;5497434704 -->\n",
       "<g id=\"edge46\" class=\"edge\">\n",
       "<title>5497434704*&#45;&gt;5497434704</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-568.5C2824.87,-568.5 2833.79,-568.5 2843.26,-568.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.08,-572 2853.08,-568.5 2843.08,-565 2843.08,-572\"/>\n",
       "</g>\n",
       "<!-- 5495108224 -->\n",
       "<g id=\"node119\" class=\"node\">\n",
       "<title>5495108224</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-715.5 1895.5,-751.5 2076.25,-751.5 2076.25,-715.5 1895.5,-715.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-716 1915.25,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.2094</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-716 1995,-751.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-728.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108224&#45;&gt;5497425680* -->\n",
       "<g id=\"edge180\" class=\"edge\">\n",
       "<title>5495108224&#45;&gt;5497425680*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-733.5C2088.08,-733.5 2099.16,-733.5 2109.05,-733.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2108.93,-737 2118.93,-733.5 2108.93,-730 2108.93,-737\"/>\n",
       "</g>\n",
       "<!-- 5495108272 -->\n",
       "<g id=\"node120\" class=\"node\">\n",
       "<title>5495108272</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2217.38,-880.5 2217.38,-916.5 2402.63,-916.5 2402.63,-880.5 2217.38,-880.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2227.25\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2237.13,-881 2237.13,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.3571</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2321.38,-881 2321.38,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2362\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108272&#45;&gt;5497434320* -->\n",
       "<g id=\"edge169\" class=\"edge\">\n",
       "<title>5495108272&#45;&gt;5497434320*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2388.45,-880.16C2393.45,-877.63 2398.24,-874.76 2402.63,-871.5 2426.08,-854.08 2419.38,-838.48 2438.63,-816.5 2440.74,-814.09 2443.06,-811.69 2445.46,-809.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2447.53,-812.22 2452.61,-802.92 2442.84,-807.03 2447.53,-812.22\"/>\n",
       "</g>\n",
       "<!-- 5495108320 -->\n",
       "<g id=\"node121\" class=\"node\">\n",
       "<title>5495108320</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-550.5 2543.76,-586.5 2724.51,-586.5 2724.51,-550.5 2543.76,-550.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-551 2563.51,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.6169</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-551 2643.26,-586.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-563.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108320&#45;&gt;5497434704* -->\n",
       "<g id=\"edge134\" class=\"edge\">\n",
       "<title>5495108320&#45;&gt;5497434704*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2724.94,-568.5C2734.11,-568.5 2743.04,-568.5 2751.16,-568.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2750.91,-572 2760.91,-568.5 2750.91,-565 2750.91,-572\"/>\n",
       "</g>\n",
       "<!-- 5497434896 -->\n",
       "<g id=\"node122\" class=\"node\">\n",
       "<title>5497434896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3164.01,-633.5 3164.01,-669.5 3349.26,-669.5 3349.26,-633.5 3164.01,-633.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3173.88\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3183.76,-634 3183.76,-669.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3225.88\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0504</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3268.01,-634 3268.01,-669.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3308.63\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497434944ReLU -->\n",
       "<g id=\"node126\" class=\"node\">\n",
       "<title>5497434944ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3418.69\" cy=\"-651.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3418.69\" y=\"-646.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497434896&#45;&gt;5497434944ReLU -->\n",
       "<g id=\"edge133\" class=\"edge\">\n",
       "<title>5497434896&#45;&gt;5497434944ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3349.35,-651.5C3357.69,-651.5 3365.86,-651.5 3373.48,-651.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3373.45,-655 3383.45,-651.5 3373.45,-648 3373.45,-655\"/>\n",
       "</g>\n",
       "<!-- 5497434896+&#45;&gt;5497434896 -->\n",
       "<g id=\"edge47\" class=\"edge\">\n",
       "<title>5497434896+&#45;&gt;5497434896</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3128.41,-651.5C3135.45,-651.5 3143.5,-651.5 3152.06,-651.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3152.04,-655 3162.04,-651.5 3152.04,-648 3152.04,-655\"/>\n",
       "</g>\n",
       "<!-- 5495108368 -->\n",
       "<g id=\"node124\" class=\"node\">\n",
       "<title>5495108368</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-880.5 1895.5,-916.5 2076.25,-916.5 2076.25,-880.5 1895.5,-880.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-881 1915.25,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-881 1995,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108368&#45;&gt;5497425488+ -->\n",
       "<g id=\"edge170\" class=\"edge\">\n",
       "<title>5495108368&#45;&gt;5497425488+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2049.69,-916.97C2059.38,-920.03 2069.24,-923.26 2078.5,-926.5 2090.17,-930.58 2102.79,-935.43 2113.98,-939.89\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2112.62,-943.11 2123.2,-943.61 2115.24,-936.62 2112.62,-943.11\"/>\n",
       "</g>\n",
       "<!-- 5497434944 -->\n",
       "<g id=\"node125\" class=\"node\">\n",
       "<title>5497434944</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-633.5 3490.38,-669.5 3671.13,-669.5 3671.13,-633.5 3490.38,-633.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-634 3510.13,-669.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-634 3589.88,-669.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-646.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437680* -->\n",
       "<g id=\"node189\" class=\"node\">\n",
       "<title>5497437680*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3736.38\" cy=\"-596.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3736.38\" y=\"-591.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497434944&#45;&gt;5497437680* -->\n",
       "<g id=\"edge80\" class=\"edge\">\n",
       "<title>5497434944&#45;&gt;5497437680*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3648.17,-633.11C3656.72,-630.4 3665.3,-627.5 3673.38,-624.5 3683.36,-620.8 3693.99,-616.23 3703.58,-611.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3704.91,-615.11 3712.51,-607.73 3701.96,-608.77 3704.91,-615.11\"/>\n",
       "</g>\n",
       "<!-- 5497434944ReLU&#45;&gt;5497434944 -->\n",
       "<g id=\"edge48\" class=\"edge\">\n",
       "<title>5497434944ReLU&#45;&gt;5497434944</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3452.47,-651.5C3460.49,-651.5 3469.5,-651.5 3478.92,-651.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3478.62,-655 3488.62,-651.5 3478.62,-648 3478.62,-655\"/>\n",
       "</g>\n",
       "<!-- 5495108464 -->\n",
       "<g id=\"node127\" class=\"node\">\n",
       "<title>5495108464</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-358.5 1569.13,-394.5 1754.38,-394.5 1754.38,-358.5 1569.13,-358.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-359 1588.88,-394.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.3538</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-359 1673.13,-394.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497435232* -->\n",
       "<g id=\"node132\" class=\"node\">\n",
       "<title>5497435232*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1823.81\" cy=\"-376.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1823.81\" y=\"-371.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5495108464&#45;&gt;5497435232* -->\n",
       "<g id=\"edge137\" class=\"edge\">\n",
       "<title>5495108464&#45;&gt;5497435232*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1754.47,-376.5C1765.27,-376.5 1775.79,-376.5 1785.2,-376.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.96,-380 1794.96,-376.5 1784.96,-373 1784.96,-380\"/>\n",
       "</g>\n",
       "<!-- 5495108560 -->\n",
       "<g id=\"node128\" class=\"node\">\n",
       "<title>5495108560</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1893.25,-330.5 1893.25,-366.5 2078.5,-366.5 2078.5,-330.5 1893.25,-330.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1903.13\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1913,-331 1913,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.5785</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1997.25,-331 1997.25,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2037.88\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108560&#45;&gt;5497435616* -->\n",
       "<g id=\"edge175\" class=\"edge\">\n",
       "<title>5495108560&#45;&gt;5497435616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2078.6,-348.5C2089.4,-348.5 2099.91,-348.5 2109.33,-348.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2109.09,-352 2119.09,-348.5 2109.09,-345 2109.09,-352\"/>\n",
       "</g>\n",
       "<!-- 5495108608 -->\n",
       "<g id=\"node129\" class=\"node\">\n",
       "<title>5495108608</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1569.13,-165.5 1569.13,-201.5 1754.38,-201.5 1754.38,-165.5 1569.13,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1579\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1588.88,-166 1588.88,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.7798</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1673.13,-166 1673.13,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1713.75\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108608&#45;&gt;5497409056* -->\n",
       "<g id=\"edge139\" class=\"edge\">\n",
       "<title>5495108608&#45;&gt;5497409056*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1754.47,-183.5C1765.27,-183.5 1775.79,-183.5 1785.2,-183.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1784.96,-187 1794.96,-183.5 1784.96,-180 1784.96,-187\"/>\n",
       "</g>\n",
       "<!-- 5495108704 -->\n",
       "<g id=\"node130\" class=\"node\">\n",
       "<title>5495108704</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-165.5 1247.25,-201.5 1428,-201.5 1428,-165.5 1247.25,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-166 1267,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-166 1346.75,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108704&#45;&gt;5497408384+ -->\n",
       "<g id=\"edge141\" class=\"edge\">\n",
       "<title>5495108704&#45;&gt;5497408384+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1401.43,-165.03C1411.12,-161.97 1420.99,-158.74 1430.25,-155.5 1441.92,-151.42 1454.54,-146.57 1465.73,-142.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1466.99,-145.38 1474.95,-138.39 1464.37,-138.89 1466.99,-145.38\"/>\n",
       "</g>\n",
       "<!-- 5497435232 -->\n",
       "<g id=\"node131\" class=\"node\">\n",
       "<title>5497435232</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1893.25,-275.5 1893.25,-311.5 2078.5,-311.5 2078.5,-275.5 1893.25,-275.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1903.13\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1913,-276 1913,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1997.25,-276 1997.25,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2037.88\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497435424+ -->\n",
       "<g id=\"node134\" class=\"node\">\n",
       "<title>5497435424+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2147.94\" cy=\"-293.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2147.94\" y=\"-288.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497435232&#45;&gt;5497435424+ -->\n",
       "<g id=\"edge123\" class=\"edge\">\n",
       "<title>5497435232&#45;&gt;5497435424+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2078.6,-293.5C2089.4,-293.5 2099.91,-293.5 2109.33,-293.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2109.09,-297 2119.09,-293.5 2109.09,-290 2109.09,-297\"/>\n",
       "</g>\n",
       "<!-- 5497435232*&#45;&gt;5497435232 -->\n",
       "<g id=\"edge49\" class=\"edge\">\n",
       "<title>5497435232*&#45;&gt;5497435232</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1839.57,-361.43C1852.63,-348.87 1872.76,-331.33 1893.25,-320.5 1895.98,-319.06 1898.79,-317.69 1901.66,-316.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1903.01,-319.61 1910.87,-312.51 1900.3,-313.15 1903.01,-319.61\"/>\n",
       "</g>\n",
       "<!-- 5497435424 -->\n",
       "<g id=\"node133\" class=\"node\">\n",
       "<title>5497435424</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-275.5 2219.63,-311.5 2400.38,-311.5 2400.38,-275.5 2219.63,-275.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-276 2239.38,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-276 2319.13,-311.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-288.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497435808+ -->\n",
       "<g id=\"node143\" class=\"node\">\n",
       "<title>5497435808+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2472.07\" cy=\"-348.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2472.07\" y=\"-343.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497435424&#45;&gt;5497435808+ -->\n",
       "<g id=\"edge110\" class=\"edge\">\n",
       "<title>5497435424&#45;&gt;5497435808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2376.33,-311.86C2385.22,-314.62 2394.18,-317.54 2402.63,-320.5 2414.47,-324.66 2427.25,-329.72 2438.52,-334.39\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2436.82,-337.47 2447.39,-338.13 2439.53,-331.02 2436.82,-337.47\"/>\n",
       "</g>\n",
       "<!-- 5497435424+&#45;&gt;5497435424 -->\n",
       "<g id=\"edge50\" class=\"edge\">\n",
       "<title>5497435424+&#45;&gt;5497435424</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-293.5C2184.72,-293.5 2195.89,-293.5 2207.71,-293.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2207.71,-297 2217.71,-293.5 2207.71,-290 2207.71,-297\"/>\n",
       "</g>\n",
       "<!-- 5495108896 -->\n",
       "<g id=\"node135\" class=\"node\">\n",
       "<title>5495108896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2219.63,-495.5 2219.63,-531.5 2400.38,-531.5 2400.38,-495.5 2219.63,-495.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2229.5\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2239.38,-496 2239.38,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.5471</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2319.13,-496 2319.13,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2359.75\" y=\"-508.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108896&#45;&gt;5497436000* -->\n",
       "<g id=\"edge194\" class=\"edge\">\n",
       "<title>5495108896&#45;&gt;5497436000*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2400.86,-513.5C2412.21,-513.5 2423.29,-513.5 2433.17,-513.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.05,-517 2443.05,-513.5 2433.05,-510 2433.05,-517\"/>\n",
       "</g>\n",
       "<!-- 5495108944 -->\n",
       "<g id=\"node136\" class=\"node\">\n",
       "<title>5495108944</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-165.5 2543.76,-201.5 2724.51,-201.5 2724.51,-165.5 2543.76,-165.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-166 2563.51,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.9807</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-166 2643.26,-201.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-178.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108944&#45;&gt;5497436384* -->\n",
       "<g id=\"edge92\" class=\"edge\">\n",
       "<title>5495108944&#45;&gt;5497436384*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2702.08,-201.95C2710.45,-204.63 2718.85,-207.5 2726.76,-210.5 2736.85,-214.32 2747.58,-219.08 2757.23,-223.63\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2755.69,-226.77 2766.21,-227.97 2758.73,-220.47 2755.69,-226.77\"/>\n",
       "</g>\n",
       "<!-- 5495108992 -->\n",
       "<g id=\"node137\" class=\"node\">\n",
       "<title>5495108992</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-220.5 1895.5,-256.5 2076.25,-256.5 2076.25,-220.5 1895.5,-220.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-221 1915.25,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-221 1995,-256.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-233.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495108992&#45;&gt;5497435424+ -->\n",
       "<g id=\"edge138\" class=\"edge\">\n",
       "<title>5495108992&#45;&gt;5497435424+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2052.21,-256.86C2061.09,-259.62 2070.05,-262.54 2078.5,-265.5 2090.34,-269.66 2103.12,-274.72 2114.39,-279.39\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2112.69,-282.47 2123.26,-283.13 2115.4,-276.02 2112.69,-282.47\"/>\n",
       "</g>\n",
       "<!-- 5497435616 -->\n",
       "<g id=\"node138\" class=\"node\">\n",
       "<title>5497435616</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2217.38,-330.5 2217.38,-366.5 2402.63,-366.5 2402.63,-330.5 2217.38,-330.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2227.25\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2237.13,-331 2237.13,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2321.38,-331 2321.38,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2362\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497435616&#45;&gt;5497435808+ -->\n",
       "<g id=\"edge188\" class=\"edge\">\n",
       "<title>5497435616&#45;&gt;5497435808+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2402.72,-348.5C2413.53,-348.5 2424.04,-348.5 2433.45,-348.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.21,-352 2443.21,-348.5 2433.21,-345 2433.21,-352\"/>\n",
       "</g>\n",
       "<!-- 5497435616*&#45;&gt;5497435616 -->\n",
       "<g id=\"edge51\" class=\"edge\">\n",
       "<title>5497435616*&#45;&gt;5497435616</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2175.37,-348.5C2184.23,-348.5 2194.71,-348.5 2205.84,-348.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2205.64,-352 2215.64,-348.5 2205.64,-345 2205.64,-352\"/>\n",
       "</g>\n",
       "<!-- 5495109184 -->\n",
       "<g id=\"node140\" class=\"node\">\n",
       "<title>5495109184</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-358.5 3490.38,-394.5 3671.13,-394.5 3671.13,-358.5 3490.38,-358.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-359 3510.13,-394.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.4687</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-359 3589.88,-394.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-371.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495109184&#45;&gt;5497436912* -->\n",
       "<g id=\"edge142\" class=\"edge\">\n",
       "<title>5495109184&#45;&gt;5497436912*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3671.57,-388.19C3680.83,-389.4 3689.83,-390.57 3698.02,-391.64\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3697.49,-395.1 3707.86,-392.92 3698.39,-388.15 3697.49,-395.1\"/>\n",
       "</g>\n",
       "<!-- 5495109280 -->\n",
       "<g id=\"node141\" class=\"node\">\n",
       "<title>5495109280</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-468.5 3490.38,-504.5 3671.13,-504.5 3671.13,-468.5 3490.38,-468.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-469 3510.13,-504.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.9936</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-469 3589.88,-504.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-481.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495109280&#45;&gt;5497437296* -->\n",
       "<g id=\"edge86\" class=\"edge\">\n",
       "<title>5495109280&#45;&gt;5497437296*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3671.57,-495.85C3680.83,-496.82 3689.83,-497.76 3698.02,-498.61\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3697.54,-502.08 3707.85,-499.63 3698.27,-495.12 3697.54,-502.08\"/>\n",
       "</g>\n",
       "<!-- 5497435808 -->\n",
       "<g id=\"node142\" class=\"node\">\n",
       "<title>5497435808</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-330.5 2543.76,-366.5 2724.51,-366.5 2724.51,-330.5 2543.76,-330.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-331 2563.51,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-331 2643.26,-366.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-343.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436192+ -->\n",
       "<g id=\"node147\" class=\"node\">\n",
       "<title>5497436192+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"2789.76\" cy=\"-349.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"2789.76\" y=\"-344.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497435808&#45;&gt;5497436192+ -->\n",
       "<g id=\"edge78\" class=\"edge\">\n",
       "<title>5497435808&#45;&gt;5497436192+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2724.94,-349.08C2734.11,-349.14 2743.04,-349.2 2751.16,-349.26\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2750.89,-352.75 2760.91,-349.32 2750.93,-345.75 2750.89,-352.75\"/>\n",
       "</g>\n",
       "<!-- 5497435808+&#45;&gt;5497435808 -->\n",
       "<g id=\"edge52\" class=\"edge\">\n",
       "<title>5497435808+&#45;&gt;5497435808</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2499.5,-348.5C2508.85,-348.5 2520.01,-348.5 2531.84,-348.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2531.84,-352 2541.84,-348.5 2531.84,-345 2531.84,-352\"/>\n",
       "</g>\n",
       "<!-- 5497436000 -->\n",
       "<g id=\"node144\" class=\"node\">\n",
       "<title>5497436000</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2543.76,-440.5 2543.76,-476.5 2724.51,-476.5 2724.51,-440.5 2543.76,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2553.63\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2563.51,-441 2563.51,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2603.38\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0772</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2643.26,-441 2643.26,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2683.88\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436000&#45;&gt;5497436192+ -->\n",
       "<g id=\"edge155\" class=\"edge\">\n",
       "<title>5497436000&#45;&gt;5497436192+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2711.87,-440.04C2717.1,-437.54 2722.12,-434.71 2726.76,-431.5 2747.13,-417.41 2763.63,-394.51 2774.47,-376.48\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2777.49,-378.26 2779.44,-367.84 2771.42,-374.77 2777.49,-378.26\"/>\n",
       "</g>\n",
       "<!-- 5497436000*&#45;&gt;5497436000 -->\n",
       "<g id=\"edge53\" class=\"edge\">\n",
       "<title>5497436000*&#45;&gt;5497436000</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2495.4,-504.17C2508.69,-498.75 2525.95,-491.94 2541.51,-486.5 2547.3,-484.48 2553.32,-482.46 2559.39,-480.48\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2560.42,-483.82 2568.88,-477.43 2558.29,-477.15 2560.42,-483.82\"/>\n",
       "</g>\n",
       "<!-- 5497436192 -->\n",
       "<g id=\"node146\" class=\"node\">\n",
       "<title>5497436192</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-331.5 2855.01,-367.5 3035.76,-367.5 3035.76,-331.5 2855.01,-331.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-344.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-332 2874.76,-367.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-344.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0772</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-332 2954.51,-367.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-344.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436576+ -->\n",
       "<g id=\"node156\" class=\"node\">\n",
       "<title>5497436576+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3101.01\" cy=\"-321.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3101.01\" y=\"-316.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497436192&#45;&gt;5497436576+ -->\n",
       "<g id=\"edge181\" class=\"edge\">\n",
       "<title>5497436192&#45;&gt;5497436576+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3036.19,-333.13C3045.74,-331.39 3055.01,-329.7 3063.4,-328.17\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3063.9,-331.64 3073.11,-326.4 3062.64,-324.75 3063.9,-331.64\"/>\n",
       "</g>\n",
       "<!-- 5497436192+&#45;&gt;5497436192 -->\n",
       "<g id=\"edge54\" class=\"edge\">\n",
       "<title>5497436192+&#45;&gt;5497436192</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-349.5C2824.87,-349.5 2833.79,-349.5 2843.26,-349.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.08,-353 2853.08,-349.5 2843.08,-346 2843.08,-353\"/>\n",
       "</g>\n",
       "<!-- 5497436384 -->\n",
       "<g id=\"node148\" class=\"node\">\n",
       "<title>5497436384</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2855.01,-221.5 2855.01,-257.5 3035.76,-257.5 3035.76,-221.5 2855.01,-221.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2864.88\" y=\"-234.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2874.76,-222 2874.76,-257.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2914.63\" y=\"-234.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2954.51,-222 2954.51,-257.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2995.13\" y=\"-234.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436384&#45;&gt;5497436576+ -->\n",
       "<g id=\"edge168\" class=\"edge\">\n",
       "<title>5497436384&#45;&gt;5497436576+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3019.6,-257.86C3025.98,-260.41 3032.2,-263.28 3038.01,-266.5 3053.05,-274.84 3067.56,-287.38 3078.76,-298.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3076.07,-300.64 3085.57,-305.32 3081.06,-295.73 3076.07,-300.64\"/>\n",
       "</g>\n",
       "<!-- 5497436384*&#45;&gt;5497436384 -->\n",
       "<g id=\"edge55\" class=\"edge\">\n",
       "<title>5497436384*&#45;&gt;5497436384</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2817.16,-239.5C2824.87,-239.5 2833.79,-239.5 2843.26,-239.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2843.08,-243 2853.08,-239.5 2843.08,-236 2843.08,-243\"/>\n",
       "</g>\n",
       "<!-- 5495109904 -->\n",
       "<g id=\"node150\" class=\"node\">\n",
       "<title>5495109904</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3488.13,-578.5 3488.13,-614.5 3673.38,-614.5 3673.38,-578.5 3488.13,-578.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3498.01\" y=\"-591.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3507.88,-579 3507.88,-614.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-591.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.1469</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3592.13,-579 3592.13,-614.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3632.76\" y=\"-591.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495109904&#45;&gt;5497437680* -->\n",
       "<g id=\"edge159\" class=\"edge\">\n",
       "<title>5495109904&#45;&gt;5497437680*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3673.78,-596.5C3682.15,-596.5 3690.27,-596.5 3697.72,-596.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3697.66,-600 3707.66,-596.5 3697.66,-593 3697.66,-600\"/>\n",
       "</g>\n",
       "<!-- 5495109952 -->\n",
       "<g id=\"node151\" class=\"node\">\n",
       "<title>5495109952</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1571.38,-838.5 1571.38,-874.5 1752.13,-874.5 1752.13,-838.5 1571.38,-838.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1581.25\" y=\"-851.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1591.13,-839 1591.13,-874.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-851.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.7856</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1670.88,-839 1670.88,-874.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1711.5\" y=\"-851.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495109952&#45;&gt;5497423616* -->\n",
       "<g id=\"edge192\" class=\"edge\">\n",
       "<title>5495109952&#45;&gt;5497423616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1752.61,-849.2C1764.06,-848.27 1775.24,-847.36 1785.18,-846.56\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1785.45,-850.05 1795.14,-845.75 1784.89,-843.07 1785.45,-850.05\"/>\n",
       "</g>\n",
       "<!-- 5495298416 -->\n",
       "<g id=\"node152\" class=\"node\">\n",
       "<title>5495298416</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2.25,-908.5 2.25,-944.5 183,-944.5 183,-908.5 2.25,-908.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"12.12\" y=\"-921.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"22,-909 22,-944.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"61.88\" y=\"-921.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"101.75,-909 101.75,-944.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"142.38\" y=\"-921.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298416&#45;&gt;5495298464* -->\n",
       "<g id=\"edge93\" class=\"edge\">\n",
       "<title>5495298416&#45;&gt;5495298464*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M183.43,-942.28C192.98,-943.96 202.26,-945.59 210.64,-947.07\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"209.89,-950.49 220.35,-948.77 211.11,-943.59 209.89,-950.49\"/>\n",
       "</g>\n",
       "<!-- 5495298464 -->\n",
       "<g id=\"node153\" class=\"node\">\n",
       "<title>5495298464</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"311.25,-935.5 311.25,-971.5 496.5,-971.5 496.5,-935.5 311.25,-935.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"321.12\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"331,-936 331,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.5971</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"415.25,-936 415.25,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"455.88\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298464&#45;&gt;5495298656+ -->\n",
       "<g id=\"edge116\" class=\"edge\">\n",
       "<title>5495298464&#45;&gt;5495298656+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M496.9,-953.5C505.27,-953.5 513.39,-953.5 520.84,-953.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"520.78,-957 530.78,-953.5 520.78,-950 520.78,-957\"/>\n",
       "</g>\n",
       "<!-- 5495298464*&#45;&gt;5495298464 -->\n",
       "<g id=\"edge56\" class=\"edge\">\n",
       "<title>5495298464*&#45;&gt;5495298464</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M275.66,-953.5C282.7,-953.5 290.75,-953.5 299.3,-953.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"299.29,-957 309.29,-953.5 299.29,-950 299.29,-957\"/>\n",
       "</g>\n",
       "<!-- 5497436576 -->\n",
       "<g id=\"node155\" class=\"node\">\n",
       "<title>5497436576</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3166.26,-303.5 3166.26,-339.5 3347.01,-339.5 3347.01,-303.5 3166.26,-303.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3176.13\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3186.01,-304 3186.01,-339.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3225.88\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0772</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3265.76,-304 3265.76,-339.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3306.38\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497436624ReLU -->\n",
       "<g id=\"node158\" class=\"node\">\n",
       "<title>5497436624ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3418.69\" cy=\"-321.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3418.69\" y=\"-316.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5497436576&#45;&gt;5497436624ReLU -->\n",
       "<g id=\"edge91\" class=\"edge\">\n",
       "<title>5497436576&#45;&gt;5497436624ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3347.49,-321.5C3356.49,-321.5 3365.32,-321.5 3373.52,-321.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3373.51,-325 3383.51,-321.5 3373.51,-318 3373.51,-325\"/>\n",
       "</g>\n",
       "<!-- 5497436576+&#45;&gt;5497436576 -->\n",
       "<g id=\"edge57\" class=\"edge\">\n",
       "<title>5497436576+&#45;&gt;5497436576</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3128.41,-321.5C3136.12,-321.5 3145.04,-321.5 3154.51,-321.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3154.33,-325 3164.33,-321.5 3154.33,-318 3154.33,-325\"/>\n",
       "</g>\n",
       "<!-- 5497436624 -->\n",
       "<g id=\"node157\" class=\"node\">\n",
       "<title>5497436624</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3490.38,-303.5 3490.38,-339.5 3671.13,-339.5 3671.13,-303.5 3490.38,-303.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3500.26\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3510.13,-304 3510.13,-339.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0772</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3589.88,-304 3589.88,-339.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3630.51\" y=\"-316.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497438064* -->\n",
       "<g id=\"node197\" class=\"node\">\n",
       "<title>5497438064*</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"3736.38\" cy=\"-321.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"3736.38\" y=\"-316.45\" font-family=\"Times,serif\" font-size=\"14.00\">*</text>\n",
       "</g>\n",
       "<!-- 5497436624&#45;&gt;5497438064* -->\n",
       "<g id=\"edge94\" class=\"edge\">\n",
       "<title>5497436624&#45;&gt;5497438064*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3671.57,-321.5C3680.74,-321.5 3689.66,-321.5 3697.78,-321.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3697.54,-325 3707.54,-321.5 3697.54,-318 3697.54,-325\"/>\n",
       "</g>\n",
       "<!-- 5497436624ReLU&#45;&gt;5497436624 -->\n",
       "<g id=\"edge58\" class=\"edge\">\n",
       "<title>5497436624ReLU&#45;&gt;5497436624</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3452.47,-321.5C3460.49,-321.5 3469.5,-321.5 3478.92,-321.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3478.62,-325 3488.62,-321.5 3478.62,-318 3478.62,-325\"/>\n",
       "</g>\n",
       "<!-- 5495298656 -->\n",
       "<g id=\"node159\" class=\"node\">\n",
       "<title>5495298656</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-935.5 622.5,-971.5 807.75,-971.5 807.75,-935.5 622.5,-935.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-936 642.25,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.5971</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-936 726.5,-971.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-948.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299088+ -->\n",
       "<g id=\"node171\" class=\"node\">\n",
       "<title>5495299088+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"870.75\" cy=\"-898.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"870.75\" y=\"-893.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5495298656&#45;&gt;5495299088+ -->\n",
       "<g id=\"edge108\" class=\"edge\">\n",
       "<title>5495298656&#45;&gt;5495299088+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M782.54,-935.11C791.08,-932.4 799.67,-929.5 807.75,-926.5 817.73,-922.8 828.36,-918.23 837.95,-913.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"839.28,-917.11 846.88,-909.73 836.33,-910.77 839.28,-917.11\"/>\n",
       "</g>\n",
       "<!-- 5495298656+&#45;&gt;5495298656 -->\n",
       "<g id=\"edge59\" class=\"edge\">\n",
       "<title>5495298656+&#45;&gt;5495298656</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M586.91,-953.5C593.95,-953.5 602,-953.5 610.55,-953.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"610.54,-957 620.54,-953.5 610.54,-950 610.54,-957\"/>\n",
       "</g>\n",
       "<!-- 5497436912 -->\n",
       "<g id=\"node161\" class=\"node\">\n",
       "<title>5497436912</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3799.38,-378.5 3799.38,-414.5 3980.13,-414.5 3980.13,-378.5 3799.38,-378.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3809.26\" y=\"-391.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3819.13,-379 3819.13,-414.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3859.01\" y=\"-391.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0211</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3898.88,-379 3898.88,-414.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3939.51\" y=\"-391.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437104+ -->\n",
       "<g id=\"node168\" class=\"node\">\n",
       "<title>5497437104+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"4108.76\" cy=\"-447.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"4108.76\" y=\"-442.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497436912&#45;&gt;5497437104+ -->\n",
       "<g id=\"edge173\" class=\"edge\">\n",
       "<title>5497436912&#45;&gt;5497437104+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3969.41,-414.98C4004.43,-423.22 4043.95,-432.5 4071.76,-439.04\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4070.9,-442.43 4081.44,-441.31 4072.51,-435.62 4070.9,-442.43\"/>\n",
       "</g>\n",
       "<!-- 5497436912*&#45;&gt;5497436912 -->\n",
       "<g id=\"edge60\" class=\"edge\">\n",
       "<title>5497436912*&#45;&gt;5497436912</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3763.75,-396.5C3770.9,-396.5 3779.08,-396.5 3787.77,-396.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3787.49,-400 3797.49,-396.5 3787.49,-393 3787.49,-400\"/>\n",
       "</g>\n",
       "<!-- 5495298848 -->\n",
       "<g id=\"node163\" class=\"node\">\n",
       "<title>5495298848</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"313.5,-825.5 313.5,-861.5 494.25,-861.5 494.25,-825.5 313.5,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"323.38\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"333.25,-826 333.25,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 3.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"413,-826 413,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"453.62\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298848&#45;&gt;5495298896* -->\n",
       "<g id=\"edge129\" class=\"edge\">\n",
       "<title>5495298848&#45;&gt;5495298896*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M468.28,-861.84C477.8,-864.89 487.45,-868.16 496.5,-871.5 506.34,-875.13 516.87,-879.52 526.4,-883.69\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"524.71,-886.77 535.27,-887.64 527.56,-880.38 524.71,-886.77\"/>\n",
       "</g>\n",
       "<!-- 5495110432 -->\n",
       "<g id=\"node164\" class=\"node\">\n",
       "<title>5495110432</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-605.5 1895.5,-641.5 2076.25,-641.5 2076.25,-605.5 1895.5,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-606 1915.25,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.5993</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-606 1995,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495110432&#45;&gt;5497424000* -->\n",
       "<g id=\"edge148\" class=\"edge\">\n",
       "<title>5495110432&#45;&gt;5497424000*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-623.5C2088.08,-623.5 2099.16,-623.5 2109.05,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2108.93,-627 2118.93,-623.5 2108.93,-620 2108.93,-627\"/>\n",
       "</g>\n",
       "<!-- 5495298896 -->\n",
       "<g id=\"node165\" class=\"node\">\n",
       "<title>5495298896</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-880.5 622.5,-916.5 807.75,-916.5 807.75,-880.5 622.5,-880.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-881 642.25,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;2.6726</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-881 726.5,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495298896&#45;&gt;5495299088+ -->\n",
       "<g id=\"edge120\" class=\"edge\">\n",
       "<title>5495298896&#45;&gt;5495299088+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M808.15,-898.5C816.52,-898.5 824.64,-898.5 832.09,-898.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"832.03,-902 842.03,-898.5 832.03,-895 832.03,-902\"/>\n",
       "</g>\n",
       "<!-- 5495298896*&#45;&gt;5495298896 -->\n",
       "<g id=\"edge61\" class=\"edge\">\n",
       "<title>5495298896*&#45;&gt;5495298896</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M586.91,-898.5C593.95,-898.5 602,-898.5 610.55,-898.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"610.54,-902 620.54,-898.5 610.54,-895 610.54,-902\"/>\n",
       "</g>\n",
       "<!-- 5497437104 -->\n",
       "<g id=\"node167\" class=\"node\">\n",
       "<title>5497437104</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4239.63,-431.5 4239.63,-467.5 4420.38,-467.5 4420.38,-431.5 4239.63,-431.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4249.51\" y=\"-444.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4259.38,-432 4259.38,-467.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4299.26\" y=\"-444.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0211</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4339.13,-432 4339.13,-467.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4379.76\" y=\"-444.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437488+ -->\n",
       "<g id=\"node179\" class=\"node\">\n",
       "<title>5497437488+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"4485.63\" cy=\"-464.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"4485.63\" y=\"-459.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497437104&#45;&gt;5497437488+ -->\n",
       "<g id=\"edge196\" class=\"edge\">\n",
       "<title>5497437104&#45;&gt;5497437488+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4420.82,-458.27C4430.08,-459.17 4439.08,-460.05 4447.27,-460.85\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4446.81,-464.32 4457.1,-461.81 4447.49,-457.36 4446.81,-464.32\"/>\n",
       "</g>\n",
       "<!-- 5497437104+&#45;&gt;5497437104 -->\n",
       "<g id=\"edge62\" class=\"edge\">\n",
       "<title>5497437104+&#45;&gt;5497437104</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4136.2,-447.74C4159.23,-447.95 4194.12,-448.27 4227.93,-448.58\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4227.79,-452.08 4237.82,-448.67 4227.85,-445.08 4227.79,-452.08\"/>\n",
       "</g>\n",
       "<!-- 5495110576 -->\n",
       "<g id=\"node169\" class=\"node\">\n",
       "<title>5495110576</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"2217.38,-660.5 2217.38,-696.5 2402.63,-696.5 2402.63,-660.5 2217.38,-660.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2227.25\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2237.13,-661 2237.13,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2279.25\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.1981</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"2321.38,-661 2321.38,-696.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2362\" y=\"-673.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495110576&#45;&gt;5497424384* -->\n",
       "<g id=\"edge201\" class=\"edge\">\n",
       "<title>5495110576&#45;&gt;5497424384*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2402.72,-678.5C2413.53,-678.5 2424.04,-678.5 2433.45,-678.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2433.21,-682 2443.21,-678.5 2433.21,-675 2433.21,-682\"/>\n",
       "</g>\n",
       "<!-- 5495299088 -->\n",
       "<g id=\"node170\" class=\"node\">\n",
       "<title>5495299088</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-880.5 933.75,-916.5 1119,-916.5 1119,-880.5 933.75,-880.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-881 953.5,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.2697</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-881 1037.75,-916.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-893.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299520+ -->\n",
       "<g id=\"node183\" class=\"node\">\n",
       "<title>5495299520+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1182\" cy=\"-843.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1182\" y=\"-838.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5495299088&#45;&gt;5495299520+ -->\n",
       "<g id=\"edge200\" class=\"edge\">\n",
       "<title>5495299088&#45;&gt;5495299520+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1093.79,-880.11C1102.33,-877.4 1110.92,-874.5 1119,-871.5 1128.98,-867.8 1139.61,-863.23 1149.2,-858.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1150.53,-862.11 1158.13,-854.73 1147.58,-855.77 1150.53,-862.11\"/>\n",
       "</g>\n",
       "<!-- 5495299088+&#45;&gt;5495299088 -->\n",
       "<g id=\"edge63\" class=\"edge\">\n",
       "<title>5495299088+&#45;&gt;5495299088</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M898.16,-898.5C905.2,-898.5 913.25,-898.5 921.8,-898.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"921.79,-902 931.79,-898.5 921.79,-895 921.79,-902\"/>\n",
       "</g>\n",
       "<!-- 5497437296 -->\n",
       "<g id=\"node172\" class=\"node\">\n",
       "<title>5497437296</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4018.38,-484.5 4018.38,-520.5 4199.13,-520.5 4199.13,-484.5 4018.38,-484.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4028.26\" y=\"-497.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4038.13,-485 4038.13,-520.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4078.01\" y=\"-497.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4117.88,-485 4117.88,-520.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4158.51\" y=\"-497.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437296&#45;&gt;5497437488+ -->\n",
       "<g id=\"edge150\" class=\"edge\">\n",
       "<title>5497437296&#45;&gt;5497437488+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4199.44,-498.36C4262.17,-494.73 4347.8,-488.29 4422.63,-477.5 4431.09,-476.28 4440.12,-474.6 4448.6,-472.86\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4449.2,-476.31 4458.24,-470.79 4447.73,-469.47 4449.2,-476.31\"/>\n",
       "</g>\n",
       "<!-- 5497437296*&#45;&gt;5497437296 -->\n",
       "<g id=\"edge64\" class=\"edge\">\n",
       "<title>5497437296*&#45;&gt;5497437296</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3763.72,-502.5C3813.6,-502.5 3924.21,-502.5 4006.63,-502.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4006.55,-506 4016.55,-502.5 4006.55,-499 4006.55,-506\"/>\n",
       "</g>\n",
       "<!-- 5495110816 -->\n",
       "<g id=\"node174\" class=\"node\">\n",
       "<title>5495110816</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"936,-605.5 936,-641.5 1116.75,-641.5 1116.75,-605.5 936,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"945.88\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"955.75,-606 955.75,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1035.5,-606 1035.5,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1076.12\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495110816&#45;&gt;5497406944+ -->\n",
       "<g id=\"edge95\" class=\"edge\">\n",
       "<title>5495110816&#45;&gt;5497406944+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1090.78,-641.84C1100.3,-644.89 1109.95,-648.16 1119,-651.5 1128.84,-655.13 1139.37,-659.52 1148.9,-663.69\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1147.21,-666.77 1157.77,-667.64 1150.06,-660.38 1147.21,-666.77\"/>\n",
       "</g>\n",
       "<!-- 5495299280 -->\n",
       "<g id=\"node175\" class=\"node\">\n",
       "<title>5495299280</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-770.5 622.5,-806.5 807.75,-806.5 807.75,-770.5 622.5,-770.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-771 642.25,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-771 726.5,-806.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-783.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299280&#45;&gt;5495299328* -->\n",
       "<g id=\"edge182\" class=\"edge\">\n",
       "<title>5495299280&#45;&gt;5495299328*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M779.53,-806.84C789.05,-809.89 798.7,-813.16 807.75,-816.5 817.59,-820.13 828.12,-824.52 837.65,-828.69\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"835.96,-831.77 846.52,-832.64 838.81,-825.38 835.96,-831.77\"/>\n",
       "</g>\n",
       "<!-- 5495299328 -->\n",
       "<g id=\"node176\" class=\"node\">\n",
       "<title>5495299328</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"933.75,-825.5 933.75,-861.5 1119,-861.5 1119,-825.5 933.75,-825.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"943.62\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"953.5,-826 953.5,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.6956</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1037.75,-826 1037.75,-861.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1078.38\" y=\"-838.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299328&#45;&gt;5495299520+ -->\n",
       "<g id=\"edge205\" class=\"edge\">\n",
       "<title>5495299328&#45;&gt;5495299520+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1119.4,-843.5C1127.77,-843.5 1135.89,-843.5 1143.34,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.28,-847 1153.28,-843.5 1143.28,-840 1143.28,-847\"/>\n",
       "</g>\n",
       "<!-- 5495299328*&#45;&gt;5495299328 -->\n",
       "<g id=\"edge65\" class=\"edge\">\n",
       "<title>5495299328*&#45;&gt;5495299328</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M898.16,-843.5C905.2,-843.5 913.25,-843.5 921.8,-843.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"921.79,-847 931.79,-843.5 921.79,-840 921.79,-847\"/>\n",
       "</g>\n",
       "<!-- 5497437488 -->\n",
       "<g id=\"node178\" class=\"node\">\n",
       "<title>5497437488</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4548.63,-446.5 4548.63,-482.5 4729.38,-482.5 4729.38,-446.5 4548.63,-446.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4558.51\" y=\"-459.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4568.38,-447 4568.38,-482.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4608.26\" y=\"-459.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0211</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4648.13,-447 4648.13,-482.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4688.76\" y=\"-459.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437872+ -->\n",
       "<g id=\"node191\" class=\"node\">\n",
       "<title>5497437872+</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"4792.38\" cy=\"-464.5\" rx=\"27\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"4792.38\" y=\"-459.45\" font-family=\"Times,serif\" font-size=\"14.00\">+</text>\n",
       "</g>\n",
       "<!-- 5497437488&#45;&gt;5497437872+ -->\n",
       "<g id=\"edge143\" class=\"edge\">\n",
       "<title>5497437488&#45;&gt;5497437872+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4729.81,-464.5C4738.15,-464.5 4746.25,-464.5 4753.69,-464.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4753.63,-468 4763.63,-464.5 4753.63,-461 4753.63,-468\"/>\n",
       "</g>\n",
       "<!-- 5497437488+&#45;&gt;5497437488 -->\n",
       "<g id=\"edge66\" class=\"edge\">\n",
       "<title>5497437488+&#45;&gt;5497437488</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4513,-464.5C4520.15,-464.5 4528.33,-464.5 4537.02,-464.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4536.74,-468 4546.74,-464.5 4536.74,-461 4536.74,-468\"/>\n",
       "</g>\n",
       "<!-- 5495111008 -->\n",
       "<g id=\"node180\" class=\"node\">\n",
       "<title>5495111008</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"936,-83.5 936,-119.5 1116.75,-119.5 1116.75,-83.5 936,-83.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"945.88\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"955.75,-84 955.75,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"995.62\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.3933</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1035.5,-84 1035.5,-119.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1076.12\" y=\"-96.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495111008&#45;&gt;5497408192* -->\n",
       "<g id=\"edge130\" class=\"edge\">\n",
       "<title>5495111008&#45;&gt;5497408192*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1117.18,-117.28C1126.73,-118.96 1136.01,-120.59 1144.39,-122.07\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1143.64,-125.49 1154.1,-123.77 1144.86,-118.59 1143.64,-125.49\"/>\n",
       "</g>\n",
       "<!-- 5495111056 -->\n",
       "<g id=\"node181\" class=\"node\">\n",
       "<title>5495111056</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1247.25,-605.5 1247.25,-641.5 1428,-641.5 1428,-605.5 1247.25,-605.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1257.12\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1267,-606 1267,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.7947</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1346.75,-606 1346.75,-641.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1387.38\" y=\"-618.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495111056&#45;&gt;5497407616* -->\n",
       "<g id=\"edge163\" class=\"edge\">\n",
       "<title>5495111056&#45;&gt;5497407616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1428.48,-623.5C1439.83,-623.5 1450.91,-623.5 1460.79,-623.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1460.68,-627 1470.68,-623.5 1460.68,-620 1460.68,-627\"/>\n",
       "</g>\n",
       "<!-- 5495299520 -->\n",
       "<g id=\"node182\" class=\"node\">\n",
       "<title>5495299520</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1245,-797.5 1245,-833.5 1430.25,-833.5 1430.25,-797.5 1245,-797.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1254.88\" y=\"-810.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1264.75,-798 1264.75,-833.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1306.88\" y=\"-810.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;3.9653</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1349,-798 1349,-833.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1389.62\" y=\"-810.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299568ReLU -->\n",
       "<g id=\"node186\" class=\"node\">\n",
       "<title>5495299568ReLU</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"1499.69\" cy=\"-808.5\" rx=\"33.44\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"1499.69\" y=\"-803.45\" font-family=\"Times,serif\" font-size=\"14.00\">ReLU</text>\n",
       "</g>\n",
       "<!-- 5495299520&#45;&gt;5495299568ReLU -->\n",
       "<g id=\"edge156\" class=\"edge\">\n",
       "<title>5495299520&#45;&gt;5495299568ReLU</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1430.34,-811.49C1438.69,-811.12 1446.86,-810.77 1454.48,-810.43\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1454.6,-813.93 1464.44,-810 1454.3,-806.94 1454.6,-813.93\"/>\n",
       "</g>\n",
       "<!-- 5495299520+&#45;&gt;5495299520 -->\n",
       "<g id=\"edge67\" class=\"edge\">\n",
       "<title>5495299520+&#45;&gt;5495299520</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1208.37,-838.88C1215.7,-837.54 1224.19,-835.99 1233.24,-834.34\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1233.81,-837.8 1243.02,-832.56 1232.56,-830.91 1233.81,-837.8\"/>\n",
       "</g>\n",
       "<!-- 5495111104 -->\n",
       "<g id=\"node184\" class=\"node\">\n",
       "<title>5495111104</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3488.13,-248.5 3488.13,-284.5 3673.38,-284.5 3673.38,-248.5 3488.13,-248.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3498.01\" y=\"-261.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3507.88,-249 3507.88,-284.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3550.01\" y=\"-261.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.5138</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3592.13,-249 3592.13,-284.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3632.76\" y=\"-261.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495111104&#45;&gt;5497438064* -->\n",
       "<g id=\"edge135\" class=\"edge\">\n",
       "<title>5495111104&#45;&gt;5497438064*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3648.17,-284.89C3656.72,-287.6 3665.3,-290.5 3673.38,-293.5 3683.36,-297.2 3693.99,-301.77 3703.58,-306.13\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"3701.96,-309.23 3712.51,-310.27 3704.91,-302.89 3701.96,-309.23\"/>\n",
       "</g>\n",
       "<!-- 5495299568 -->\n",
       "<g id=\"node185\" class=\"node\">\n",
       "<title>5495299568</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1571.38,-783.5 1571.38,-819.5 1752.13,-819.5 1752.13,-783.5 1571.38,-783.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1581.25\" y=\"-796.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1591.13,-784 1591.13,-819.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1631\" y=\"-796.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1670.88,-784 1670.88,-819.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1711.5\" y=\"-796.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299568&#45;&gt;5497421936* -->\n",
       "<g id=\"edge85\" class=\"edge\">\n",
       "<title>5495299568&#45;&gt;5497421936*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1683.96,-783.06C1703.89,-764.98 1733.65,-735.91 1754.38,-706.5 1779.26,-671.18 1799.99,-625.48 1811.9,-596.57\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1815.1,-597.99 1815.6,-587.41 1808.61,-595.37 1815.1,-597.99\"/>\n",
       "</g>\n",
       "<!-- 5495299568&#45;&gt;5497423616* -->\n",
       "<g id=\"edge107\" class=\"edge\">\n",
       "<title>5495299568&#45;&gt;5497423616*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1732.96,-819.92C1751.75,-824.85 1771.27,-829.98 1787.3,-834.18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1786.27,-837.53 1796.83,-836.68 1788.05,-830.76 1786.27,-837.53\"/>\n",
       "</g>\n",
       "<!-- 5495299568&#45;&gt;5497425296* -->\n",
       "<g id=\"edge128\" class=\"edge\">\n",
       "<title>5495299568&#45;&gt;5497425296*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1738.92,-819.93C1744.39,-822.69 1749.62,-825.86 1754.38,-829.5 1769.01,-840.7 1793.19,-882.71 1808.55,-911.18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1805.46,-912.83 1813.25,-920.01 1811.64,-909.53 1805.46,-912.83\"/>\n",
       "</g>\n",
       "<!-- 5495299568&#45;&gt;5497435232* -->\n",
       "<g id=\"edge166\" class=\"edge\">\n",
       "<title>5495299568&#45;&gt;5497435232*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1687.26,-783.3C1708.62,-766.02 1738.56,-738.04 1754.38,-706.5 1798.78,-617.93 1768.32,-583.09 1790.38,-486.5 1796.76,-458.53 1806.24,-427.19 1813.3,-405.14\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1816.62,-406.26 1816.38,-395.67 1809.96,-404.1 1816.62,-406.26\"/>\n",
       "</g>\n",
       "<!-- 5495299568ReLU&#45;&gt;5495299568 -->\n",
       "<g id=\"edge68\" class=\"edge\">\n",
       "<title>5495299568ReLU&#45;&gt;5495299568</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1533.47,-807.07C1541.49,-806.72 1550.49,-806.32 1559.91,-805.91\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"1559.78,-809.42 1569.61,-805.49 1559.47,-802.43 1559.78,-809.42\"/>\n",
       "</g>\n",
       "<!-- 5495111152 -->\n",
       "<g id=\"node187\" class=\"node\">\n",
       "<title>5495111152</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"3799.38,-433.5 3799.38,-469.5 3980.13,-469.5 3980.13,-433.5 3799.38,-433.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3809.26\" y=\"-446.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3819.13,-434 3819.13,-469.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3859.01\" y=\"-446.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"3898.88,-434 3898.88,-469.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"3939.51\" y=\"-446.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495111152&#45;&gt;5497437104+ -->\n",
       "<g id=\"edge117\" class=\"edge\">\n",
       "<title>5495111152&#45;&gt;5497437104+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3980.58,-449.84C4011.85,-449.27 4045.24,-448.65 4069.96,-448.2\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4069.91,-451.7 4079.84,-448.01 4069.78,-444.7 4069.91,-451.7\"/>\n",
       "</g>\n",
       "<!-- 5497437680 -->\n",
       "<g id=\"node188\" class=\"node\">\n",
       "<title>5497437680</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4237.38,-531.5 4237.38,-567.5 4422.63,-567.5 4422.63,-531.5 4237.38,-531.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4247.26\" y=\"-544.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4257.13,-532 4257.13,-567.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4299.26\" y=\"-544.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4341.38,-532 4341.38,-567.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4382.01\" y=\"-544.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437680&#45;&gt;5497437872+ -->\n",
       "<g id=\"edge89\" class=\"edge\">\n",
       "<title>5497437680&#45;&gt;5497437872+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4422.73,-543.75C4504.49,-536.89 4626.74,-522.37 4729.38,-492.5 4739.76,-489.48 4750.65,-485.04 4760.35,-480.58\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4761.84,-483.75 4769.34,-476.27 4758.81,-477.43 4761.84,-483.75\"/>\n",
       "</g>\n",
       "<!-- 5497437680*&#45;&gt;5497437680 -->\n",
       "<g id=\"edge69\" class=\"edge\">\n",
       "<title>5497437680*&#45;&gt;5497437680</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3763.57,-594.42C3843.2,-588.09 4085.59,-568.84 4225.81,-557.7\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4225.78,-561.21 4235.47,-556.93 4225.22,-554.23 4225.78,-561.21\"/>\n",
       "</g>\n",
       "<!-- 5497437872 -->\n",
       "<g id=\"node190\" class=\"node\">\n",
       "<title>5497437872</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4855.38,-408.5 4855.38,-444.5 5036.13,-444.5 5036.13,-408.5 4855.38,-408.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4865.26\" y=\"-421.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4875.13,-409 4875.13,-444.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4915.01\" y=\"-421.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0211</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4954.88,-409 4954.88,-444.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4995.51\" y=\"-421.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497437872&#45;&gt;5497450608+ -->\n",
       "<g id=\"edge147\" class=\"edge\">\n",
       "<title>5497437872&#45;&gt;5497450608+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M5036.56,-411.08C5045.17,-409.6 5053.52,-408.16 5061.16,-406.85\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"5061.74,-410.3 5071,-405.16 5060.56,-403.4 5061.74,-410.3\"/>\n",
       "</g>\n",
       "<!-- 5497437872+&#45;&gt;5497437872 -->\n",
       "<g id=\"edge70\" class=\"edge\">\n",
       "<title>5497437872+&#45;&gt;5497437872</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4818.04,-458.31C4829.95,-455.32 4845.02,-451.54 4860.57,-447.63\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4861.07,-451.12 4869.92,-445.29 4859.37,-444.33 4861.07,-451.12\"/>\n",
       "</g>\n",
       "<!-- 5495111392 -->\n",
       "<g id=\"node192\" class=\"node\">\n",
       "<title>5495111392</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"1895.5,-440.5 1895.5,-476.5 2076.25,-476.5 2076.25,-440.5 1895.5,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1905.38\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1915.25,-441 1915.25,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"1955.13\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 0.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"1995,-441 1995,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"2035.63\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495111392&#45;&gt;5497422128+ -->\n",
       "<g id=\"edge171\" class=\"edge\">\n",
       "<title>5495111392&#45;&gt;5497422128+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M2076.73,-458.5C2088.08,-458.5 2099.16,-458.5 2109.05,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"2108.93,-462 2118.93,-458.5 2108.93,-455 2108.93,-462\"/>\n",
       "</g>\n",
       "<!-- 5495299856 -->\n",
       "<g id=\"node193\" class=\"node\">\n",
       "<title>5495299856</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"313.5,-413.5 313.5,-449.5 494.25,-449.5 494.25,-413.5 313.5,-413.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"323.38\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"333.25,-414 333.25,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"373.12\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">data 2.0000</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"413,-414 413,-449.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"453.62\" y=\"-426.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299856&#45;&gt;5495299904* -->\n",
       "<g id=\"edge121\" class=\"edge\">\n",
       "<title>5495299856&#45;&gt;5495299904*</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M494.68,-447.28C504.23,-448.96 513.51,-450.59 521.89,-452.07\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"521.14,-455.49 531.6,-453.77 522.36,-448.59 521.14,-455.49\"/>\n",
       "</g>\n",
       "<!-- 5495299904 -->\n",
       "<g id=\"node194\" class=\"node\">\n",
       "<title>5495299904</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"622.5,-440.5 622.5,-476.5 807.75,-476.5 807.75,-440.5 622.5,-440.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"632.38\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"642.25,-441 642.25,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"684.38\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;1.0965</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"726.5,-441 726.5,-476.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"767.12\" y=\"-453.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5495299904&#45;&gt;5497405504+ -->\n",
       "<g id=\"edge124\" class=\"edge\">\n",
       "<title>5495299904&#45;&gt;5497405504+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M808.15,-458.5C816.52,-458.5 824.64,-458.5 832.09,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"832.03,-462 842.03,-458.5 832.03,-455 832.03,-462\"/>\n",
       "</g>\n",
       "<!-- 5495299904*&#45;&gt;5495299904 -->\n",
       "<g id=\"edge71\" class=\"edge\">\n",
       "<title>5495299904*&#45;&gt;5495299904</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M586.91,-458.5C593.95,-458.5 602,-458.5 610.55,-458.5\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"610.54,-462 620.54,-458.5 610.54,-455 610.54,-462\"/>\n",
       "</g>\n",
       "<!-- 5497438064 -->\n",
       "<g id=\"node196\" class=\"node\">\n",
       "<title>5497438064</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"4016.13,-355.5 4016.13,-391.5 4201.38,-391.5 4201.38,-355.5 4016.13,-355.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4026.01\" y=\"-368.7\" font-family=\"Times,serif\" font-size=\"14.00\"> </text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4035.88,-356 4035.88,-391.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4078.01\" y=\"-368.7\" font-family=\"Times,serif\" font-size=\"14.00\">data &#45;0.0397</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"4120.13,-356 4120.13,-391.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"4160.76\" y=\"-368.7\" font-family=\"Times,serif\" font-size=\"14.00\">grad 0.0000</text>\n",
       "</g>\n",
       "<!-- 5497438064&#45;&gt;5497450608+ -->\n",
       "<g id=\"edge96\" class=\"edge\">\n",
       "<title>5497438064&#45;&gt;5497450608+</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M4201.8,-375.03C4277.69,-376.16 4388.14,-377.5 4484.63,-377.5 4484.63,-377.5 4484.63,-377.5 4793.38,-377.5 4889.97,-377.5 5003.3,-389.13 5060.72,-395.85\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"5060.25,-399.32 5070.59,-397.03 5061.07,-392.37 5060.25,-399.32\"/>\n",
       "</g>\n",
       "<!-- 5497438064*&#45;&gt;5497438064 -->\n",
       "<g id=\"edge72\" class=\"edge\">\n",
       "<title>5497438064*&#45;&gt;5497438064</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M3763.13,-325.12C3812.3,-332.02 3922,-347.42 4004.52,-359.01\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"4003.82,-362.44 4014.21,-360.37 4004.8,-355.51 4003.82,-362.44\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x1478b9520>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "draw_dot(m(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5zHfhu1xP3-C",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Training Loop\n",
    "So after manually iterating, we put it all together in a training loop. We can repeatedly execute the next cell to continue training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define 4 different sets of inputs\n",
    "xs = [\n",
    "    [2.0, 3.0, -1.0],\n",
    "    [3.0, -1.0, 0.5],\n",
    "    [0.5, 1.0, 1.0],\n",
    "    [1.0, 1.0, -1.0]\n",
    "]\n",
    "\n",
    "# For each input set, we have a desired target value -- binary classification\n",
    "# ys = [1.0, -1.0, -1.0, 1.0]\n",
    "ys = [1.0, 0.0, 0.0, 1.0]\n",
    "\n",
    "# Manually seed the Random Number Generator for Reproducibility\n",
    "# You can comment the next line out see the variability\n",
    "np.random.seed(1)\n",
    "\n",
    "# Initialize an MLP with random weights\n",
    "m = MLP(3, [4, 4, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "id": "oUjx298hP3-C",
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final Loss:  0.005576204735206388\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Loss Per Iteration')"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHFCAYAAAAOmtghAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABICklEQVR4nO3de3gU5d3/8c/uJtlADgtJIBAIIYgIEs4UBERQBB9AKtpW1FbEQytVK4eiheIPhaqxVnnQKmgrSFVErCK1PohERY5WjlFOFZRDAiTEcMgJSLK78/sjyZIlAUPY3Uk279d1zUV25p6Z7w7UfHrf98xYDMMwBAAAECSsZhcAAADgS4QbAAAQVAg3AAAgqBBuAABAUCHcAACAoEK4AQAAQYVwAwAAggrhBgAABBXCDQAACCqEG6AeWrhwoSwWizZv3mx2KRf0xBNPyGKxeJawsDAlJydrwoQJOnnypN/PP27cOEVGRnqtmzt3rhYuXOj3c/+Y89Vx4MABWSyWOlEjUF+FmF0AgOC3YsUKORwOFRQUaPny5XrhhRe0ceNGbdiwQRaLJaC1zJ07V3FxcRo3blxAz1vTOlq2bKkvv/xSl112mTmFAUGAcAPA73r16qW4uDhJ0tChQ3Xs2DG9+eab2rBhgwYMGHBJxz59+rQaNWrkizJrzTAMnTlzxid12O12XXXVVT6oCmi4GJYCgti6des0ZMgQRUVFqXHjxurfv7/+7//+z6vNqVOnNGXKFCUnJys8PFwxMTHq3bu3Fi9e7Gmzb98+3XbbbUpISJDdbld8fLyGDBmi9PT0WtVV8cv74MGDkqSSkhI9+eST6tixo+x2u5o1a6a7775bP/zwg9d+bdu21Y033qilS5eqR48eCg8P18yZM2t83rZt22rnzp1avXq1Z6isbdu2nu35+fmeaxEWFqZWrVpp4sSJKioq8jqOxWLRQw89pFdeeUWdOnWS3W7XP/7xD0nSzJkz1bdvX8XExCg6Olo9e/bU/PnzVfkdxReq43zDUjX5u6wYrly1apV++9vfKi4uTrGxsbrlllt05MiRGl8noL6j5wYIUqtXr9bQoUPVtWtXzZ8/X3a7XXPnztWoUaO0ePFijRkzRpI0efJkvfnmm3ryySfVo0cPFRUVaceOHTp27JjnWCNGjJDL5dKzzz6rNm3aKDc3Vxs2bKj1vJnvvvtOktSsWTO53W7ddNNNWrt2rR599FH1799fBw8e1OOPP67Bgwdr8+bNXj0iW7du1e7du/XYY48pOTlZERERNT7vBx98oJ///OdyOByaO3eupLKeEqks5A0aNEiHDh3SH//4R3Xt2lU7d+7UjBkztH37dn366adeQ2jLli3T2rVrNWPGDLVo0ULNmzeXVBZO7r//frVp00aS9J///Ee/+93vdPjwYc2YMeNH66hOTf8uK9x3330aOXKk3n77bWVmZuqRRx7Rr371K33++ec1vlZAvWYAqHdef/11Q5KxadOm87a56qqrjObNmxsFBQWedU6n00hJSTFat25tuN1uwzAMIyUlxRg9evR5j5Obm2tIMubMmXPRdT7++OOGJCM7O9soLS01Tpw4Ybz11ltGo0aNjMTEROP06dPG4sWLDUnG+++/77Xvpk2bDEnG3LlzPeuSkpIMm81mfPvttzU6/1133WVERER4revcubMxaNCgKm1TU1MNq9Va5Zq+9957hiRj+fLlnnWSDIfDYRw/fvyC53e5XEZpaakxa9YsIzY21nPNL1TH/v37DUnG66+/7llX07/Lin8XDzzwgNcxn332WUOSkZWVdcF6gWDBsBQQhIqKivTVV1/p5z//udfdQjabTXfeeacOHTqkb7/9VpLUp08fffzxx5o6daq++OILnT592utYMTExuuyyy/SXv/xFs2fP1rZt2+R2uy+qnhYtWig0NFRNmzbVr371K/Xs2VMrVqxQeHi4PvroIzVp0kSjRo2S0+n0LN27d1eLFi30xRdfeB2ra9eu6tChQ+0uzAV89NFHSklJUffu3b3quOGGG2SxWKrUcd1116lp06ZVjvP555/r+uuvl8PhkM1mU2hoqGbMmKFjx44pJyfnouu6mL/LCj/96U+9Pnft2lXS2WFAINgRboAgdOLECRmGoZYtW1bZlpCQIEmeYacXX3xRf/jDH7Rs2TJde+21iomJ0ejRo7V3715JZfNLPvvsM91www169tln1bNnTzVr1kwPP/ywCgoKalTPp59+qk2bNik9PV25ublat26drrzySknS0aNHdfLkSYWFhSk0NNRryc7OVm5urtexqvtOvnD06FF98803VWqIioqSYRg1qmPjxo0aNmyYJOnvf/+71q9fr02bNmn69OmSVCU41sTF/F1WiI2N9fpcMeRVm/MD9RFzboAg1LRpU1mtVmVlZVXZVjGxtOLupYiICM2cOVMzZ87U0aNHPb04o0aN0n//+19JUlJSkubPny9J2rNnj95991098cQTKikp0SuvvPKj9XTr1s1zvnNVTHpdsWJFtdujoqK8Pvvr1vG4uDg1atRICxYsOO/2H6vjnXfeUWhoqD766COFh4d71i9btqzWdV3M3yWAMoQbIAhFRESob9++Wrp0qZ577jnPhFy326233npLrVu3rnZoJz4+XuPGjdPXX3+tOXPm6NSpU2rcuLFXmw4dOuixxx7T+++/r61bt15yrTfeeKPeeecduVwu9e3b95KP92Psdnu1PRg33nijnn76acXGxio5OblWx7ZYLAoJCZHNZvOsO336tN58880a13Gu2v5dAg0Z4Qaoxz7//HMdOHCgyvoRI0YoNTVVQ4cO1bXXXqspU6YoLCxMc+fO1Y4dO7R48WJPz0Pfvn114403qmvXrmratKl2796tN998U/369VPjxo31zTff6KGHHtIvfvELXX755QoLC9Pnn3+ub775RlOnTr3k73Dbbbdp0aJFGjFihCZMmKA+ffooNDRUhw4d0qpVq3TTTTfp5ptvvuTzVOjSpYveeecdLVmyRO3atVN4eLi6dOmiiRMn6v3339c111yjSZMmqWvXrnK73crIyNDKlSv1+9///kfD18iRIzV79mzdcccd+s1vfqNjx47pueeeq/ZOqPPVUZ2a/l0CKGfyhGYAtVBxV8z5lv379xuGYRhr1641rrvuOiMiIsJo1KiRcdVVVxn//ve/vY41depUo3fv3kbTpk0Nu91utGvXzpg0aZKRm5trGIZhHD161Bg3bpzRsWNHIyIiwoiMjDS6du1q/O///q/hdDovWGfF3VI//PDDBduVlpYazz33nNGtWzcjPDzciIyMNDp27Gjcf//9xt69ez3tkpKSjJEjR9b4OlV3t9SBAweMYcOGGVFRUYYkIykpybOtsLDQeOyxx4wrrrjCCAsLMxwOh9GlSxdj0qRJRnZ2tqedJOPBBx+s9pwLFiwwrrjiCs+1TE1NNebPn+/193KhOqq7W8owavZ3eb676FatWmVIMlatWlWzCwfUcxbDqPRkKQAAgHqOu6UAAEBQIdwAAICgQrgBAABBhXADAACCCuEGAAAEFcINAAAIKg3uIX5ut1tHjhxRVFQUD74CAKCeMAxDBQUFSkhIkNV64b6ZBhdujhw5osTERLPLAAAAtZCZmanWrVtfsE2DCzcVL+HLzMxUdHS0ydUAAICayM/PV2JiYpWX6VanwYWbiqGo6Ohowg0AAPVMTaaUMKEYAAAEFcINAAAIKoQbAAAQVAg3AAAgqBBuAABAUCHcAACAoGJquFmzZo1GjRqlhIQEWSwWLVu27Ef3KS4u1vTp05WUlCS73a7LLrtMCxYs8H+xAACgXjD1OTdFRUXq1q2b7r77bv3sZz+r0T633nqrjh49qvnz56t9+/bKycmR0+n0c6UAAKC+MDXcDB8+XMOHD69x+xUrVmj16tXat2+fYmJiJElt27b1U3UAAKA+qldzbj788EP17t1bzz77rFq1aqUOHTpoypQpOn369Hn3KS4uVn5+vtcCAACCV716/cK+ffu0bt06hYeH64MPPlBubq4eeOABHT9+/LzzblJTUzVz5swAVwoAAMxSr3pu3G63LBaLFi1apD59+mjEiBGaPXu2Fi5ceN7em2nTpikvL8+zZGZmBrhqAAAQSPWq56Zly5Zq1aqVHA6HZ12nTp1kGIYOHTqkyy+/vMo+drtddrvd77W53IaOFRbrVIlLbeMi/H4+AABQvXrVczNgwAAdOXJEhYWFnnV79uyR1WpV69atTaxMyso7rT5Pf6Zhc9aYWgcAAA2dqeGmsLBQ6enpSk9PlyTt379f6enpysjIkFQ2pDR27FhP+zvuuEOxsbG6++67tWvXLq1Zs0aPPPKI7rnnHjVq1MiMr+ARaS/rBCtxulXidJtaCwAADZmp4Wbz5s3q0aOHevToIUmaPHmyevTooRkzZkiSsrKyPEFHkiIjI5WWlqaTJ0+qd+/e+uUvf6lRo0bpxRdfNKX+yiLsZ0f4iop57g4AAGaxGIZhmF1EIOXn58vhcCgvL0/R0dE+PXbH//exzpS6tfbRa5UY09inxwYAoCG7mN/f9WrOTV1XMTRVSM8NAACmIdz4EOEGAADzEW58KIJwAwCA6Qg3PuTpuTlDuAEAwCyEGx+qCDfcLQUAgHkINz4UGc6wFAAAZiPc+BBzbgAAMB/hxoeimHMDAIDpCDc+5JlzU0K4AQDALIQbH6oYliqg5wYAANMQbnyICcUAAJiPcOND3AoOAID5CDc+FMmwFAAApiPc+FAEE4oBADAd4caHosK5FRwAALMRbnzI03NT7DK5EgAAGi7CjQ9VzLkpcblV7CTgAABgBsKND1WEG4neGwAAzEK48SGb1aJGoTZJzLsBAMAshBsf40F+AACYi3DjY5G8GRwAAFMRbnzsbLgpNbkSAAAaJsKNj0XYy+fcMKEYAABTEG58LNIeKokJxQAAmIVw42OR5T03vDwTAABzEG58rOJuqQLCDQAApiDc+FjFsBQ9NwAAmINw42MVw1LMuQEAwByEGx/z3ApeQrgBAMAMhBsfq3gzOD03AACYg3DjY1G8fgEAAFMRbnysoueGCcUAAJjD1HCzZs0ajRo1SgkJCbJYLFq2bFmN912/fr1CQkLUvXt3v9VXGxVzbgoYlgIAwBSmhpuioiJ169ZNL7300kXtl5eXp7Fjx2rIkCF+qqz2KsJNEROKAQAwRYiZJx8+fLiGDx9+0fvdf//9uuOOO2Sz2S6qtycQKh7iV3jGKcMwZLFYTK4IAICGpd7NuXn99df1/fff6/HHH69R++LiYuXn53st/lTRc+N0Gyp2uv16LgAAUFW9Cjd79+7V1KlTtWjRIoWE1KzTKTU1VQ6Hw7MkJib6tcaIsLN1cccUAACBV2/Cjcvl0h133KGZM2eqQ4cONd5v2rRpysvL8yyZmZl+rFKyWi2KCOPlmQAAmMXUOTcXo6CgQJs3b9a2bdv00EMPSZLcbrcMw1BISIhWrlyp6667rsp+drtddrs9oLVG2ENUVOLijikAAExQb8JNdHS0tm/f7rVu7ty5+vzzz/Xee+8pOTnZpMqqigwPUU5BMT03AACYwNRwU1hYqO+++87zef/+/UpPT1dMTIzatGmjadOm6fDhw3rjjTdktVqVkpLitX/z5s0VHh5eZb3ZPO+XItwAABBwpoabzZs369prr/V8njx5siTprrvu0sKFC5WVlaWMjAyzyqs1wg0AAOaxGIZhmF1EIOXn58vhcCgvL0/R0dF+Ocev39istF1H9dTNKfpl3yS/nAMAgIbkYn5/15u7peqTKN4MDgCAaQg3flDxlGImFAMAEHiEGz+oeDN4AeEGAICAI9z4geflmYQbAAACjnDjB9wtBQCAeQg3fnA23LhMrgQAgIaHcOMHEZ67pUpNrgQAgIaHcOMHUeEMSwEAYBbCjR9EeCYUMywFAECgEW78oGLOTQHDUgAABBzhxg8qhqWKSlxqYG+3AADAdIQbP6gYlnK5DZ0pdZtcDQAADQvhxg8ah9pksZT9zKRiAAACi3DjB1arRRFh3DEFAIAZCDd+wisYAAAwB+HGTyLsNklSwRnCDQAAgUS48ZPI8FBJ9NwAABBohBs/iSzvuWHODQAAgUW48RPPg/wINwAABBThxk8i7QxLAQBgBsKNn3iGpZhQDABAQBFu/CSSN4MDAGAKwo2fVLyCgXADAEBgEW78JIqH+AEAYArCjZ/QcwMAgDkIN34SSbgBAMAUhBs/8YQb7pYCACCgCDd+wt1SAACYg3DjJ8y5AQDAHIQbP6l8t5RhGCZXAwBAw0G48ZOKYSm3IZ0udZlcDQAADYep4WbNmjUaNWqUEhISZLFYtGzZsgu2X7p0qYYOHapmzZopOjpa/fr10yeffBKYYi9So1CbrJayn5lUDABA4JgaboqKitStWze99NJLNWq/Zs0aDR06VMuXL9eWLVt07bXXatSoUdq2bZufK714FouFeTcAAJggxMyTDx8+XMOHD69x+zlz5nh9fvrpp/Wvf/1L//73v9WjRw8fV3fpIu0hKjjjJNwAABBA9XrOjdvtVkFBgWJiYswupVo8yA8AgMAztefmUj3//PMqKirSrbfeet42xcXFKi4u9nzOz88PRGmSKt0OzpwbAAACpt723CxevFhPPPGElixZoubNm5+3XWpqqhwOh2dJTEwMWI1R5XdMFZUQbgAACJR6GW6WLFmie++9V++++66uv/76C7adNm2a8vLyPEtmZmaAqpQiwui5AQAg0OrdsNTixYt1zz33aPHixRo5cuSPtrfb7bLb7QGorKqKZ90UMOcGAICAMTXcFBYW6rvvvvN83r9/v9LT0xUTE6M2bdpo2rRpOnz4sN544w1JZcFm7NixeuGFF3TVVVcpOztbktSoUSM5HA5TvsOF8PJMAAACz9Rhqc2bN6tHjx6e27gnT56sHj16aMaMGZKkrKwsZWRkeNq/+uqrcjqdevDBB9WyZUvPMmHCBFPq/zFNGodKkk6cKjW5EgAAGg5Te24GDx58wfcuLVy40OvzF1984d+CfKxZVNlwWG5h8Y+0BAAAvlIvJxTXF3GRZeHmhwLCDQAAgUK48aOKnhvCDQAAgUO48aNmFT03hcUXHH4DAAC+Q7jxo4qemxKnW/ncMQUAQEAQbvwoPNTmeUoxQ1MAAAQG4cbPuGMKAIDAItz4WTPumAIAIKAIN34Wxx1TAAAEFOHGzyrfMQUAAPyPcONnPOsGAIDAItz4GeEGAIDAItz4GeEGAIDAItz4WcWcG24FBwAgMAg3fta8vOfmWFGJXG5ewQAAgL8RbvwsJiJMFovkchs6carE7HIAAAh6hBs/C7FZFdM4TBLzbgAACATCTQAwqRgAgMAh3AQA4QYAgMAh3AQATykGACBwCDcB4HkzOD03AAD4HeEmADzDUvTcAADgd4SbAIiLZM4NAACBQrgJACYUAwAQOISbAGBYCgCAwCHcBEDF3VInT5WqxOk2uRoAAIIb4SYAHI1CFWqzSJKOFdF7AwCAPxFuAsBqtTCpGACAACHcBAiTigEACAzCTYDQcwMAQGAQbgKkGeEGAICAINwECLeDAwAQGISbAGHODQAAgWFquFmzZo1GjRqlhIQEWSwWLVu27Ef3Wb16tXr16qXw8HC1a9dOr7zyiv8L9QHPyzPpuQEAwK9MDTdFRUXq1q2bXnrppRq1379/v0aMGKGBAwdq27Zt+uMf/6iHH35Y77//vp8rvXT03AAAEBghZp58+PDhGj58eI3bv/LKK2rTpo3mzJkjSerUqZM2b96s5557Tj/72c/8VKVvMKEYAIDAqFdzbr788ksNGzbMa90NN9ygzZs3q7S0tNp9iouLlZ+f77WYIa6856aoxKWiYqcpNQAA0BDUq3CTnZ2t+Ph4r3Xx8fFyOp3Kzc2tdp/U1FQ5HA7PkpiYGIhSq4gIs6lRqE0S824AAPCnehVuJMlisXh9Ngyj2vUVpk2bpry8PM+SmZnp9xqrY7FYmHcDAEAAmDrn5mK1aNFC2dnZXutycnIUEhKi2NjYavex2+2y2+2BKO9HNYuyK+P4KcINAAB+VK96bvr166e0tDSvdStXrlTv3r0VGhpqUlU1VzGpmGEpAAD8x9RwU1hYqPT0dKWnp0squ9U7PT1dGRkZksqGlMaOHetpP378eB08eFCTJ0/W7t27tWDBAs2fP19Tpkwxo/yLxrAUAAD+Z+qw1ObNm3Xttdd6Pk+ePFmSdNddd2nhwoXKysryBB1JSk5O1vLlyzVp0iS9/PLLSkhI0IsvvljnbwOvwCsYAADwP1PDzeDBgz0TgquzcOHCKusGDRqkrVu3+rEq/+HN4AAA+F+9mnNT3zEsBQCA/xFuAohwAwCA/xFuAujsyzNLLjgcBwAAao9wE0BxkWGSpBKXW3mnq39dBAAAuDSEmwCyh9gUG1EWcA6fPG1yNQAABCfCTYC1btpIknToBOEGAAB/INwEWOuYxpKkzOOnTK4EAIDgRLgJMHpuAADwL8JNgCU2Leu5OXSCnhsAAPyBcBNg9NwAAOBfhJsAS6w054Zn3QAA4HuEmwBr1aSs56aoxKWTp3jWDQAAvka4CbDwUJualz+pOJN5NwAA+BzhxgTMuwEAwH8INyZI5Fk3AAD4DeHGBPTcAADgP4QbE1Q864Y5NwAA+B7hxgStPQ/yo+cGAABfI9yYIDGmYliKZ90AAOBrhBsTtHQ0ksUinSl1K7ewxOxyAAAIKoQbE4SFWNUyOlwS824AAPA1wo1JmHcDAIB/EG5M0rp83g3PugEAwLcINyah5wYAAP8g3JgksenZO6YAAIDv1CrcZGZm6tChQ57PGzdu1MSJE/W3v/3NZ4UFO3puAADwj1qFmzvuuEOrVq2SJGVnZ2vo0KHauHGj/vjHP2rWrFk+LTBYVTzr5vCJ03K7edYNAAC+Uqtws2PHDvXp00eS9O677yolJUUbNmzQ22+/rYULF/qyvqDVIjpcNqtFJS63cgqKzS4HAICgUatwU1paKrvdLkn69NNP9dOf/lSS1LFjR2VlZfmuuiAWYrMqoQnPugEAwNdqFW46d+6sV155RWvXrlVaWpr+53/+R5J05MgRxcbG+rTAYNa6ScW8G8INAAC+Uqtw8+c//1mvvvqqBg8erNtvv13dunWTJH344Yee4Sr8uETPs26YVAwAgK+E1GanwYMHKzc3V/n5+WratKln/W9+8xs1btzYZ8UFu7N3TNFzAwCAr9Sq5+b06dMqLi72BJuDBw9qzpw5+vbbb9W8efOLOtbcuXOVnJys8PBw9erVS2vXrr1g+0WLFqlbt25q3LixWrZsqbvvvlvHjh2rzdcwHT03AAD4Xq3CzU033aQ33nhDknTy5En17dtXzz//vEaPHq158+bV+DhLlizRxIkTNX36dG3btk0DBw7U8OHDlZGRUW37devWaezYsbr33nu1c+dO/fOf/9SmTZt033331eZrmM7Tc3OSnhsAAHylVuFm69atGjhwoCTpvffeU3x8vA4ePKg33nhDL774Yo2PM3v2bN17772677771KlTJ82ZM0eJiYnnDUj/+c9/1LZtWz388MNKTk7W1Vdfrfvvv1+bN2+uzdcwXWJ5uDly8oycLrfJ1QAAEBxqFW5OnTqlqKgoSdLKlSt1yy23yGq16qqrrtLBgwdrdIySkhJt2bJFw4YN81o/bNgwbdiwodp9+vfvr0OHDmn58uUyDENHjx7Ve++9p5EjR573PMXFxcrPz/da6ormUXaF2axyuQ1l558xuxwAAIJCrcJN+/bttWzZMmVmZuqTTz7xBJScnBxFR0fX6Bi5ublyuVyKj4/3Wh8fH6/s7Oxq9+nfv78WLVqkMWPGKCwsTC1atFCTJk3017/+9bznSU1NlcPh8CyJiYk1/Jb+Z7Va1Kop824AAPClWoWbGTNmaMqUKWrbtq369Omjfv36SSrrxenRo8dFHctisXh9NgyjyroKu3bt0sMPP6wZM2Zoy5YtWrFihfbv36/x48ef9/jTpk1TXl6eZ8nMzLyo+vytNS/QBADAp2p1K/jPf/5zXX311crKyvI840aShgwZoptvvrlGx4iLi5PNZqvSS5OTk1OlN6dCamqqBgwYoEceeUSS1LVrV0VERGjgwIF68skn1bJlyyr72O12z9OU66KKScWZvEATAACfqFXPjSS1aNFCPXr00JEjR3T48GFJUp8+fdSxY8ca7R8WFqZevXopLS3Na31aWpr69+9f7T6nTp2S1epdss1mk1TW41MfeXpujtNzAwCAL9Qq3Ljdbs2aNUsOh0NJSUlq06aNmjRpoj/96U9yu2t+18/kyZP12muvacGCBdq9e7cmTZqkjIwMzzDTtGnTNHbsWE/7UaNGaenSpZo3b5727dun9evX6+GHH1afPn2UkJBQm69iuuS4CEnS97lFJlcCAEBwqNWw1PTp0zV//nw988wzGjBggAzD0Pr16/XEE0/ozJkzeuqpp2p0nDFjxujYsWOaNWuWsrKylJKSouXLlyspKUmSlJWV5fXMm3HjxqmgoEAvvfSSfv/736tJkya67rrr9Oc//7k2X6NOaN88UpL0fU7hBecbAQCAmrEYtRjPSUhI0CuvvOJ5G3iFf/3rX3rggQc8w1R1UX5+vhwOh/Ly8mp8Z5c/lTjd6jRjhVxuQ/+ZNkQtHOFmlwQAQJ1zMb+/azUsdfz48Wrn1nTs2FHHjx+vzSEbrLAQq5JiyiYVf/9DocnVAABQ/9Uq3HTr1k0vvfRSlfUvvfSSunbteslFNTTtmpUNTX2XQ7gBAOBS1WrOzbPPPquRI0fq008/Vb9+/WSxWLRhwwZlZmZq+fLlvq4x6LVvHqlPdx8l3AAA4AO16rkZNGiQ9uzZo5tvvlknT57U8ePHdcstt2jnzp16/fXXfV1j0KuYVEy4AQDg0tVqQvH5fP311+rZs6dcLpevDulzdW1CsSSlZ57U6JfXq3mUXRunX292OQAA1Dl+n1AM32rXrOxZNzkFxco/U2pyNQAA1G+EmzogOjxU8dFlr4hgaAoAgEtDuKkjmHcDAIBvXNTdUrfccssFt588efJSamnQ2jeL1PrvjvGsGwAALtFFhRuHw/Gj2yu/Cwo1d1ml1zAAAIDau6hww23e/tOeB/kBAOATzLmpIyrm3GQcP6UzpXX3VnoAAOo6wk0d0SzKrqjwELkN6eCxU2aXAwBAvUW4qSMsFosuY2gKAIBLRripQ7gdHACAS0e4qUM84YbbwQEAqDXCTR1ScccUt4MDAFB7hJs6pOJZN/tyC+V2++x9pgAANCiEmzoksWkjhdmsOlPq1uGTp80uBwCAeolwU4eE2KxKjit7QzjzbgAAqB3CTR3TntcwAABwSQg3dcxlzcp7bgg3AADUCuGmjrmMZ90AAHBJCDd1jGdYijk3AADUCuGmjmkXFymLRTpxqlTHCovNLgcAgHqHcFPHNAqzqXXTRpKkb7MLTK4GAID6h3BTB3Vu6ZAk7TiSZ3IlAADUP4SbOqhL67Jws/1wvsmVAABQ/xBu6qCUVmXhZudhem4AALhYhJs6KCUhWpK0L7dIBWdKTa4GAID6hXBTB8VG2pXgCJck7TzC0BQAABfD9HAzd+5cJScnKzw8XL169dLatWsv2L64uFjTp09XUlKS7Ha7LrvsMi1YsCBA1QZOxdDUDoamAAC4KCFmnnzJkiWaOHGi5s6dqwEDBujVV1/V8OHDtWvXLrVp06bafW699VYdPXpU8+fPV/v27ZWTkyOn0xngyv2vSyuHVu46SrgBAOAimRpuZs+erXvvvVf33XefJGnOnDn65JNPNG/ePKWmplZpv2LFCq1evVr79u1TTEyMJKlt27aBLDlgUjx3TBFuAAC4GKYNS5WUlGjLli0aNmyY1/phw4Zpw4YN1e7z4Ycfqnfv3nr22WfVqlUrdejQQVOmTNHp06fPe57i4mLl5+d7LfVBSkJZuNmXW6TC4uDrmQIAwF9MCze5ublyuVyKj4/3Wh8fH6/s7Oxq99m3b5/WrVunHTt26IMPPtCcOXP03nvv6cEHHzzveVJTU+VwODxLYmKiT7+HvzSLsqtFdLgMQ9qdVT8CGQAAdYHpE4otFovXZ8Mwqqyr4Ha7ZbFYtGjRIvXp00cjRozQ7NmztXDhwvP23kybNk15eXmeJTMz0+ffwV8qJhVvP8TQFAAANWVauImLi5PNZqvSS5OTk1OlN6dCy5Yt1apVKzkcDs+6Tp06yTAMHTp0qNp97Ha7oqOjvZb6IqVVWa1MKgYAoOZMCzdhYWHq1auX0tLSvNanpaWpf//+1e4zYMAAHTlyRIWFhZ51e/bskdVqVevWrf1arxm6tOIdUwAAXCxTh6UmT56s1157TQsWLNDu3bs1adIkZWRkaPz48ZLKhpTGjh3raX/HHXcoNjZWd999t3bt2qU1a9bokUce0T333KNGjRqZ9TX8piLcfJdTqFMlTCoGAKAmTL0VfMyYMTp27JhmzZqlrKwspaSkaPny5UpKSpIkZWVlKSMjw9M+MjJSaWlp+t3vfqfevXsrNjZWt956q5588kmzvoJfNY8OV/Mou3IKirU7K1+9kmLMLgkAgDrPYhiGYXYRgZSfny+Hw6G8vLx6Mf/mnoWb9Pl/czTzp511V/+2ZpcDAIApLub3t+l3S+HCPHdMMakYAIAaIdzUcV14xxQAABeFcFPHVdwOvjenUGdKXSZXAwBA3Ue4qeNaRIcrLjJMLrfBk4oBAKgBwk0dZ7FYPPNuGJoCAODHEW7qgS5MKgYAoMYIN/VARc/N15mEGwAAfgzhph7oldRUkrQnp0B5p0pNrgYAgLqNcFMPxEXalRwXIcOQtmacMLscAADqNMJNPdG7vPdm04HjJlcCAEDdRripJ37Stuy9UpsP0HMDAMCFEG7qid5ty3puvj50UsVOHuYHAMD5EG7qieS4CMVGhKnY6daOwzzMDwCA8yHc1BMWi8Vz19Rm5t0AAHBehJt6pGLezSbm3QAAcF6Em3qkYt7NloPHZRiGydUAAFA3EW7qkc4JDtlDrDpxqlTf/1BkdjkAANRJhJt6JCzEqu6JTSQx7wYAgPMh3NQzzLsBAODCCDf1TOV5NwAAoCrCTT3TM6mpLBbpwLFTyik4Y3Y5AADUOYSbeiY6PFRXxEdJkrYwNAUAQBWEm3qIeTcAAJwf4aYeYt4NAADnR7iph3qX99zsOJKvUyVOk6sBAKBuIdzUQ62aNFKCI1wut6FtGSfNLgcAgDqFcFNPXdUuVpK0/rtckysBAKBuIdzUU1dfHidJWruXcAMAQGWEm3rq6vZl4WbHkTwdKyw2uRoAAOoOwk091Tw6XB1bRMkwpPXfHzO7HAAA6gzCTT12TYdmkqR1e38wuRIAAOoOwk09VjE0tXZvrgzDMLkaAADqBtPDzdy5c5WcnKzw8HD16tVLa9eurdF+69evV0hIiLp37+7fAuuwPskxCguxKivvjL7/odDscgAAqBNMDTdLlizRxIkTNX36dG3btk0DBw7U8OHDlZGRccH98vLyNHbsWA0ZMiRAldZN4aE29U0ue6Dfmj3cNQUAgGRyuJk9e7buvfde3XffferUqZPmzJmjxMREzZs374L73X///brjjjvUr1+/AFVadw0svyV8Hc+7AQBAkonhpqSkRFu2bNGwYcO81g8bNkwbNmw4736vv/66vv/+ez3++OM1Ok9xcbHy8/O9lmBydfuyScVffn9MxU6XydUAAGA+08JNbm6uXC6X4uPjvdbHx8crOzu72n327t2rqVOnatGiRQoJCanReVJTU+VwODxLYmLiJddel3RsEaW4SLtOl7q09eBJs8sBAMB0pk8otlgsXp8Nw6iyTpJcLpfuuOMOzZw5Ux06dKjx8adNm6a8vDzPkpmZeck11yVWq8UzNLWWW8IBAFDNuj/8IC4uTjabrUovTU5OTpXeHEkqKCjQ5s2btW3bNj300EOSJLfbLcMwFBISopUrV+q6666rsp/dbpfdbvfPl6gjBl4epw+2Hda673L1qNnFAABgMtN6bsLCwtSrVy+lpaV5rU9LS1P//v2rtI+Ojtb27duVnp7uWcaPH68rrrhC6enp6tu3b6BKr3Mqnnez/XCejheVmFwNAADmMq3nRpImT56sO++8U71791a/fv30t7/9TRkZGRo/fryksiGlw4cP64033pDValVKSorX/s2bN1d4eHiV9Q1NxasY/ptdoPXf5WpUtwSzSwIAwDSmhpsxY8bo2LFjmjVrlrKyspSSkqLly5crKSlJkpSVlfWjz7xBmYGXx+m/2QVau/cHwg0AoEGzGA3suf35+flyOBzKy8tTdHS02eX4zJo9P2jsgo2Kj7bry6lDZLVWnZQNAEB9dTG/v02/Wwq+0bddjKLCQ3Q0v1ibD54wuxwAAExDuAkS9hCbbujcQpL076+PmFwNAADmIdwEkRu7tpQkfbwjS06X2+RqAAAwB+EmiAxoH6emjUOVW1iir/YfN7scAABMQbgJIqE2q/4npWxo6qNvGJoCADRMhJsgc2PXstvAP96RrVKGpgAADRDhJsj0TY5RXGSYTp4q1frvcs0uBwCAgCPcBJkQm1XDU8omFv/76yyTqwEAIPAIN0Go4q6plbuyVex0mVwNAACBRbgJQj9pG6P4aLsKzji1Zg9DUwCAhoVwE4SsVotGdimbWMxdUwCAhoZwE6Ru7FY2NPXprqM6U8rQFACg4SDcBKkeiU3UqkkjFZW49NnuHLPLAQAgYAg3Qcpisein3cuGpv65JdPkagAACBzCTRAb0ztRkrR6zw86fPK0ydUAABAYhJsg1jYuQv3axcowpHc30XsDAGgYCDdB7rY+Zb03/9ycKZfbMLkaAAD8j3AT5G7o3EJNGofqSN4Zrdn7g9nlAADgd4SbIBceatPNPVpJkt7ZmGFyNQAA+B/hpgG47SdtJEmf7c5RTsEZk6sBAMC/CDcNwBUtotSjTRM53Ybe33LY7HIAAPArwk0DcXt5782STRkyDCYWAwCCF+GmgRjZtaUi7SE6cOyU/rPvuNnlAADgN4SbBiLCHqJR3cqeWPzOJiYWAwCCF+GmAbm9/Jk3H2/P1tF8JhYDAIIT4aYB6dLKoZ+0baoSl1t/W7PP7HIAAPALwk0DYrFY9OC17SVJb3+VoeNFJSZXBACA7xFuGphBHZqpSyuHTpe6tGDdfrPLAQDA5wg3DUzl3pt/bDigvNOlJlcEAIBvEW4aoGFXxqtDfKQKip1688sDZpcDAIBPEW4aIKv1bO/N/HX7VVTsNLkiAAB8x/RwM3fuXCUnJys8PFy9evXS2rVrz9t26dKlGjp0qJo1a6bo6Gj169dPn3zySQCrDR4ju7RUUmxjnThVqsW8UBMAEERMDTdLlizRxIkTNX36dG3btk0DBw7U8OHDlZFR/S/bNWvWaOjQoVq+fLm2bNmia6+9VqNGjdK2bdsCXHn9F2Kz6reDLpMkvbpmn86UukyuCAAA37AYJr5oqG/fvurZs6fmzZvnWdepUyeNHj1aqampNTpG586dNWbMGM2YMaNG7fPz8+VwOJSXl6fo6Oha1R0sSpxuDfrLKmXlndETo67UuAHJZpcEAEC1Lub3t2k9NyUlJdqyZYuGDRvmtX7YsGHasGFDjY7hdrtVUFCgmJgYf5QY9MJCrHqgfO7N7LQ9OlZYbHJFAABcOtPCTW5urlwul+Lj473Wx8fHKzs7u0bHeP7551VUVKRbb731vG2Ki4uVn5/vteCs23+SqE4to5V/xqlnV3xrdjkAAFwy0ycUWywWr8+GYVRZV53FixfriSee0JIlS9S8efPztktNTZXD4fAsiYmJl1xzMAmxWfWnmzpLkpZsztS2jBMmVwQAwKUxLdzExcXJZrNV6aXJycmp0ptzriVLlujee+/Vu+++q+uvv/6CbadNm6a8vDzPkpmZecm1B5vebWP0s56tJUkz/rVTLrdp07AAALhkpoWbsLAw9erVS2lpaV7r09LS1L9///Put3jxYo0bN05vv/22Ro4c+aPnsdvtio6O9lpQ1dThHRUVHqLth/O4NRwAUK+ZOiw1efJkvfbaa1qwYIF2796tSZMmKSMjQ+PHj5dU1usyduxYT/vFixdr7Nixev7553XVVVcpOztb2dnZysvLM+srBI1mUXZNHtpBkvSXT77lpZoAgHrL1HAzZswYzZkzR7NmzVL37t21Zs0aLV++XElJSZKkrKwsr2fevPrqq3I6nXrwwQfVsmVLzzJhwgSzvkJQufOqJHVsEaW806X6yyf/NbscAABqxdTn3JiB59xc2Mb9x3Xrq19Kkt66t6+uvjzO5IoAAKgnz7lB3dQnOUZ39G0jSZr8bjrDUwCAeodwgyr+38grdVmzCOUUFOvR975RA+vcAwDUc4QbVNEozKYXbuuhMJtVn+4+qkVfcfcUAKD+INygWimtHHr0f66QJD35f7u092iByRUBAFAzhBuc1z0DkjXw8jidKXXr4XfSVezkzeEAgLqPcIPzslotev7WboqNCNPurHw9/q+dzL8BANR5hBtcUPOocD33i26yWKR3NmXq+ZV7zC4JAIALItzgR13bsbmeHJ0iSXpp1Xd6be0+kysCAOD8CDeokV/2TdIjN1RMMN6t97ccMrkiAACqR7hBjT0w+DLde3WyJOnR97/Rp7uOmlwRAABVEW5QYxaLRdNHdNItPVvJ5Tb04Ntb9cnObLPLAgDAC+EGF8VqtejPP+uqYVfGq9jp1m/f2qI3vjxgdlkAAHgQbnDRQm1Wzf1lT93ep43chjTjXzuV+vFuud3cJg4AMB/hBrUSYrPq6ZtTPJOMX129TxOX8KA/AID5CDeoNYvFogevba/Zt3ZTiNWiD78+ojGv/keZx0+ZXRoAoAEj3OCS3dKztRbe3UdR4SFKzzypES+s1b/SD5tdFgCggSLcwCeuvjxOH08YqN5JTVVQ7NSEd9L1+3e/VmGx0+zSAAANDOEGPtO6aWO985urNGHI5bJapPe3HtLIF9fq4+1ZTDYGAASMxWhgb0LMz8+Xw+FQXl6eoqOjzS4naG3cf1wT39mmI3lnJElXtozW5KEdNKRTc1ksFpOrAwDUNxfz+5twA7/JP1Oq19bu14J1+z3DU90Sm2jikMs1+IpmhBwAQI0Rbi6AcBN4J4pK9OqaffrHhgM6XVp2q3jHFlG6f1A73dg1QaE2RkcBABdGuLkAwo15figo1t/WfK+3v8pQUUlZyGnVpJHuuTpZt/RopaYRYSZXCACoqwg3F0C4MV/eqVK99dVBvb7+gHILiyVJoTaLBl/RXDf3aKXrOjZXeKjN5CoBAHUJ4eYCCDd1x5lSl5ZuPay3/nNQu7LyPeujwkN0Xcfm6pscqz7JMbqsWQTzcwCggSPcXADhpm76NrtAy9IP61/bDnvusKoQFxmmn7SN8SydWkYphHk6ANCgEG4ugHBTt7ndhjYdOK713x/Txv3HtC3jpIqdbq82EWE29Uxqqh5tmqpzQrQ6J0SrVZNG9O4AQBAj3FwA4aZ+KXa6tP1Qnr7af1ybDxzX5oMnVHCm6lOPHY1C1TkhWpc3j9RlzSPVLi5SlzWPUIvocEIPAAQBws0FEG7qN5fb0J6jBdp04Li+OZSnnUfytfdogZzneQKyPcSqlo5wtXQ0UktHuFo4whUfHa74aLuaR4ereZRdzaLssocwgRkA6rKL+f0dEqCaAJ+wWS3q1DJanVqe/Ydd7HRp79FC7crK1/c/FOr7nCLtyy3UwWOnVOx068CxUzpw7MJvKo8OD1FcpF2xkWGKjbCraUSomjQOU9PGFX+GKSo8RFHhIYoOD1V0eKiiwkNktdIrBAB1DeEG9Z49xKaUVg6ltHJ4rS9xupWVd1rZeWeUVb5k553W0fxi5RSc8fxZ6jKUf8ap/DNO7cstqvF5rRapaeMwxUaGKSaibIm0h6hxWIgi7DY1DgtR4zCbGoXaFO5ZrN4/h3ivD7NZCUwAcIkINwhaYSFWJcVGKCk24rxtDMPQyVOlOlZUomOFxTpWVKLcwmKdKCrViVMlOnmqRCdOlerk6VIVnClVwRmnCs6U6kypW25DZfsVlfi87vAQq+yhNtlDykKPPcRavpQFIXuITaEhVoXaLAqzWRVisyjUZvX6uWyxKMRqLWtrtSik0rqydpV/tirEWv5nxX628n3K97VZz+4TarMwnwlAnUS4QYNmsVjUNCJMTSPC1L55ZI33K3G6dfJUWbA57glEJSoqcelUiVNFxS4VFTt1qtSl4lKXzpS6dbrUpdMlLp1xulRc6taZUlfZ4nTLVWnOUInTrRKnW6pm4nRdY7XIE35s5cHIZrUoxGqRrTwE2awW2Sxl20Ns5e3K15eFKIts5WGpYt+Q8qDlOVbl7edsq/zZ5rWPxXNua/k6a6VtFbV5PleqJeSc44RYrbJadfb7WC2yWkS4A+oo08PN3Llz9Ze//EVZWVnq3Lmz5syZo4EDB563/erVqzV58mTt3LlTCQkJevTRRzV+/PgAVgyU9a40jw5X8+hwnxzP6XLrjPNs4Cl2ulVc6laxsywYnXG6VOJ0l68vC0SlTrdKXWVLictQqcstp8ut0vKfyz4bKnUbKnW65XSXbXO63Sp1Gip1l293ueV0G559ne6ysFXqKl9Xvq26OdtuozyM+eQq1D9Wi8qDztnwdG6gqhK2rFbZrPIEvor9zw1nVktZ4PJss5T1lNmsZee0WM6GRmv5es85zwl0tnOOX/mYZTVLVsvZ9VbL2c/W8m3e7c89hry3l2+zVLo+FkvZd644rsVzTp3zHQmNuHSmhpslS5Zo4sSJmjt3rgYMGKBXX31Vw4cP165du9SmTZsq7ffv368RI0bo17/+td566y2tX79eDzzwgJo1a6af/exnJnwDwDdCbFZF2qyKtJv+/zfOy+02ykJQRUhylYcgtyGXqywslYWis+HIbRhyVvzpNuQqD1QV+1Ucw1mxlAcyl1H+s7tsf8++bsNz7MqfnZ5jSS63Wy5DnmO7jbI2lddVLKVut9xueQJdxfk8bcr3Pe81MSS3y5BkqDhwfxVBryI0Vg5wFUGpcsg7N4BVhKjKoalyW0v5epulanCrvO3c4597TFU5R6VaLGUh89warJX3qbTdcs7xLTob8CrvY/GEz7M/Vz7/ue2lyvvJ08Yi75o9f6qiLu92Z49/zmedrf3cY0pSiM2ilo5GZv0TMvdW8L59+6pnz56aN2+eZ12nTp00evRopaamVmn/hz/8QR9++KF2797tWTd+/Hh9/fXX+vLLL2t0Tm4FB3AxDONsgPKENNfZ4OO1GIbclUJR5eBUuY3TbXjCYuX2FYv7nDYutyHDUKXjGnKXf3ZXPu85dbrd3iHNK+yVB0G3Ic86t1FWs9so/7nStnPrclVq633c8trK1xkVxyj/GQ1Dsyi7Nk2/3qfHrBe3gpeUlGjLli2aOnWq1/phw4Zpw4YN1e7z5ZdfatiwYV7rbrjhBs2fP1+lpaUKDQ2tsk9xcbGKi8/+f6r8/PwqbQDgfCzlw0M8CunSGeeEHa8g5S4Pa0bV8OYVvsqD07mBTyoPY+6zoaziXJXPUfGzq/x8xjnnNyqFuopwaEhexyr7ueyzJK/ajErH9xy7vGbpnODoPlujofLPkqeGs+sqXZ/yNpXP56p8HOPs96io2zjn2lf+05D39/Ucu/zB8K7y81d857PHPbufKmqtdMzwUHNfkWNauMnNzZXL5VJ8fLzX+vj4eGVnZ1e7T3Z2drXtnU6ncnNz1bJlyyr7pKamaubMmb4rHABQK54hGFnMn/CJoGb62wfPnThmGMYFJ5NV17669RWmTZumvLw8z5KZmXmJFQMAgLrMtPAcFxcnm81WpZcmJyenSu9MhRYtWlTbPiQkRLGxsdXuY7fbZbfbfVM0AACo80zruQkLC1OvXr2UlpbmtT4tLU39+/evdp9+/fpVab9y5Ur17t272vk2AACg4TF1WGry5Ml67bXXtGDBAu3evVuTJk1SRkaG57k106ZN09ixYz3tx48fr4MHD2ry5MnavXu3FixYoPnz52vKlClmfQUAAFDHmDqna8yYMTp27JhmzZqlrKwspaSkaPny5UpKSpIkZWVlKSMjw9M+OTlZy5cv16RJk/Tyyy8rISFBL774Is+4AQAAHqY+58YMPOcGAID652J+f5t+txQAAIAvEW4AAEBQIdwAAICgQrgBAABBhXADAACCCuEGAAAEFcINAAAIKoQbAAAQVBrcW+crnlmYn59vciUAAKCmKn5v1+TZww0u3BQUFEiSEhMTTa4EAABcrIKCAjkcjgu2aXCvX3C73Tpy5IiioqJksVh8euz8/HwlJiYqMzOTVzv4Gdc6cLjWgcO1DhyudeD46lobhqGCggIlJCTIar3wrJoG13NjtVrVunVrv54jOjqa/7EECNc6cLjWgcO1DhyudeD44lr/WI9NBSYUAwCAoEK4AQAAQYVw40N2u12PP/647Ha72aUEPa514HCtA4drHThc68Ax41o3uAnFAAAguNFzAwAAggrhBgAABBXCDQAACCqEGwAAEFQINz4yd+5cJScnKzw8XL169dLatWvNLqneS01N1U9+8hNFRUWpefPmGj16tL799luvNoZh6IknnlBCQoIaNWqkwYMHa+fOnSZVHDxSU1NlsVg0ceJEzzqute8cPnxYv/rVrxQbG6vGjRure/fu2rJli2c719o3nE6nHnvsMSUnJ6tRo0Zq166dZs2aJbfb7WnDta69NWvWaNSoUUpISJDFYtGyZcu8ttfk2hYXF+t3v/ud4uLiFBERoZ/+9Kc6dOjQpRdn4JK98847RmhoqPH3v//d2LVrlzFhwgQjIiLCOHjwoNml1Ws33HCD8frrrxs7duww0tPTjZEjRxpt2rQxCgsLPW2eeeYZIyoqynj//feN7du3G2PGjDFatmxp5Ofnm1h5/bZx40ajbdu2RteuXY0JEyZ41nOtfeP48eNGUlKSMW7cOOOrr74y9u/fb3z66afGd99952nDtfaNJ5980oiNjTU++ugjY//+/cY///lPIzIy0pgzZ46nDde69pYvX25Mnz7deP/99w1JxgcffOC1vSbXdvz48UarVq2MtLQ0Y+vWrca1115rdOvWzXA6nZdUG+HGB/r06WOMHz/ea13Hjh2NqVOnmlRRcMrJyTEkGatXrzYMwzDcbrfRokUL45lnnvG0OXPmjOFwOIxXXnnFrDLrtYKCAuPyyy830tLSjEGDBnnCDdfad/7whz8YV1999Xm3c619Z+TIkcY999zjte6WW24xfvWrXxmGwbX2pXPDTU2u7cmTJ43Q0FDjnXfe8bQ5fPiwYbVajRUrVlxSPQxLXaKSkhJt2bJFw4YN81o/bNgwbdiwwaSqglNeXp4kKSYmRpK0f/9+ZWdne117u92uQYMGce1r6cEHH9TIkSN1/fXXe63nWvvOhx9+qN69e+sXv/iFmjdvrh49eujvf/+7ZzvX2neuvvpqffbZZ9qzZ48k6euvv9a6des0YsQISVxrf6rJtd2yZYtKS0u92iQkJCglJeWSr3+De3Gmr+Xm5srlcik+Pt5rfXx8vLKzs02qKvgYhqHJkyfr6quvVkpKiiR5rm911/7gwYMBr7G+e+edd7R161Zt2rSpyjaute/s27dP8+bN0+TJk/XHP/5RGzdu1MMPPyy73a6xY8dyrX3oD3/4g/Ly8tSxY0fZbDa5XC499dRTuv322yXx79qfanJts7OzFRYWpqZNm1Zpc6m/Pwk3PmKxWLw+G4ZRZR1q76GHHtI333yjdevWVdnGtb90mZmZmjBhglauXKnw8PDztuNaXzq3263evXvr6aefliT16NFDO3fu1Lx58zR27FhPO671pVuyZIneeustvf322+rcubPS09M1ceJEJSQk6K677vK041r7T22urS+uP8NSlyguLk42m61KyszJyamSWFE7v/vd7/Thhx9q1apVat26tWd9ixYtJIlr7wNbtmxRTk6OevXqpZCQEIWEhGj16tV68cUXFRIS4rmeXOtL17JlS1155ZVe6zp16qSMjAxJ/Lv2pUceeURTp07Vbbfdpi5duujOO+/UpEmTlJqaKolr7U81ubYtWrRQSUmJTpw4cd42tUW4uURhYWHq1auX0tLSvNanpaWpf//+JlUVHAzD0EMPPaSlS5fq888/V3Jystf25ORktWjRwuval5SUaPXq1Vz7izRkyBBt375d6enpnqV379765S9/qfT0dLVr145r7SMDBgyo8kiDPXv2KCkpSRL/rn3p1KlTslq9f83ZbDbPreBca/+pybXt1auXQkNDvdpkZWVpx44dl379L2k6MgzDOHsr+Pz5841du3YZEydONCIiIowDBw6YXVq99tvf/tZwOBzGF198YWRlZXmWU6dOedo888wzhsPhMJYuXWps377duP3227mN00cq3y1lGFxrX9m4caMREhJiPPXUU8bevXuNRYsWGY0bNzbeeustTxuutW/cddddRqtWrTy3gi9dutSIi4szHn30UU8brnXtFRQUGNu2bTO2bdtmSDJmz55tbNu2zfMYlJpc2/HjxxutW7c2Pv30U2Pr1q3Gddddx63gdcnLL79sJCUlGWFhYUbPnj09tyuj9iRVu7z++uueNm6323j88ceNFi1aGHa73bjmmmuM7du3m1d0EDk33HCtfeff//63kZKSYtjtdqNjx47G3/72N6/tXGvfyM/PNyZMmGC0adPGCA8PN9q1a2dMnz7dKC4u9rThWtfeqlWrqv1v9F133WUYRs2u7enTp42HHnrIiImJMRo1amTceOONRkZGxiXXZjEMw7i0vh8AAIC6gzk3AAAgqBBuAABAUCHcAACAoEK4AQAAQYVwAwAAggrhBgAABBXCDQAACCqEGwANQtu2bTVnzhyzywAQAIQbAD43btw4jR49WpI0ePBgTZw4MWDnXrhwoZo0aVJl/aZNm/Sb3/wmYHUAME+I2QUAQE2UlJQoLCys1vs3a9bMh9UAqMvouQHgN+PGjdPq1av1wgsvyGKxyGKx6MCBA5KkXbt2acSIEYqMjFR8fLzuvPNO5ebmevYdPHiwHnroIU2ePFlxcXEaOnSoJGn27Nnq0qWLIiIilJiYqAceeECFhYWSpC+++EJ333238vLyPOd74oknJFUdlsrIyNBNN92kyMhIRUdH69Zbb9XRo0c925944gl1795db775ptq2bSuHw6HbbrtNBQUFnjbvvfeeunTpokaNGik2NlbXX3+9ioqK/HQ1AdQU4QaA37zwwgvq16+ffv3rXysrK0tZWVlKTExUVlaWBg0apO7du2vz5s1asWKFjh49qltvvdVr/3/84x8KCQnR+vXr9eqrr0qSrFarXnzxRe3YsUP/+Mc/9Pnnn+vRRx+VJPXv319z5sxRdHS053xTpkypUpdhGBo9erSOHz+u1atXKy0tTd9//73GjBnj1e7777/XsmXL9NFHH+mjjz7S6tWr9cwzz0iSsrKydPvtt+uee+7R7t279cUXX+iWW24Rr+sDzMewFAC/cTgcCgsLU+PGjdWiRQvP+nnz5qlnz556+umnPesWLFigxMRE7dmzRx06dJAktW/fXs8++6zXMSvP30lOTtaf/vQn/fa3v9XcuXMVFhYmh8Mhi8Xidb5zffrpp/rmm2+0f/9+JSYmSpLefPNNde7cWZs2bdJPfvITSZLb7dbChQsVFRUlSbrzzjv12Wef6amnnlJWVpacTqduueUWJSUlSZK6dOlyCVcLgK/QcwMg4LZs2aJVq1YpMjLSs3Ts2FFSWW9Jhd69e1fZd9WqVRo6dKhatWqlqKgojR07VseOHbuo4aDdu3crMTHRE2wk6corr1STJk20e/duz7q2bdt6go0ktWzZUjk5OZKkbt26aciQIerSpYt+8Ytf6O9//7tOnDhR84sAwG8INwACzu12a9SoUUpPT/da9u7dq2uuucbTLiIiwmu/gwcPasSIEUpJSdH777+vLVu26OWXX5YklZaW1vj8hmHIYrH86PrQ0FCv7RaLRW63W5Jks9mUlpamjz/+WFdeeaX++te/6oorrtD+/ftrXAcA/yDcAPCrsLAwuVwur3U9e/bUzp071bZtW7Vv395rOTfQVLZ582Y5nU49//zzuuqqq9ShQwcdOXLkR893riuvvFIZGRnKzMz0rNu1a5fy8vLUqVOnGn83i8WiAQMGaObMmdq2bZvCwsL0wQcf1Hh/AP5BuAHgV23bttVXX32lAwcOKDc3V263Ww8++KCOHz+u22+/XRs3btS+ffu0cuVK3XPPPRcMJpdddpmcTqf++te/at++fXrzzTf1yiuvVDlfYWGhPvvsM+Xm5urUqVNVjnP99dera9eu+uUvf6mtW7dq48aNGjt2rAYNGlTtUFh1vvrqKz399NPavHmzMjIytHTpUv3www8XFY4A+AfhBoBfTZkyRTabTVdeeaWaNWumjIwMJSQkaP369XK5XLrhhhuUkpKiCRMmyOFwyGo9/3+WunfvrtmzZ+vPf/6zUlJStGjRIqWmpnq16d+/v8aPH68xY8aoWbNmVSYkS2U9LsuWLVPTpk11zTXX6Prrr1e7du20ZMmSGn+v6OhorVmzRiNGjFCHDh302GOP6fnnn9fw4cNrfnEA+IXF4L5FAAAQROi5AQAAQYVwAwAAggrhBgAABBXCDQAACCqEGwAAEFQINwAAIKgQbgAAQFAh3AAAgKBCuAEAAEGFcAMAAIIK4QYAAAQVwg0AAAgq/x8wG2p6TRZ32AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "losses = []\n",
    "niters = 100\n",
    "step_size = 0.01\n",
    "\n",
    "for k in range(niters):\n",
    "\n",
    "    # Training Step 1: forward pass\n",
    "    ypred = [m(x) for x in xs]\n",
    "    \n",
    "    # Training Step 2: Calculate the loss\n",
    "    loss = sum((yout - ygt)**2 for ygt, yout in zip(ys, ypred))\n",
    "    losses.append(loss.data)\n",
    "\n",
    "    # Training Step 3: Zero the gradients and run the backward pass\n",
    "    m.zero_grad()\n",
    "    loss.backward()\n",
    "\n",
    "    # Training Step 4: Update parameters\n",
    "    for p in m.parameters():\n",
    "        p.data += -step_size * p.grad\n",
    "\n",
    "    # print(k, loss.data)\n",
    "\n",
    "print(\"Final Loss: \", loss.data)\n",
    "    \n",
    "plt.plot(losses)\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.xlabel(\"Iterations\")\n",
    "plt.title(\"Loss Per Iteration\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "id": "Yynp96CBP3-C",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Value(data=1.0366133797239414, grad=0.07322675944788282),\n",
       " Value(data=0.00698935172551762, grad=0.01397870345103524),\n",
       " Value(data=0.028369351100668988, grad=0.056738702201337976),\n",
       " Value(data=0.9418450858397948, grad=-0.11630982832041048)]"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ypred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Build and Train the Equivalent MLP in PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ptMLP(\n",
      "  (layers): Sequential(\n",
      "    (0): Linear(in_features=3, out_features=4, bias=True)\n",
      "    (1): ReLU()\n",
      "    (2): Linear(in_features=4, out_features=4, bias=True)\n",
      "    (3): ReLU()\n",
      "    (4): Linear(in_features=4, out_features=1, bias=True)\n",
      "  )\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torch.optim import SGD\n",
    "\n",
    "# Manually seed the Random Number Generator for Reproducibility\n",
    "# You can comment the next line out see the variability\n",
    "torch.manual_seed(99)\n",
    "\n",
    "# Step 1: Define the MLP model\n",
    "class ptMLP(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(ptMLP, self).__init__()\n",
    "        self.layers = nn.Sequential(\n",
    "            nn.Linear(3, 4),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(4, 4),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(4, 1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.layers(x)\n",
    "\n",
    "model = ptMLP()\n",
    "print(model)\n",
    "\n",
    "# Step 2: Define a loss function and an optimizer\n",
    "criterion = nn.MSELoss(reduction='sum')\n",
    "optimizer = SGD(model.parameters(), lr=0.01)\n",
    "\n",
    "# Step 3: Create a tiny dataset\n",
    "xs = [\n",
    "    [2.0, 3.0, -1.0],\n",
    "    [3.0, -1.0, 0.5],\n",
    "    [0.5, 1.0, 1.0],\n",
    "    [1.0, 1.0, -1.0]\n",
    "]\n",
    "\n",
    "# we had to transpose ys for torch.tensor\n",
    "ys_transpose = [[1.0], \n",
    "      [0.0], \n",
    "      [0.0], \n",
    "      [1.0]]\n",
    "\n",
    "inputs = torch.tensor(xs)\n",
    "outputs = torch.tensor(ys_transpose)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now run the training loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final Loss: 0.06534020602703094\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Loss Per Iteration')"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHFCAYAAAAOmtghAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIHklEQVR4nO3de3hU1b3/8c/cMrmQDJCQGwQSRAG5X1RAuSiKFaVSbau2irSnp8WKihxqBf15V6zHetAqUC2Cioo9DXKsIhXlIhYVAkEBES8EEiAhJEASEkgyM/v3R5KBMSGEMDM7mbxfz7OfZNZee+abHTSfZ+219rYYhmEIAAAgTFjNLgAAACCQCDcAACCsEG4AAEBYIdwAAICwQrgBAABhhXADAADCCuEGAACEFcINAAAIK4QbAAAQVgg3QCu0aNEiWSwWZWVlmV1Kox566CFZLBbfFhERoYyMDN111106cuRI0D9/8uTJateunV/b3LlztWjRoqB/9umcqo7du3fLYrG0iBqB1spudgEAwt+KFSvkcrlUVlam5cuX69lnn9WGDRu0fv16WSyWkNYyd+5cJSQkaPLkySH93KbWkZKSok8//VTnnHOOOYUBYYBwAyDohgwZooSEBEnSFVdcoeLiYr322mtav369Lr744rN672PHjikqKioQZTabYRg6fvx4QOpwOp0aNmxYAKoC2i4uSwFh7JNPPtHYsWMVGxur6OhojRgxQu+9955fn4qKCs2YMUMZGRmKjIxUx44dNXToUL355pu+Prt27dKNN96o1NRUOZ1OJSUlaezYsdqyZUuz6qr7471nzx5JUlVVlR577DH16tVLTqdTnTp10q9+9SsdPHjQ77j09HRdc801Wrp0qQYNGqTIyEg9/PDDTf7c9PR0bd++XWvXrvVdKktPT/ftLy0t9Z2LiIgIde7cWdOmTVN5ebnf+1gsFk2dOlXz589X79695XQ69corr0iSHn74YV100UXq2LGj4uLiNHjwYC1YsEAnP6O4sTpOdVmqKb/LusuVq1ev1m233aaEhATFx8fruuuu0/79+5t8noDWjpEbIEytXbtWV1xxhfr3768FCxbI6XRq7ty5mjBhgt58803dcMMNkqTp06frtdde02OPPaZBgwapvLxc27ZtU3Fxse+9xo8fL4/Ho6eeekpdu3ZVUVGR1q9f3+x5M999950kqVOnTvJ6vbr22mu1bt063XPPPRoxYoT27NmjBx98UGPGjFFWVpbfiMjmzZu1Y8cO3X///crIyFBMTEyTP/ftt9/WT3/6U7lcLs2dO1dSzUiJVBPyRo8erb1792rWrFnq37+/tm/frgceeEBbt27Vhx9+6HcJbdmyZVq3bp0eeOABJScnKzExUVJNOPnd736nrl27SpI+++wz3XHHHdq3b58eeOCB09bRkKb+Luv85je/0dVXX6033nhDeXl5+sMf/qCbb75Zq1atavK5Alo1A0Crs3DhQkOSsXHjxlP2GTZsmJGYmGiUlZX52txut9G3b1+jS5cuhtfrNQzDMPr27WtMnDjxlO9TVFRkSDLmzJlzxnU++OCDhiSjoKDAqK6uNg4fPmwsXrzYiIqKMtLS0oxjx44Zb775piHJyMzM9Dt248aNhiRj7ty5vrZu3boZNpvN2LlzZ5M+/9ZbbzViYmL82vr06WOMHj26Xt/Zs2cbVqu13jn9xz/+YUgyli9f7muTZLhcLuPQoUONfr7H4zGqq6uNRx55xIiPj/ed88bqyMnJMSQZCxcu9LU19XdZ9+/i97//vd97PvXUU4YkIz8/v9F6gXDBZSkgDJWXl+vzzz/XT3/6U7/VQjabTbfccov27t2rnTt3SpIuvPBCvf/++7r33nu1Zs0aHTt2zO+9OnbsqHPOOUf//d//rWeeeUbZ2dnyer1nVE9ycrIcDoc6dOigm2++WYMHD9aKFSsUGRmpd999V+3bt9eECRPkdrt928CBA5WcnKw1a9b4vVf//v113nnnNe/ENOLdd99V3759NXDgQL86rrzySlkslnp1XHbZZerQoUO991m1apUuv/xyuVwu2Ww2ORwOPfDAAyouLlZhYeEZ13Umv8s6P/7xj/1e9+/fX9KJy4BAuCPcAGHo8OHDMgxDKSkp9falpqZKku+y03PPPac//vGPWrZsmS699FJ17NhREydO1LfffiupZn7JRx99pCuvvFJPPfWUBg8erE6dOunOO+9UWVlZk+r58MMPtXHjRm3ZskVFRUX65JNPdP7550uSDhw4oCNHjigiIkIOh8NvKygoUFFRkd97NfQzBcKBAwf05Zdf1qshNjZWhmE0qY4NGzZo3LhxkqSXXnpJ//73v7Vx40bdd999klQvODbFmfwu68THx/u9rrvk1ZzPB1oj5twAYahDhw6yWq3Kz8+vt69uYmnd6qWYmBg9/PDDevjhh3XgwAHfKM6ECRP09ddfS5K6deumBQsWSJK++eYb/f3vf9dDDz2kqqoqzZ8//7T1DBgwwPd5P1Q36XXFihUN7o+NjfV7Hayl4wkJCYqKitLLL798yv2nq2PJkiVyOBx69913FRkZ6WtftmxZs+s6k98lgBqEGyAMxcTE6KKLLtLSpUv19NNP+ybker1eLV68WF26dGnw0k5SUpImT56sL774QnPmzFFFRYWio6P9+px33nm6//77lZmZqc2bN591rddcc42WLFkij8ejiy666Kzf73ScTmeDIxjXXHONnnjiCcXHxysjI6NZ722xWGS322Wz2Xxtx44d02uvvdbkOn6oub9LoC0j3ACt2KpVq7R79+567ePHj9fs2bN1xRVX6NJLL9WMGTMUERGhuXPnatu2bXrzzTd9Iw8XXXSRrrnmGvXv318dOnTQjh079Nprr2n48OGKjo7Wl19+qalTp+pnP/uZzj33XEVERGjVqlX68ssvde+99571z3DjjTfq9ddf1/jx43XXXXfpwgsvlMPh0N69e7V69Wpde+21+slPfnLWn1OnX79+WrJkid566y11795dkZGR6tevn6ZNm6bMzEyNGjVKd999t/r37y+v16vc3Fx98MEH+q//+q/Thq+rr75azzzzjH7xi1/ot7/9rYqLi/X00083uBLqVHU0pKm/SwC1TJ7QDKAZ6lbFnGrLyckxDMMw1q1bZ1x22WVGTEyMERUVZQwbNsz45z//6fde9957rzF06FCjQ4cOhtPpNLp3727cfffdRlFRkWEYhnHgwAFj8uTJRq9evYyYmBijXbt2Rv/+/Y3/+Z//Mdxud6N11q2WOnjwYKP9qqurjaefftoYMGCAERkZabRr187o1auX8bvf/c749ttvff26detmXH311U0+Tw2tltq9e7cxbtw4IzY21pBkdOvWzbfv6NGjxv3332/07NnTiIiIMFwul9GvXz/j7rvvNgoKCnz9JBm33357g5/58ssvGz179vSdy9mzZxsLFizw+700VkdDq6UMo2m/y1Otolu9erUhyVi9enXTThzQylkM46Q7SwEAALRyrJYCAABhhXADAADCCuEGAACEFcINAAAIK4QbAAAQVgg3AAAgrLS5m/h5vV7t379fsbGx3PgKAIBWwjAMlZWVKTU1VVZr42MzbS7c7N+/X2lpaWaXAQAAmiEvL09dunRptE+bCzd1D+HLy8tTXFycydUAAICmKC0tVVpaWr2H6TakzYWbuktRcXFxhBsAAFqZpkwpYUIxAAAIK4QbAAAQVgg3AAAgrBBuAABAWCHcAACAsEK4AQAAYYVwAwAAwgrhBgAAhBXCDQAACCuEGwAAEFYINwAAIKwQbgAAQFgh3ASI12voYFmlvis8anYpAAC0aYSbAMk7XKELHv9QP37+E7NLAQCgTSPcBEh8O6ckqaLKo4oqt8nVAADQdhFuAiQmwianveZ0Fh+tMrkaAADaLsJNgFgsFiXUjt4UHa00uRoAANouwk0AJbSLkMTIDQAAZiLcBFDdvJvickZuAAAwC+EmgOJjakZuihi5AQDANISbAIpnzg0AAKYj3AQQc24AADAf4SaAEphzAwCA6Qg3ARTPyA0AAKYj3ARQfEzdnBvCDQAAZiHcBFDdnJtD5ZXyeg2TqwEAoG0i3ARQh9ql4F5DOlzB6A0AAGYg3ASQw2ZVh2iHJKm4nHADAIAZCDcBxr1uAAAwF+EmwOruUsyKKQAAzEG4CTDfvW4YuQEAwBSEmwCru9cNy8EBADCHqeFm3rx56t+/v+Li4hQXF6fhw4fr/fffb/SYtWvXasiQIYqMjFT37t01f/78EFXbNHX3uuEuxQAAmMPUcNOlSxc9+eSTysrKUlZWli677DJde+212r59e4P9c3JyNH78eI0cOVLZ2dmaNWuW7rzzTmVmZoa48lNLiGXkBgAAM9nN/PAJEyb4vX788cc1b948ffbZZ+rTp0+9/vPnz1fXrl01Z84cSVLv3r2VlZWlp59+Wtdff30oSj4t38gNc24AADBFi5lz4/F4tGTJEpWXl2v48OEN9vn00081btw4v7Yrr7xSWVlZqq6ubvCYyspKlZaW+m3B5HsyOPe5AQDAFKaHm61bt6pdu3ZyOp2aMmWK3n77bZ1//vkN9i0oKFBSUpJfW1JSktxut4qKiho8Zvbs2XK5XL4tLS0t4D/DyeJ9q6UINwAAmMH0cNOzZ09t2bJFn332mW677Tbdeuut+uqrr07Z32Kx+L02DKPB9jozZ85USUmJb8vLywtc8Q2oWy11tNKt49WeoH4WAACoz9Q5N5IUERGhHj16SJKGDh2qjRs36tlnn9Vf//rXen2Tk5NVUFDg11ZYWCi73a74+PgG39/pdMrpdAa+8FOIddoVYbOqyuNV0dFKdekQHbLPBgAALWDk5ocMw1BlZcOTcYcPH66VK1f6tX3wwQcaOnSoHA5HKMo7LYvFcmLeDZemAAAIOVPDzaxZs7Ru3Trt3r1bW7du1X333ac1a9bol7/8paSaS0qTJk3y9Z8yZYr27Nmj6dOna8eOHXr55Ze1YMECzZgxw6wfoUG+eTfc6wYAgJAz9bLUgQMHdMsttyg/P18ul0v9+/fXihUrdMUVV0iS8vPzlZub6+ufkZGh5cuX6+6779YLL7yg1NRUPffccy1mGXgd7lIMAIB5TA03CxYsaHT/okWL6rWNHj1amzdvDlJFgVF3rxueDA4AQOi1uDk34YA5NwAAmIdwEwTxvnDDyA0AAKFGuAmCBN+EYkZuAAAINcJNENStlmJCMQAAoUe4CYL4GC5LAQBgFsJNEJx8WcrrNUyuBgCAtoVwEwQda0duPF5DJccaflo5AAAIDsJNEETYrYqLrLmFEHcpBgAgtAg3QZIQy6RiAADMQLgJkoTauxRzIz8AAEKLcBMkJ54vxWUpAABCiXATJNylGAAAcxBugsT38EzuUgwAQEgRboKkbkIxIzcAAIQW4SZIEmJ4MjgAAGYg3ARJPA/PBADAFISbIPGtlirjshQAAKFEuAmSuvvclFW6dbzaY3I1AAC0HYSbIImLssths0iSDnFpCgCAkCHcBInFYvEtB2dSMQAAoUO4CSLfvBsengkAQMgQboKobsUUk4oBAAgdwk0Q+e51w5wbAABChnATRHV3KT7IyA0AACFDuAmixNpwc6D0uMmVAADQdhBugijFFSVJKigh3AAAECqEmyBKdkVKkvIJNwAAhAzhJojqwk1h2XF5vYbJ1QAA0DYQboIoMdYpi0Wq9hismAIAIEQIN0HksFnVqfZeN8y7AQAgNAg3QVZ3aaqAFVMAAIQE4SbIkuNqw03JMZMrAQCgbSDcBFkKIzcAAIQU4SbIklgODgBASBFugsw3ckO4AQAgJAg3QZYcV3uXYi5LAQAQEoSbIEs+aeTGMLiRHwAAwUa4CbK61VIVVR6VHnebXA0AAOGPcBNkURE2tY92SOLp4AAAhALhJgTqRm9YMQUAQPARbkLgxLwbbuQHAECwEW5C4MRy8EqTKwEAIPyZGm5mz56tCy64QLGxsUpMTNTEiRO1c+fORo9Zs2aNLBZLve3rr78OUdVnLqnuEQyljNwAABBspoabtWvX6vbbb9dnn32mlStXyu12a9y4cSovLz/tsTt37lR+fr5vO/fcc0NQcfOkcJdiAABCxm7mh69YscLv9cKFC5WYmKhNmzZp1KhRjR6bmJio9u3bB7G6wEl21d7Ij3ADAEDQtag5NyUlJZKkjh07nrbvoEGDlJKSorFjx2r16tWn7FdZWanS0lK/LdR8TwZnKTgAAEHXYsKNYRiaPn26LrnkEvXt2/eU/VJSUvTiiy8qMzNTS5cuVc+ePTV27Fh9/PHHDfafPXu2XC6Xb0tLSwvWj3BKdauljlRU63i1J+SfDwBAW2IxWsgzAW6//Xa99957+uSTT9SlS5czOnbChAmyWCx655136u2rrKxUZeWJVUqlpaVKS0tTSUmJ4uLizrrupjAMQ30e/JcqqjxaM2OM0hNiQvK5AACEi9LSUrlcrib9/W4RIzd33HGH3nnnHa1evfqMg40kDRs2TN9++22D+5xOp+Li4vy2ULNYLNzIDwCAEDE13BiGoalTp2rp0qVatWqVMjIymvU+2dnZSklJCXB1geW7kR/LwQEACCpTV0vdfvvteuONN/R///d/io2NVUFBgSTJ5XIpKqpmhdHMmTO1b98+vfrqq5KkOXPmKD09XX369FFVVZUWL16szMxMZWZmmvZzNEUyN/IDACAkTA038+bNkySNGTPGr33hwoWaPHmyJCk/P1+5ubm+fVVVVZoxY4b27dunqKgo9enTR++9957Gjx8fqrKbxbdiikcwAAAQVKaGm6bMZV60aJHf63vuuUf33HNPkCoKHm7kBwBAaLSICcVtQd2N/A5wrxsAAIKKcBMirJYCACA0CDchUjeh+ODRSlV7vCZXAwBA+CLchEh8TIQcNosMQzpYxoopAACChXATIlarRYmxXJoCACDYCDchVLdiiknFAAAED+EmhJJZDg4AQNARbkKIG/kBABB8hJsQOvF8KSYUAwAQLISbEEqpvZEfIzcAAAQP4SaEkl1OScy5AQAgmAg3IVT3CIbC0kp5vad/rhYAADhzhJsQSox1ymKRqjxeFZUz7wYAgGAg3ISQw2ZVau3oTd6hCpOrAQAgPBFuQiw9IVqSlFNEuAEAIBgINyHWLT5GkrSnuNzkSgAACE+EmxBLj68ZudldzMgNAADBQLgJsfTakZvdRYzcAAAQDISbEEtPqA03xeUyDJaDAwAQaISbEOvaMVoWi1R23K3DFdVmlwMAQNgh3IRYpMOmlNoHaOZwaQoAgIAj3JiAFVMAAAQP4cYEJ+bdsGIKAIBAI9yYwLccnMtSAAAEHOHGBFyWAgAgeAg3JsiovSyVU8RycAAAAo1wY4KuHWsuS5Ued+sIy8EBAAgowo0JoiJsSq5dDr6bS1MAAAQU4cYkdU8HJ9wAABBYhBuTnHjGFMvBAQAIJMKNSVgxBQBAcBBuTJJRe1kqhxv5AQAQUIQbkzByAwBAcBBuTNKt9i7FRyqqdaSiyuRqAAAIH4Qbk0RH2JUU55TEM6YAAAgkwo2JuDQFAEDgEW5MVPcAzRweoAkAQMAQbkyUnlA3csNlKQAAAoVwYyLfjfy4LAUAQMAQbkxUt2JqN5elAAAIGMKNiepGbg5XVKuEp4MDABAQhBsTxTjt6hRbsxx8zyFGbwAACATCjclYMQUAQGCZGm5mz56tCy64QLGxsUpMTNTEiRO1c+fO0x63du1aDRkyRJGRkerevbvmz58fgmqDIz2eFVMAAASSqeFm7dq1uv322/XZZ59p5cqVcrvdGjdunMrLTz2KkZOTo/Hjx2vkyJHKzs7WrFmzdOeddyozMzOElQdO3XJwRm4AAAgMu5kfvmLFCr/XCxcuVGJiojZt2qRRo0Y1eMz8+fPVtWtXzZkzR5LUu3dvZWVl6emnn9b1118f7JIDrkdiO0nSzoIykysBACA8tKg5NyUlJZKkjh07nrLPp59+qnHjxvm1XXnllcrKylJ1df0VR5WVlSotLfXbWpLeyXGSpO8Kj8rt8ZpcDQAArV+LCTeGYWj69Om65JJL1Ldv31P2KygoUFJSkl9bUlKS3G63ioqK6vWfPXu2XC6Xb0tLSwt47WejS4coxUTYVOXxcmkKAIAAaDHhZurUqfryyy/15ptvnravxWLxe20YRoPtkjRz5kyVlJT4try8vMAUHCBWq0U9k2MlSTu4NAUAwFlrEeHmjjvu0DvvvKPVq1erS5cujfZNTk5WQUGBX1thYaHsdrvi4+Pr9Xc6nYqLi/PbWppeKTU1fZ3fsi6ZAQDQGpkabgzD0NSpU7V06VKtWrVKGRkZpz1m+PDhWrlypV/bBx98oKFDh8rhcASr1KDqVTty8zUjNwAAnDVTw83tt9+uxYsX64033lBsbKwKCgpUUFCgY8eO+frMnDlTkyZN8r2eMmWK9uzZo+nTp2vHjh16+eWXtWDBAs2YMcOMHyEgetVOKmbFFAAAZ8/UcDNv3jyVlJRozJgxSklJ8W1vvfWWr09+fr5yc3N9rzMyMrR8+XKtWbNGAwcO1KOPPqrnnnuuVS4Dr1M352bfkWMqOcYzpgAAOBum3uembiJwYxYtWlSvbfTo0dq8eXMQKjKHK8qhzu2jtO/IMe0sKNOFGadeCg8AABrXIiYU4+R5N0wqBgDgbBBuWoieTCoGACAgCDctBMvBAQAIDMJNC9G7duRmZ0GZvN7Tz0UCAAANI9y0EBkJMYqwWVVe5dHew8dOfwAAAGgQ4aaFsNusOjep5gnhO5hUDABAsxFuWpCeJ12aAgAAzUO4aUF6196pmOXgAAA0H+GmBemVUrscPJ+RGwAAmotw04LUPWMqp7hcx6o8JlcDAEDrRLhpQTrFOpXQLkKGIX1byOgNAADNQbhpYXx3KubSFAAAzUK4aWHqLk2xHBwAgOYh3LQwvRi5AQDgrBBuWpjeKSeWgxsGj2EAAOBMEW5amB6J7WS1SIcrqlVYVml2OQAAtDqEmxYm0mHTOZ1qHsOwbV+JydUAAND6EG5aoIFp7SVJ2blHTK0DAIDWiHDTAg3q2kGStCXviLmFAADQChFuWqC6kZsteUfk8TKpGACAM9GscJOXl6e9e/f6Xm/YsEHTpk3Tiy++GLDC2rLzktopOsKmo5VufX/wqNnlAADQqjQr3PziF7/Q6tWrJUkFBQW64oortGHDBs2aNUuPPPJIQAtsi+w2q/p1dkmStjDvBgCAM9KscLNt2zZdeOGFkqS///3v6tu3r9avX6833nhDixYtCmR9bVbdvJvsvMMmVwIAQOvSrHBTXV0tp9MpSfrwww/14x//WJLUq1cv5efnB666NowVUwAANE+zwk2fPn00f/58rVu3TitXrtSPfvQjSdL+/fsVHx8f0ALbqkFd20uSvjlQpvJKt7nFAADQijQr3PzpT3/SX//6V40ZM0Y33XSTBgwYIEl65513fJercHaS4iLVuX2UvIb05V5u5gcAQFPZm3PQmDFjVFRUpNLSUnXo0MHX/tvf/lbR0dEBK66tG5jWXvuOHFN23mENP4cRMQAAmqJZIzfHjh1TZWWlL9js2bNHc+bM0c6dO5WYmBjQAtuyuktTzLsBAKDpmhVurr32Wr366quSpCNHjuiiiy7Sn//8Z02cOFHz5s0LaIFt2ck38+MJ4QAANE2zws3mzZs1cuRISdI//vEPJSUlac+ePXr11Vf13HPPBbTAtqxvZ5fsVosOllVq35FjZpcDAECr0KxwU1FRodjYWEnSBx98oOuuu05Wq1XDhg3Tnj17AlpgWxbpsKl3SpwknjMFAEBTNSvc9OjRQ8uWLVNeXp7+9a9/ady4cZKkwsJCxcXFBbTAto55NwAAnJlmhZsHHnhAM2bMUHp6ui688EINHz5cUs0ozqBBgwJaYFt3Itxwp2IAAJqiWUvBf/rTn+qSSy5Rfn6+7x43kjR27Fj95Cc/CVhxkAam1axI27a/VFVuryLsPMgdAIDGNCvcSFJycrKSk5O1d+9eWSwWde7cmRv4BUF6fLTaRzt0pKJaO/JLNaB2BRUAAGhYs4YBvF6vHnnkEblcLnXr1k1du3ZV+/bt9eijj8rr9Qa6xjbNYrH4LQkHAACNa1a4ue+++/T888/rySefVHZ2tjZv3qwnnnhCf/nLX/T//t//C3SNbd6g2ktTm5l3AwDAaTXrstQrr7yiv/3tb76ngUvSgAED1LlzZ/3+97/X448/HrACIQ3u1l6SlLX7sAzDkMViMbcgAABasGaN3Bw6dEi9evWq196rVy8dOnTorIuCvyHdOshhs2jfkWPKPVRhdjkAALRozQo3AwYM0PPPP1+v/fnnn1f//v3Puij4i46w++bdfPp9sbnFAADQwjXrstRTTz2lq6++Wh9++KGGDx8ui8Wi9evXKy8vT8uXLw90jZA0/JwEbdx9WOu/L9aNF3Y1uxwAAFqsZo3cjB49Wt98841+8pOf6MiRIzp06JCuu+46bd++XQsXLgx0jZA0vHu8JOnTXcU8RBMAgEY0+45wqampevzxx5WZmamlS5fqscce0+HDh/XKK680+T0+/vhjTZgwQampqbJYLFq2bFmj/desWSOLxVJv+/rrr5v7Y7Qag7q2V4TdqoNllfr+4FGzywEAoMUy9Xa35eXlp5y/05idO3cqPz/ft5177rlBqrDliHTYNLRbzZJw5t0AAHBqzb5DcSBcddVVuuqqq874uMTERLVv3z7wBbVww7vHa/33xfp0V7FuGZ5udjkAALRIrfJBRYMGDVJKSorGjh2r1atXN9q3srJSpaWlfltrNaJH7byb74vl9TLvBgCAhpzRyM11113X6P4jR46cTS2nlZKSohdffFFDhgxRZWWlXnvtNY0dO1Zr1qzRqFGjGjxm9uzZevjhh4NaV6j079Je0RE2Ha6o1s4DZeqdEmd2SQAAtDhnFG5cLtdp90+aNOmsCmpMz5491bNnT9/r4cOHKy8vT08//fQpw83MmTM1ffp03+vS0lKlpaUFrcZgctisGpreUR9/c1Drvy8m3AAA0IAzCjctcZn3sGHDtHjx4lPudzqdcjqdIawouEacE6+PvzmoT78v1n9ckmF2OQAAtDitcs7NybKzs5WSkmJ2GSFTd7+bz3OK5WHeDQAA9Zi6Wuro0aP67rvvfK9zcnK0ZcsWdezYUV27dtXMmTO1b98+vfrqq5KkOXPmKD09XX369FFVVZUWL16szMxMZWZmmvUjhFyf1DjFRtpVdtyt7ftL1L9Le7NLAgCgRTE13GRlZenSSy/1va6bG3Prrbdq0aJFys/PV25urm9/VVWVZsyYoX379ikqKkp9+vTRe++9p/Hjx4e8drPYbVZdlNFRH+4o1Prviwk3AAD8gMVoY/fyLy0tlcvlUklJieLiWueE3AWf5OjRd7/S6PM66ZVfX2h2OQAABN2Z/P1u9XNu2qK6eTcbdx9StcdrcjUAALQshJtWqFdyrDpEO1RR5dEXeUfMLgcAgBaFcNMKWa0WjTgnQZK09puDJlcDAEDLQrhppS7tlShJWvV1ocmVAADQshBuWqkxPTvJYpG27y9VQclxs8sBAKDFINy0UgntnBpQuwx89U5GbwAAqEO4acUu49IUAAD1EG5asbpw8+/vilTp9phcDQAALQPhphXrkxqnxFinKqo8+nzXIbPLAQCgRSDctGIWi0WX9uTSFAAAJyPctHKX9T4RbtrYkzQAAGgQ4aaVu6RHgiJsVuUeqtD3B8vNLgcAANMRblq5GKddF3XvKElazaUpAAAIN+GAeTcAAJxAuAkDdUvCN+4+pNLj1SZXAwCAuQg3YSA9IUbdO8XI7TX0ybdFZpcDAICpCDdh4jIuTQEAIIlwEzZOfhSD2+M1uRoAAMxDuAkTF2Z0VIdohw6VV2lDDncrBgC0XYSbMGG3WXVln2RJ0ntb802uBgAA8xBuwshV/VIkSf/aXiCPl7sVAwDaJsJNGBlxTrzaRztUdLRKn+cUm10OAACmINyEEYfNqnHnJ0mS3t9aYHI1AACYg3ATZsbXXpp6fxuXpgAAbRPhJsxc3CNBriiHio5WauNuVk0BANoewk2YOfnS1HJWTQEA2iDCTRji0hQAoC0j3IShi3skKC7SroNlldq057DZ5QAAEFKEmzAUYbfqivNrbujHpSkAQFtDuAlTV/evCTfvb8uXl0tTAIA2hHATpi7ukaDYSLsOlFZqUy6XpgAAbQfhJkw57TZdUbtq6v+27DO5GgAAQodwE8auG9RFkvTPL/JV6faYXA0AAKFBuAljw8+JV4orUiXHqvXRjkKzywEAICQIN2HMZrVo4qDOkqSlm/eaXA0AAKFBuAlz1w+uuTS1ZudBFR2tNLkaAACCj3AT5nokttOAtPZyew29s2W/2eUAABB0hJs24PrBNZemMrk0BQBoAwg3bcCE/qly2Czavr9UXxeUml0OAABBRbhpAzrEROiyXomSpKWbuecNACC8EW7aiLqJxW9n75Pb4zW5GgAAgodw00aM6ZmojjEROlhWqU++KzK7HAAAgoZw00ZE2K368YBUSVIml6YAAGHM1HDz8ccfa8KECUpNTZXFYtGyZctOe8zatWs1ZMgQRUZGqnv37po/f37wCw0TdZem/rW9QEcqqkyuBgCA4DA13JSXl2vAgAF6/vnnm9Q/JydH48eP18iRI5Wdna1Zs2bpzjvvVGZmZpArDQ99O8fp/JQ4Vbm9+scmloUDAMKT3cwPv+qqq3TVVVc1uf/8+fPVtWtXzZkzR5LUu3dvZWVl6emnn9b1118fpCrDh8Vi0c3DumnW21v1+ue5+vXFGbJaLWaXBQBAQLWqOTeffvqpxo0b59d25ZVXKisrS9XV1Q0eU1lZqdLSUr+tLbt2YKraOe3KKSrX+u+LzS4HAICAa1XhpqCgQElJSX5tSUlJcrvdKipqeAXQ7Nmz5XK5fFtaWlooSm2xYpx2/aT2YZqLP9tjcjUAAAReqwo3Us2llZMZhtFge52ZM2eqpKTEt+Xl5QW9xpbu5mHdJEkrdxzQgdLjJlcDAEBgtapwk5ycrIKCAr+2wsJC2e12xcfHN3iM0+lUXFyc39bW9UyO1QXpHeTxGlqygbAHAAgvrSrcDB8+XCtXrvRr++CDDzR06FA5HA6Tqmqd6kZv3tyQyx2LAQBhxdRwc/ToUW3ZskVbtmyRVLPUe8uWLcrNzZVUc0lp0qRJvv5TpkzRnj17NH36dO3YsUMvv/yyFixYoBkzZphRfqv2o77Jio+JUEHpcX30daHZ5QAAEDCmhpusrCwNGjRIgwYNkiRNnz5dgwYN0gMPPCBJys/P9wUdScrIyNDy5cu1Zs0aDRw4UI8++qiee+45loE3g9Nu08+G1kyuZmIxACCcWIy6GbltRGlpqVwul0pKStr8/Ju8QxUa9d+rZRjSmhljlJ4QY3ZJAAA06Ez+freqOTcIrLSO0Rp9XidJ0muM3gAAwgThpo27dUS6JGnJhlyVHGv4RogAALQmhJs2bsx5nXRuYjuVV3m0ZEPu6Q8AAKCFI9y0cRaLRf85qrskaeG/d6vKzbJwAEDrRriBrh2Yqk6xThWUHte7X+43uxwAAM4K4QZy2m2aXDv35sWPd6mNLaADAIQZwg0kSTdf1E3RETZ9XVCmT75r+CGkAAC0BoQbSJJc0Q7dcEHNTf1e/HiXydUAANB8hBv4/PriDFkt0rpvi7Qjv9TscgAAaBbCDXzSOkZrfL8USdJL6xi9AQC0ToQb+Plt7bLwd7bs197DFSZXAwDAmSPcwE//Lu11cY94ub2GXlj9vdnlAABwxgg3qGfa5edJkv43K4/RGwBAq0O4QT0XpHfUJT0SakdvvjO7HAAAzgjhBg2advm5kqT/zdqrvEOM3gAAWg/CDRo0NL2jRp7L6A0AoPUh3OCU6kZv/rGJ0RsAQOtBuMEpDel2YvTmL6u+NbscAACahHCDRt19Rc3KqczN+5RbzOgNAKDlI9ygUYO7dtDo8zrJ4zU058NvzC4HAIDTItzgtP5rXM3ozdLsfdq2r8TkagAAaBzhBqfVv0t7TRyYKkl67L2vZBiGyRUBAHBqhBs0yR9+1EtOu1Wf7TqklV8dMLscAABOiXCDJuncPkr/cUmGJGn2+1+ryu01uSIAABpGuEGT3TbmHCW0i1BOUble/3yP2eUAANAgwg2aLDbS4Vsa/uxH36qkotrkigAAqI9wgzNyw9A0nZfUTkcqqvX8am7sBwBoeQg3OCN2m1WzxveWJC1av1u7Dh41uSIAAPwRbnDGxvRM1JienVTtMXTf29tYGg4AaFEIN2iWR6/tq0iHVZ/uKlbm5n1mlwMAgA/hBs2S1jFad42tmVz8+Htf6VB5lckVAQBQg3CDZvvNyAz1So7V4YpqPf7eDrPLAQBAEuEGZ8Fhs+qJ6/rJYpEyN+/V+u+KzC4JAADCDc7O4K4ddPNF3SRJ9y3bpuPVHpMrAgC0dYQbnLU//KinEmOdyikq119Wce8bAIC5CDc4a3GRDj1ybR9J0rw132vTnkMmVwQAaMsINwiIH/VN0U8GdZbXkKa9tUVlx3k0AwDAHIQbBMzD1/ZR5/ZRyjt0TA+985XZ5QAA2ijCDQImLtKhOTcOlLV29dS7X+43uyQAQBtEuEFAXZDeUbdf2kOSNGvpVu0/cszkigAAbQ3hBgF359hzNSCtvUqPuzX971vk9fLsKQBA6BBuEHAOm1Vzbhio6AibPtt1SHM+Ynk4ACB0TA83c+fOVUZGhiIjIzVkyBCtW7fulH3XrFkji8VSb/v6669DWDGaIiMhRo9N7CtJeu6jb7ViW4HJFQEA2gpTw81bb72ladOm6b777lN2drZGjhypq666Srm5uY0et3PnTuXn5/u2c889N0QV40xcN7iLfnVxuiTpv/6+Rd8cKDO3IABAm2BquHnmmWf0H//xH/rNb36j3r17a86cOUpLS9O8efMaPS4xMVHJycm+zWazhahinKlZ43trePd4lVd59NtXs1RSwf1vAADBZVq4qaqq0qZNmzRu3Di/9nHjxmn9+vWNHjto0CClpKRo7NixWr16daN9KysrVVpa6rchdBw2q57/xSB1bh+l3cUVuuutbHmYYAwACCLTwk1RUZE8Ho+SkpL82pOSklRQ0PD8jJSUFL344ovKzMzU0qVL1bNnT40dO1Yff/zxKT9n9uzZcrlcvi0tLS2gPwdOL76dU3+9ZYgiHVat2XlQT/2LOVIAgOCxm12AxWLxe20YRr22Oj179lTPnj19r4cPH668vDw9/fTTGjVqVIPHzJw5U9OnT/e9Li0tJeCYoG9nl/50fX/dtWSL/rp2l5LjIvWrizPMLgsAEIZMG7lJSEiQzWarN0pTWFhYbzSnMcOGDdO33556qbHT6VRcXJzfBnNcO7Cz/uuK8yRJD//zKy3L3mdyRQCAcGRauImIiNCQIUO0cuVKv/aVK1dqxIgRTX6f7OxspaSkBLo8BMnUy3po8oh0SdKM//1Cq3cWmlsQACDsmHpZavr06brllls0dOhQDR8+XC+++KJyc3M1ZcoUSTWXlPbt26dXX31VkjRnzhylp6erT58+qqqq0uLFi5WZmanMzEwzfwycAYvFogeuOV9HKqq0bMt+3bZ4k17/zUUa0q2j2aUBAMKEqeHmhhtuUHFxsR555BHl5+erb9++Wr58ubp16yZJys/P97vnTVVVlWbMmKF9+/YpKipKffr00Xvvvafx48eb9SOgGaxWi/77ZwN05Fi11uw8qF8t3Kg3fztMfVJdZpcGAAgDFsMw2tS63NLSUrlcLpWUlDD/xmTHqjy6ecHn2rTnsOIi7Vr4qws1pFsHs8sCALRAZ/L32/THL6DtioqwaeGvLtDQbh1UetytWxZ8rvXfFZldFgCglSPcwFRxkQ69+h8XauS5Caqo8mjyoo1a9fUBs8sCALRihBuYLjrCrr/dOlTjzk9Sldur3766Sf/8Yr/ZZQEAWinCDVoEp92mF345WBMHpsrtNXTHm9n6y0ffqo1NCQMABADhBi2Gw2bVMz8f6HuS+J9XfqOpb2brWJXH3MIAAK0K4QYtitVq0YMT+ujJ6/rJYbPovS/z9bO/rtf+I8fMLg0A0EoQbtAi3XhhV73+m2GKj4nQtn2l+vHz/9Znu4rNLgsA0AoQbtBiXZjRUf839WL1So5V0dFK3fTSZ/rvf32tao/X7NIAAC0Y4QYtWpcO0cq8bYR+PrSLDEN6YfX3+um89copKje7NABAC0W4QYsX47TrqZ8O0NxfDpYryqEv9pbo6ufWacmGXFZTAQDqIdyg1RjfL0Xv3zVSw7p3VEWVR/cu3aobX/xM3xWWmV0aAKAFIdygVUltH6XXfzNM943vrSiHTZ/nHNJVz67TUyu+Zsk4AEAS4QatkM1q0X+O6q6V00fp8t6JqvYYmrvme13xP2v1/tZ8LlUBQBvHU8HR6n2wvUAPvbNd+0uOS5IGdHHpjz/qpRE9EkyuDAAQKGfy95twg7BQXunWX9d+r799kqOK2stTI89N0D1X9lK/Li6TqwMAnC3CTSMIN+HtYFmlXlj9nV7/fI+qPTX/tEeem6Apo8/RiHPiZbFYTK4QANAchJtGEG7ahtziCv3Ph9/o/7bsk7f2X3i/zi79bnR3/ahPsuw2ppsBQGtCuGkE4aZtyTtUob+t26W3svJ0vLrmzsYprkj9fGiabrggTanto0yuEADQFISbRhBu2qZD5VV69dPdevXTPTpUXiVJslqkS3sm6qYLu2p0z05yMJoDAC0W4aYRhJu2rdLt0YptBXpzQ64+23XI194+2qGr+qboxwNSdWFGR9mszM0BgJaEcNMIwg3qfH/wqJZsyNXb2ftVdLTS154U59RVfVM0tneiLsqIV4SdER0AMBvhphGEG/yQx2vos13FemfLfr2/LV+lx92+fbFOu0b17KTLeyfqkh6d1CnWaWKlANB2EW4aQbhBYyrdHn38TZE+/OqAPvq60G9ER5J6JsXq4h4JuuTceF2Q3lGxkQ6TKgWAtoVw0wjCDZrK6zX0xd4j+mhHoVZ9Xaiv8kv99lstUq/kOF2Q3kFD0ztqaHoHpbhYfQUAwUC4aQThBs11qLxKn35frE++K9K/vytS7qGKen0SY53q36W9+ndxqX8Xl/p1dim+HZeyAOBsEW4aQbhBoBSUHFfWnkPK2n1YWXsO6av9pb4bBp6sU6xTvVPi1Ds5Vr1SYnVuYqzO6dROURG20BcNAK0U4aYRhBsES0WVW1/tL9WXe0v05d4j+nJviXKKy9XQf2EWi9S5fZR6JLbTOZ3aKT0hRhnxMUpPiFaqK0pWlqIDgB/CTSMINwil8kq3vjlQph35Zfq6oFRf55fp28IyHa6oPuUxEXarunSIUlqHaKV1rPsardT2UUptH6mEGCfhB0CbcyZ/v+0hqglok2Kcdg3q2kGDunbway8+WqnvCo/qu4NHtetguXYXlSunuFx5hypU5fZq18Fy7TpY3uB7RtisSmkfqeS4SCXFRSrZVfs1LlKJcU51audUYpxT0RH85w2gbeL/foAJ4ts5Fd/OqYu6x/u1uz1e7T9yXHmHK5R3qKL26zHlHa5Q/pHjKiw7riqPV3uKK7SnuP6E5pO1c9qV0C5CCe2civd9dSo+JkIdYiIUHxOhjrVb+2iHnHbmAAEID4QboAWx26zqGh+trvHRDe6v9nh1oPS49h85rvySYzpQelwFJZU6UHpcB0qP6+DRShWWVupYtUdHK906WunW7tOEoDrRETZ1iK4JOu2jHWofFSFXtEPtoxxy1W5xdV8jHYqLsis20qHYSDvP5QLQohBugFbEYbOqS4dodenQcPiRJMMwVF7lUWHpcRUdrVLR0UoVH63UwaNVKj5aqUPlVX7b4YoqeQ2posqjiqpj2nfk2BnXFeWwKTbSrthIu9pFOhRX973TrnZOh9o5bWoXaVeMs6YtJuKk7502xThrXkc7bMwnAnDWCDdAmLFYLDWholM7de90+v5er6Gy424drqgJOkcqqlVyrFpHKqp05Fi1jlRUq/RYtUqP17SXHnOr5Fi1yo5Xq7zKI0k6Vu3RsWqPCssqT/NppxflsCnGaVN0bQCKibApKsKmmAi7ouu+d9oV5bApOsJW21a7z1Gzv25fpMPmOybSTnAC2grCDdDGWa0WuaIdckU7lK6YMzrW7fGq7LhbZcfdKj1erbLjNZfCyk76/milW0d97W6VV7pVUVXzurzSo/JKt8qr3L57BNUFJakq4D9rpMNaE4AcNkXWhSFHTQiKdNSFIGvNV1+7VZH22n2130c6bHI6rDX77bXtJ/V32m08WR4wEeEGQLPZbVZ1qJ2gfDYMw1Cl21sTdCo9Kq+qCUB14aeiyqOKao8q6r6vqvl6rMrj23esyl0TjOraa7+vdHt9n3O82qvj1V4d1qmX4geKw2ZRpN0mp8Mmp93qCz11X521QcnpsNbur+n3wz51bU67tfa1TRF2qyJsNa8jbFZF2E+0O2s3i4VwhbaLcAPAdBaLxTfyEd8usO/t9Ro67j4RhI7XjgzVBaDK2tfHq701weiHbdU1xxyv9qrSfeL4mqBU217t0XG3R9WeE7cNq/YYqva4VVbpbqS64Imw1YScusBzcgD6YThyOmwnhaQT+yMa+N7p18f2gz4WX5vDZvHbR9hCKBFuAIQ1q9Wi6Ai7oiPsij9997Pi8Rq1Aagm+FS6/b/WfV/prgtENV8rT/p6vNqjKo9XldXeBl6f1MftrWmvfb+qk0aoJKnKU7NfZz8NKiBODkh1wcdhs/qFsJPbTg5VjpO/nnzsSV+dtd+faLf4He+wnehTt7/us5iLFX4INwAQIDZfkAr9ZxuGURNo3N56AajqpADk+95TE6jqgtOJfjXHVXuME8d4vKqqDVZVfm3+71f3utrjlfsHD1praWHrZDarxS/sOGxWOewW3/cnAtIP+5wIUY6TjnPWfm+3nQhydqt/qKt7v4iT+p3cXve5dqtFjtqgZ7daZLNaGAVrAsINAIQBi8VSOzfHplizi1HNKFa1x+sXkKpPDkGeE0GoXmCq19fw9as+xfHVHuPEsSftOxHSPLWXCusHL4/XkMdr6Hi19xQ/TcthscgXsOwnha267+1Wiy8U2X+wz+HrU3MJ0W6t2VfXp6b9xLGOeu95Ulttf4etpq/daqkXzJLiIk07T4QbAEDA2awW2aw186haGq/X8I0kuX8QnOoCUOVJwcnt9arKfWJkzO3xqtprqNp3zIkAVtdWddJ7nRzA3F6vqmvfq/rkz/d91olj3B6jXhAzDPnCYEuW0M6prPsvN+3zCTcAgDbFarUosoUGrx/yeg1Ve/1DkNsvNNWNRvmHqWqP4devLlj5ApS7JqC5Pf6hzu058Xl1oa7mvX/Qp3YEzH3Se7s9hq++qAhz71pOuAEAoIWyWi1yWm1y8tf6jPBAGAAAEFZMDzdz585VRkaGIiMjNWTIEK1bt67R/mvXrtWQIUMUGRmp7t27a/78+SGqFAAAtAamhpu33npL06ZN03333afs7GyNHDlSV111lXJzcxvsn5OTo/Hjx2vkyJHKzs7WrFmzdOeddyozMzPElQMAgJbKYhiGcfpuwXHRRRdp8ODBmjdvnq+td+/emjhxombPnl2v/x//+Ee988472rFjh69typQp+uKLL/Tpp5826TNLS0vlcrlUUlKiuLi4s/8hAABA0J3J32/TRm6qqqq0adMmjRs3zq993LhxWr9+fYPHfPrpp/X6X3nllcrKylJ1dcPPiqmsrFRpaanfBgAAwpdp4aaoqEgej0dJSUl+7UlJSSooKGjwmIKCggb7u91uFRUVNXjM7Nmz5XK5fFtaWlpgfgAAANAimT6h+Ie3kTYMo9FbSzfUv6H2OjNnzlRJSYlvy8vLO8uKAQBAS2bayvmEhATZbLZ6ozSFhYX1RmfqJCcnN9jfbrcrPr7hR+I5nU45nc7AFA0AAFo800ZuIiIiNGTIEK1cudKvfeXKlRoxYkSDxwwfPrxe/w8++EBDhw6Vw+EIWq0AAKD1MPWy1PTp0/W3v/1NL7/8snbs2KG7775bubm5mjJliqSaS0qTJk3y9Z8yZYr27Nmj6dOna8eOHXr55Ze1YMECzZgxw6wfAQAAtDCm3tD5hhtuUHFxsR555BHl5+erb9++Wr58ubp16yZJys/P97vnTUZGhpYvX667775bL7zwglJTU/Xcc8/p+uuvN+tHAAAALYyp97kxA/e5AQCg9WkV97kBAAAIBsINAAAIK23uIep1V+G4UzEAAK1H3d/tpsymaXPhpqysTJK4UzEAAK1QWVmZXC5Xo33a3IRir9er/fv3KzY2ttE7ITdHaWmp0tLSlJeXx2TlIONchw7nOnQ416HDuQ6dQJ1rwzBUVlam1NRUWa2Nz6ppcyM3VqtVXbp0CepnxMXF8R9LiHCuQ4dzHTqc69DhXIdOIM716UZs6jChGAAAhBXCDQAACCuEmwByOp168MEHeVBnCHCuQ4dzHTqc69DhXIeOGee6zU0oBgAA4Y2RGwAAEFYINwAAIKwQbgAAQFgh3AAAgLBCuAmQuXPnKiMjQ5GRkRoyZIjWrVtndkmt3uzZs3XBBRcoNjZWiYmJmjhxonbu3OnXxzAMPfTQQ0pNTVVUVJTGjBmj7du3m1Rx+Jg9e7YsFoumTZvma+NcB86+fft08803Kz4+XtHR0Ro4cKA2bdrk28+5Dgy32637779fGRkZioqKUvfu3fXII4/I6/X6+nCum+/jjz/WhAkTlJqaKovFomXLlvntb8q5rays1B133KGEhATFxMToxz/+sfbu3Xv2xRk4a0uWLDEcDofx0ksvGV999ZVx1113GTExMcaePXvMLq1Vu/LKK42FCxca27ZtM7Zs2WJcffXVRteuXY2jR4/6+jz55JNGbGyskZmZaWzdutW44YYbjJSUFKO0tNTEylu3DRs2GOnp6Ub//v2Nu+66y9fOuQ6MQ4cOGd26dTMmT55sfP7550ZOTo7x4YcfGt99952vD+c6MB577DEjPj7eePfdd42cnBzjf//3f4127doZc+bM8fXhXDff8uXLjfvuu8/IzMw0JBlvv/223/6mnNspU6YYnTt3NlauXGls3rzZuPTSS40BAwYYbrf7rGoj3ATAhRdeaEyZMsWvrVevXsa9995rUkXhqbCw0JBkrF271jAMw/B6vUZycrLx5JNP+vocP37ccLlcxvz5880qs1UrKyszzj33XGPlypXG6NGjfeGGcx04f/zjH41LLrnklPs514Fz9dVXG7/+9a/92q677jrj5ptvNgyDcx1IPww3TTm3R44cMRwOh7FkyRJfn3379hlWq9VYsWLFWdXDZamzVFVVpU2bNmncuHF+7ePGjdP69etNqio8lZSUSJI6duwoScrJyVFBQYHfuXc6nRo9ejTnvpluv/12XX311br88sv92jnXgfPOO+9o6NCh+tnPfqbExEQNGjRIL730km8/5zpwLrnkEn300Uf65ptvJElffPGFPvnkE40fP14S5zqYmnJuN23apOrqar8+qamp6tu371mf/zb34MxAKyoqksfjUVJSkl97UlKSCgoKTKoq/BiGoenTp+uSSy5R3759Jcl3fhs693v27Al5ja3dkiVLtHnzZm3cuLHePs514OzatUvz5s3T9OnTNWvWLG3YsEF33nmnnE6nJk2axLkOoD/+8Y8qKSlRr169ZLPZ5PF49Pjjj+umm26SxL/rYGrKuS0oKFBERIQ6dOhQr8/Z/v0k3ASIxWLxe20YRr02NN/UqVP15Zdf6pNPPqm3j3N/9vLy8nTXXXfpgw8+UGRk5Cn7ca7Pntfr1dChQ/XEE09IkgYNGqTt27dr3rx5mjRpkq8f5/rsvfXWW1q8eLHeeOMN9enTR1u2bNG0adOUmpqqW2+91dePcx08zTm3gTj/XJY6SwkJCbLZbPVSZmFhYb3Eiua544479M4772j16tXq0qWLrz05OVmSOPcBsGnTJhUWFmrIkCGy2+2y2+1au3atnnvuOdntdt/55FyfvZSUFJ1//vl+bb1791Zubq4k/l0H0h/+8Afde++9uvHGG9WvXz/dcsstuvvuuzV79mxJnOtgasq5TU5OVlVVlQ4fPnzKPs1FuDlLERERGjJkiFauXOnXvnLlSo0YMcKkqsKDYRiaOnWqli5dqlWrVikjI8Nvf0ZGhpKTk/3OfVVVldauXcu5P0Njx47V1q1btWXLFt82dOhQ/fKXv9SWLVvUvXt3znWAXHzxxfVuafDNN9+oW7dukvh3HUgVFRWyWv3/zNlsNt9ScM518DTl3A4ZMkQOh8OvT35+vrZt23b25/+spiPDMIwTS8EXLFhgfPXVV8a0adOMmJgYY/fu3WaX1qrddttthsvlMtasWWPk5+f7toqKCl+fJ5980nC5XMbSpUuNrVu3GjfddBPLOAPk5NVShsG5DpQNGzYYdrvdePzxx41vv/3WeP31143o6Ghj8eLFvj6c68C49dZbjc6dO/uWgi9dutRISEgw7rnnHl8fznXzlZWVGdnZ2UZ2drYhyXjmmWeM7Oxs321QmnJup0yZYnTp0sX48MMPjc2bNxuXXXYZS8FbkhdeeMHo1q2bERERYQwePNi3XBnNJ6nBbeHChb4+Xq/XePDBB43k5GTD6XQao0aNMrZu3Wpe0WHkh+GGcx04//znP42+ffsaTqfT6NWrl/Hiiy/67edcB0Zpaalx1113GV27djUiIyON7t27G/fdd59RWVnp68O5br7Vq1c3+P/oW2+91TCMpp3bY8eOGVOnTjU6duxoREVFGddcc42Rm5t71rVZDMMwzm7sBwAAoOVgzg0AAAgrhBsAABBWCDcAACCsEG4AAEBYIdwAAICwQrgBAABhhXADAADCCuEGQJuQnp6uOXPmmF0GgBAg3AAIuMmTJ2vixImSpDFjxmjatGkh++xFixapffv29do3btyo3/72tyGrA4B57GYXAABNUVVVpYiIiGYf36lTpwBWA6AlY+QGQNBMnjxZa9eu1bPPPiuLxSKLxaLdu3dLkr766iuNHz9e7dq1U1JSkm655RYVFRX5jh0zZoymTp2q6dOnKyEhQVdccYUk6ZlnnlG/fv0UExOjtLQ0/f73v9fRo0clSWvWrNGvfvUrlZSU+D7voYceklT/slRubq6uvfZatWvXTnFxcfr5z3+uAwcO+PY/9NBDGjhwoF577TWlp6fL5XLpxhtvVFlZma/PP/7xD/Xr109RUVGKj4/X5ZdfrvLy8iCdTQBNRbgBEDTPPvushg8frv/8z/9Ufn6+8vPzlZaWpvz8fI0ePVoDBw5UVlaWVqxYoQMHDujnP/+53/GvvPKK7Ha7/v3vf+uvf/2rJMlqteq5557Ttm3b9Morr2jVqlW65557JEkjRozQnDlzFBcX5/u8GTNm1KvLMAxNnDhRhw4d0tq1a7Vy5Up9//33uuGGG/z6ff/991q2bJneffddvfvuu1q7dq2efPJJSVJ+fr5uuukm/frXv9aOHTu0Zs0aXXfddeJxfYD5uCwFIGhcLpciIiIUHR2t5ORkX/u8efM0ePBgPfHEE762l19+WWlpafrmm2903nnnSZJ69Oihp556yu89T56/k5GRoUcffVS33Xab5s6dq4iICLlcLlksFr/P+6EPP/xQX375pXJycpSWliZJeu2119SnTx9t3LhRF1xwgSTJ6/Vq0aJFio2NlSTdcsst+uijj/T4448rPz9fbrdb1113nbp16yZJ6tev31mcLQCBwsgNgJDbtGmTVq9erXbt2vm2Xr16SaoZLakzdOjQeseuXr1aV1xxhTp37qzY2FhNmjRJxcXFZ3Q5aMeOHUpLS/MFG0k6//zz1b59e+3YscPXlp6e7gs2kpSSkqLCwkJJ0oABAzR27Fj169dPP/vZz/TSSy/p8OHDTT8JAIKGcAMg5LxeryZMmKAtW7b4bd9++61GjRrl6xcTE+N33J49ezR+/Hj17dtXmZmZ2rRpk1544QVJUnV1dZM/3zAMWSyW07Y7HA6//RaLRV6vV5Jks9m0cuVKvf/++zr//PP1l7/8RT179lROTk6T6wAQHIQbAEEVEREhj8fj1zZ48GBt375d6enp6tGjh9/2w0BzsqysLLndbv35z3/WsGHDdN5552n//v2n/bwfOv/885Wbm6u8vDxf21dffaWSkhL17t27yT+bxWLRxRdfrIcffljZ2dmKiIjQ22+/3eTjAQQH4QZAUKWnp+vzzz/X7t27VVRUJK/Xq9tvv12HDh3STTfdpA0bNmjXrl364IMP9Otf/7rRYHLOOefI7XbrL3/5i3bt2qXXXntN8+fPr/d5R48e1UcffaSioiJVVFTUe5/LL79c/fv31y9/+Utt3rxZGzZs0KRJkzR69OgGL4U15PPPP9cTTzyhrKws5ebmaunSpTp48OAZhSMAwUG4ARBUM2bMkM1m0/nnn69OnTopNzdXqamp+ve//y2Px6Mrr7xSffv21V133SWXyyWr9dT/Wxo4cKCeeeYZ/elPf1Lfvn31+uuva/bs2X59RowYoSlTpuiGG25Qp06d6k1IlmpGXJYtW6YOHTpo1KhRuvzyy9W9e3e99dZbTf654uLi9PHHH2v8+PE677zzdP/99+vPf/6zrrrqqqafHABBYTFYtwgAAMIIIzcAACCsEG4AAEBYIdwAAICwQrgBAABhhXADAADCCuEGAACEFcINAAAIK4QbAAAQVgg3AAAgrBBuAABAWCHcAACAsEK4AQAAYeX/AzwX51yMvT4QAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Step 4: Write the training loop\n",
    "losses = []\n",
    "niters = 100\n",
    "\n",
    "for epoch in range(niters):\n",
    "\n",
    "    # Training Step 1: Forward pass\n",
    "    predictions = model(inputs)\n",
    "\n",
    "    # Training Step 2: Calculate the loss\n",
    "    loss = criterion(predictions, outputs)\n",
    "\n",
    "    # Training Step 3: Zero the gradient and run backward pass\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "\n",
    "    # Training Step 4: Update parameters\n",
    "    optimizer.step()\n",
    "\n",
    "    losses.append(loss.item())\n",
    "    # print(f'Epoch {epoch+1}, Loss: {loss.item()}')\n",
    "\n",
    "print(f'Final Loss: {loss.item()}')\n",
    "\n",
    "plt.plot(losses)\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.xlabel(\"Iterations\")\n",
    "plt.title(\"Loss Per Iteration\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.1306],\n",
       "        [0.0364],\n",
       "        [0.0176],\n",
       "        [0.7840]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## To Dig a Little Deeper\n",
    "\n",
    "[PyTorch Quick Start Tutorial](https://pytorch.org/tutorials/beginner/basics/intro.html)\n",
    "\n",
    "[TensorFlow Playground](https://playground.tensorflow.org/#activation=relu&batchSize=30&dataset=gauss&regDataset=reg-plane&learningRate=0.01&regularizationRate=0&noise=0&networkShape=4,4&seed=0.75152&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BLrr-ZCNP3-C",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Summary\n",
    "\n",
    "So today we...\n",
    "\n",
    "* got a glimpse of the wide applications of neural networks\n",
    "* revisited loss functions\n",
    "* developed the notion of gradient descent first by intuition, then in the univariate case, then the multivariate case\n",
    "* defined artificial neurons\n",
    "* implemented a computation graph and visualization\n",
    "* implemented the chain rule as backpropagation on the computation graph\n",
    "* defined Neuron, Layer and MLP modules which completes are homegrown Neural Network Framework\n",
    "* then trained a small MLP on a tiny dataset\n",
    "* finally implemented the same in PyTorch"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}