AI Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Jupyter Notebook Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "IPython.notebook.set_autosave_interval(60000)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Autosaving every 60 seconds\n",
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%autosave 60\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tictactoe import *\n",
    "from utils import Node\n",
    "\n",
    "import torch \n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.autograd import Variable\n",
    "\n",
    "import copy \n",
    "import json\n",
    "import pickle \n",
    "from collections import deque\n",
    "from time import time \n",
    "from multiprocessing import Pool \n",
    "from tqdm.notebook import tqdm, trange\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "import random \n",
    "import torch.optim as optim\n",
    "from matplotlib import pyplot as plt\n",
    "import csv\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Todo\n",
    "1. Save both the network and game trees to csv\n",
    "2. Figure out what the distibution is for exploratory play"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variants of Tic-Tac-Toe \n",
    "\n",
    "Many of these variants are listed at https://en.wikipedia.org/wiki/Tic-tac-toe_variants.\n",
    "\n",
    "$m,n,k$ game = play on $m$ by $n$ board to try to get $k$ in a row.\n",
    "\n",
    "We have that TicTacToe extends Game, meaning it must have a checkGameOver function. We modify it so that in the initializer it takes in an additional parameter, 'gameOverChecker', a function that takes a board and logger and returns $0$ if nobody won, $1$ if Player one won, or $2$ if Player $2$ won. \n",
    "\n",
    "This allows us to easily implement any variants of the standard 3x3 game that only modify that winning conditions. For example, Misere Tic-tac-toe, or 'inverse' Tic-Tac-Toe, is the game where Player 1 wins if Player 2 gets 3 in a row (we make wrappers for each of the game versions)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we would like to support variants of the game where instead of alternating turns, there is some different rule for check whose turn it is. For this, we will add another parameter to TicTacToe constructor, turnChooser. This is passed into the Game superclass. Note that $turnChooser$ is a function that takes in the current player and gives the next player. This lets us support $random$ $turn$ $tic$-$tac$-$toe$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we would like to support larger boards. To do this, we pass in an additional parameter $dimension$. This then creates a board of size $dimension$ by $dimension$. Although this allows us to define boards of arbitrary size, we create $FourByFourTicTacToe$ and $FiveByFiveTicTacToe$. The most logical win condition for an $n$ by $n$ board is $n$ in a row. For this reason we modify the $checkGameOver$ function to take in $n$ - the number of pieces in a row required for a win. Note that other variants are possible - for example, getting a diamond in $4$ by $4$ could also be considered a win. We can then create an $n$ in a row win condition function by using partial functions with $checkGameOver$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we would like to support games in 3 dimensions. We do this by adding yet another paremeter to the initialization - $threeDims$. In order to make it easier to handle 2D vs 3D games, we will always assume the board is 3D - n by n by n, but for 2D we can just get the n by n board by calling board[0]."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We would like easily define combinations of these variants to create custom games. In order to do this, we define a TicTacToeConfig class. Now, TicTacToe simply takes a TicTacToeConfig. This config has all of the default values so we can set any combinations of the ones we want. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing the Board\n",
    "\n",
    "Looking at the board in the command line is annoying, and we would like some way of seeing what the algorithm is actually doing in a way that is easier to interpret. We will use PyGame to do this. First, we create a `display` method in TicTacToe.\n",
    "\n",
    "The correct way to do this would be to have some event that is triggered when we make a move to update the display. Due to lazyness, we will just spawn a different thread. This then renders the grid 60 times per second and colors it according to the current board state. \n",
    "\n",
    "http://programarcadegames.com/index.php?lang=en&chapter=array_backed_grids\n",
    "https://www.pygame.org/docs/tut/ChimpLineByLine.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning the Value Function with a Neural Network\n",
    "\n",
    "First, we define the neural network. We use tanh to bound the result between -1 and 1 (since this is the bound of our value function). We use standard activation functions, testing first ReLu and then Leaky ReLu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regular Feed forward network with only dense layers \n",
    "class DenseNetRELU(nn.Module):\n",
    "    def __init__(self, dimension):\n",
    "        super(DenseNetRELU, self).__init__()\n",
    "        # 9 input features (each of the positions in the board), with a bias\n",
    "        # 1 hidden layer with 9 inputs, 1 output (the value of the state)\n",
    "        numStates = dimension * dimension\n",
    "        self.first = nn.Linear(numStates, numStates, True)\n",
    "        self.hiddenOne = nn.Linear(numStates, numStates, True)\n",
    "        self.hiddenTwo = nn.Linear(numStates, 1, True)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.first(x)) \n",
    "        x = F.relu(self.hiddenOne(x)) \n",
    "        x = torch.tanh(self.hiddenTwo(x)) \n",
    "        return x\n",
    "    \n",
    "    def evaluate(self, board): \n",
    "        return self(torch.FloatTensor(board).reshape(-1))\n",
    "    \n",
    "class DenseNetLeakyRELU(nn.Module):\n",
    "    def __init__(self, dimension):\n",
    "        super(DenseNetLeakyRELU, self).__init__()\n",
    "        \n",
    "        numStates = dimension * dimension\n",
    "        self.first = nn.Linear(numStates, numStates, True)\n",
    "        self.hiddenOne = nn.Linear(numStates, numStates, True)\n",
    "        self.hiddenTwo = nn.Linear(numStates, 1, True)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.leaky_relu(self.first(x)) \n",
    "        x = F.leaky_relu(self.hiddenOne(x)) \n",
    "        x = torch.tanh(self.hiddenTwo(x)) \n",
    "        return x\n",
    "    \n",
    "    def evaluate(self, board): \n",
    "        return self(torch.FloatTensor(board).reshape(-1))\n",
    "    \n",
    "# Convolutional neural network (for 2D)\n",
    "class ConvNet(nn.Module):\n",
    "    def __init__(self, dimension): \n",
    "        super(ConvNet, self).__init__()\n",
    "        \n",
    "        # input is a single channel, we perform 5 convolutions on each entry \n",
    "        # with kernel size of 2 \n",
    "        if dimension == 3:\n",
    "            # 1 in_channel, \n",
    "            # 5 out_channels\n",
    "            self.first = nn.Conv2d(1, 5, kernel_size=2, padding=0) # This gives us 5 2x2 squares\n",
    "            self.flattenLayer = nn.Flatten()\n",
    "            self.hidden = nn.Linear(20, 1, True)\n",
    "        else: \n",
    "            # TODO \n",
    "            self.first = nn.Conv2d(1, 5, 3, padding=1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.first(x))\n",
    "        x = self.flattenLayer(x)\n",
    "        x = torch.tanh(self.hidden(x)) # Make sure output is between -1 and 1 \n",
    "        return x \n",
    "    \n",
    "    def evaluate(self, board):\n",
    "        return self(torch.FloatTensor(board).reshape(1, 1, 3, 3))\n",
    "    \n",
    "class ConvNetLeaky(nn.Module):\n",
    "    def __init__(self, dimension):\n",
    "        super(ConvNetLeaky, self).__init__()\n",
    "        \n",
    "        if dimension == 3:\n",
    "            self.first = nn.Conv2d(1, 5, 2, padding=0) # This gives us a 2x2\n",
    "            self.flattenLayer = nn.Flatten()\n",
    "            self.hidden = nn.Linear(20, 1, True)\n",
    "        else:\n",
    "            # TODO\n",
    "            self.first = nn.Conv2d(1, 5, 3, padding=1)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.leaky_relu(self.first(x))\n",
    "        x = self.flattenLayer(x)\n",
    "        x = torch.tanh(self.hidden(x)) # Make sure output is between -1 and 1\n",
    "        return x \n",
    "        \n",
    "    def evaluate(self, board):\n",
    "        return self(torch.FloatTensor(board).reshape(1, 1, 3, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we initialize all of the neural networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "nets = dict()\n",
    "for dim in range(3, 5): \n",
    "    netOne   = DenseNetRELU(dim)\n",
    "    netTwo   = DenseNetLeakyRELU(dim)\n",
    "    netThree = ConvNet(dim)\n",
    "    netFour  = ConvNetLeaky(dim)\n",
    "    \n",
    "    nets[dim] = [netOne, netTwo, netThree, netFour]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We take the most successful neural network and turn it into the network we use for MCTS by adding the policy head. Instead of having a single value output, we output the entire probability distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MCTSNet(nn.Module):\n",
    "    def __init__(self, dimension):\n",
    "        super(MCTSNet, self).__init__()\n",
    "        \n",
    "        numStates = dimension * dimension\n",
    "        self.first = nn.Linear(numStates, numStates, True)\n",
    "        self.hiddenOne = nn.Linear(numStates, numStates, True)\n",
    "        self.valueHead = nn.Linear(numStates, 1, True)          # Give us a value for the board\n",
    "        self.policyHead = nn.Linear(numStates, numStates, True) # Give us a probability for each action\n",
    "        \n",
    "        # Maybe do this in an outside loop?\n",
    "        LEARNING_RATE = 0.05\n",
    "        self.criterionVal = nn.MSELoss()\n",
    "        self.criterionPol = nn.MSELoss()\n",
    "        self.optimizer = optim.SGD(self.parameters(), lr=LEARNING_RATE)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.leaky_relu(self.first(x)) \n",
    "        x = F.leaky_relu(self.hiddenOne(x)) \n",
    "        value = torch.tanh(self.valueHead(x)) \n",
    "        policy = torch.sigmoid(self.policyHead(x)) # Probabilities must be between 0 and 1\n",
    "#         policy /= policy.sum()                # Normalize so that they add up to 1 \n",
    "#         return torch.cat((value, policy))\n",
    "        return value, policy\n",
    "    \n",
    "    def predict(self, board): \n",
    "        return self(torch.FloatTensor(board).reshape(-1))\n",
    "    \n",
    "    def update(self, state, value, probabilities):\n",
    "        # https://discuss.pytorch.org/t/a-model-with-multiple-outputs/10440\n",
    "        self.optimizer.zero_grad()\n",
    "        \n",
    "        predictedVal, predictedPol    = self.predict(state)\n",
    "        actualVal,    actualPol       = torch.FloatTensor([value]), torch.FloatTensor(probabilities)\n",
    "        lossVal = self.criterionVal(actualVal, predictedVal)\n",
    "        lossPol = self.criterionPol(actualPol, predictedPol) \n",
    "        loss = lossVal + lossPol\n",
    "        \n",
    "        loss.backward()\n",
    "        self.optimizer.step()\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = MCTSNet(3)\n",
    "n.update(torch.FloatTensor([0]*9), 0, [0, 0, 0, 0, 0, 0, 0, 0, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then make a generic function which will take some starting node and a game constructor (which must create an object that extends the Game abstractclass) and fills out the game tree. It does this more efficiently by using a thread pool. Note that this also returns a list of all the nodes in the tree, which allows us to choose a random node much more easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def boardToTuple(board):\n",
    "    return tuple([tuple([tuple(j) for j in l]) for l in board])\n",
    "\n",
    "def boardToList(board):\n",
    "    return [list([list(j) for j in l]) for l in board]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]\n"
     ]
    }
   ],
   "source": [
    "n = Node(boardToTuple([[[0,0,0],[0,0,0],[0,0,0]]]))\n",
    "a = boardToList(n.board)\n",
    "print(a)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def findAllGraphSearch(game, startingNode, num_nodes): \n",
    "    '''\n",
    "    findAllGraphSearch performs BFS from the startingNode but will not expand the same node twice. \n",
    "    '''\n",
    "    count = 0 \n",
    "    with tqdm(total = num_nodes) as pbar: \n",
    "        visited = dict()\n",
    "        \n",
    "        # Initialize a queue with the starting node. \n",
    "        unvisited = deque()\n",
    "        unvisited.append(startingNode) \n",
    "        visited[startingNode.board] = startingNode\n",
    "        \n",
    "        # Continue until there are no more unvisited nodes. \n",
    "        while len(unvisited) > 0:\n",
    "            count += 1\n",
    "            pbar.update() \n",
    "            \n",
    "            currNode = unvisited.popleft()\n",
    "            \n",
    "            # Check if the current game is over. If it is, skip this node\n",
    "            # This does not check if the game ends in a draw.\n",
    "            game.board = currNode.board\n",
    "            if game.checkGameOver() != 0: \n",
    "                continue\n",
    "                \n",
    "            # Find all of the children of the current node we are looking at\n",
    "            pieceToPlay = 1 if currNode.currPlayer == 0 else 2\n",
    "            allActions = game.getAllActions()\n",
    "            currNode.children = [0] * len(allActions)\n",
    "            for actionIndex, action in enumerate(allActions):\n",
    "                # Create the child node\n",
    "                child = Node(boardToList(currNode.board))\n",
    "                child.currPlayer = game.turnChooser(currNode.currPlayer)\n",
    "                child.board[action[0]][action[1]][action[2]] = pieceToPlay\n",
    "                \n",
    "                # Convert to hashable\n",
    "                child.board = boardToTuple(child.board)\n",
    "                \n",
    "                # Check if we have already queued the same node to be visited\n",
    "                if child.board not in visited: \n",
    "                    visited[child.board] = child     # Save the fact that we will look at this node later.  \n",
    "                    unvisited.append(child)          # Look at this child node in some later iteration\n",
    "                \n",
    "                # Add this child node as a child of the current node\n",
    "                currNode.children[actionIndex] = visited[child.board] \n",
    "        \n",
    "        pbar.update(num_nodes - count)\n",
    "        return visited"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we generate the game tree for all game variants so we have data to train the neural network. Note that for many games the game tree is small enough to store in memory. There are 9 places to place the first piece, then 8 to place the second, etc.., so there are $9! = 362880$ states, many of which are not reachable because someone would win. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean up the data (if we allocated too much space)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cleanUpData(vertices):\n",
    "    with tqdm(total = len(vertices), desc=\"Removing Unused Allocated Space\") as pbar: \n",
    "        for v in range(len(vertices)):\n",
    "            pbar.update() \n",
    "            if vertices[v] == 0:\n",
    "                pbar.update(len(vertices) - v - 1) \n",
    "                vertices = vertices[0:v]\n",
    "                break \n",
    "\n",
    "    for v in tqdm(vertices, desc=\"Converting to Tensors\"): \n",
    "        v.board_torch = torch.FloatTensor(v.board) \n",
    "        \n",
    "    return vertices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split into Training Set, Validation Set, and Test Set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def splitDataFromGraph(vertices): \n",
    "    keys = list(vertices.keys())\n",
    "    \n",
    "    random.shuffle(keys)\n",
    "\n",
    "    size = len(keys)\n",
    "    training_set_size = int(size*0.8)\n",
    "    training_set = keys[0:training_set_size]\n",
    "\n",
    "    test_set_end_index = int(0.9 * size)\n",
    "    test_set = keys[training_set_size:test_set_end_index]\n",
    "\n",
    "    validation_set = keys[test_set_end_index:]\n",
    "\n",
    "    return test_set, training_set, validation_set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we implement minimax so that we can evaluate a state. We define player 2 winning as -1, and player 1 winning as 1. If currPlayer is 0, it's player 1 to move."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "memo = dict()\n",
    "def minimax(game, vertex, maxDepth = None, neuralNet = None): \n",
    "    \n",
    "    # Check if we already have the value for this vertex. \n",
    "    if vertex in memo and maxDepth is None: \n",
    "        return memo[vertex] \n",
    "    \n",
    "    game.board = vertex.board \n",
    "    res = game.getAllActions()\n",
    "    res2 = game.checkGameOver()\n",
    "    if res2 == 1:\n",
    "        memo[vertex] = 1\n",
    "        return 1 \n",
    "    elif res2 == 2: \n",
    "        memo[vertex] = -1 \n",
    "        return -1\n",
    "    elif len(res) == 0: \n",
    "        memo[vertex] = 0 \n",
    "        return 0\n",
    "    \n",
    "    # If we are not a terminal state but the maxDepth is 0, just return 0, or use the value heuristic. \n",
    "    if maxDepth == 0: \n",
    "        if neuralNet is None: \n",
    "            return 0\n",
    "    \n",
    "        return net.evaluate(game.board)\n",
    "    if maxDepth is not None: \n",
    "        maxDepth -= 1 \n",
    "    \n",
    "    children = vertex.children\n",
    "    if vertex.currPlayer == 0: \n",
    "        val = -float('inf')\n",
    "    else: \n",
    "        val = float('inf')\n",
    "    \n",
    "    \n",
    "    for child in children: \n",
    "        child_val = minimax(game, child)\n",
    "        if vertex.currPlayer == 0: \n",
    "            val = max(val, child_val)\n",
    "        else: \n",
    "            val = min(val, child_val)\n",
    "\n",
    "    # Once we have computed the value for this vertex, save it. \n",
    "    memo[vertex] = val\n",
    "    return val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we can train it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training on the GPU\n",
    "\n",
    "A large part of this information comes from https://medium.com/dsnet/training-deep-neural-networks-on-a-gpu-with-pytorch-11079d89805."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "def get_default_device():\n",
    "    if torch.cuda.is_available():\n",
    "        return torch.device('cuda')\n",
    "    else:\n",
    "        return torch.device('cpu')\n",
    "    \n",
    "device = get_default_device()\n",
    "device = torch.device('cpu')\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getExpectedResults(data, game, batchSize, flatten): \n",
    "    expected = [0] * batchSize\n",
    "    nodes = random.sample(data, batchSize) \n",
    "    for i in range(batchSize): \n",
    "        if nodes[i] not in memo: \n",
    "            minimax(game, nodes[i])\n",
    "        expected[i] = torch.FloatTensor([memo[nodes[i]]])\n",
    "        \n",
    "        # Put the board in a format the neural net can use \n",
    "        if flatten:\n",
    "            nodes[i] = nodes[i].board_torch.reshape(-1)\n",
    "        else:\n",
    "            nodes[i] = nodes[i].board_torch.reshape(1, 3, 3)\n",
    "            \n",
    "    return nodes, expected \n",
    "        \n",
    "def train(data, game, net, trainCount, batchCount, num_epochs, start_lr, lrStepFactor, flatten=True):\n",
    "    # Move to GPU if necessary \n",
    "    net.to(device)\n",
    "    \n",
    "    lr_step = (start_lr / num_epochs) / lrStepFactor\n",
    "    \n",
    "    epoch = 0 \n",
    "    learning_rates = np.arange(start_lr, start_lr - num_epochs * lr_step, -lr_step)\n",
    "    losses = []\n",
    "    criterion = nn.MSELoss() # Using mean square error \n",
    "    batchSize = trainCount // batchCount\n",
    "    \n",
    "    for lr_var in tqdm(learning_rates, desc=f\"Neural Net Training\"):\n",
    "        epoch += 1\n",
    "        optimizer = optim.SGD(net.parameters(), lr=lr_var) #  create the optimizer  \n",
    "        L = [0] * batchCount \n",
    "        \n",
    "        allNodes = []\n",
    "        allExpected = []\n",
    "        for batch in range(batchCount): \n",
    "            nodes, expected = getExpectedResults(data, game, batchSize, flatten)\n",
    "            allNodes.append(torch.stack(nodes).to(device))\n",
    "            allExpected.append(expected)\n",
    "        \n",
    "        for batch in range(batchCount):\n",
    "            # Get training data for one batch \n",
    "            outputs  = [0] * batchSize\n",
    "            nodes, expected = allNodes[batch], allExpected[batch] \n",
    "            \n",
    "            # Move training data to GPU if necessary \n",
    "            outputs = net(nodes) \n",
    "\n",
    "            # Update the neural network\n",
    "            optimizer.zero_grad()\n",
    "            expected = torch.stack(expected)\n",
    "            expected = expected.to(device)\n",
    "            loss = criterion(outputs, expected) \n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            L[batch] = loss.item()\n",
    "                \n",
    "        losses.append(L)\n",
    "        \n",
    "    return losses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiments (Results)\n",
    "\n",
    "First, we begin with the simplests case - 3x3. We determined empirically that training the first neural network with the parameters listed below would be sufficient to reach minimal loss. Then, we use these parameters to train the 4 different network topologies. After testing it seems that ```findAll``` is _slightly_ faster than ```findAllGraphSearch```, however, there are 2 benefits to using ```findAllGraphSearch```. \n",
    "\n",
    "1. It outputs a hashmap mapping board states to nodes. Although this does not currently give us a speed increase, we expect that this would enable speed increases in the training loop (e.g. by allowing us to use data loaders). \n",
    "\n",
    "\n",
    "2. It looks at significantly fewer nodes, which makes it faster overall. Further, since there is less duplicate data, it uses less memory to store all of the nodes. While the ```findAll``` looks at 986410 nodes, ```findAllGraphSearch``` looks at:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "memo = dict()\n",
    "def getGameTree(game, dim): \n",
    "    root = Node([[[0] * dim for _ in range(dim)]])\n",
    "    root.board = boardToTuple(root.board)\n",
    "    NUM_NODES = 3**(dim**2)\n",
    "    if dim == 4: \n",
    "        NUM_NODES = 9810255\n",
    "    \n",
    "    vertices = findAllGraphSearch(game, root, NUM_NODES) \n",
    "    test_set, training_set, validation_set = splitDataFromGraph(vertices)\n",
    "    return test_set, training_set, validation_set, vertices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d03d3743277c403592e797e4e1f65dce",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=19683), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "80d9be271f1d4a8189eaa5bc7e26d4ac",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=4496, style=ProgressStyle(descrip…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "15325e9c03f24ca4852e3a556d670ccc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=562, style=ProgressStyle(descript…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fda589356ed642f6bb2abc70473919ef",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=562, style=ProgressStyle(descript…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "gameOverCheck = partial(checkGameOver, n=3)\n",
    "config = TicTacToeConfig(turnChooser=defaultTurnChooser, gameOverChecker=gameOverCheck, dimension=dim, threeDims=False)\n",
    "game = TicTacToe(None, None, None, config)\n",
    "    \n",
    "test_set_one, training_set_one, validation_set_one, vertices = getGameTree(game, 3)\n",
    "\n",
    "# For now, don't take advantage of the hashtable (this is only needed for graph search)\n",
    "training_set_one = [vertices[i] for i in training_set_one]\n",
    "for v in tqdm(training_set_one, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board) \n",
    "\n",
    "validation_set_one = [vertices[i] for i in validation_set_one]\n",
    "for v in tqdm(validation_set_one, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board)\n",
    "    \n",
    "test_set_one = [vertices[i] for i in test_set_one]\n",
    "for v in tqdm(test_set_one, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4496 562 562\n",
      "5620\n"
     ]
    }
   ],
   "source": [
    "print(f\"{len(training_set_one)} {len(validation_set_one)} {len(test_set_one)}\")\n",
    "print(len(training_set_one) + len(validation_set_one) + len(test_set_one))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "387ee4a024c8427b8e5ed7b4932418bb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Neural Net Training', max=2, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ed278e990cb044c6b570d77cd2714657",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Neural Net Training', max=2, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6fd59b93af3f4aabb5e17b277cb31e35",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Neural Net Training', max=2, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2be5837ac46b490597828c088551010d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Neural Net Training', max=2, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "trainCount = len(training_set_one)\n",
    "batchSize = 32\n",
    "batchCount = trainCount // batchSize \n",
    "startLR = 0.04\n",
    "numEpochs = 2\n",
    "lrStepFactor = 1.5 \n",
    "\n",
    "results = dict() \n",
    "for netIndex in range(4):\n",
    "    results[(3, netIndex)] = train(training_set_one, game, nets[3][netIndex],\n",
    "                                   trainCount, batchCount, numEpochs,\n",
    "                                   startLR, lrStepFactor, (netIndex == 0 or netIndex == 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we try training it for a 4x4 board (using the exact same neural network and same parameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a09e4a72469545018ee15f005c7936a0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=9810255), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "gameOverCheck = partial(checkGameOver, n=4)\n",
    "config = TicTacToeConfig(turnChooser=defaultTurnChooser, gameOverChecker=gameOverCheck, dimension=4, threeDims=False)\n",
    "game = TicTacToe(None, None, None, config)\n",
    "test_set, training_set, validation_set, vertices = getGameTree(game, 4)\n",
    "# Approximately 8500 iterations per second "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7848204 981026 981025\n",
      "9810255\n"
     ]
    }
   ],
   "source": [
    "print(f\"{len(training_set)} {len(validation_set)} {len(test_set)}\")\n",
    "print(len(training_set) + len(validation_set) + len(test_set))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert to the correct format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "756fb10347404b3ba83d4214832852b0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=7848204, style=ProgressStyle(desc…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "02f1e8cb7e0e49cb9d77450ce48ef7b1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=981026, style=ProgressStyle(descr…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d036fa1de02040aa8f4829651e1a280a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Converting to Tensors', max=981025, style=ProgressStyle(descr…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "training_set = [vertices[i] for i in training_set]\n",
    "for v in tqdm(training_set, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board) \n",
    "\n",
    "validation_set = [vertices[i] for i in validation_set]\n",
    "for v in tqdm(validation_set, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board)\n",
    "    \n",
    "test_set = [vertices[i] for i in test_set]\n",
    "for v in tqdm(test_set, desc=\"Converting to Tensors\"): \n",
    "    v.board_torch = torch.FloatTensor(v.board)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save the data out to an external file, using Pickle as this lets us save external objects. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = open(\"training_set_4by4.pkl\",\"wb\")\n",
    "pickle.dump(training_set, f)\n",
    "f.close()\n",
    "\n",
    "print(\"Done with training set. \")\n",
    "\n",
    "f = open(\"validation_set_4by4.pkl\", \"wb\")\n",
    "pickle.dump(validation_set, f)\n",
    "f.close()\n",
    "\n",
    "print(\"Done with validation set. \")\n",
    "\n",
    "f = open(\"test_set_4by4.pkl\", \"wb\")\n",
    "pickle.dump(test_set, f)\n",
    "f.close()\n",
    "\n",
    "print(\"Done with test set. \")\n",
    "\n",
    "f = open(\"vertices_4by4.pkl\", \"wb\") \n",
    "pickle.dump(vertices, f) \n",
    "f.close()\n",
    "\n",
    "print(\"Done with vertices. \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the data already exists in the external file, we can load directly from the file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filehandler = open(\"training_set_4by4.pkl\", 'r') \n",
    "training_set = pickle.load(filehandler)\n",
    "\n",
    "filehandler = open(\"validation_set_4by4.pkl\", 'r')\n",
    "validation_set = pickle.load(filehandler)\n",
    "\n",
    "filehandler = open(\"test_set_4by4.pkl\", 'r')\n",
    "test_set = pickle.load(filehandler)\n",
    "\n",
    "filehandler = open(\"vertices_4by4.pkl\", 'r')\n",
    "vertices = pickle.load(filehandler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model for the 4x4 version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "95137bd600ec4bc19a113f0b4f2b5627",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Neural Net Training', max=5, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "ename": "AttributeError",
     "evalue": "'tuple' object has no attribute 'board'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-91-0d3e924ab3ad>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mnetIndex\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m4\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m     results[(4, netIndex)] = train(training_set, game, nets[4][netIndex], trainCount, batchCount, numEpochs, startLR,\n\u001b[1;32m---> 10\u001b[1;33m                                    lrStepFactor, (netIndex == 0 or netIndex == 1))\n\u001b[0m",
      "\u001b[1;32m<ipython-input-45-21efe63b4d36>\u001b[0m in \u001b[0;36mtrain\u001b[1;34m(data, game, net, trainCount, batchCount, num_epochs, start_lr, lrStepFactor, flatten)\u001b[0m\n\u001b[0;32m     35\u001b[0m         \u001b[0mallExpected\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     36\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0mbatch\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatchCount\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 37\u001b[1;33m             \u001b[0mnodes\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mexpected\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgetExpectedResults\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgame\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbatchSize\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mflatten\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     38\u001b[0m             \u001b[0mallNodes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstack\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnodes\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     39\u001b[0m             \u001b[0mallExpected\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mexpected\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-45-21efe63b4d36>\u001b[0m in \u001b[0;36mgetExpectedResults\u001b[1;34m(data, game, batchSize, flatten)\u001b[0m\n\u001b[0;32m      4\u001b[0m     \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatchSize\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mnodes\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mmemo\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 6\u001b[1;33m             \u001b[0mminimax\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mgame\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnodes\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      7\u001b[0m         \u001b[0mexpected\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mFloatTensor\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mmemo\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mnodes\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-43-2e9e5b968033>\u001b[0m in \u001b[0;36mminimax\u001b[1;34m(game, vertex, maxDepth)\u001b[0m\n\u001b[0;32m      6\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mmemo\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mvertex\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m     \u001b[0mgame\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mboard\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvertex\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mboard\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      9\u001b[0m     \u001b[0mres\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgame\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgetAllActions\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     10\u001b[0m     \u001b[0mres2\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgame\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcheckGameOver\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mAttributeError\u001b[0m: 'tuple' object has no attribute 'board'"
     ]
    }
   ],
   "source": [
    "trainCount = 1000\n",
    "batchSize = 32\n",
    "batchCount = trainCount // batchSize \n",
    "startLR = 0.04\n",
    "numEpochs = 5\n",
    "lrStepFactor = 1.5 \n",
    "\n",
    "for netIndex in range(4):\n",
    "    results[(4, netIndex)] = train(training_set, game, nets[4][netIndex], trainCount, batchCount, numEpochs, startLR,\n",
    "                                   lrStepFactor, (netIndex == 0 or netIndex == 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we try training it for a 5x5 board (using the exact same neural networks and same parameters)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gameOverCheck = partial(checkGameOver, n=5)\n",
    "config = TicTacToeConfig(turnChooser=defaultTurnChooser, gameOverChecker=gameOverCheck, dimension=7, threeDims=False)\n",
    "game = TicTacToe(None, None, None, config)\n",
    "vertices = getGameTree(game, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, this tree is way too large to even store in memory and create. Finding the minimax values for nodes in the tree would also be extremely slow, even if we had the entire tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing the Results\n",
    "\n",
    "First, we plot the loss as the network is being trained: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def do_plot(L, chunkSize, showEpochs = True): \n",
    "    L2 = []\n",
    "    epoch_index = 0 \n",
    "    for epoch in L:\n",
    "        if showEpochs: \n",
    "            plt.axvline(x=len(L2), color=\"r\", linestyle='dashed')\n",
    "        for i in range(0, len(epoch), chunkSize):\n",
    "            if len(epoch) - i < chunkSize:\n",
    "                amount = len(epoch) - i\n",
    "            else: \n",
    "                amount = playGames(agentsPlayerOne, True, \"value-minimax\")\n",
    " \n",
    "            L2.append(sum(epoch[i:i+amount])/amount)\n",
    "        \n",
    "    plt.plot(L2)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hUddrG8e+ThNCrBEVAgwJiBAUZkRp0JTRXYkHFgtiwIgLuWra66u66uNIURSyIXcQWsNDUhC4B6UUCgiBIojQRKZHn/SMDbxYiGSDJZJL7c125Muec35nz/Chzzzkz84y5OyIiIrlFhbsAEREpfhQOIiJyGIWDiIgcRuEgIiKHUTiIiMhhYsJdQEGoWbOmx8fHh7sMEZGIMm/evB/cPS6vbSUiHOLj40lPTw93GSIiEcXM1v3WNl1WEhGRwygcRETkMAoHERE5jMJBREQOE1I4mFkXM1tpZhlm9uARxvUwMzezQHA51sxGm9liM1toZhfksU+KmS3JtVzDzCab2arg7+rHMC8RETkO+YaDmUUDI4CuQAJwjZkl5DGuMtAPmJNrdR8Ad28KJAFPmllUrn0uB3YeclcPAlPdvSEwNbgsIiJFKJQzh5ZAhruvcfe9wFtAch7jHgUGAbtzrUsg5wEed88EtgEHzioqAQOBxw65n2RgTPD2GODSkGYiIiIFJpRwqAOsz7W8IbjuIDNrDtRz9wmH7LsQSDazGDOrD7QA6gW3PQo8Cew6ZJ8T3X0TQPB3rbyKMrPbzCzdzNKzsrJCmMbhVmft5MlJK9m979dj2l9EpKQKJRwsj3UHvwQieJloCHBfHuNeIidM0oGhwEwg28yaAQ3c/f2jrvhAAe6j3D3g7oG4uDw/4Jevycs289RnGVw8fBrz1m051lJEREqcUMJhA///bB+gLrAx13JloAnwhZmtBVoBKWYWcPdsdx/g7s3cPRmoBqwCWgMtguOnA43M7Ivg/W02s9oAwd+Zxzq5/NzR4XTG3NyS3fv202PkLB5OWcrPe7IL63AiIhEjlHCYCzQ0s/pmFgv0BFIObHT37e5e093j3T0emA10d/d0M6tgZhUBzCwJyHb3Ze7+rLufHBzfDvja3S8I3mUK0Dt4uzfw4fFP87d1aBTHxAGJ3NDqVMbMWkunIWmkfX1sl6lEREqKfMPB3bOBvsBEYDkw1t2XmtkjZtY9n91rAfPNbDnwANArhJoeB5LMbBU573B6PIR9jkulsjH8I7kJY29vTdkyUdzw0pf84Z2FbNu1t7APLSJSLFlJ+A7pQCDgBdV4b/e+Xxk+dRXPpa2heoVYHk0+i65NaxfIfYuIFCdmNs/dA3lt0yekD1GuTDT3d2nMh3e3pVblstz5+nzufG0emT/tzn9nEZESQuHwG5rUqcqHfdtyf5czmLoik6TBabyTvp6ScKYlIpIfhcMRlImO4q4LGvDJve1pdGIl/jhuETe89CXrtxz60QwRkZJF4RCC0+Mq8fZtrXkk+Szmr9tK56FpvDzjG/bv11mEiJRMCocQRUUZN7SOZ+KARALxNXh4/DKufG4WGZk/hbs0EZECp3A4SnWrV2DMTefx5JXnkJG5k27DpjPi8wz2/bo/3KWJiBQYhcMxMDOuaFGXKQM70DGhFk9MXEny0zNY8t32cJcmIlIgFA7HIa5yWZ65rgUjr29B1s49JI+YwX8+XaFGfiIS8RQOBaBLk5OYMqADV5xbh2e/WE23YdP48hs18hORyKVwKCBVK5RhUI9zeO2W89n7636uem4Wf/1gCTvVyE9EIpDCoYC1a1iTif0TualtPK/NWUenwal8vrLQGsuKiBQKhUMhqFg2hr9fchbj7mhDhbIx3DR6LgPfXsDWn9XIT0Qig8KhELU4tTof9WvHPb9rQMrCjSQNSeWjRZvUgkNEij2FQyErGxPNfZ3OIKVvO2pXLc/db8zn9lfnkblDjfxEpPhSOBSRhJOr8P5dbXioa2NSv87iosGpjJ2rRn4iUjwpHIpQTHQUt3c4nU/ubc+Ztatw/7uLuP7FOXz7oxr5iUjxonAIg9PiKvFWn1Y8dmkTFq7fTuehabw4/Rt+VSM/ESkmFA5hEhVlXN/qVCYNSOT802rw6IRl9Bg5k1Wb1chPRMJP4RBmJ1crz+gbz2Po1c1Y+8PPXDx8OsOnrmJvthr5iUj4KByKATPj0uZ1mDywA52bnMTgyV/T/enpLNqwLdyliUgppXAoRmpWKstT1zTn+RsCbN21l0tHzODfHy/nl71q5CciRUvhUAwlJZzIpAEduPq8ejyXtoauw9KYvebHcJclIqWIwqGYqlq+DP++/GzeuPV89jv0HDWbP7+/mJ927wt3aSJSCigcirk2DWryaf/23NquPm9++S2dhqTx2YrN4S5LREq4kMLBzLqY2UozyzCzB48wroeZuZkFgsuxZjbazBab2UIzuyDX2E+D65aa2Ugziw6uf9jMvjOzBcGfbsc5x4hXITaGv/w+gXfvbEPlcjHc/HI6/d/6ii1q5CcihSTfcAg+aI8AugIJwDVmlpDHuMpAP2BOrtV9ANy9KZAEPGlmB455lbufAzQB4oArc+03xN2bBX8+PvpplUzNT6nOhHvac+9FDflo8SY6Dk4lZeFGteAQkQIXyplDSyDD3de4+17gLSA5j3GPAoOA3B3lEoCpAO6eCWwDAsHlHcExMUAsoEe4EMTGRDEgqRHj72lHverl6ffmV/R5ZR7fb1cjPxEpOKGEQx1gfa7lDcF1B5lZc6Ceu084ZN+FQLKZxZhZfaAFUC/XfhOBTOAnYFyu/fqa2SIze8nMqudVlJndZmbpZpaelZUVwjRKlsYnVeG9u9ry525nMj0ji6TBqbz55bc6ixCRAhFKOFge6w4+AgUvEw0B7stj3EvkhEk6MBSYCRz83kx37wzUBsoCvwuufhY4HWgGbAKezKsodx/l7gF3D8TFxYUwjZInOsrok3gan96byFl1qvDQe4u59vk5rPvx53CXJiIRLpRw2ECuZ/tAXWBjruXK5Lxu8IWZrQVaASlmFnD3bHcfEHztIBmoBqzKfefuvhtIIXipyt03u/uv7r4feJ6cy1pyBPE1K/LGra349+VNWfJdTiO/F6atUSM/ETlmoYTDXKChmdU3s1igJzkP5gC4+3Z3r+nu8e4eD8wGurt7uplVMLOKAGaWBGS7+zIzq2RmtYPrY4BuwIrgcu1cx74MWHL80yz5oqKMa1qewuSBHWjXoCaPfbScy5+dycrv1chPRI5evuHg7tlAX2AisBwY6+5LzewRM+uez+61gPlmthx4AOgVXF+RnLOLReS8LpEJjAxuGxR86+si4EJgwNFOqjQ7qWo5nr8hwPBrmrN+yy5+/9Q0hkz+Wo38ROSoWEl4ATMQCHh6enq4yyh2tvy8l3+MX8qHCzbS6MRKDOpxDs3qVQt3WSJSTJjZPHcP5LVNn5AuwWpUjGVYz+a82DvAjl+yufyZGTw2YZka+YlIvhQOpcBFZ57IpIGJ9Gx5Ci9M/4bOQ9OYufqHcJclIsWYwqGUqFKuDP+6rClv9mlFlMG1z8/hofcWsUON/EQkDwqHUqb16Sfwyb2J3J54Gm/PXU/S4FSmLFMjPxH5XwqHUqh8bDQPdTuTD+5uS/UKsdz6Sjr3vPkVP+zcE+7SRKSYUDiUYmfXrUZK33YMTGrEp0s2kTQ4lQ+++k4tOERE4VDaxcZE0e+ihnzUrz2nnlCR/m8v4JYx6Wzc9ku4SxORMFI4CACNTqzMu3e24a+/T2DW6h/pNCSN12avY79acIiUSgoHOSg6yrilXX0m9k/knHpV+csHS7jm+dl884Ma+YmUNgoHOcwpJ1TgtVvOZ9AVZ7Ns0w66DE3judTVZP+qFhwipYXCQfJkZlx1Xj2mDOxAYqM4/v3JCi57ZibLNu7If2cRiXgKBzmiE6uUY1SvFoy49lw2bf+F7k9P58lJK9mTrRYcIiWZwkHyZWZcfHZtJg/oQPdzTuapzzK4ePh05q3bGu7SRKSQKBwkZNUrxjL46maMvuk8du3JpsfImfxj/FJ27c3Of2cRiSgKBzlqF55Ri0kDO9Cr1amMnrGWTkPSmL5KjfxEShKFgxyTSmVjeCS5CWNvb02Z6Ciuf3EO949byPZf1MhPpCRQOMhxaVm/Bp/c2547Lzidd+d/R9LgVCYu/T7cZYnIcVI4yHErVyaaB7o05oO72nJCpbLc/uo87n59Plk/qZGfSKRSOEiBaVq3Kil92/LHzmcwedlmOg5O5d15G9TITyQCKRykQJWJjuLuCxvw8b3taFCrEve9s5AbR8/lOzXyE4koCgcpFA1qVead21vz8CUJzF27hU6DU3ll1lo18hOJEAoHKTRRUcaNbXMa+Z17anX+9uFSrh41i9VZO8NdmojkQ+Egha5ejQq8cnNLnuhxNiu//4muw6bxzBcZ7FMjP5FiS+EgRcLMuDJQjyn3deB3Z9Ri0KcruXTEDJZ8tz3cpYlIHkIKBzPrYmYrzSzDzB48wrgeZuZmFggux5rZaDNbbGYLzeyCXGM/Da5bamYjzSw6uL6GmU02s1XB39WPc45SjNSqXI6RvVrw7HXnsnnHHpJHzOCJiSvYvU+N/ESKk3zDIfigPQLoCiQA15hZQh7jKgP9gDm5VvcBcPemQBLwpJkdOOZV7n4O0ASIA64Mrn8QmOruDYGpwWUpYbo2rc2UgYlc1rwOIz5fTbfh00hfuyXcZYlIUChnDi2BDHdf4+57gbeA5DzGPQoMAnbnWpdAzgM87p4JbAMCweUDXwwQA8QCB97GkgyMCd4eA1wa6mQkslSrEMt/rzyHV25uyZ59+7nyuVk8nLKUn/eokZ9IuIUSDnWA9bmWNwTXHWRmzYF67j7hkH0XAslmFmNm9YEWQL1c+00EMoGfgHHB1Se6+yaA4O9aoU9HIlFiozgmDUikd+t4xszKaeSX+nVWuMsSKdVCCQfLY93BN6sHLxMNAe7LY9xL5IRJOjAUmAkcfFro7p2B2kBZ4HchV51z3NvMLN3M0rOy9EAS6SqWjeHh7mfxzu2tKVsmit4vfcl9YxeybdfecJcmUiqFEg4byPVsH6gLbMy1XJmc1w2+MLO1QCsgxcwC7p7t7gPcvZm7JwPVgFW579zddwMp/P+lqs1mVhsg+Dszr6LcfZS7B9w9EBcXF8I0JBIE4mvwcb/23H3h6Xyw4Ds6Dk7jk8Wbwl2WSKkTSjjMBRqaWX0ziwV6kvNgDoC7b3f3mu4e7+7xwGygu7unm1kFM6sIYGZJQLa7LzOzSrkCIAboBqwI3mUK0Dt4uzfw4fFPUyJJuTLR/LFzY1L6tuXEKmW58/X53PHqPDJ37M5/ZxEpEPmGg7tnA32BicByYKy7LzWzR8ysez671wLmm9ly4AGgV3B9RXLOLhaR87pEJjAyuO1xIMnMVpHzDqfHj3JOUkKcdXJVPry7LQ90acxnKzPpODiVd9LXq5GfSBGwkvAfLRAIeHp6erjLkEK0OmsnD767iLlrt9K+YU3+dVlT6tWoEO6yRCKamc1z90Be2/QJaYkIp8dV4u3bWvNo8lnMX7eVzkPTeHnGN/yqRn4ihULhIBEjKsro1TqeiQMSOS++Bg+PX8ZVz80iI/OncJcmUuIoHCTi1K1egZdvOo/BV53D6qyddBs2nac/W6VGfiIFSOEgEcnMuPzcukwe0IGks07kv5O+pvvTauQnUlAUDhLR4iqXZcS15/Jcrxb8sDOnkd/jn6iRn8jxUjhIidD5rJOYMqADPc6ty8jU1XQbNo0vv1EjP5FjpXCQEqNqhTL8p8fZvHbL+ez9dT9XPTeLv36whJ927wt3aSIRR+EgJU67hjWZNCCRm9vW57U56+g8JI3PV+bZhUVEfoPCQUqkCrEx/O2SBMbd0YaKZWO4afRcBr69gK0/q5GfSCgUDlKitTi1OhP6taPf7xqQsnAjHQenMmHRRrXgEMmHwkFKvLIx0QzsdAbj72nHydXK0/eNr7j91XlsViM/kd+kcJBS48zaVXj/rjY81LUxqV9n0XFwKm/P/VZnESJ5UDhIqRITHcXtHU7n0/6JnFm7Cg+8u5jrXpjDtz/uCndpIsWKwkFKpfo1K/JWn1b887ImLNqwnc5D03hxuhr5iRygcJBSKyrKuO78U5k8MJHWp5/AoxOWccWzM/l6sxr5iSgcpNSrXbU8L/YOMKxnM9b9+DMXD5/G8Kmr2JutRn5SeikcRMhp5JfcrA5TBnagS5PaDJ78Nd2fns7C9dvCXZpIWCgcRHI5oVJZnrqmOc/fEGDrrr1c9swM/vXxcn7Zq0Z+UrooHETykJRwIpMHduDq8+oxKm0NXYelMWv1j+EuS6TIKBxEfkOVcmX49+Vn88at57Pf4ZrnZ/On9xezQ438pBRQOIjko02Dmkzsn0if9vV568tv6TQ4jc9WbA53WSKFSuEgEoLysdH8+eIE3rurLVXLl+Hml9O5962v+HHnnnCXJlIoFA4iR6FZvWqMv6cd/Ts25OPFm0gakkbKQjXyk5JH4SBylGJjoujfsRET7mlPvRoV6PfmV/R5JZ1N238Jd2kiBSakcDCzLma20swyzOzBI4zrYWZuZoHgcqyZjTazxWa20MwuCK6vYGYfmdkKM1tqZo/nuo8bzSzLzBYEf249zjmKFIozTqrMe3e24S8Xn8n0jB/oNDiNN+Z8y3614JASIN9wMLNoYATQFUgArjGzhDzGVQb6AXNyre4D4O5NgSTgSTM7cMz/untjoDnQ1sy65trvbXdvFvx54RjmJVIkoqOMW9ufxsT+iTSpU5U/vb+Ya1+Yzdoffg53aSLHJZQzh5ZAhruvcfe9wFtAch7jHgUGAbmb5CcAUwHcPRPYBgTcfZe7fx5cvxeYD9Q95lmIhNmpJ1TkjT7n8/jlTVn63Q66DEvj+bQ1auQnESuUcKgDrM+1vCG47iAzaw7Uc/cJh+y7EEg2sxgzqw+0AOodsm814BKCIRJ0hZktMrNxZvY/40WKKzOjZ8tTmDywA+0a1OSfHy/n8mdmsPJ7NfKTyBNKOFge6w4+HQpeJhoC3JfHuJfICZN0YCgwE8jOtW8M8CYw3N3XBFePB+Ld/WxgCjAmz6LMbjOzdDNLz8rKCmEaIkXjpKrleP6GAE9d05wNW3/h909NY8jkr9mTrRYcEjksv7fgmVlr4GF37xxcfgjA3f8dXK4KrAZ2Bnc5CdgCdHf39EPuayZwq7svCy6/BOx0936/cexoYIu7Vz1SjYFAwNPT0480RCQstvy8l0fGL+WDBRtpdGIl/nPF2TQ/pXq4yxIBwMzmuXsgr22hnDnMBRqaWX0ziwV6AikHNrr7dnev6e7x7h4PzCYYDMF3JVUMFpEEZOcKhseAqkD/Q4qtnWuxO7A81ImKFDc1KsYytGdzXroxwE+7s7n82Zk8OmEZu/Zm57+zSBjlGw7ung30BSaS80A91t2XmtkjZtY9n91rAfPNbDnwANALwMzqAn8m5wXr+Ye8ZbVf8O2tC8l599ONxzAvkWLld41PZNKARK47/xRenP4NXYZOY2bGD+EuS+Q35XtZKRLospJEktlrfuTBdxex9sdd9DyvHg91O5Oq5cuEuywphY73spKIFKBWp53Ap/0Tub3DaYxNX0+nIalMXqZGflK8KBxEwqBcmWge6nomH9zdluoVYunzSjp935jPD2rkJ8WEwkEkjM6uW42Uvu24L6kRk5ZupuPgVN7/aoMa+UnYKRxEwiw2Jop7LmrIR/3aUb9mRQa8vZCbX57Lxm1q5Cfho3AQKSYanliZcXe04W+/T2D2mi10GpLGq7PXqZGfhIXCQaQYiY4ybm5Xn0kDEmlWrxp//WAJPZ+fzTdq5CdFTOEgUgzVq1GBV29pyaArzmb5ph10GZrGyNTVZP+6P9ylSSmhcBAppsyMq86rx5SBHejQKI7HP1nBZc/MZNnGHeEuTUoBhYNIMXdilXI816sFz1x3Lpu2/0L3p6fz5KSVauQnhUrhIBIBzIxuTWszeUAHujc7mac+y+Di4dOZt25ruEuTEkrhIBJBqleMZfBVzXj5pvP4Ze+v9Bg5k3+MX8rPe9TITwqWwkEkAl1wRi0mDkikV6tTGT1jLZ2HpjFtlb7XRAqOwkEkQlUqG8MjyU0Ye3trYqOj6PXil9w/biHbd+0Ld2lSAigcRCJcy/o1+Pje9tx5wem8O/87Og5J5dMl34e7LIlwCgeREqBcmWge6NKYD+9uS1ylstzx2jzuen0emT/tDndpEqEUDiIlSJM6Vfmwb1v+2PkMpizPJGlwGu/OUyM/OXoKB5ESpkx0FHdf2ICP+7WnQa1K3PfOQnqPnsuGrbvCXZpEEIWDSAnVoFYl3rm9Nf/ofhbpa7fQeUgar8xaq0Z+EhKFg0gJFhVl9G4Tz8T+iZx7anX+9uFSrh41i9VZO8NdmhRzCgeRUqBejQq8cnNL/nvlOXy9eSddh01jxOcZ7FMjP/kNCgeRUsLM6NGiLpMHJtLxzFo8MXEll46YwZLvtoe7NCmGFA4ipUytyuV45roWjLz+XDbv2EPyiBkM+nQFu/epkZ/8P4WDSCnVpUltpg7swOXN6/DMF6vpNnwa6Wu3hLssKSYUDiKlWNUKZXjiynN45eaW7Nm3nyufm8XfP1zCTjXyK/UUDiJCYqM4Jg1IpHfreF6ZvY7OQ9JI/VqN/EqzkMLBzLqY2UozyzCzB48wroeZuZkFgsuxZjbazBab2UIzuyC4voKZfWRmK8xsqZk9nus+yprZ28FjzTGz+OOaoYiEpGLZGB7ufhbj7mhNuTJR9H7pSwaOXcC2XXvDXZqEQb7hYGbRwAigK5AAXGNmCXmMqwz0A+bkWt0HwN2bAknAk2Z24Jj/dffGQHOgrZl1Da6/Bdjq7g2AIcB/jmViInJsWpxag4/6tafvhQ1IWbCRjoNT+XjxpnCXJUUslDOHlkCGu69x973AW0ByHuMeBQYBuTt9JQBTAdw9E9gGBNx9l7t/Hly/F5gP1A3ukwyMCd4eB1xkZnZUsxKR41KuTDR/6HwGH/Zty0lVy3HX6/O549V5ZO5QI7/SIpRwqAOsz7W8IbjuIDNrDtRz9wmH7LsQSDazGDOrD7QA6h2ybzXgEoIhkvt47p4NbAdOOLQoM7vNzNLNLD0rS9dGRQrDWSdX5YO72vJAl8Z8tjKTjoNTGZu+Xo38SoFQwiGvZ+0H/2UELxMNAe7LY9xL5IRJOjAUmAlk59o3BngTGO7ua0I53sEV7qPcPeDugbi4uBCmISLHIiY6ijsvOJ1P721P45OqcP+4Rdzw0pes36JGfiVZKOGwgf99tl8X2JhruTLQBPjCzNYCrYAUMwu4e7a7D3D3Zu6eDFQDVuXadxSwyt2H5nW8YHhUBfTma5EwOy2uEm/d1opHk89i/rqtdB6axugZ3/CrGvmVSKGEw1ygoZnVN7NYoCeQcmCju29395ruHu/u8cBsoLu7pwfflVQRwMySgGx3XxZcfoycB/7+hxwvBegdvN0D+Mx1DitSLERFGb1axzNpYAda1q/BP8Yv48qRM8nI/CncpUkByzccgtf9+wITgeXAWHdfamaPmFn3fHavBcw3s+XAA0AvADOrC/yZnBes55vZAjO7NbjPi8AJZpYBDAR+862zIhIedaqVZ/SN5zHk6nNY88PPdBs2nac/W6VGfiWIlYQn5YFAwNPT08Ndhkip9MPOPfw9ZSkfLdpE45Mq80SPc2hat2q4y5IQmNk8dw/ktU2fkBaR41KzUllGXHsuz/VqwZaf93LpMzN4/BM18ot0CgcRKRCdzzqJyQM70OPcuoxMXU3XYdOYs+bHcJclx0jhICIFpmr5Mvynx9m8fuv5ZO/fz9WjZvOXDxbz0+594S5NjpLCQUQKXNsGNZnYP5Fb2tXn9Tnf0nlIGp+vyAx3WXIUFA4iUigqxMbw198n8O6dbahYNoabXp7LgLcXsOVnNfKLBAoHESlU555SnQn92tHvooaMX7iRpMGpTFi0US04ijmFg4gUurIx0QxMasT4e9pRp3p5+r7xFbe9Oo/NauRXbCkcRKTInFm7Cu/d2YY/dWtM2tdZdBycyltffquziGJI4SAiRSomOorbEk9nYv9EEmpX4cH3FnPdC3P49kc18itOFA4iEhbxNSvyZp9W/OuypizasJ1OQ1N5YdoaNfIrJhQOIhI2UVHGteefwuSBibQ5vSaPfbScK56dydeb1cgv3BQOIhJ2tauW58XeAYb1bMa3W3Zx8fBpDJuyir3ZauQXLgoHESkWzIzkZnWYPCCRrk1qM2TK13R/ejoL128Ld2mlksJBRIqVEyqVZfg1zXnhhgDbdu3jsmdm8K+Pl/PLXjXyK0oKBxEpljomnMikgYn0bHkKo9LW0GVYGrNWq5FfUVE4iEixVaVcGf51WVPe6HM+ANc8P5uH3lvMDjXyK3QKBxEp9tqcXpNP703ktsTTeHvut3QanMbU5ZvDXVaJpnAQkYhQPjaaP3U7k/fuakvV8mW4ZUw6/d78ih937gl3aSWSwkFEIkqzetUYf087BnRsxCdLNpE0JI0PF3ynFhwFTOEgIhEnNiaKezs25KN+7TmlRgXufWsBt45JZ9P2X8JdWomhcBCRiNXoxMq8e2cb/nLxmcxY/QNJg9N4fc469qsFx3FTOIhIRIuOMm5tfxqT+nfg7LpV+fP7S7j2hdms/eHncJcW0RQOIlIinHJCBV6/9Xwev7wpS7/bQeehaYxKW032r2rBcSwUDiJSYpgZPVuewuSBHWjfMI5/fbyCK56dyYrvd4S7tIgTUjiYWRczW2lmGWb24BHG9TAzN7NAcDnWzEab2WIzW2hmF+Qa+08zW29mOw+5jxvNLMvMFgR/bj3GuYlIKXVS1XI8f0MLnr62ORu2/sLvh09n8OSv2ZOtFhyhyjcczCwaGAF0BRKAa8wsIY9xlYF+wJxcq/sAuHtTIAl40swOHHM80PI3Dvu2uzcL/rwQ6mRERA4wM35/9slMGdiBS845meFTV3HJU9P56tut4S4tIoRy5tASyHD3Ne6+F3gLSM5j3KPAICD3l8ImAFMB3D0T2AYEgsuz3X3TcdQuIpKv6hVjGW2dhJMAAAuISURBVHJ1M0bfeB4/7c7m8mdn8uiEZezamx3u0oq1UMKhDrA+1/KG4LqDzKw5UM/dJxyy70Ig2cxizKw+0AKoF8IxrzCzRWY2zszyHG9mt5lZupmlZ2VlhXCXIlKaXdi4FpMGJHLd+afw4vRv6Dw0jRkZP4S7rGIrlHCwPNYdfBNx8DLREOC+PMa9RE6YpANDgZlAfnE9Hoh397OBKcCYvAa5+yh3D7h7IC4uLt9JiIhULleGxy5tytu3tSImKorrXpjDg+8uYvsvauR3qFDCYQP/+2y/LrAx13JloAnwhZmtBVoBKWYWcPdsdx8QfO0gGagGrDrSwdz9R3c/0CzleXLONkRECsz5p53AJ/e25/YOpzE2fT1Jg1OZtPT7cJdVrIQSDnOBhmZW38xigZ5AyoGN7r7d3Wu6e7y7xwOzge7unm5mFcysIoCZJQHZ7r7sSAczs9q5FrsDy49uSiIi+StXJpqHup7JB3e3pUbFWG57dR5935jPD2rkB4QQDu6eDfQFJpLzQD3W3Zea2SNm1j2f3WsB881sOfAA0OvABjMbZGYbgApmtsHMHg5u6mdmS81sITnvfrrxaCclIhKqs+vmNPL7Q6dGTFq6mY6DU3n/qw2lvpGflYQ/gEAg4Onp6eEuQ0QiXEbmT9w/bhHzv93GBWfE8c/LmlKnWvlwl1VozGyeuwfy2qZPSIuIBDWoVZl37mjD3y9JYM6aLXQanMqrs0tnIz+Fg4hILtFRxk1t6zNpQCLNT6nOXz9YQs9Rs1mTtTP/nUsQhYOISB7q1ajAq7e0ZFCPs1nx/Q66DpvGyNTS08hP4SAi8hvMjKsC9ZgysAMXnBHH45+s4NJnZrBsY8lv5KdwEBHJR60q5XiuV4BnrzuX77fvofvT0/nvxJXs3ldyG/kpHEREQtS1aW2mDEwkuVkdnv48g4uHT2Peui3hLqtQKBxERI5CtQqxPHnVOYy5uSW79+2nx8hZPJyylJ/3lKxGfgoHEZFj0KFRHBMHJHJDq1N5eeZaOg9NY9qqktMEVOEgInKMKpWN4R/JTXjnjtbExkTR68Uv+eM7C9m+K/Ib+SkcRESO03nxNfi4X3vuuuB03vvqOzoOSeXTJZH9dTUKBxGRAlCuTDT3d2nMh3e3Ja5SWe54bT53vjaPzJ92579zMaRwEBEpQE3qVOXDvm35Y+czmLoik6TBaYybF3mN/BQOIiIFrEx0FHdf2ICP+7WnYa1K/OGdhfQePZcNW3eFu7SQKRxERApJg1qVGHt7a/7R/SzS126h05A0xsxcGxGN/BQOIiKFKCrK6N0mnkkDEgnE1+DvKUu56rlZZGQW70Z+CgcRkSJQt3oFxtx0Hk9eeQ6rMnfSbdg0Rnyewb5i2shP4SAiUkTMjCta1GXKwA50TKjFExNXkvz0DJZ8tz3cpR1G4SAiUsTiKpflmetaMPL6c8nauYfkETP4z6crilUjP4WDiEiYdGlSmykDOnB58zo8+8Vqug2bxty1xaORn8JBRCSMqlYowxNXnsOrt7Rk76/7uXLkLP724RJ2hrmRn8JBRKQYaN8wjon9E7mpbTyvzl5H5yFpfLEyM2z1KBxERIqJimVj+PslZzHujjaUj43mxtFzGTh2AVt/3lvktSgcRESKmRanVuejfu2453cNSFmwkaQhqXy8eFORtuBQOIiIFENlY6K5r9MZpPRtR+2q5bnr9fnc8do8MncUTSO/kMLBzLqY2UozyzCzB48wroeZuZkFgsuxZjbazBab2UIzuyDX2H+a2Xoz23nIfZQ1s7eDx5pjZvHHNDMRkRIg4eQqvH9XGx7s2pgvVmbRcXAqY9PXF/pZRL7hYGbRwAigK5AAXGNmCXmMqwz0A+bkWt0HwN2bAknAk2Z24JjjgZZ5HPIWYKu7NwCGAP8JeTYiIiVQTHQUd3Q4nU/ubU/j2lW4f9wier34Jeu3FF4jv1DOHFoCGe6+xt33Am8ByXmMexQYBOQ+50kApgK4eyawDQgEl2e7e17fhpEMjAneHgdcZGYWQp0iIiXaaXGVeKtPKx67tAkL1m+j05A0xi/cWCjHCiUc6gDrcy1vCK47yMyaA/XcfcIh+y4Eks0sxszqAy2AeqEez92zge3ACYcOMrPbzCzdzNKzskrO97aKiBxJVJRxfatTmTQgkbYNalK/ZsVCOU5MCGPyetZ+8GJX8DLREODGPMa9BJwJpAPrgJlAfp/sOOLxDq5wHwWMAggEAsW//62ISAE6uVp5XugdKLT7DyUcNvC/z/brArnPYyoDTYAvgld/TgJSzKy7u6cDAw4MNLOZwKoQj7fBzGKAqkDx+Dy5iEgpEcplpblAQzOrb2axQE8g5cBGd9/u7jXdPd7d44HZQHd3TzezCmZWEcDMkoBsd1+Wz/FSgN7B2z2AzzzSvl9PRCTC5RsOwev+fYGJwHJgrLsvNbNHzKx7PrvXAuab2XLgAaDXgQ1mNsjMNgAVzGyDmT0c3PQicIKZZQADgd9866yIiBQOKwlPygOBgKenp4e7DBGRiGJm89w9zxcu9AlpERE5jMJBREQOo3AQEZHDKBxEROQwJeIFaTPLIudDdseiJvBDAZYTCTTn0kFzLh2OZ86nuntcXhtKRDgcDzNL/61X60sqzbl00JxLh8Kasy4riYjIYRQOIiJyGIVDsHlfKaM5lw6ac+lQKHMu9a85iIjI4XTmICIih1E4iIjIYUpNOJhZFzNbaWYZZnZYp1czK2tmbwe3zzGz+KKvsmCFMOeBZrbMzBaZ2VQzOzUcdRak/Oaca1wPM3Mzi/i3PYYyZzO7Kvh3vdTM3ijqGgtaCP+2TzGzz83sq+C/727hqLOgmNlLZpZpZkt+Y7uZ2fDgn8ciMzv3uA/q7iX+B4gGVgOnAbHkfH1pwiFj7gJGBm/3BN4Od91FMOcLgQrB23eWhjkHx1UG0sj57pFAuOsugr/nhsBXQPXgcq1w110Ecx4F3Bm8nQCsDXfdxznnROBcYMlvbO8GfELON2m2AuYc7zFLy5lDSyDD3de4+17gLSD5kDHJwJjg7XHARRb8arsIle+c3f1zd98VXJxNzrf8RbJQ/p4BHgUGAbuLsrhCEsqc+wAj3H0rgLtnFnGNBS2UOTtQJXi7Kv/77ZURx93TOPI3YiYDr3iO2UA1M6t9PMcsLeFQB1ifa3lDcF2eYzznC462AycUSXWFI5Q553YLOc88Ilm+czaz5kA9d59QlIUVolD+nhsBjcxshpnNNrMuRVZd4Qhlzg8D1we/UOxj4J6iKS1sjvb/e75C+Q7pkiCvM4BD38MbyphIEvJ8zOx6IAB0KNSKCt8R52xmUcAQ4MaiKqgIhPL3HEPOpaULyDk7nGZmTdx9WyHXVlhCmfM1wMvu/qSZtQZeDc55f+GXFxYF/vhVWs4cNgD1ci3X5fDTzINjzCyGnFPRI53GFXehzBkz6wj8mZzv/d5TRLUVlvzmXBloAnxhZmvJuTabEuEvSof6b/tDd9/n7t8AK8kJi0gVypxvAcYCuPssoBw5DepKqpD+vx+N0hIOc4GGZlbfzGLJecE55ZAxKUDv4O0ewGcefKUnQuU75+AllufICYZIvw4N+czZ3be7e013j3f3eHJeZ+nu7pH8HbOh/Nv+gJw3H2BmNcm5zLSmSKssWKHM+VvgIgAzO5OccMgq0iqLVgpwQ/BdS62A7e6+6XjusFRcVnL3bDPrC0wk550OL7n7UjN7BEh39xTgRXJOPTPIOWPoGb6Kj1+Ic34CqAS8E3zt/Vt37x62oo9TiHMuUUKc80Sgk5ktA34F/ujuP4av6uMT4pzvA543swHkXF65MZKf7JnZm+RcFqwZfB3l70AZAHcfSc7rKt2ADGAXcNNxHzOC/7xERKSQlJbLSiIichQUDiIichiFg4iIHEbhICIih1E4iIjIYRQOIiJyGIWDiIgc5v8AnSyf+lmV0+4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhU9fn+8fcT9kBAVtkJyA5B0WFxVxTFDVS07msVa2ttrQvgVhStilZrW1rFKupPq62AmuKCdcF9IahkYzEEhLCGLawh2/P7Y0a/kQYykGUyM/fruriunHM+c+b5kOTcc86ZeWLujoiIxJ+ESBcgIiKRoQAQEYlTCgARkTilABARiVMKABGROFU/0gXsjzZt2nhycnKkyxARiRrz58/f4O5tK9oWVQGQnJxMWlpapMsQEYkaZvb93rbpEpCISJxSAIiIxCkFgIhInFIAiIjEKQWAiEicUgCIiMQpBYCISJxSAIiI1GHzlm/iiQ+X1si+o+qDYCIi8WL77hKmvL2I5z//nq6tErn8yG4kNqzeQ7YCQESkjvlwST63z8pgdcEurjo6mVtO6VPtB39QAIiI1BmbdxQx+Y1sZn29ip7tmjHjF0dxRLeWNfZ8CgARkQhzd97KXMvdr2eyZWcxvx7RkxtG9KRR/Xo1+rwKABGRCFq/tZC7Xs9kTtY6Ujq14Pmrh9G/Y/NaeW4FgIhIBLg7r8zP477Z2ewuKWPCaX255pju1K9Xe2/OVACIiNSylZt2MnFWBp/kbGBociseHJtCj7bNar0OBYCISC0pLXOe+2w5D89ZTL0EY/LZA7lkaFcSEiwi9SgARERqwXfrtjF+Zjpfr9jCCX3a8odzUuh4UJOI1qQAEBGpQcWlZTwxdyl/eT+Hpo3q8acLDmPMYR0xi8yr/vIUACIiNSQjr4BbZyxg0dptnDmoA5NGD6BNs0aRLutHCgARkWpWWFzKY+8u4amPcmnTrBHTLjuCUwa0j3RZ/yOs9xuZ2SgzW2xmOWY2YR/jzjMzN7NAaLmhmU03swwzW2BmJ5Qbe0RofY6Z/dnqwvmQiEgVfZm7kdMe/5gnP8zlgiFd+O/vjq+TB38I4wzAzOoBU4GRQB4wz8xS3T17j3FJwI3Al+VWXwvg7ilm1g54y8yGuHsZ8HdgHPAF8CYwCnir6lMSEal92wqLeejtRbzwxQq6tkrkn9cM46iebSJd1j6FcwloKJDj7rkAZvYyMAbI3mPcZGAKcEu5df2B9wDcfb2ZbQECZrYSaO7un4f2+TxwNgoAEYlCHyxaz+2vZrBuayHXHNOd353Su0aat1W3cC4BdQJWllvOC637kZkNBrq4++w9HrsAGGNm9c2sO3AE0CX0+Lx97bPcvseZWZqZpeXn54dRrohI7di0o4jfvvwNVz07j2aN6jPz+qO488z+UXHwh/DOACq6Nu8/bjRLAB4Drqxg3DNAPyAN+B74DCipbJ8/Wek+DZgGEAgEKhwjIlKb3J3Z6WuYlJpFwa5ifnNSL3554iE13rytuoUTAHkEX7X/oDOwutxyEjAQmBu6j9seSDWz0e6eBtz0w0Az+wz4Dtgc2s/e9ikiUiet21rIHa9m8u7CdQzq3IIXrx1G3/a107ytuoUTAPOAXqFLOKuAC4GLf9jo7gXAj3c6zGwucIu7p5lZImDuvsPMRgIlP9w8NrNtZjac4E3jy4G/VNOcRESqnbvzr3kruf/NhRSXlnHH6f246ujkWm3eVt0qDQB3LzGzG4A5QD3gGXfPMrN7gTR3T93Hw9sBc8ysjGB4XFZu2/XAs0ATgjd/dQNYROqk7zfuYOKsDD5bupHhPVrx4LmDSG7TNNJlVZm5R89l9UAg4GlpaZEuQ0TiRGmZM/3TZTzyzmIaJCQw8fR+XDikS8Satx0IM5vv7oGKtkXHrWoRkVq2eO02bpuZzoKVWzipbzvuO2cgHVpEtnlbdVMAiIiUU1RSxt/m5jD1gxySGjfg8QsPY/ShdaN5W3VTAIiIhCxYuYXbZqSzeN02xhzWkbvP7E/rOtS8rbopAEQk7u0qKuXR/y7m6U+W0S6pMU9fEeCkfgdHuqwapwAQkbj22dINTJiZwYpNO7l4WFcmnNaX5o0bRLqsWqEAEJG4tLWwmAfeXMRLX62gW+tEXrp2OEce0jrSZdUqBYCIxJ13s9dxx2sZ5G/bzbjjenDTyb1p0jC62jhUBwWAiMSNjdt3c89/skldsJq+7ZOYdlmAQ7scFOmyIkYBICIxz91JXbCaSalZbN9dwu9G9uYXxx9Cw/rR28ahOigARCSmrSnYxZ2vZvLeovUc1uUgppw3iN4HJ0W6rDpBASAiMamszHlp3goeeHMRpWXOXWf258qjkqkXRW0capoCQERizrINO5gwM50vl23i6J6teeCcQXRtnRjpsuocBYCIxIyS0jKe+XQZf3xnCQ3rJ/DQ2BR+FugSk20cqoMCQERiwsI1Wxk/M530vAJG9j+Y+84eyMHNG0e6rDpNASAiUW13SSlT38/hb3OX0qJJA/568WDOSOmgV/1hUACISNT6esVmxs9I57v12zl3cCfuOrM/LZs2jHRZUUMBICJRZ2dRCY/MWcL0z5bRoXljpl81hBP7tIt0WVFHASAiUeXTnA1MmJXOyk27uGx4N24b1YekOGneVt0UACISFQp2FfOHNxbyr7SVdG/TlH+NG86wHvHVvK26KQBEpM57J2std76WycYdRfzi+EP47cm9aNwg/pq3VTcFgIjUWfnbdjPpP1m8kb6Gfh2a8/QVQ0jp3CLSZcUMBYCI1DnuzqvfrOLe2dns3F3Kraf2YdxxPWhQL76bt1W3sP43zWyUmS02sxwzm7CPceeZmZtZILTcwMyeM7MMM1toZhPLjb3JzLLMLNPMXjIzfWJDRFi1ZRdXPTuP3/17AT3aNOXN3xzDr07sqYN/Daj0DMDM6gFTgZFAHjDPzFLdPXuPcUnAjcCX5VafDzRy9xQzSwSyzewloDg0tr+77zKzfwMXAs9Ww5xEJAqVlTkvfvk9D761CAcmndWfy45U87aaFM4loKFAjrvnApjZy8AYIHuPcZOBKcAt5dY50NTM6gNNgCJga+jr+kATMysGEoHVVZiHiESx3PztTJiZwVfLN3Fsrzb84ZwUurRS87aaFk4AdAJWllvOA4aVH2Bmg4Eu7j7bzMoHwAyCYbGG4EH+JnffFHrMI8AKYBfwjru/c8CzEJGoVFJaxlMfL+Oxd5fQuH4CD583iPOO6Kw2DrUknACo6DvhP240SwAeA66sYNxQoBToCLQEPjazd4HNBIOhO7AFeMXMLnX3F/7nyc3GAeMAunbtGka5IhINslYXMH5mOpmrtjJqQHvuHTOAdmreVqvCCYA8oEu55c789HJNEjAQmBtK7fZAqpmNBi4G3nb3YmC9mX0KBAgGyDJ3zwcws1nAUcD/BIC7TwOmAQQCAd9zu4hEl8LiUv7y/nc88WEuLRMb8vdLDue0lA6RLisuhRMA84BeZtYdWEXwZu3FP2x09wKgzQ/LZjYXuMXd08zsJGCEmb1A8BLQcOBPBO8BDA/dGN4FnASkVcuMRKTOmv/9Jm6bkc7S/B2MPbwzd53Zj4MS1bwtUioNAHcvMbMbgDlAPeAZd88ys3uBNHdP3cfDpwLTgUyCl5Kmu3s6gJnNAL4GSoBvCL3KF5HYs2N3CQ/PWcxzny+nY4smPHf1UI7v3TbSZcU9c4+eqyqBQMDT0nSiIBJNPlqSz8RZGawu2MXlw7tx66i+NGukz6DWFjOb7+6BirbpuyAiNWLLziLue2MhM+bn0aNtU1657kgCya0iXZaUowAQkWr3VsYa7no9i807i/jViYfw6xFq3lYXKQBEpNqs31bI71/P4q3MtQzo2Jznrh7CgI5q3lZXKQBEpMrcnRnz87jvjYXsKi7ltlF9uPZYNW+r6xQAIlIlKzft5PZXM/j4uw0MSW7Jg2MHcUjbZpEuS8KgABCRA1JW5jz/+XKmzFmMAfeOGcClw7qRoOZtUUMBICL7LWf9dibMTCft+80c37st958zkM4t1bwt2igARCRsxaVlTPsol8ff/Y7ERvV49GeHcs7gTmreFqUUACISlsxVBdw2I53sNVs5I6UDk0YPoG1So0iXJVWgABCRfSosLuXx975j2ke5tGrakCcuPYJRA9tHuiypBgoAEdmrecs3MX5GOrkbdvCzQGfuOL0/LRIbRLosqSYKABH5H9t3lzDl7UU8//n3dG7ZhBd+PoxjerWp/IESVRQAIvITHyxezx2zMliztZCrj+7Ozaf0pqmat8UkfVdFBIDNO4qYPDubWd+some7Zsz4xVEc0a1lpMuSGqQAEIlz7s6bGWv5fWomW3YWc+OInvxqRE8a1VfztlinABCJY+u3FnLna5m8k72OlE4teP7qYfTv2DzSZUktUQCIxCF355W0PCa/kU1RSRkTT+vLz4/pTn01b4srCgCROLNiY7B52yc5GxjavRUPnptCDzVvi0sKAJE4UVrmPPvZch6Zs5h6CcZ9Zw/k4qFd1bwtjikAROLAd+u2cdvMdL5ZsYUT+7Tl/nNS6HhQk0iXJRGmABCJYUUlZTzx4VL++n4OTRvV408XHMaYwzqqeZsACgCRmJWet4XbZqSzaO02zjq0I78/qz9tmql5m/wfBYBIjNlVVMqf3l3CUx/n0japEU9dHmBk/4MjXZbUQWG958vMRpnZYjPLMbMJ+xh3npm5mQVCyw3M7DkzyzCzhWY2sdzYg8xshpktCm07surTEYlvX+Ru5LTHP+LJj3K5YEgX3rnpeB38Za8qPQMws3rAVGAkkAfMM7NUd8/eY1wScCPwZbnV5wON3D3FzBKBbDN7yd2XA48Db7v7eWbWENCfExI5QNsKi3nwrUW8+OUKurZK5J/XDOOonmreJvsWziWgoUCOu+cCmNnLwBgge49xk4EpwC3l1jnQ1MzqA02AImCrmTUHjgOuBHD3otA2EdlP7y9axx2vZrJuayHXHNOdm0/pQ5OGauMglQvnElAnYGW55bzQuh+Z2WCgi7vP3uOxM4AdwBpgBfCIu28CegD5wHQz+8bM/mFmTSt6cjMbZ2ZpZpaWn58f1qRE4sGmHUX89uVvuPrZNJIa12fm9Udx55n9dfCXsIUTABW9X8x/3GiWADwG3FzBuKFAKdAR6A7cbGY9CJ55HA783d0HEwyJCu8tuPs0dw+4e6Bt27ZhlCsS29yd1AWrOfnRD3kjYw2/OakXs399LIO7qnOn7J9wLgHlAV3KLXcGVpdbTgIGAnND7y1uD6Sa2WjgYoLX+YuB9Wb2KRAAPgLy3P2H+wUz2EsAiMj/WVsQbN727sJ1HNq5BQ+dN4y+7dW8TQ5MOGcA84BeZtY9dLP2QiD1h43uXuDubdw92d2TgS+A0e6eRvCyzwgLagoMBxa5+1pgpZn1Ce3mJP73noKIhLg7L321gpGPfsgnOfnceUY/Zv3yaB38pUoqPQNw9xIzuwGYA9QDnnH3LDO7F0hz99R9PHwqMB3IJHgpabq7p4e2/Rp4MRQqucBVVZiHSMz6fuMOJszM4PPcjRzZozUPjk2hW+sKb5mJ7Bdz98pH1RGBQMDT0tIiXYZIrSgtc6Z/uoxH3llMg4QEbj+jHxcO6aI2DrJfzGy+uwcq2qZPAovUQYvXBpu3LVi5hZP7teO+s1No36JxpMuSGKMAEKlDikrKmPpBDn+bm0NS4wb8+aLBnDWog171S41QAIjUEd+u3MJtMxawZN12xhzWkd+fNYBWTRtGuiyJYQoAkQjbVVTKH99ZzDOfLqNdUmOeviLASf3Uv0dqngJAJII+W7qBCTMzWLFpJ5cM68qE0/qS1LhBpMuSOKEAEImArYXFPPDmQl76aiXJrRN5edxwhvdoHemyJM4oAERq2bvZ67jjtQzyt+3muuN68NuTe6t/j0SEAkCklmzYvpt7/pPNfxaspm/7JJ66PMCgzgdFuiyJYwoAkRrm7rz+7Wru+U8W23eX8LuRvfnF8YfQsH5Yf49JpMYoAERq0Ootu7jztUzeX7SewV0P4qGxg+h9cFKkyxIBFAAiNaKszPnnVyt48K1FlJY5d5/ZnyuOSqZegj7QJXWHAkCkmi3bsIMJM9P5ctkmju7ZmgfOGUTX1vqLp1L3KABEqklJaRlPf7KMR/+7hIb1E5gydhDnBzqrjYPUWQoAkWqQvXor42emk7GqgJH9D+a+swdycHM1b5O6TQEgUgW7S0r56/s5/H3uUg5KbMDUiw/n9JT2etUvUUEBIHKA5n+/mfEz08lZv51zD+/EXWf0p6Wat0kUUQCI7KedRSU8PGcxz362nA7NGzP9qiGc2KddpMsS2W8KAJH98Ml3G5gwK528zbu4/Mhu3DaqL80a6ddIopN+ckXCULCzmPvfzObfaXl0b9OUf193JEO7t4p0WSJVogAQqcTbmWu56/VMNu0o4voTDuE3J/WicQM1b5PopwAQ2Yv8bbuZlJrFGxlr6N+hOdOvHMLATi0iXZZItVEAiOzB3Zn19SrunZ3NrqJSbj21D+OO60GDemreJrFFASBSzqotu7h9VgYfLsnniG4teWjsIHq2axbpskRqRFgvacxslJktNrMcM5uwj3HnmZmbWSC03MDMnjOzDDNbaGYT9xhfz8y+MbPZVZuGSNWUlTnPf76cUx79kHnLNzHprP68ct2ROvhLTKv0DMDM6gFTgZFAHjDPzFLdPXuPcUnAjcCX5VafDzRy9xQzSwSyzewld18e2v4bYCHQvMozETlAS/O3M2FmOvOWb+bYXm34wzkpdGml5m0S+8I5AxgK5Lh7rrsXAS8DYyoYNxmYAhSWW+dAUzOrDzQBioCtAGbWGTgD+MeBly9y4IpLy/jb3BxOe/xjFq/dxsPnDeL5q4fq4C9xI5x7AJ2AleWW84Bh5QeY2WCgi7vPNrNbym2aQTAs1gCJwE3uvim07U/AbcA+/zqGmY0DxgF07do1jHJFKpe5qoDxM9PJWr2V0wa2554xA2iXpOZtEl/CCYCKulr5jxvNEoDHgCsrGDcUKAU6Ai2Bj83sXaA/sN7d55vZCft6cnefBkwDCAQCvq+xIpUpLC7lL+9/xxMf5tIysSF/v+RwTkvpEOmyRCIinADIA7qUW+4MrC63nAQMBOaGOiC2B1LNbDRwMfC2uxcD683sUyAADAZGm9npQGOguZm94O6XVnVCInuTtnwTt81MJzd/B+cd0Zk7z+jHQYlq3ibxK5x7APOAXmbW3cwaAhcCqT9sdPcCd2/j7snungx8AYx29zRgBTDCgpoCw4FF7j7R3TuHxl8IvK+Dv9SUHbtLmJSaxflPfs7u4jKev3ooj5x/qA7+EvcqPQNw9xIzuwGYA9QDnnH3LDO7F0hz99R9PHwqMB3IJHgpabq7p1dD3SJh+XBJPrfPymB1wS6uODKZW0/tQ1M1bxMBwNyj57J6IBDwtLS0SJchUWDLziImz17IzK/zOKRtUx4aO4hAspq3Sfwxs/nuHqhom14KScx5K2MNd72exeadRdxwYk9uGNFTzdtEKqAAkJixfmshd7+exdtZaxnQsTnPXT2EAR3VvE1kbxQAEvXcnRnz85g8O5vCkjLGj+rLtcd2p76at4nskwJAotrKTTu5/dUMPv5uA0OSW/Lg2EEc0lb9e0TCoQCQqFQaat728JzFGDB5zAAuGdaNhISKPrcoIhVRAEjUyVm/jfEzM5j//WaO792WP5ybQqeDmkS6LJGoowCQqFFcWsaTHy7lz+/lkNioHo/+7FDOGdyJ0CfQRWQ/KQAkKmSuKuDWGeksXLOVMwZ1YNJZA2ib1CjSZYlENQWA1GmFxaX86d3veOrjXFo1bciTlx3BqQPaR7oskZigAJA668vcjUyYlcGyDTu4INCF20/vR4vEBpEuSyRmKACkztlWWMyUtxfz/774ns4tm/DCz4dxTK82kS5LJOYoAKRO+WDxeu6YlcGarYVcfXR3bjm1N4kN9WMqUhP0myV1wuYdRUyenc2sb1bRq10zZl5/FId3bRnpskRimgJAIsrdeSNjDb9/PYuCXcXcOKInvxrRk0b11bxNpKYpACRi1m0t5M7XMvlv9jpSOrXghWuG0a9D80iXJRI3FABS69ydf6et5L43FlJUUsbE0/ry82PUvE2ktikApFat2LiTCbPS+WzpRoZ2b8VDYwfRvU3TSJclEpcUAFIrSsucZz9bziNzFlMvwbj/nIFcNKSrmreJRJACQGrcknXbuG1GOt+u3MKIvu24/5yBdGih5m0ikaYAkBpTVFLGEx8u5S/vf0ezRvV5/MLDGH1oRzVvE6kjFABSIxas3ML4meksWruNsw7tyKSz+tO6mZq3idQlCgCpVruKSnns3SX84+Nc2iY14qnLA4zsf3CkyxKRCigApNp8vnQjE2els3zjTi4a2pWJp/eleWM1bxOpq8J647WZjTKzxWaWY2YT9jHuPDNzMwuElhuY2XNmlmFmC81sYmh9FzP7ILQuy8x+Uz3TkUjYWljM7a9mcNFTX+DAP68dxgPnpujgL1LHVXoGYGb1gKnASCAPmGdmqe6evce4JOBG4Mtyq88HGrl7ipklAtlm9hKwG7jZ3b8OPW6+mf13z31K3ff+onXcPiuT9dsKufbY7vxuZB+aNFQbB5FoEM4loKFAjrvnApjZy8AYYM+D9WRgCnBLuXUONDWz+kAToAjY6u6bgDUA7r7NzBYCnSrYp9RRG7fv5t7Z2bz+7Wr6HJzEE5cdwWFdDop0WSKyH8IJgE7AynLLecCw8gPMbDDQxd1nm1n5AJhBMCzWAInATaGDf/nHJgOD+emZQ/nt44BxAF27dg2jXKlJ7k7qgtXc859sthUW89uTe/HLE3rSsL7aOIhEm3ACoKI3bfuPG80SgMeAKysYNxQoBToCLYGPzezdcmcTzYCZwG/dfWtFT+7u04BpAIFAwCsaI7VjTcEu7nw1k/cWrefQLgcxZewg+rRPinRZInKAwgmAPKBLueXOwOpyy0nAQGBu6AM+7YFUMxsNXAy87e7FwHoz+xQIALlm1oDgwf9Fd59V5ZlIjSkrc16et5IH3lxIcVkZd57Rj6uO7k49tXEQiWrhBMA8oJeZdQdWARcSPLAD4O4FwI9/r8/M5gK3uHuamZ0EjDCzFwheAhoO/MmCSfE0sNDdH62uyUj1W75hBxNmpfNF7iaO7NGaB8em0K21mreJxIJKA8DdS8zsBmAOUA94xt2zzOxeIM3dU/fx8KnAdCCT4KWk6e6ebmbHAJcBGWb2bWjs7e7+ZlUmI9WntMx55pNl/PG/i2mQkMCD56ZwwZAuauMgEkPMPXouqwcCAU9LS4t0GTFv0dqtjJ+RzoK8Ak7u1477zk6hfYvGkS5LRA6Amc1390BF2/RJYPnR7pJSpn6wlL99kEOLJg34y0WDOXNQB73qF4lRCgAB4JsVmxk/M50l67Zz9mEdufusAbRq2jDSZYlIDVIAxLmdRSX88Z0lPPPpMto3b8wzVwYY0VfN20TigQIgjn2Ws4EJszJYsWknlw7vyvhRfUlS/x6RuKEAiEMFu4p54M2FvDxvJcmtE3l53HCG92gd6bJEpJYpAOLMO1lrufO1TDZs3811x/fgppN707iBmreJxCMFQJzYsH03k1KzmJ2+hr7tk/jHFQEGdVbzNpF4pgCIce7Oa9+u4p7/ZLNzdyk3j+zNdccfouZtIqIAiGWrt+zijlcz+GBxPoO7Bpu39TpYzdtEJEgBEIPKypwXv1rBQ28torTMufvM/lxxVLKat4nITygAYkxu/nYmzMrgq2WbOKZnGx44N4UurRIjXZaI1EEKgBhRUlrGPz5ZxmP/XULD+glMGTuI8wOd1cZBRPZKARADsldv5baZC8hctZVT+h/M5LMHcnBzNW8TkX1TAESx3SWl/PX9HP4+dykHJTbgb5cczmkD2+tVv4iERQEQpeZ/H2zelrN+O+ce3om7zuhPSzVvE5H9oACIMjt2l/DIO4t59rPldGzRhGevGsIJfdpFuiwRiUIKgCjy8Xf5TJyVQd7mXVx+ZDduG9WXZo30LRSRA6OjRxQo2FnMfW9k88r8PHq0acq/rzuSod1bRbosEYlyCoA67u3Mtdz1eiabdhRx/QmH8JuTeql5m4hUCwVAHbV+WyGTUrN4M2Mt/Ts0Z/qVQxjYqUWkyxKRGKIAqGPcnVlfr+Le2dnsKi7l1lP7MO64HjSop+ZtIlK9FAB1SN7mndz+aiYfLcnniG4teWjsIHq2axbpskQkRoX1stLMRpnZYjPLMbMJ+xh3npm5mQVCyw3M7DkzyzCzhWY2cX/3GQ/KypznPlvOKY99RNryTdwzegCvXHekDv4iUqMqPQMws3rAVGAkkAfMM7NUd8/eY1wScCPwZbnV5wON3D3FzBKBbDN7CVgZzj7jwdL87YyfkU7a95s5tlcb/nCOmreJSO0I5xLQUCDH3XMBzOxlYAyw58F6MjAFuKXcOgeamll9oAlQBGzdj33GrOLSMqZ9lMvj731Hkwb1eOT8Qxl7eCe1cRCRWhNOAHQi+Ir9B3nAsPIDzGww0MXdZ5tZ+QCYQfDAvgZIBG5y901mVuk+Y1nmqgLGz0wna/VWTk9pz6TRA2iXpOZtIlK7wgmAil6S+o8bzRKAx4ArKxg3FCgFOgItgY/N7N3K9vmTJzcbB4wD6Nq1axjl1l2FxaX8+b3vePKjXFomNuSJSw9n1MAOkS5LROJUOAGQB3Qpt9wZWF1uOQkYCMwNXb5oD6Sa2WjgYuBtdy8G1pvZp0CA4Kv/fe3zR+4+DZgGEAgEKgyJaDBv+SbGz0wnN38H5x/RmTvP6E+LxAaRLktE4lg47wKaB/Qys+5m1hC4EEj9YaO7F7h7G3dPdvdk4AtgtLunASuAERbUFBgOLKpsn7Fk++4S7n49k/Of+JzdxWU8f/VQHj7/UB38RSTiKj0DcPcSM7sBmAPUA55x9ywzuxdIc/d9HbinAtOBTIKXfaa7ezpARfus2lTqng+X5HP7rAxWF+ziyqOSufXUPjRV8zYRqSPMPXquqgQCAU9LS4t0GZXasrOIe2dnM+vrVRzStikPjR1EIFnN2+lhWNEAAAfTSURBVESk9pnZfHcPVLRNL0er2ZsZa7j79Uy27CzmhhN7csOInmreJiJ1kgKgmqzfWshdr2cyJ2sdAzs157mrhzKgo5q3iUjdpQCoInfnlfl53Dc7m8KSMsaP6su1x3anvpq3iUgdpwCogpWbdjJxVgaf5GxgaHIrHhybQo+26t8jItFBAXAASsuc5z9fzpS3F5NgMHnMAC4Z1o2EBLVxEJHooQDYTznrt3HbjHS+XrGFE/q05f5zUuh0UJNIlyUist8UAGEqLi3jyQ+X8uf3ckhsVI/HLjiUsw9T8zYRiV4KgDBk5BVw64wFLFq7jTMGdeCe0QNo06xRpMsSEakSBcA+FBaX8ti7S3jqo1zaNGvEk5cdwakD2ke6LBGRaqEA2IsvczcyYVYGyzbs4IJAF24/ox8tmqh/j4jEDgXAHrYVFvPQ24t44YsVdGnVhBevGcbRPdtEuiwRkWqnACjng0XruePVDNZsLeTnx3Tn5lN6k9hQ/0UiEpt0dAM27Shi8uxsXv1mFb3aNWPm9UdxeNeWkS5LRKRGxXUAuDuz09cwKTWLgl3F3HhSL3514iE0qq/mbSIS++I2ANZtLeSOVzN5d+E6BnVuwQvXDKNfh+aRLktEpNbEXQC4O/+at5L731xIUUkZt5/el6uPVvM2EYk/cRUAKzbuZMKsdD5bupFh3Vvx0NhBJLdpGumyREQiIi4CoLTMmf7pMh55ZzH1ExK4/5yBXDSkq5q3iUhci/kAKNhZzBXTv+LblVsY0bcd958zkA4t1LxNRCTmA6B5k/p0a53IVUcnM/rQjmreJiISEvMBYGY8fuHgSJchIlLn6K0vIiJxSgEgIhKnwgoAMxtlZovNLMfMJuxj3Hlm5mYWCC1fYmbflvtXZmaHhbZdZGYZZpZuZm+bmTquiYjUokoDwMzqAVOB04D+wEVm1r+CcUnAjcCXP6xz9xfd/TB3Pwy4DFju7t+aWX3gceBEdx8EpAM3VMeEREQkPOGcAQwFctw9192LgJeBMRWMmwxMAQr3sp+LgJdCX1voX1MLvi2nObB6fwoXEZGqCScAOgEryy3nhdb9yMwGA13cffY+9nMBoQBw92LgeiCD4IG/P/B0RQ8ys3FmlmZmafn5+WGUKyIi4QgnACp647z/uNEsAXgMuHmvOzAbBux098zQcgOCATAY6EjwEtDEih7r7tPcPeDugbZt24ZRroiIhCOcAMgDupRb7sxPL9ckAQOBuWa2HBgOpP5wIzjkQv7v8g/AYQDuvtTdHfg3cNR+Vy8iIgfMgsfffQwI3rBdApwErALmARe7e9Zexs8FbnH3tNByArACOM7dc0PrOgLzgUHunm9mk4FEd9/rWUTocfnA9+FP7yfaABsO8LHRSnOOffE2X9Cc91c3d6/w8kmlnwR29xIzuwGYA9QDnnH3LDO7F0hz99RKdnEckPfDwT+0z9Vmdg/wkZkVEzyoXxlGLQd8DcjM0tw9UPnI2KE5x754my9oztW638rOAGKFfmjiQ7zNOd7mC5pzddIngUVE4lQ8BcC0SBcQAZpz7Iu3+YLmXG3i5hKQiIj8VDydAYiISDkKABGROBVzAVBZ51Iza2Rm/wpt/9LMkmu/yuoTxnx/Z2bZoa6r75lZt0jUWZ0OtDttNAtnzmb2s9D3OsvM/lnbNVa3MH62u5rZB2b2Tejn+/RI1FldzOwZM1tvZpl72W5m9ufQ/0e6mR1e5Sd195j5R/BzCkuBHkBDYAHQf48xvwSeCH19IfCvSNddw/M9keCH7CDYfiNq5xvunEPjkoCPgC+AQKTrroXvcy/gG6BlaLldpOuuhTlPA64Pfd2fYLfhiNdehTkfBxwOZO5l++nAWwTb8wwHvqzqc8baGUA4nUvHAM+Fvp4BnGTR+4eCK52vu3/g7jtDi18QbOURzaqrO200CWfO1wJT3X0zgLuvr+Uaq1s4c3aCnYQBWhDlHYXd/SNg0z6GjAGe96AvgIPMrENVnjPWAqDSzqXlx7h7CVAAtK6V6qpfOPMt7+cEX0FEs+rqThtNwvk+9wZ6m9mnZvaFmY2qtepqRjhzngRcamZ5wJvAr2untIjZ39/3SsXaH4XfZ+fS/RgTLcKei5ldCgSA42u0opoXbnfaK2uroFoQzve5PsHLQCcQPMv72MwGuvuWGq6tpoQz54uAZ939j2Z2JPD/QnMuq/nyIqLaj12xdgZQWefSn4wJNbprwb5Pu+qycOaLmZ0M3AGMdvfdtVRbTamO7rTRJtyf69fdvdjdlwGLCQZCtApnzj8n2EkYd/8caEywaVqsCuv3fX/EWgDMA3qZWXcza0jwJu+ezepSgStCX58HvO+hOyxRqNL5hi6HPEnw4B/t14Whkjm7e4G7t3H3ZHdPJnjfY7SHutNGqXB+rl8jeMOf0N/X7g3kEr3CmfMKgl2KMbN+BAMglv9qVCpweejdQMOBAndfU5UdxtQlIA+vc+nTBE8Vcwi+8r8wchVXTZjzfRhoBrwSute9wt1HR6zoKgpzzjElzDnPAU4xs2ygFLjV3TdGruqqCXPONwNPmdlNBC+FXBnFL+Yws5cIXsJrE7qv8XugAYC7P0HwPsfpQA6wE7iqys8Zxf9fIiJSBbF2CUhERMKkABARiVMKABGROKUAEBGJUwoAEZE4pQAQEYlTCgARkTj1/wFgyV6O8mao0QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD7CAYAAACBiVhwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hVdbbG8e9KIXQECYiAhirSBIl0EkfpjmABBRt2RBEFxhm90xydO+OgE0RFEQtiB7FFVJolAQQkKL0GpCNEQVCQvu4fOWAuCeQASU6S836eJ0/O3vu391k/EvJm733Oirk7IiIiWUWEugARESl8FA4iIpKNwkFERLJROIiISDYKBxERyUbhICIi2QQVDmbW1cxWmFm6mT14gnG9zMzNLD6wHG1mY81skZktM7OHAutrmtkXgXVLzOy+LMeoZGZTzWxV4HPF052kiIicnFzDwcwigZFAN6Ah0NfMGuYwrhwwCJiTZXVvIMbdmwAtgP5mFgccBIa6+/lAa+CeLMd8EPjM3esBnwWWRUSkAEUFMaYlkO7uawDM7G2gJ7D0mHGPAsOAP2RZ50AZM4sCSgH7gV3uvh3YAuDuP5vZMqB64Jg9gYsD+48FvgT+dKICK1eu7HFxcUFMRUREjpg3b94P7h6b07ZgwqE6sCHL8kagVdYBZtYcqOnuE80sazhMIPOH/RagNDA4EAxZ940DmvPbGUdVdz8SHFvMrEpuBcbFxZGWlhbEVERE5AgzW3e8bcHcc7Ac1h3tuWFmEcBwYGgO41oCh4CzgVrAUDOrnWXfssC7wP3uviuIWn4ryuxOM0szs7SMjIyT2VVERHIRTDhsBGpmWa4BbM6yXA5oDHxpZmvJvIeQHLgpfR0wyd0PuPs2YCZw9GY1mcHwhru/l+V4W82sWmBMNWBbTkW5+2h3j3f3+NjYHM+KRETkFAUTDnOBemZWy8xKAH2A5CMb3X2nu1d29zh3jwNmAz3cPQ1YD1ximcqQGRzLzcyAl4Bl7p50zPMlA/0Cj/sBH57G/ERE5BTkGg7ufhAYCEwGlgHj3X2JmT1iZj1y2X0kUBZYTGbIjHH3hUA74EYyg2N+4KN7YJ/HgE5mtgroFFgWEZECZMWhZXd8fLzrhrSIyMkxs3nuHp/TNr1DWkREslE4iIhINmEdDqszfuG/U1aw98ChUJciIlKohHU4TF26lac/T+eyp6Yzb9323HcQEQkTYR0OdyXWYeytLdl74DC9Rs3i4eQl7N53MNRliYiEXFiHA0Bi/VgmD07gptbnMnbWWjoPTyV1pd5xLSLhLezDAaBsTBT/6NmY8f3bEBMdwU0vf80f3lnAT3v2h7o0EZGQUDhkcVFcJT4Z1IG7L67D+99uomNSKp8u2hLqskRECpzC4RgloyP5Y9cGfHhPO6qUi2HAG98w4PV5bPt5b6hLExEpMAqH42hcvQIfDmzHH7uex2fLt9EpKZV30jZQHN5RLiKSG4XDCURHRnD3xXX59L4O1K9algcmLOSml79mw/Y9oS5NRCRfKRyCUCe2LOPubMMjPRvxzboddHkylVdmfsfhwzqLEJHiSeEQpIgI46Y2cUwenEB8XCUe/mgpvZ+fRfq2n0NdmohInlM4nKQaFUsz9paL+G/vC0jf9gvdR8xg5BfpHDh0ONSliYjkGYXDKTAzrm5Rg2lDEunYsAqPT15Bz2dmsnjTzlCXJiKSJxQOpyG2XAzPXt+CUTe0IOOXffQcOZP/TFquRn4iUuQpHPJA18ZnMW1wIldfWJ3nvlxN9xHT+fo7NfITkaJL4ZBHKpSOZlivC3j9tlbsP3SYa56fxV8/WMwvauQnIkWQwiGPta9Xmcn3J3BLuzhen7OOzkkpfLFiW6jLEhE5KQqHfFAmJoq/X96ICXe1pXRMFLeMmcuQcfPZsVuN/ESkaFA45KMW51bk40HtufeSuiQv2Eyn4Sl8vHCLWnCISKGncMhnMVGRDO18HskD21OtQinuefMb+r82j2271MhPRAovhUMBaXh2ed6/uy0PdWtAysoMLk1KYfxcNfITkcJJ4VCAoiIj6J9Yh0/v68D51crzx3cXcsNLc1j/oxr5iUjhonAIgdqxZXn7jtb884rGLNiwky5PpvLSjO84pEZ+IlJIBBUOZtbVzFaYWbqZPXiCcb3MzM0sPrAcbWZjzWyRmS0zs4eyjH3ZzLaZ2eJjjvGwmW0ys/mBj+6nOrnCLCLCuKH1uUwZnECr2pV4dOJSeo36ilVb1chPREIv13Aws0hgJNANaAj0NbOGOYwrBwwC5mRZ3RuIcfcmQAugv5nFBba9AnQ9ztMOd/dmgY9PgptK0XT2GaUYc/NFPHltM9b+sJvLnprBU5+tYv9BNfITkdAJ5syhJZDu7mvcfT/wNtAzh3GPAsOArC/DcaCMmUUBpYD9wC4Ad08F1GOCzEZ+VzSvztQhiXRpfBZJU1fS45kZLNz4U6hLE5EwFUw4VAc2ZFneGFh3lJk1B2q6+8Rj9p0A7Aa2AOuBJ9w9mEAYaGYLA5eeKuY0wMzuNLM0M0vLyMgI4pCFX+WyMTzdtzkv3BTPjj37uWLkTP79yTJ+3a9GfiJSsIIJB8th3dE7p2YWAQwHhuYwriVwCDgbqAUMNbPauTzfc0AdoBmZofLfnAa5+2h3j3f3+NjY2FwnUZR0aliVKYMTufaimjyfuoZuI1KZvebHUJclImEkmHDYCNTMslwD2JxluRzQGPjSzNYCrYHkwE3p64BJ7n7A3bcBM4H4Ez2Zu29190Pufhh4gcyACTsVSkXz76ua8ubtrTjs0Gf0bP78/iJ+3nsg1KWJSBgIJhzmAvXMrJaZlQD6AMlHNrr7Tnev7O5x7h4HzAZ6uHsamZeSLrFMZcgMjuUnejIzq5Zl8Upg8fHGhoO2dSsz6f4O3N6+Fm99vZ7Ow1P5fPnWUJclIsVcruHg7geBgcBkYBkw3t2XmNkjZtYjl91HAmXJ/AE/Fxjj7gsBzOwtYBZwnpltNLPbAvsMC7z0dSHwO2DwqUysOCldIoq//L4h7w5oS7mSUdz6Shr3v/0t29XIT0TyiRWH9g3x8fGelpYW6jIKxP6Dhxn5RTrPfplOuZLRPNyjEZc3rYZZTreGRESOz8zmuXuOl/r1DukipkRUBIM71eeje9tTs2IpBr31LXe8Oo/vd6qRn4jkHYVDEdXgrPK8d3c7/tz9fGakZ9ApKYW3vl6vRn4ikicUDkVYZIRxR0JtJt2XQKPq5XnovUVc98Ic1v24O9SliUgRp3AoBuIql+HN21vz76uasHhTZiO/F6evUSM/ETllCodiIiLC6NvyHKYOSaR93cr88+NlXPXcV6z4Xo38ROTkKRyKmbMqlOSFm+J5qm9zNmzfw++fns7wqSvVyE9ETorCoRgyM3pccDbThiTSvUk1Rny2it8/PZ35G9TIT0SCo3AoxiqVKcGIPs15qV88u349yFXPzuSfE5eqkZ+I5ErhEAYuPb8qU4Yk0KflObw44zu6PJnKV6t/CHVZIlKIKRzCRPmS0fzryia8dUdrIgyue2EOD723kF1q5CciOVA4hJk2dc7k0/sS6J9Qm3FzN9ApKYVpS9XIT0T+P4VDGCpVIpKHup/PB/e0o2LpEtz+ahr3vvUtP/yyL9SliUghoXAIY01rnEHywPYM6VSfSYu30CkphQ++3aQWHCKicAh3JaIiGHRpPT4e1IFzzyzD/ePmc9vYNDb/9GuoSxOREFI4CAD1q5bj3QFt+evvGzJr9Y90Hp7K67PXcVgtOETCksJBjoqMMG5rX4vJ9ydwQc0K/OWDxfR9YTbf/aBGfiLhRuEg2ZxzZmlev60Vw65uytItu+j6ZCrPp6zm4CG14BAJFwoHyZGZcc1FNZk2JJGE+rH8+9PlXPnsVyzdvCvUpYlIAVA4yAlVLV+S0Te2YOR1F7Jl56/0eGYG/52ygn0H1YJDpDhTOEiuzIzLmlZj6uBEelxwNk9/ns5lT81g3rodoS5NRPKJwkGCVrFMCZKubcaYWy5iz76D9Br1Ff/4aAl79h8MdWkikscUDnLSfndeFaYMSeTG1ucyZuZaOg9PZcYqNfITKU4UDnJKysZE8UjPxozv34boyAhueGkOf5ywgJ2/qpGfSHGgcJDT0rJWJT69rwMDLq7Du99solNSCpOXfB/qskTkNAUVDmbW1cxWmFm6mT14gnG9zMzNLD6wHG1mY81skZktM7OHsox92cy2mdniY45RycymmtmqwOeKpzo5KRgloyP5U9cGfHB3O84sG0P/1+ZxzxvfkPGzGvmJFFW5hoOZRQIjgW5AQ6CvmTXMYVw5YBAwJ8vq3kCMuzcBWgD9zSwusO0VoGsOT/kg8Jm71wM+CyxLEdCkRgWSB7bjgS7nMXXpVjompfDuvI1q5CdSBAVz5tASSHf3Ne6+H3gb6JnDuEeBYcDeLOscKGNmUUApYD+wC8DdU4HtORynJzA28HgscEUQNUohER0ZwT2/q8sn97WnbpWyDH1nATePmcsmNfITKVKCCYfqwIYsyxsD644ys+ZATXefeMy+E4DdwBZgPfCEu+cUCFlVdfctAIHPVXIaZGZ3mlmamaVlZGQEMQ0pSHWrlOOd/m14+PKGzF27nc5JKbw6a60a+YkUEcGEg+Ww7uj/cDOLAIYDQ3MY1xI4BJwN1AKGmlntU6gzewHuo9093t3jY2Nj8+KQksciIoyb22U28rvw3Ir87cMlXDt6Fqszfgl1aSKSi2DCYSNQM8tyDWBzluVyQGPgSzNbC7QGkgM3pa8DJrn7AXffBswE4nN5vq1mVg0g8HlbMBORwqtmpdK8emtLHu/VlBXf/0y3EdN59st0DqiRn0ihFUw4zAXqmVktMysB9AGSj2x0953uXtnd49w9DpgN9HD3NDIvJV1imcqQGRzLc3m+ZKBf4HE/4MOTmpEUSmZG7/iaTBuayCXnVWHYpBVcMXImizftDHVpIpKDXMPB3Q8CA4HJwDJgvLsvMbNHzKxHLruPBMoCi8kMmTHuvhDAzN4CZgHnmdlGM7stsM9jQCczWwV0CixLMVGlXElG3diC566/kK279tFz5Ewen7ycvQfUyE+kMLHi8DLD+Ph4T0tLC3UZcpJ+2rOff368jAnzNlI7tgzDrm5KfFylUJclEjbMbJ6753ipX++QlpA5o3QJnuh9Aa/e2pJ9Bw7T+/lZPJy8hN371MhPJNQUDhJyCfVjmTI4gX5t4hg7K7ORX8pKvTxZJJQUDlIolImJ4uEejXinfxtioiPo9/LXDB2/gJ/27A91aSJhSeEghUp8XCU+GdSBe35Xhw/mb6JjUiqfLtoS6rJEwo7CQQqdktGRPNClAckD21G1fAwD3viGu16bx7Zde3PfWUTyhMJBCq1GZ1fgw3va8aeuDfh8xTY6JqXwTtoGNfITKQAKBynUoiIjGHBxHT69rwPnnVWOByYs5KaXv2bD9j2hLk2kWFM4SJFQJ7Ys4+5sw6M9G/HNuh10eTKVV2Z+xyE18hPJFwoHKTIiIowb28QxeXACF8VV4uGPlnLN87NI3/ZzqEsTKXYUDlLk1KhYmlduuYikay5gdcYvdB8xg2c+X6VGfiJ5SOEgRZKZcdWFNZg6OJFOjaryxJSV9HhGjfxE8orCQYq02HIxjLzuQp6/sQU//JLZyO+xT9XIT+R0KRykWOjS6CymDU6k14U1GJWymu4jpvP1d7n90UEROR6FgxQbFUpH859eTXn9tlbsP3SYa56fxV8/WMzPew+EujSRIkfhIMVO+3qVmTI4gVvb1eL1OevoMjyVL1boDwqKnAyFgxRLpUtE8bfLGzLhrraUiYniljFzGTJuPjt2q5GfSDAUDlKstTi3IhMHtWfQJXVJXrCZjkkpTFy4WS04RHKhcJBiLyYqkiGdz+Oje9tz9hmlGPjmt/R/bR5b1chP5LgUDhI2zq9WnvfvbstD3RqQsjKDjkkpjJu7XmcRIjlQOEhYiYqMoH9iHSbdn8D51crzp3cXcf2Lc1j/oxr5iWSlcJCwVKtyGd6+ozX/e2VjFm7cSZcnU3lphhr5iRyhcJCwFRFhXN/qXKYOSaBNnTN5dOJSrn7uK1ZuVSM/EYWDhL1qFUrxUr94RvRpxrofd3PZU9N56rNV7D+oRn4SvhQOImQ28uvZrDrThiTStXE1kqaupMczM1iw4adQlyYSEkGFg5l1NbMVZpZuZg+eYFwvM3Mziw8sR5vZWDNbZGbLzOyh3I5pZq+Y2XdmNj/w0ex0JihyMs4sG8PTfZvzwk3x7Niznyufncm/PlnGr/vVyE/CS67hYGaRwEigG9AQ6GtmDXMYVw4YBMzJsro3EOPuTYAWQH8ziwvimA+4e7PAx/xTnJvIKevUsCpThyRy7UU1GZ26hm4jUpm1+sdQlyVSYII5c2gJpLv7GnffD7wN9Mxh3KPAMCDrO4scKGNmUUApYD+w6ySOKRIy5UtG8++rmvLm7a047ND3hdn8z/uL2KVGfhIGggmH6sCGLMsbA+uOMrPmQE13n3jMvhOA3cAWYD3whLtvD+KY/2tmC81suJnFBDUTkXzStm5lJt+fwB0davH21+vpnJTK58u3hroskXwVTDhYDuuOvhjczCKA4cDQHMa1BA4BZwO1gKFmVjuXYz4ENAAuAioBf8qxKLM7zSzNzNIyMjKCmIbIqStVIpI/X9aQ9+5uR4VS0dz6Shr3vf0tP/6yL9SlieSLYMJhI1Azy3INYHOW5XJAY+BLM1sLtAaSAzelrwMmufsBd98GzATiT3RMd9/imfYBY8gMmGzcfbS7x7t7fGxsbBDTEDl9zWqewUf3tuf+jvX4ZNEWOg1PJXmBGvlJ8RNMOMwF6plZLTMrAfQBko9sdPed7l7Z3ePcPQ6YDfRw9zQyLyVdYpnKkBkcy090TDOrFvhswBXA4jyaq0ieKBEVwf0d6zPx3g7UrFSaQW99yx2vprFl56+hLk0kz+QaDu5+EBgITAaWAePdfYmZPWJmPXLZfSRQlswf8HOBMe6+8HjHDOzzhpktAhYBlYF/nsK8RPLdeWeV470BbfnLZeczI/0HOiel8uac9RxWCw4pBqw4nA7Hx8d7WlpaqMuQMLbux908+O4iZq35kda1K/HYVU2Jq1wm1GWJnJCZzXP3+Jy26R3SInng3DPL8OYdrXjsqiYs2bSLriNSeSF1jRr5SZGlcBDJI2ZGn5bnMHVIIu3rVuZ/P1nGVc/OZMX3auQnRY/CQSSPnVWhJC/cFM/TfZuzccev/P7p6QyfupJ9B9WCQ4oOhYNIPjAzLr/gbKYOSeSyJtUY8dkqLn96Bt+u3xHq0kSConAQyUeVypTgyT7NefnmeH7ee5CrnvuKRycuZc/+g6EuTeSEFA4iBeCSBlWZMjiB61udw0szvqPrk9P5Kv2HUJclclwKB5ECUq5kNP+8oglv39maCIPrXpzDg+8uZOevauQnhY/CQaSAta59JpPuT6B/Ym3Gp22g8/AUpi5VIz8pXBQOIiFQMjqSh7qdzwf3tKNi6RLc8WoaA9/8hh/UyE8KCYWDSAg1rXEGyQPbM7RTfaYs2UrHpBTe/3ajGvlJyCkcREKsRFQE915aj48HtadW5TIMHreAW1+Zy+af1MhPQkfhIFJI1Ktajgl3teVvv2/I7DXb6Tw8lddmr1MjPwkJhYNIIRIZYdzavhZTBifQrOYZ/PWDxfR5YTbf/bA71KVJmFE4iBRCNSuV5rXbWjLs6qYs27KLrk+mMiplNQcPHQ51aRImFA4ihZSZcc1FNZk2JJHE+rE89ulyrnz2K5Zu3hXq0iQMKBxECrmq5Uvy/I0tePb6C9my81d6PDOD/05ZoUZ+kq8UDiJFgJnRvUk1pg5OpEezs3n683Que2oG89apkZ/kD4WDSBFSsUwJkq5pxiu3XMSv+w/Ra9RX/OOjJezep0Z+krcUDiJF0MXnVWHy4ARubH0uY2aupcuTqUxflRHqsqQYUTiIFFFlY6J4pGdjxvdvQ4nICG586Wv+OGEBO/eokZ+cPoWDSBHXslYlPrmvAwMursO732yi4/AUJi3+PtRlSRGncBApBkpGR/Knrg348J52xJaN4a7X53H3G/PY9vPeUJcmRZTCQaQYaVy9Ah8ObMcDXc5j2rJtdEpK5d15auQnJ0/hIFLMREdGcM/v6vLJoA7UrVKWoe8soN+YuWzcsSfUpUkRonAQKabqVinLO/3b8I8ejUhbu50uw1N5ddZaNfKToAQVDmbW1cxWmFm6mT14gnG9zMzNLD6wHG1mY81skZktM7OHcjummdUyszlmtsrMxplZidOZoEg4i4gw+rWNY/L9CVx4bkX+9uESrh09i9UZv4S6NCnkcg0HM4sERgLdgIZAXzNrmMO4csAgYE6W1b2BGHdvArQA+ptZXC7H/A8w3N3rATuA2051ciKSqWal0rx6a0ue6H0BK7f+QrcR0xn5RToH1MhPjiOYM4eWQLq7r3H3/cDbQM8cxj0KDAOyvjzCgTJmFgWUAvYDu453TDMz4BJgQmD/scAVJz8tETmWmdGrRQ2mDkmg4/lVeHzyCq4YOZPFm3aGujQphIIJh+rAhizLGwPrjjKz5kBNd594zL4TgN3AFmA98IS7bz/BMc8EfnL3g8esz8bM7jSzNDNLy8jQO0NFglWlXEmevb4Fo264kK279tFz5EyGTVrO3gNq5Ce/CSYcLId1R+9omVkEMBwYmsO4lsAh4GygFjDUzGqf4JgnfK7/t9J9tLvHu3t8bGzsiWcgItl0bVyNz4YkclXz6jz75Wq6PzWdtLXbQ12WFBLBhMNGoGaW5RrA5izL5YDGwJdmthZoDSQHbkpfB0xy9wPuvg2YCcSf4Jg/AGcELkPl9FwikocqlI7m8d4X8OqtLdl34DC9n5/F3z9czC9q5Bf2ggmHuUC9wKuISgB9gOQjG919p7tXdvc4d48DZgM93D2NzEtJl1imMmQGx/LjHdMz36nzBdArcPh+wId5MlMROa6E+rFMGZxAvzZxvDp7HV2Gp5KyUpdrw1mu4RC4/j8QmAwsA8a7+xIze8TMeuSy+0igLLCYzEAY4+4Lj3fMwD5/AoaYWTqZ9yBeOoV5ichJKhMTxcM9GjHhrjaUjI6g38tfM2T8fH7asz/UpUkIWHF4W318fLynpaWFugyRYmPvgUM883k6o1JWc0bpaB7p2ZjuTaqFuizJY2Y2z93jc9qmd0iLSDYloyP5Q5fz+HBgO86qUJK73/iGu16bx7ZdauQXLhQOInJcjc6uwAd3t+NPXRvw+YptdExKYXzaBjXyCwMKBxE5oajICAZcXIdJ93WgwVnl+eOEhdz08tds2K5GfsWZwkFEglI7tixv39maR3s24pt1O+jyZCpjZn7HITXyK5YUDiIStIgI48Y2cUwZkkjLWpX4x0dL6T3qK9K3/Rzq0iSPKRxE5KRVP6MUY26+iOHXXsCaH3bTfcQMnvl8lRr5FSMKBxE5JWbGlc1rMG1IIp0aVeWJKSu5/OkZLNqoRn7FgcJBRE5L5bIxjLzuQp6/sQXbd+/nimdn8tinauRX1CkcRCRPdGl0FlOHJNLrwhqMSllNtxHTmbPmx1CXJadI4SAieaZCqWj+06spb9zeioOHD3Pt6Nn85YNF/Lz3QKhLk5OkcBCRPNeubmUm35/Abe1r8cac9XQZnsoXy7eFuiw5CQoHEckXpUtE8dffN+TdAW0pExPFLa/MZfC4+WzfrUZ+RYHCQUTy1YXnVGTioPYMurQeHy3YTKekFCYu3KwWHIWcwkFE8l1MVCRDOtXno3vbU71iKQa++S13vjaPrWrkV2gpHESkwJxfrTzvDWjL/3RvQOrKDDompfD21+t1FlEIKRxEpEBFRUZwZ0IdJt+fQMNq5XnwvUVc/+Ic1v+oRn6FicJBREIirnIZ3rqjNf+6sgkLN+6k85MpvDh9jRr5FRIKBxEJmYgI47pW5zB1SAJt61Tmnx8v4+rnvmLlVjXyCzWFg4iEXLUKpXipXzwj+jRj/fY9XPbUdEZMW8X+g2rkFyoKBxEpFMyMns2qM3VwAt0aV2P4tJX0eGYGCzb8FOrSwpLCQUQKlTPLxvBU3+a8eFM8P+05wJXPzuRfnyzj1/1q5FeQFA4iUih1bFiVKUMS6NPyHEanrqHriFRmrVYjv4KicBCRQqt8yWj+dWUT3ryjFQB9X5jNQ+8tYpca+eU7hYOIFHpt61Rm0n0J3JlQm3Fz19M5KZXPlm0NdVnFWlDhYGZdzWyFmaWb2YMnGNfLzNzM4gPL15vZ/Cwfh82sWWDbtWa20MyWmNmwLMe42cwysuxz++lOUkSKvlIlIvmf7ufz3t3tqFAqmtvGpjHorW/58Zd9oS6tWMo1HMwsEhgJdAMaAn3NrGEO48oBg4A5R9a5+xvu3szdmwE3Amvdfb6ZnQk8Dlzq7o2AqmZ2aZbDjTuyn7u/eDoTFJHipVnNM/jo3vYM7lifTxdvodPwVD6cv0ktOPJYMGcOLYF0d1/j7vuBt4GeOYx7FBgGHK+TVl/grcDj2sBKd88ILE8Drg66ahEJayWiIrivYz0+HtSBcyqV5r6353P72DS27Pw11KUVG8GEQ3VgQ5bljYF1R5lZc6Cmu088wXGu5bdwSAcamFmcmUUBVwA1s4y9OnDJaYKZ1Tz2QCIiAPWrluPdAW35y2XnM3P1D3RKSuWNOes4rBYcpy2YcLAc1h39lzezCGA4MPS4BzBrBexx98UA7r4DGACMA6YDa4GDgeEfAXHu3pTMM4qxxznmnWaWZmZpGRkZOQ0RkTAQGWHc3qE2U+5PpGmNCvz5/cVc9+Js1v6wO9SlFWnBhMNG/v9v9TWAzVmWywGNgS/NbC3QGkg+clM6oA+/nTUA4O4fuXsrd28DrABWBdb/6O5H7jC9ALTIqSh3H+3u8e4eHxsbG8Q0RKQ4O+fM0rxxeyseu6oJSzbtosuTqYxOXc3BQ2rBcSqCCYe5QD0zq2VmJcj8QZ98ZKO773T3yu4e5+5xwGygh7unwdEzi95k3qs4ysyqBD5XBO4GXgwsV8syrAew7BTnJiJhxszo0/Icpg5JpEO9WP71yXKufu4rln+/K9SlFTm5hoO7HwQGAhaTnWwAAAsOSURBVJPJ/EE93t2XmNkjZtYjiOdIADa6+5pj1o8ws6XATOAxd18ZWD8o8PLWBWS++unmIOciIgLAWRVK8sJNLXjmuuZs3PErv39qBklTV7LvoFpwBMuKw8u/4uPjPS0tLdRliEghtGP3fh6ZuJT3v91E/apl+c/VTWl+TsVQl1UomNk8d4/PaZveIS0ixVrFMiUYfm0zxtx8ET/vPchVz33FoxOXsmf/wdx3DmMKBxEJC79rUIUpgxO4vtU5vDTjO7o8mcrM9B9CXVahpXAQkbBRrmQ0/7yiCePubE1URATXvziHB99dyM5f1cjvWAoHEQk7rWqfyaf3daB/Ym3Gp22gU1IKU5Z8H+qyChWFg4iEpZLRkTzU7Xw+uKcdlcqU4M7X5jHwzW/4QY38AIWDiIS5pjUyG/n9oXN9pizZSsekFN7/dmPYN/JTOIhI2IuOjGDgJfX45L721K5chsHjFnDLK3PZ9FP4NvJTOIiIBNStUo537mrL3y9vyJw12+mclMJrs8OzkZ/CQUQki8gI45Z2tZgyOIHm51Tkrx8sps/o2azJ+CXUpRUohYOISA5qVirNa7e1ZFivpiz/fhfdRkxnVEr4NPJTOIiIHIeZcU18TaYNSeTi82J57NPlXPHsTJZuLv6N/BQOIiK5qFK+JM/fGM9z11/I9zv30eOZGTwxeQV7DxTfRn4KBxGRIHVrUo1pQxLo2aw6z3yRzmVPTWfeuu2hLitfKBxERE7CGaVL8N9rLmDsrS3Ze+AwvUbN4uHkJezeV7wa+SkcREROQWL9WCYPTuCm1ufyyldr6fJkKtNXFZ8/WaxwEBE5RWVjovhHz8a8c1cbSkRFcONLX/PAOwvYuafoN/JTOIiInKaL4irxyaAO3H1xHd77dhMdh6cwafGWUJd1WhQOIiJ5oGR0JH/s2oAP72lHbNkY7nr9Gwa8Po9tP+8NdWmnROEgIpKHGlevwIcD2/FAl/P4bPk2OiWlMmFe0Wvkp3AQEclj0ZER3PO7unwyqAP1qpTlD+8soN+YuWzcsSfUpQVN4SAikk/qVinL+P5t+EePRqSt3U7n4amM/WptkWjkp3AQEclHERFGv7ZxTBmcQHxcJf6evIRrnp9F+rbC3chP4SAiUgBqVCzN2Fsu4r+9L2DVtl/oPmI6I79I50AhbeSncBARKSBmxtUtajBtSCIdG1bh8ckr6PnMTBZv2hnq0rJROIiIFLDYcjE8e30LRt1wIRm/7KPnyJn8Z9LyQtXIL6hwMLOuZrbCzNLN7METjOtlZm5m8YHl681sfpaPw2bWLLDtWjNbaGZLzGxYlmPEmNm4wHPNMbO405uiiEjh1LVxNaYNTuSq5tV57svVdB8xnblrC0cjv1zDwcwigZFAN6Ah0NfMGuYwrhwwCJhzZJ27v+Huzdy9GXAjsNbd55vZmcDjwKXu3gioamaXBna7Ddjh7nWB4cB/TmuGIiKFWIXS0Tze+wJeu60l+w8dpveoWfztw8X8EuJGfsGcObQE0t19jbvvB94GeuYw7lFgGHC8twP2Bd4KPK4NrHT3I12qpgFXBx73BMYGHk8ALjUzC6JOEZEiq0O9WCbfn8At7eJ4bfY6ugxP5csV20JWTzDhUB3YkGV5Y2DdUWbWHKjp7hNPcJxr+S0c0oEGZhZnZlHAFUDNY5/P3Q8CO4Ezjz2Ymd1pZmlmlpaRUXw6IYpI+CoTE8XfL2/EhLvaUqpEJDePmcuQ8fPZsXt/gdcSTDjk9Fv70XdwmFkEmZd/hh73AGatgD3uvhjA3XcAA4BxwHRgLXDkHOqEz3d0hftod4939/jY2NggpiEiUjS0OLciHw9qz72X1CV5/mY6DU/hk0VbCrQFRzDhsJHffqsHqAFszrJcDmgMfGlma4HWQPKRm9IBffjtrAEAd//I3Vu5extgBbDq2OcLnFVUAArHHRoRkQISExXJ0M7nkTywPdUqlOLuN77hrtfnsW1XwTTyCyYc5gL1zKyWmZUg8wd98pGN7r7T3Su7e5y7xwGzgR7ungZHzyx6k3mv4igzqxL4XBG4G3gxsCkZ6Bd43Av43ItaxyoRkTzS8OzyvH93Wx7s1oAvV2TQMSmF8Wkb8v0sItdwCFz3HwhMBpYB4919iZk9YmY9gniOBGCju685Zv0IM1sKzAQec/eVgfUvAWeaWTowBDjuS2dFRMJBVGQEdyXW4dP7OtCgWnn+OGEhN770NRu2518jPysOv5THx8d7WlpaqMsQEcl3hw87b369nsc+Xc6hw86wXk25/IKzT+lYZjbP3eNz2qZ3SIuIFCEREcYNrc9lyuAE2tWtTK3KZfLleaLy5agiIpKvzj6jFC/2y/GX/jyhMwcREclG4SAiItkoHEREJBuFg4iIZKNwEBGRbBQOIiKSjcJBRESyUTiIiEg2xaJ9hpllAOtOcffKwA95WE5RoDmHB805PJzOnM919xz/5kGxCIfTYWZpx+stUlxpzuFBcw4P+TVnXVYSEZFsFA4iIpKNwgFGh7qAENCcw4PmHB7yZc5hf89BRESy05mDiIhkEzbhYGZdzWyFmaWbWbY/PWpmMWY2LrB9jpnFFXyVeSuIOQ8xs6VmttDMPjOzc0NRZ17Kbc5ZxvUyMzezIv/KlmDmbGbXBL7WS8zszYKuMa8F8b19jpl9YWbfBr6/u4eizrxiZi+b2TYzW3yc7WZmTwX+PRaa2YWn/aTuXuw/gEhgNVAbKAEsABoeM+ZuYFTgcR9gXKjrLoA5/w4oHXg8IBzmHBhXDkgFZgPxoa67AL7O9YBvgYqB5SqhrrsA5jwaGBB43BBYG+q6T3POCcCFwOLjbO8OfAoY0BqYc7rPGS5nDi2BdHdf4+77gbeBnseM6QmMDTyeAFxqZlaANea1XOfs7l+4+5G/UD4bqFHANea1YL7OAI8Cw4C9BVlcPglmzncAI919B4C7byvgGvNaMHN2oHzgcQVgcwHWl+fcPRXYfoIhPYFXPdNs4Awzq3Y6zxku4VAd2JBleWNgXY5j3P0gsBM4s0Cqyx/BzDmr28j8zaMoy3XOZtYcqOnuEwuysHwUzNe5PlDfzGaa2Wwz61pg1eWPYOb8MHCDmW0EPgHuLZjSQuZk/7/nKlz+hnROZwDHvkwrmDFFSdDzMbMbgHggMV8ryn8nnLOZRQDDgZsLqqACEMzXOYrMS0sXk3l2ON3MGrv7T/lcW34JZs59gVfc/b9m1gZ4LTDnw/lfXkjk+c+vcDlz2AjUzLJcg+ynmUfHmFkUmaeiJzqNK+yCmTNm1hH4M9DD3fcVUG35Jbc5lwMaA1+a2Voyr80mF/Gb0sF+b3/o7gfc/TtgBZlhUVQFM+fbgPEA7j4LKElmD6LiKqj/7ycjXMJhLlDPzGqZWQkybzgnHzMmGegXeNwL+NwDd3qKqFznHLjE8jyZwVDUr0NDLnN2953uXtnd49w9jsz7LD3cPS005eaJYL63PyDzxQeYWWUyLzOtKdAq81Ywc14PXApgZueTGQ4ZBVplwUoGbgq8aqk1sNPdt5zOAcPispK7HzSzgcBkMl/p8LK7LzGzR4A0d08GXiLz1DOdzDOGPqGr+PQFOefHgbLAO4F77+vdvUfIij5NQc65WAlyzpOBzma2FDgEPODuP4au6tMT5JyHAi+Y2WAyL6/cXJR/2TOzt8i8LFg5cB/l70A0gLuPIvO+SncgHdgD3HLaz1mE/71ERCSfhMtlJREROQkKBxERyUbhICIi2SgcREQkG4WDiIhko3AQEZFsFA4iIpKNwkFERLL5P3jspNxGm0yhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hUZdrH8e+dSq8JiPTee+iQ6EpoKiCioghiAUURTLao777uurrVXUNRBEFBxQKIBbDQXE3oEHrvHZQISJUSeN4/MvpmWSQDKSfJ/D7XNRdzzjznzP2QZH5zzpncMeccIiISeIK8LkBERLyhABARCVAKABGRAKUAEBEJUAoAEZEAFeJ1AdciIiLCValSxesyRETylBUrVnzvnIu8fH2eCoAqVaqQnJzsdRkiInmKme250nqdAhIRCVAKABGRAKUAEBEJUAoAEZEApQAQEQlQCgARkQClABARCVABEQDvLN5N4tYUr8sQEclV8tQvgl2PCxcv8f7SvWz+9iR3NqvAc7fVpUShMK/LEhHxXL4/AggNDuLTJ9ox5OYafLr6AB0Tkvhy3SGvyxIR8Vy+DwCAAqHB/KZzbWYMaUfZYuEMfm8lj01aweETZ70uTUTEMwERAD+pf2Nxpj/Rjqe71OHfWw7TMSGRqcn70J/FFJFAFFABABASHMTgm6rz5bAO1L6hKL+btpb+E5ax7+gZr0sTEclRARcAP6keWYQpg9rwYo/6rNxzjM4jkpi4cBcXL+loQEQCQ8AGAEBQkNGvTRXmxMfQokop/jRzI3e/vpjth096XZqISLYL6AD4SfkSBXnrwRYk3N2YHSmn6DZyAa/+exsXLl7yujQRkWyjAPAxM3o1q8DcuBhi65flX3O20v3Vhazbf9zr0kREsoUC4DKRRcMZfV8zXu/XnO9PnaPnawv5+5ebOXvhoteliYhkKQXAL+hc/wbmxcXQu1kFxibuoOvI+SzdecTrskREsowC4CqKFwrlH70b8e7Drbhw8RL3jFvCc5+u5+TZC16XJiKSaQoAP7SvGcGcuGgealeVd5fuofPwJL7ectjrskREMkUB4KdCYSH84fZ6fDS4LYXDQ3hw4nLipqzm6OnzXpcmInJdFADXqFmlknw2tD1Df1WDmWsOEpuQyGdrD6qdhIjkOQqA6xAeEkx8p9rMfLI9N5YoyJD3VzFo0gq+U3M5EclDFACZULdcMT55vC3Pdq1D0tYUOiYkMmX5Xh0NiEieoADIpJDgIB6Nqc6sp6KpW64YT3+0jr5vLGXvETWXE5HcTQGQRapGFGbywNb85Y4GrN1/nM4jknhzgZrLiUjupQDIQkFBRt9WlZkbH02b6qV58bON3DlmEVu/U3M5Ecl9FADZoFzxgrz5QBQj+zRhz5HT3DpqPiPnbeN8qprLiUjuoQDIJmZGjyblmRcfQ5cG5Rg+byvdX13Amn0/eF2aiAigAMh2pYuE88q9TRnfP4pjZ85zx2sL+esXm/jxvJrLiYi3FAA5JLZeWebGx3BPi0qMS9pJ15FJLN6h5nIi4h0FQA4qViCUv/VqyPsDW+GAe8cv4X8+WccJNZcTEQ8oADzQtnoEs4ZFM7BDVSYv20unhCS+2vSd12WJSIBRAHikYFgwv7+1Hh8/3o7iBUN5+O1khn6wiiOnznldmogECAWAx5pULMHMJ9vzVMeafLn+ELHDk5i++oDaSYhItlMA5AJhIUE81bEWnz3ZgYqlCjFs8moeeTuZQ8d/9Lo0EcnH/AoAM+tiZlvMbLuZPXOFxweYWYqZrfbdHkn32ANmts13eyDd+m98+/xpmzJZM6W8q/YNRfl4cFv+99a6LNzxPZ0Sknh/6V4uqZ2EiGQDy+hUg5kFA1uBWGA/sBy41zm3Md2YAUCUc27IZduWApKBKMABK4DmzrljZvYN8BvnXLK/xUZFRbnkZL+H52l7jpzmmY/WsXjnEVpXK8XfezWiSkRhr8sSkTzIzFY456IuX+/PEUBLYLtzbqdz7jwwGejh5/N2BuY65446544Bc4Eu/hYdyCqXLsz7A1vx914N2XDgBF1GJjE+aSepF9VOQkSyhj8BUB7Yl255v2/d5e40s7VmNs3MKvq57UTf6Z/nzMyu9ORmNsjMks0sOSUlxY9y8w8zo0/LSsyNj6F9jQj+8sUm7hyziM3fnvC6NBHJB/wJgCu9MF9+3mgmUMU51wiYB7ztx7Z9nXMNgQ6+W78rPblzbpxzLso5FxUZGelHufnPDcULML5/FK/c25T9x37ktlELSJi7lXOpaichItfPnwDYD1RMt1wBOJh+gHPuiHPupw+wjweaZ7Stc+6A79+TwPuknWqSX2Bm3N74RubGx3Bbo3KM+mobt7+ygFV7j3ldmojkUf4EwHKgpplVNbMwoA8wI/0AMyuXbrE7sMl3fzbQycxKmllJoBMw28xCzCzCt20ocBuwPnNTCQylCocxok9TJgyI4uTZVHqNWcSLn23kzPlUr0sTkTwmJKMBzrlUMxtC2ot5MDDBObfBzF4Akp1zM4ChZtYdSAWOAgN82x41sxdJCxGAF3zrCpMWBKG+fc4j7chB/PSrOmWZE1eKf8zazJsLdjF343f8vVdD2taI8Lo0EckjMvwYaG4SSB8DvRZLdh7hmY/WsvvIGfq0qMiz3epSvGCo12WJSC6RmY+BSi7XulppZj0VzaMx1ZiavI/YhETmbPjW67JEJJdTAOQTBUKDebZrXT59oh2lCocxaNIKhry/ku/VXE5EfoECIJ9pVKEEM4a059extZiz4Ts6JiTyyar9ai4nIv9FAZAPhYUE8eQtNfl8aHuqRhQmbsoaHnprOQd/UHM5Efl/CoB8rGbZokx7rC1/uK0eS3YepdPwJCYt2aPmciICKADyveAg46H2VZkTF02TiiV47tP19Bm3hJ0pp7wuTUQ8pgAIEBVLFWLSwy156c5GbPr2BF1Hzmds4g41lxMJYAqAAGJm3N2iIvPiY4ipFcnfv9xMz9cWsvGgmsuJBCIFQAAqW6wAr/drzmt9m/Ht8bN0f3UBL8/ZouZyIgFGARCgzIxuDcsxNy6G7k1u5JV/b+fWUQtYsUfN5UQChQIgwJUsHEbC3U1468EW/Hj+Ir3HLuJPMzdw+pyay4nkdwoAAeCm2mWYHRdNv9aVmbhwN51HJDF/W2D9AR6RQKMAkJ8VCQ/hhR4NmPpoG8KCg+j35jJ+++Eajp+54HVpIpINFADyX1pWLcUXwzrw+E3V+XjVAToOT2TWejWXE8lvFAByRQVCg/ldlzpMf6IdkUXCeezdFTz+3goOnzzrdWkikkUUAHJVDcoXZ/qQdvy2c23mbTpMbEISH61QczmR/EABIBkKDQ7iiZtr8MXQDtQoU4Rff7iGByYuZ/+xM16XJiKZoAAQv9UoU4QPH23Dn7rXJ3l3WnO5txftVnM5kTxKASDXJCjIeKBtFWY/FU3zyiX544wN3P36YnaouZxInqMAkOtSsVQh3nmoJf+6qzHbDp+i68j5jP56OxfUXE4kz1AAyHUzM3o3r8Dc+Gg61i3DP2dvoefohaw/cNzr0kTEDwoAybQyRQvwWt/mjL2/Gd+dOEeP0Qt5adZmzl5QczmR3EwBIFmmS4NyfBUfQ6+m5Xntmx10Gzmf5buPel2WiPwCBYBkqeKFQvnnXY1556GWnEu9xF1jF/OH6es5peZyIrmOAkCyRXStSObERTOgbRUmLdlD5+FJJG5VczmR3EQBINmmcHgIz3evz7TH2lAgNIgHJiwjfupqfjhz3uvSRAQFgOSA5pVL8fnQDgy5uQYzVh+kY0IiX6w75HVZIgFPASA5okBoML/pXJvpQ9pxQ/ECPP7eSh6dlMzhE2ouJ+IVBYDkqPo3FufTx9vxdJc6fL0lhY4JiUxN3qfmciIeUABIjgsJDmLwTdWZNawDdW4oxu+mraXfm8vYd1TN5URykgJAPFMtsgiTB7XmxZ4NWLX3GJ2GJzFx4S4uqrmcSI5QAIingoKMfq0rMyc+hlbVSvGnmRu5a+with8+6XVpIvmeAkByhfIlCjJxQAuG39OYnd+fptvIBbz6721qLieSjRQAkmuYGXc0rcC8+Bhi65flX3O2cvsrC1i3X83lRLKDAkBynYgi4Yy+rxmv92vO0dPn6TF6AX/7cpOay4lkMQWA5Fqd69/A3PgY7o6qyOuJO+k6cj5Ldx7xuiyRfEMBILla8YKh/P3ORrz3SCtSL13innFL+N9P13Hy7AWvSxPJ8xQAkie0qxHB7Keiebh9Vd5bupfOw5P4evNhr8sSydP8CgAz62JmW8xsu5k9c4XHB5hZipmt9t0eSffYA2a2zXd7IN365ma2zrfPUWZmWTMlya8KhYXw3G31+GhwWwqHh/DgW8uJm7Kao6fVXE7kemQYAGYWDIwGugL1gHvNrN4Vhk5xzjXx3d7wbVsK+CPQCmgJ/NHMSvrGjwEGATV9ty6ZnYwEhmaVSvLZ0PYMvaUmM9ccJDYhkZlrDqqdhMg18ucIoCWw3Tm30zl3HpgM9PBz/52Buc65o865Y8BcoIuZlQOKOecWu7Sf2neAntdRvwSo8JBg4mNrMfPJ9pQvWZAnP1jFwHdW8J2ay4n4zZ8AKA/sS7e837fucnea2Vozm2ZmFTPYtrzvfkb7xMwGmVmymSWnpOgPish/qluuGB8Pbsv/dKvD/G1pzeUmL9urowERP/gTAFc6N3/5T9dMoIpzrhEwD3g7g2392WfaSufGOeeinHNRkZGRfpQrgSYkOIhB0dWZ/VQ09coV45mP19H3jaXsPaLmciJX408A7AcqpluuABxMP8A5d8Q5d863OB5onsG2+333f3GfIteqSkRhPhjYmr/e0ZC1+4/TaUQib8zfqeZyIr/AnwBYDtQ0s6pmFgb0AWakH+A7p/+T7sAm3/3ZQCczK+m7+NsJmO2cOwScNLPWvk//9AemZ3IuIgQFGfe1qsTc+GjaVo/gz59v4s4xi9jyrZrLiVwuwwBwzqUCQ0h7Md8ETHXObTCzF8ysu2/YUDPbYGZrgKHAAN+2R4EXSQuR5cALvnUAg4E3gO3ADuDLLJuVBLxyxQvy5gNRjOzThL1Hz3DbK/MZMW8r51PVXE7kJ5aXLpZFRUW55ORkr8uQPObIqXO88NlGpq8+SO2yRXmpdyMaVyzhdVkiOcbMVjjnoi5fr98ElnyvdJFwRvZpyhv9ozj+4wXueG0hf/l8Iz+eV3M5CWwKAAkYHeuVZU58NH1aVmL8/F10GZnE4h1qLieBSwEgAaVYgVD+ekdD3h/YCoB7xy/h2Y/XcULN5SQAKQAkILWtHsGsYdEMiq7GlOV76ZSQxLyN33ldlkiOUgBIwCoYFsz/dKvLx4+3o3jBUB55J5mhH6ziyKlzGW8skg8oACTgNalYgplPtieuYy2+XH+IjgmJTF99QO0kJN9TAIgAYSFBDOtYk8+HdqBy6cIMm7yaR95O5tDxH70uTSTbKABE0qlVtigfDW7L/95al4U7vic2IYn3lu7hktpJSD6kABC5THCQ8UiHasx5KoZGFYrz+0/Wc98bS9j9/WmvSxPJUgoAkV9QqXQh3nukFX/v1ZANB07QeUQS45J2kHpR7SQkf1AAiFyFmdGnZSXmxsfQoWYkf/1iM73GLGLToRNelyaSaQoAET/cULwA4/s359X7mnLg2I/c/soCEuZu5Vyq2klI3qUAEPGTmXFboxuZFx/D7Y1vZNRX27ht1AJW7j3mdWki10UBIHKNShYOY/g9TZg4oAWnzqVy55hFvPjZRs6cT/W6NJFrogAQuU431ynDnLho+raqxJsLdtF5RBILt3/vdVkiflMAiGRC0QKh/LlnQ6YMak1IUBB931jK09PWcvxHNZeT3E8BIJIFWlUrzZfDOvBYTHWmrdxPbEIiczZ863VZIlelABDJIgVCg3mmax0+fbwdpYuEM2jSCp54fyUpJ9VcTnInBYBIFmtYoTgzhrTjN51qMXfDd8QOT+STVfvVXE5yHQWASDYIDQ5iyK9q8sWw9lSLKEzclDU8+NZyDvyg5nKSeygARLJRjTJF+fCxtvzx9nos3XmUTgmJTFq8W83lJFdQAIhks+Ag48F2VZkTF03TSiV5bvoG+oxbws6UU16XJgFOASCSQyqWKsSkh1vyUu9GbP72BF1GzmfMN2ouJ95RAIjkIDPj7qiKzIuP4ebakfxj1mZ6vraQjQfVXE5yngJAxANlihXg9X5RjOnbjG+Pn6P7qwv41+wtnL2g5nKScxQAIh7q2rAc8+Kj6dGkPK9+vZ1bR81nxZ6jXpclAUIBIOKxEoXCePnuxrz9UEvOXrhE77GLeX7GBk6fU3M5yV4KAJFcIqZWJLPjounfujJvL95Np+FJJG1N8bosyccUACK5SJHwEP7UowFTH21DeGgQ/Scs4zcfruH4GTWXk6ynABDJhVpUKcUXQzvw+E3V+WTVAToOT2TW+kNelyX5jAJAJJcqEBrM77rUYfoT7YgsEs5j765k8LsrOHzyrNelST6hABDJ5RqUL870Ie34befafLX5MLEJSUxboeZyknkKAJE8IDQ4iCdursEXQztQs0wRfvPhGvpPWMa+o2e8Lk3yMAWASB5So0wRpj7ahhd61GflnmN0HpHEWwt3qbmcXBcFgEgeExRk9G9Thdlx0URVKcXzMzdy9+uL2X5YzeXk2igARPKoCiUL8faDLXj5rsZsO3yKbiPnM/rr7VxQcznxkwJAJA8zM+5sXoF58TF0rFeGf87eQo9XF7L+wHGvS5M8wK8AMLMuZrbFzLab2TNXGdfbzJyZRfmWw8xsopmtM7M1ZnZTurHf+Pa52ncrk+nZiASoyKLhvNa3OWPvb0bKqXP0GL2Qf8zarOZyclUZBoCZBQOjga5APeBeM6t3hXFFgaHA0nSrBwI45xoCscDLZpb+Ofs655r4boevfxoiAtClQTnmxcVwZ7PyjPlmB91Gzmf5bjWXkyvz5wigJbDdObfTOXcemAz0uMK4F4GXgPS/pVIP+ArA9wL/AxCVqYpF5KqKFwrlpd6NeffhVpy/eIm7xi7mD9PXc0rN5eQy/gRAeWBfuuX9vnU/M7OmQEXn3GeXbbsG6GFmIWZWFWgOVEz3+ETf6Z/nzMyuvXwR+SXta0Yw+6loHmxXhUlL9tB5eBLfbNGBtvw/fwLgSi/MP3/o2HdKZzjw6yuMm0BaYCQDI4BFwE9vQ/r6Tg118N36XfHJzQaZWbKZJaekqDOiyLUoHB7CH2+vz7TH2lIwLJgBE5cTP3U1x06f97o0yQX8CYD9/Oe79grAwXTLRYEGwDdmthtoDcwwsyjnXKpzLs53jr8HUALYBuCcO+D79yTwPmmnmv6Lc26ccy7KORcVGRl5bbMTEQCaVy7J50Pb8+SvajBj9UFihyfyxbpDaicR4PwJgOVATTOramZhQB9gxk8POueOO+cinHNVnHNVgCVAd+dcspkVMrPCAGYWC6Q65zb6TglF+NaHArcB67N2aiKSXnhIML/uVJsZQ9pTrnhBHn9vJY+9u4LDJ9RcLlBlGADOuVRgCDAb2ARMdc5tMLMXzKx7BpuXAVaa2Sbgaf7/NE84MNvM1gKrgQPA+Oucg4hcg3o3FuOTx9vybNc6fLMlhVsSEpm6fJ+OBgKQ5aUvelRUlEtOTva6DJF8Y2fKKZ75eB3Ldh2lfY0I/tarIRVLFfK6LMliZrbCOfdfn8DUbwKLBLBqkUWYPLA1f+7ZgNX7fqDT8CQmLNjFRTWXCwgKAJEAFxRk3N+6MnPiomlVrRQvfLaRu8YuYtt3J70uTbKZAkBEALixREEmDmjBiHuasOv709w6agGvfLVNzeXyMQWAiPzMzOjZtDxz42PoVL8sL8/dyu2vLGDt/h+8Lk2ygQJARP5LRJFwXr2vGeP6NefYmfP0HL2Qv32xSc3l8hkFgIj8ok71b2BOXAz3tKjI60k76TIiiSU7j3hdlmQRBYCIXFXxgqH8rVcj3n+kFZcc9Bm3hN9/so6TZy94XZpkkgJARPzStkYEs57qwCPtq/LBsr10Gp7E15vVXC4vUwCIiN8KhYXwv7fV46PBbSkSHsKDby3nqcmrOKrmcnmSAkBErlnTSiX5bGh7ht1Sk8/XHSI2IZGZaw6qnUQeowAQkesSHhJMXGwtZj7ZngolC/LkB6sY+M4Kvj2u5nJ5hQJARDKlzg3F+Pjxdvy+W10WbE8hNiGRD5bt1dFAHqAAEJFMCw4yBkZXY9awaOqXL8azH6/jvvFL2XPktNelyVUoAEQky1SJKMz7j7Tmr3c0ZP2B43QekcQb83equVwupQAQkSwVFGTc16oSc+KjaVc9gj9/voleYxax5Vs1l8ttFAAiki3KFS/IGw9EMerepuw7eobbXpnPiHlbOZ+q5nK5hQJARLKNmdG98Y3Mi4+hW8NyjJi3jdtfWcDqfWoulxsoAEQk25UqHMbIPk1584Eojv94gV6vLeQvn2/kx/NqLuclBYCI5Jhb6pZlTnw0fVpWYvz8XXQekcSiHd97XVbAUgCISI4qViCUv97RkA8GtsYM7hu/lGc/XscJNZfLcQoAEfFEm+qlmTUsmkejqzFl+V5iExKZt/E7r8sKKAoAEfFMwbBgnu1Wl0+faEfJQmE88k4yT36wiiOnznldWkBQAIiI5xpVKMGMIe2Jj63FrPWH6JiQyPTVB9ROIpspAEQkVwgLCWLoLTX5fGgHKpcuzLDJq3n47WQO/vCj16XlWwoAEclVapUtykeD2/LcbfVYvOMInYYn8d7SPVxSO4kspwAQkVwnOMh4uH1VZj8VTeOKxfn9J+u5d/wSdn2v5nJZSQEgIrlWpdKFePfhVvzjzoZsPHSCLiOSeD1xB6kX1U4iKygARCRXMzPuaVGJefExRNeK5G9fbqbXmEVsOnTC69LyPAWAiOQJZYsVYFy/5oy+rxkHf/iR219ZQMKcLZxLVTuJ66UAEJE8w8y4tVE55sbF0L3xjYz693ZuG7WAlXuPeV1anqQAEJE8p2ThMBLuacLEB1tw+lwqd45ZxAszN3LmfKrXpeUpCgARybNurl2G2XHR3N+qMhMWpjWXW7hdzeX8pQAQkTytaIFQXuzZgKmPtiEkKIi+byzl6WlrOf6jmstlRAEgIvlCy6ql+HJYBwbfVJ1pK/cTm5DI7A3fel1WrqYAEJF8o0BoME93qcOnj7ejdJFwHp20gifeW0nKSTWXuxIFgIjkOw0rFGfGkHb8tnNt5m78jtjhiXy8cr+ay11GASAi+VJocBBP3FyDL4a1p1pEYeKnruHBt5ZzQM3lfqYAEJF8rUaZonz4WFuev70ey3YdpVNCIpMW71ZzOfwMADPrYmZbzGy7mT1zlXG9zcyZWZRvOczMJprZOjNbY2Y3pRvb3Ld+u5mNMjPL9GxERK4gOMgY0C6tuVyzyiV5bvoG7hm3mB0pp7wuzVMZBoCZBQOjga5APeBeM6t3hXFFgaHA0nSrBwI45xoCscDLZvbTc44BBgE1fbcu1z8NEZGMVSxViHceask/ezdiy7cn6TpyPq99sz1gm8v5cwTQEtjunNvpnDsPTAZ6XGHci8BLwNl06+oBXwE45w4DPwBRZlYOKOacW+zSrsq8A/S8/mmIiPjHzLgrqiLzfh3Dr2qX4aVZW+j52kI2HDzudWk5zp8AKA/sS7e837fuZ2bWFKjonPvssm3XAD3MLMTMqgLNgYq+7fdfbZ/p9j3IzJLNLDklJcWPckVEMlamaAHG9mvOmL7N+Pb4Obq/upB/zt7M2QuB01zOnwC40rn5n6+e+E7pDAd+fYVxE0h7cU8GRgCLgNSM9vkfK50b55yLcs5FRUZG+lGuiIj/ujYsx7z4aHo2Kc/or3dw66j5rNhz1OuycoQ/AbCftHftP6kAHEy3XBRoAHxjZruB1sAMM4tyzqU65+Kcc02ccz2AEsA23z4rXGWfIiI5pkShMF6+uzFvP9SSsxcu0XvsYp6fsYHT5/J3czl/AmA5UNPMqppZGNAHmPHTg8654865COdcFedcFWAJ0N05l2xmhcysMICZxQKpzrmNzrlDwEkza+379E9/YHoWz01E5JrE1IpkTlw0D7SpwtuLd9NpeBJJW/PvqecMA8A5lwoMAWYDm4CpzrkNZvaCmXXPYPMywEoz2wQ8DfRL99hg4A1gO7AD+PI66hcRyVKFw0N4vnt9Pny0DeGhQfSfsIzffLiGH86c97q0LGd56Vejo6KiXHJystdliEiAOHvhIq/8extjE3dSslAYL/aoT9eG5bwu65qZ2QrnXNTl6/WbwCIiv6BAaDC/7VyHGUPaUbZYOIPfW8ngd1dw+OTZjDfOAxQAIiIZqH9jcT59oh1Pd6nDV5sPE5uQxIfJ+/J8czkFgIiIH0KDgxh8U3W+HNaBWmWL8Ntpa+k/YRn7jp7xurTrpgAQEbkG1SOLMGVQG17sUZ+Ve47ReUQSby3clSebyykARESuUVCQ0a9NFWbHRdOiSimen7mRu15fzPbDJ70u7ZooAERErlOFkoV468EWJNzdmB0pp+g2cgGjv97OhTzSXE4BICKSCWZGr2YVmBsXQ2y9svxz9hZ6vLqQ9Qdyf3M5BYCISBaILBrO6L7NGHt/c1JOnaPH6IX8Y1bubi6nABARyUJdGtzAvLgYejerwJhvdtBt5HyW7cqdzeUUACIiWax4oVD+0bsR7z7civMXL3H364t57tP1nMplzeUUACIi2aR9zQjmxEXzULuqvLt0D50SEvl6y2Gvy/qZAkBEJBsVCgvhD7fXY9pjbSkUHsKDE5cTP2U1x05731xOASAikgOaVy7J50PbM/RXNZix5iCxwxP5fO0hT9tJKABERHJIeEgw8Z1qM/PJ9pQrXpAn3l/Jo5NW8N0Jb5rLKQBERHJY3XLF+OTxtjzbtQ6JW1PomJDIlOV7c/xoQAEgIuKBkOAgHo2pzqynoqlbrhhPf7SO+99cyt4jOddcTgEgIuKhqhGFmTywNX/u2YA1+47TeUQSby7YxcUcaC6nABAR8VhQkHF/68rMiYumdbVSvPjZRnqPXcS277K3uZwCQEQkl7ixREEmDGjByD5N2P39aW4dtYBRX23jfGr2NNGJVmoAAASPSURBVJdTAIiI5CJmRo8m5ZkXH0PnBjeQMHcr3V9dkC2fFFIAiIjkQqWLhPPKvU0Z3z+KyqULEVEkPMufIyTL9ygiIlkmtl5ZYuuVzZZ96whARCRAKQBERAKUAkBEJEApAEREApQCQEQkQCkAREQClAJARCRAKQBERAKUefnXaK6VmaUAe65z8wjg+ywsJy/QnANDoM050OYLmZ9zZedc5OUr81QAZIaZJTvnoryuIydpzoEh0OYcaPOF7JuzTgGJiAQoBYCISIAKpAAY53UBHtCcA0OgzTnQ5gvZNOeAuQYgIiL/KZCOAEREJB0FgIhIgMp3AWBmXcxsi5ltN7NnrvB4uJlN8T2+1Myq5HyVWceP+cab2UYzW2tmX5lZZS/qzEoZzTnduN5m5swsz39k0J85m9ndvq/1BjN7P6drzGp+fG9XMrOvzWyV7/u7mxd1ZhUzm2Bmh81s/S88bmY2yvf/sdbMmmX6SZ1z+eYGBAM7gGpAGLAGqHfZmMeBsb77fYApXtedzfO9GSjkuz84L8/X3zn7xhUFkoAlQJTXdefA17kmsAoo6Vsu43XdOTDnccBg3/16wG6v687knKOBZsD6X3i8G/AlYEBrYGlmnzO/HQG0BLY753Y6584Dk4Eel43pAbztuz8NuMXMLAdrzEoZztc597Vz7oxvcQlQIYdrzGr+fI0BXgReArL+L2nnPH/mPBAY7Zw7BuCcO5zDNWY1f+bsgGK++8WBgzlYX5ZzziUBR68ypAfwjkuzBChhZuUy85z5LQDKA/vSLe/3rbviGOdcKnAcKJ0j1WU9f+ab3sOkvYPIyzKcs5k1BSo65z7LycKykT9f51pALTNbaGZLzKxLjlWXPfyZ8/PA/Wa2H/gCeDJnSvPMtf68Zyi//VH4K72Tv/xzrv6MySv8nouZ3Q9EATHZWlH2u+qczSwIGA4MyKmCcoA/X+cQ0k4D3UTaUd58M2vgnPshm2vLLv7M+V7gLefcy2bWBpjkm/Ol7C/PE1n+2pXfjgD2AxXTLVfgvw8Lfx5jZiGkHTpe7bArN/NnvphZR+D3QHfn3Lkcqi27ZDTnokAD4Bsz203audIZefxCsL/f19Odcxecc7uALaQFQl7lz5wfBqYCOOcWAwVIa5qWX/n1834t8lsALAdqmllVMwsj7SLvjMvGzAAe8N3vDfzb+a6w5EEZztd3OuR10l788/p5Ychgzs654865COdcFedcFdKue3R3ziV7U26W8Of7+lPSLvhjZhGknRLamaNVZi1/5rwXuAXAzOqSFgApOVplzpoB9Pd9Gqg1cNw5dygzO8xXp4Ccc6lmNgSYTdqnCCY45zaY2QtAsnNuBvAmaYeK20l759/Hu4ozx8/5/hMoAnzou9a91znX3bOiM8nPOecrfs55NtDJzDYCF4HfOueOeFd15vg5518D480sjrRTIQPy8Js5zOwD0k7hRfiua/wRCAVwzo0l7TpHN2A7cAZ4MNPPmYf/v0REJBPy2ykgERHxkwJARCRAKQBERAKUAkBEJEApAEREApQCQEQkQCkAREQC1P8B0NkE8odBM6kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "CHUNK_SIZE = 200\n",
    "for i in [3]:\n",
    "    do_plot(results[(i,0)], CHUNK_SIZE, False)\n",
    "    do_plot(results[(i,1)], CHUNK_SIZE, False)\n",
    "    do_plot(results[(i,2)], CHUNK_SIZE, False)\n",
    "    do_plot(results[(i,3)], CHUNK_SIZE, False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now compute the accuracy and loss against the validation set. The loss is computed as we normally do - by taking the value given by the network and comparing it to the value given by the minimax algorithm. The accuracy is a measure of what percentage of the time we make the optimal move. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.86\n",
      "Accuracy: 0.866\n",
      "Accuracy: 0.85\n",
      "Accuracy: 0.848\n",
      "Loss: 0.4548850655555725\n",
      "Loss: 0.455858051776886\n",
      "Loss: 0.46155235171318054\n",
      "Loss: 0.46853405237197876\n"
     ]
    }
   ],
   "source": [
    "dimNets = nets[3]\n",
    "loss = [0, 0, 0, 0]     # One loss per net\n",
    "accuracy = [0, 0, 0, 0] # One accuracy count per neural net \n",
    "criterion = nn.MSELoss()\n",
    "\n",
    "for netIndex in [0, 1, 2, 3]:\n",
    "    net = dimNets[netIndex]\n",
    "    flatten = (netIndex == 0 or netIndex == 1)\n",
    "    correctMoves = 0 \n",
    "    wrongMoves = 0 \n",
    "    \n",
    "    for node in validation_set_one:\n",
    "        res = net.evaluate(node.board)\n",
    "            \n",
    "        # For each node, look at all of the possible actions we can take\n",
    "        game.board = boardToList(node.board) \n",
    "        game.currPlayer = node.currPlayer\n",
    "        actions = game.getAllActions()\n",
    "        gameStatus = game.checkGameOver() \n",
    "        \n",
    "        # Evaluate the state of the current board\n",
    "        originalBoard = copy.deepcopy(game.board)\n",
    "        v = minimax(game, vertices[boardToTuple(game.board)])\n",
    "        expected = torch.FloatTensor([v])\n",
    "        loss[netIndex] += criterion(torch.FloatTensor([res[0].item()]), expected)\n",
    "        game.board = originalBoard\n",
    "         \n",
    "        if game.checkGameOver() != 0: \n",
    "            continue \n",
    "        \n",
    "        # Pick the action according to the greedy policy \n",
    "        maxValue = None\n",
    "        maxIndex = None \n",
    "        \n",
    "        maxValueMinimax = None\n",
    "        maxIndecesMinimax = []\n",
    "        \n",
    "        for i in range(len(actions)):\n",
    "            originalBoard = copy.deepcopy(game.board)\n",
    "            game.makeMove(actions[i])\n",
    "            \n",
    "            # Evaluate how good the board is according to the neural network\n",
    "            value = net.evaluate(game.board)[0].item()\n",
    "                \n",
    "            # See if it's better than the best move we've seen so far \n",
    "            if maxIndex is None or (game.currPlayer == 1 and value > maxValue) or (game.currPlayer == 2 and value < maxValue): \n",
    "                maxValue = value \n",
    "                maxIndex = i\n",
    "                \n",
    "            # Evaluate how good the board is according to minimax. \n",
    "            node = vertices[boardToTuple(game.board)]\n",
    "            minimiMaxValue = minimax(game, node) \n",
    "            # We have to chose SOME action\n",
    "            # If player 1, then we are trying to maximize value. If player 2, we are trying to minimize. \n",
    "            if (maxIndecesMinimax == []) or (game.currPlayer == 1 and minimiMaxValue > maxValueMinimax) or (game.currPlayer == 2 and minimiMaxValue > maxValueMinimax): \n",
    "                maxIndecesMinimax = [i] \n",
    "                maxValueMinimax = minimiMaxValue\n",
    "            elif minimiMaxValue == maxValueMinimax:\n",
    "                maxIndecesMinimax.append(i)\n",
    "            \n",
    "            # Reset the board back to initial state \n",
    "            game.board = originalBoard\n",
    "            \n",
    "        # Compare the greedy policy to the actual optimal policy (according to minimax) \n",
    "        if maxIndex in maxIndecesMinimax: \n",
    "            correctMoves += 1 \n",
    "        else: \n",
    "            wrongMoves += 1\n",
    "    \n",
    "    totalMoves = correctMoves + wrongMoves\n",
    "    accuracy[netIndex] = correctMoves / totalMoves\n",
    "\n",
    "for val in accuracy:\n",
    "    print(f\"Accuracy: {val}\")\n",
    "    \n",
    "for l in loss: \n",
    "    print(f\"Loss: {l / len(validation_set_one)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we pass this network into an agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Given a neural network, return a value function we can use for an agent. \n",
    "def netValueFunction(neuralNet, flatten): \n",
    "    # This should return an action\n",
    "    def valueFunction(board, game):     \n",
    "        actions = game.getAllActions() \n",
    "        \n",
    "        currAction = None \n",
    "        currVal = float('inf')\n",
    "        if game.currPlayer == 0:  # If first player, want to maximize the value.\n",
    "            currVal = -float('inf')\n",
    "            \n",
    "        for action in actions: \n",
    "            origBoard = copy.deepcopy(game.board)\n",
    "            origTurn = game.currTurn\n",
    "            \n",
    "            game.makeMove(action) \n",
    "            value = neuralNet.evaluate(game.board).item() \n",
    "            \n",
    "            if (game.currPlayer == 0 and value > currVal) or (game.currPlayer == 1 and value < currVal): \n",
    "                currVal = value \n",
    "                currAction = action \n",
    "            \n",
    "            game.board = origBoard\n",
    "            game.currTurn = origTurn\n",
    "            \n",
    "        return currAction \n",
    "    \n",
    "    return valueFunction\n",
    "\n",
    "def getMinimaxValueFunction(depth = None):\n",
    "    \n",
    "    def minimaxValueFunction(board, game, net = None):\n",
    "        actions = game.getAllActions() \n",
    "        currAction = None \n",
    "        if game.currPlayer == 0:  # If first player, want to maximize the value \n",
    "            currVal = -float('inf')\n",
    "        elif game.currPlayer == 1: # If the second player, want to minimize the value \n",
    "            currVal = float('inf')\n",
    "\n",
    "        for action in actions: \n",
    "            origBoard = copy.deepcopy(game.board)\n",
    "            origTurn = game.currTurn\n",
    "\n",
    "            game.makeMove(action) \n",
    "            value = minimax(game, vertices[boardToTuple(game.board)], depth, net)\n",
    "\n",
    "            if (game.currPlayer == 0 and value > currVal) or (game.currPlayer == 1 and value < currVal): \n",
    "                currVal = value \n",
    "                currAction = action \n",
    "\n",
    "            game.board = origBoard\n",
    "            game.currTurn = origTurn\n",
    "\n",
    "        return currAction \n",
    "    \n",
    "    return minimaxValueFunction                \n",
    "\n",
    "# The value agent takes a playerNum and a value function. \n",
    "agentsPlayerOne, agentsPlayerTwo = [], []\n",
    "for i in [0, 1]:\n",
    "    agentsPlayerOne.append(ValueAgent(0, netValueFunction(nets[3][i], True), 2))\n",
    "    agentsPlayerTwo.append(ValueAgent(1, netValueFunction(nets[3][i], True), 2))\n",
    "    \n",
    "for i in [2, 3]:\n",
    "    agentsPlayerOne.append(ValueAgent(0, netValueFunction(nets[3][i], False), 2))\n",
    "    agentsPlayerTwo.append(ValueAgent(1, netValueFunction(nets[3][i], False), 2))\n",
    "\n",
    "minimaxAgentPlayerOne = ValueAgent(0, getMinimaxValueFunction(None), 2)\n",
    "minimaxAgentPlayerTwo = ValueAgent(1, getMinimaxValueFunction(None), 2)\n",
    "\n",
    "DEPTH = 3\n",
    "\n",
    "minimaxDepthAgentOne = ValueAgent(0, getMinimaxValueFunction(DEPTH, None), 2)\n",
    "minimaxDepthAgentTwo = ValueAgent(1, getMinimaxValueFunction(DEPTH, None), 2)\n",
    "\n",
    "agentsMinimaxOne, agentsMinimaxTwo = [], [] \n",
    "for i in [0, 1, 2, 3]:\n",
    "    agentsMinimaxOne.append(ValueAgent(0, getMinimaxValueFunction(DEPTH, nets[3][i]), 2))\n",
    "    agentsMinimaxTwo.append(ValueAgent(1, getMinimaxValueFunction(DEPTH, nets[3][i]), 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparisons of Greedy Policy to Baseline\n",
    "\n",
    "Here, we compare the greedy policy from the learned value function to an agent that plays random policies and a minimax agent (which plays perfectly)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'agentsPlayerOne' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-44-88bfc9add397>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     47\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     48\u001b[0m \u001b[0;31m# Play as both player 1 and player 2 against random\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 49\u001b[0;31m \u001b[0mplayGames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0magentsPlayerOne\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"random\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     50\u001b[0m \u001b[0mplayGames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0magentsPlayerTwo\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"random\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     51\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'agentsPlayerOne' is not defined"
     ]
    }
   ],
   "source": [
    "NUM_GAMES = 10\n",
    "\n",
    "def playGames(agents, playerOne = True, type = None, otherAgent = None):\n",
    "    for agentNum, agent in enumerate(agents): \n",
    "        TODO = None \n",
    "        log = Log(3)\n",
    "        \n",
    "        if type == \"minimax\":\n",
    "            if playerOne:\n",
    "                game = StandardTicTacToe(agent, minimaxAgentPlayerTwo, log) \n",
    "            else: \n",
    "                game = StandardTicTacToe(minimaxAgentPlayerOne, agent, log) \n",
    "        elif type == \"depth-minimax\":\n",
    "            if playerOne: \n",
    "                game = StandardTicTacToe(agent, minimaxDepthAgentTwo, log)\n",
    "            else: \n",
    "                game = StandardTicTacToe(minimaxDepthAgentOne, agent, log)\n",
    "        elif type == \"value-minimax\":\n",
    "            if playerOne: \n",
    "                game = StandardTicTacToe(agent, otherAgent, log) \n",
    "            else: \n",
    "                game = StandardTicTacToe(otherAgent, agent, log)\n",
    "        else:\n",
    "            if playerOne: \n",
    "                game = StandardTicTacToe(agent, DumbAgent(1), log)\n",
    "            else: \n",
    "                game = StandardTicTacToe(DumbAgent(1), agent, log)\n",
    "    \n",
    "        ties = 0 \n",
    "        wins = 0 \n",
    "        losses = 0 \n",
    "        for i in range(NUM_GAMES): \n",
    "            gameResult = game.play()        \n",
    "            if gameResult == 0: \n",
    "                ties += 1\n",
    "            elif (gameResult == 1) and (playerOne):\n",
    "                wins += 1\n",
    "            elif (gameResult == 2) and not playerOne:\n",
    "                wins += 1\n",
    "            else: \n",
    "                losses += 1\n",
    "        \n",
    "        if playerOne:\n",
    "            print(f\"{type} {agentNum}-Player 1: W {wins} T {ties} L {losses}\")\n",
    "        else: \n",
    "            print(f\"{type} {agentNum}-Player 2: W {wins} T {ties} L {losses}\")\n",
    "\n",
    "# Play as both player 1 and player 2 against random\n",
    "playGames(agentsPlayerOne, True, \"random\")\n",
    "playGames(agentsPlayerTwo, False, \"random\")\n",
    "\n",
    "# Play as both player 1 and player 2 against minimax\n",
    "playGames(agentsPlayerOne, True, \"minimax\")\n",
    "playGames(agentsPlayerTwo, False, \"minimax\")\n",
    "\n",
    "# Play as both player 1 and player 2 against minimax with max depth \n",
    "playGames(agentsPlayerOne, True, \"depth-minimax\")\n",
    "playGames(agentsPlayerTwo, False, \"depth-minimax\")\n",
    "\n",
    "# Play as both player 1 and player 2 against minimax with max depth and value heuristic \n",
    "for agent in agentsMinimaxOne:\n",
    "    playGames(agentsPlayerOne, True, \"value-minimax\", agent)\n",
    "    \n",
    "for agent in agentsMinimaxTwo:\n",
    "    playGames(agentsPlayerTwo, False, \"value-minimax\", agent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "from MTCS import MCTS\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initializing\n",
      "initializing\n"
     ]
    }
   ],
   "source": [
    "mcts = MCTS(n, 2)\n",
    "agentOne = AlphaZeroAgent(0, True, 1, 3, mcts)\n",
    "agentTwo = AlphaZeroAgent(1, True, 1, 3, mcts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting game...\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "\n",
      "move: (0, 2, 1) for player 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 1 0\n",
      "\n",
      "move: (0, 2, 0) for player 1\n",
      "0 0 0\n",
      "0 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 0) for player 0\n",
      "0 0 0\n",
      "1 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 2) for player 1\n",
      "0 0 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 1) for player 0\n",
      "0 1 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 1) for player 1\n",
      "0 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 0) for player 0\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 2, 2) for player 1\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "move: (0, 0, 2) for player 0\n",
      "Game Ended!\n",
      "1 1 1\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "1\n",
      "Starting game...\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "\n",
      "move: (0, 2, 1) for player 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 1 0\n",
      "\n",
      "move: (0, 2, 0) for player 1\n",
      "0 0 0\n",
      "0 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 0) for player 0\n",
      "0 0 0\n",
      "1 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 2) for player 1\n",
      "0 0 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 1) for player 0\n",
      "0 1 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 1) for player 1\n",
      "0 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 0) for player 0\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 2, 2) for player 1\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "move: (0, 0, 2) for player 0\n",
      "Game Ended!\n",
      "1 1 1\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "for i in range(2):\n",
    "    game = StandardTicTacToe(agentOne, agentTwo, Log(0))\n",
    "    game.play()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initializing\n",
      "initializing\n"
     ]
    }
   ],
   "source": [
    "agentOne = AlphaZeroAgent(0, False, 1, 3, mcts)\n",
    "agentTwo = AlphaZeroAgent(1, False, 1, 3, mcts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting game...\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "\n",
      "move: (0, 2, 1) for player 0\n",
      "0 0 0\n",
      "0 0 0\n",
      "0 1 0\n",
      "\n",
      "move: (0, 2, 0) for player 1\n",
      "0 0 0\n",
      "0 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 0) for player 0\n",
      "0 0 0\n",
      "1 0 0\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 2) for player 1\n",
      "0 0 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 1) for player 0\n",
      "0 1 0\n",
      "1 0 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 1, 1) for player 1\n",
      "0 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 0, 0) for player 0\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 0\n",
      "\n",
      "move: (0, 2, 2) for player 1\n",
      "1 1 0\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "move: (0, 0, 2) for player 0\n",
      "Game Ended!\n",
      "1 1 1\n",
      "1 2 2\n",
      "2 1 2\n",
      "\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "for i in range(1):\n",
    "    game = StandardTicTacToe(agentOne, agentTwo, Log(0))\n",
    "    game.play()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}