5b_AttentionLSTM.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "AttentionLSTM.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SOLYaAAQ1Ceu"
      },
      "source": [
        "# LSTM + Attention\n",
        "\n",
        "Nous allons utiliser un modèle comprenant un réseau LSTM ainsi qu'une couche d'attention.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lm1iC_aQ13if"
      },
      "source": [
        "## Préparer les données"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N7eAcLKmQWxa",
        "outputId": "98346060-a480-4e13-ce76-5737a5f362e9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import os\n",
        "import sys\n",
        "import torch\n",
        "from torch.nn import functional as F\n",
        "import numpy as np\n",
        "from torchtext import data\n",
        "from torchtext import datasets\n",
        "from torchtext.vocab import Vectors, GloVe\n",
        "import pandas as pd\n",
        "import torch\n",
        "from torchtext import data\n",
        "import random\n",
        "# pour la reproductibilité\n",
        "SEED = 1234\n",
        "torch.manual_seed(SEED)\n",
        "torch.backends.cudnn.deterministic = True\n",
        "\n",
        "TEXT = data.Field(sequential=True,lower=True, tokenize = 'spacy', include_lengths=True, batch_first=True, fix_length=200)\n",
        "LABEL = data.LabelField(dtype = torch.float)\n",
        "\n",
        "train_data, test_data = datasets.IMDB.splits(TEXT, LABEL)\n",
        "test_data, valid_data = test_data.split(random_state = random.seed(1234))\n",
        "\n",
        "print(f'Taille des données train: {len(train_data)}')\n",
        "print(f'Taille des données de validation: {len(valid_data)}')\n",
        "print(f'Taille des données test: {len(test_data)}')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\raclImdb_v1.tar.gz:   0%|          | 0.00/84.1M [00:00<?, ?B/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "downloading aclImdb_v1.tar.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "aclImdb_v1.tar.gz: 100%|██████████| 84.1M/84.1M [00:03<00:00, 24.5MB/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Taille des données train: 25000\n",
            "Taille des données de validation: 7500\n",
            "Taille des données test: 17500\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aQsxGknoLCBO",
        "outputId": "4d91fd37-35eb-42ce-dc0f-7d79b7d2e368",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "MAX_VOCAB_SIZE = 50_000\n",
        "\n",
        "TEXT.build_vocab(train_data, max_size = MAX_VOCAB_SIZE, vectors=GloVe(name='6B', dim=100))\n",
        "LABEL.build_vocab(train_data)\n",
        "\n",
        "print(f\"Nombre de tokens unique dans le TEXT: {len(TEXT.vocab)}\") \n",
        "print(f\"Nombre unique de LABEL: {len(LABEL.vocab)}\")\n",
        "\n",
        "\n",
        "# utilisation du GPU si possible \n",
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "#device = 'cpu'\n",
        "BATCH_SIZE = 32\n",
        "train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(\n",
        "    (train_data, valid_data, test_data), \n",
        "    batch_size = BATCH_SIZE,\n",
        "device = device, sort_key=lambda x: len(x.text), repeat=False, shuffle=True)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            ".vector_cache/glove.6B.zip: 862MB [06:29, 2.21MB/s]                           \n",
            "100%|█████████▉| 399226/400000 [00:16<00:00, 22717.05it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Nombre de tokens unique dans le TEXT: 50002\n",
            "Nombre unique de LABEL: 2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rUunWgd_1_eH"
      },
      "source": [
        "## Construire le modèle\n",
        "\n",
        "Nous allons incorporer le mécanisme Attention dans notre modèle GRU. Dans ce nouveau modèle, nous utiliserons l'attention pour calculer le score d'alignement entre chacun des états cachés et le dernier état caché du GRU. \n",
        "\n",
        "Nous utiliserons `torch.bmm` pour la multiplication de la matrice par batch.\n",
        "\n",
        " Celui-ci est constitué d'une couche **embedding GloVe**, d'une couche **LSTM**, puis d'une couche **Attention** et enfin de deux couches **fully connected**.\n",
        "\n",
        "L'attention est calculée comme ceci:\n",
        "\n",
        "$$\n",
        "m_i = \\tanh(W*\\overrightarrow{h_i}\\overleftarrow{h_i}+b)\n",
        "$$ \n",
        "\n",
        "$$\n",
        "a_i = \\frac{m_i}{\\sum_{k}^{n}m_k}\n",
        "$$\n",
        "\n",
        "$i$ représente le $i$ème mot du commentaire. $W$ est la matrice des poids et $b$ est un vecteur de biais.\n",
        "L'attention consiste ici à prendre en compte un vecteur de contexte spécifique qui est une somme pondérée de tous les vecteurs d’état émis par le BiLSTM."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7nuMkAtFLCBT"
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.autograd import Variable\n",
        "from torch.nn import functional as F\n",
        "\n",
        "class AttentionModel(nn.Module):\n",
        "    def __init__(self, batch_size, output_size, hidden_size, vocab_size, embedding_length, word_embeddings, dropout):\n",
        "        super(AttentionModel, self).__init__()\n",
        "\n",
        "\n",
        "        self.batch_size = batch_size\n",
        "        self.output_size = output_size\n",
        "        self.hidden_size = hidden_size\n",
        "        self.vocab_size = vocab_size\n",
        "        self.embedding_length = embedding_length\n",
        "        self.weights = word_embeddings\n",
        "\n",
        "        self.word_embeddings = nn.Embedding(vocab_size, embedding_length)\n",
        "        self.word_embeddings.weights = nn.Parameter(word_embeddings, requires_grad=False)\n",
        "        self.dropout = nn.Dropout(dropout)\n",
        "        self.lstm = nn.LSTM(embedding_length, hidden_size, dropout=dropout, bidirectional=True)\n",
        "        self.W_s1 = nn.Linear(2*hidden_size, 350)\n",
        "        self.W_s2 = nn.Linear(350, 30)\n",
        "        self.fc_layer = nn.Linear(30*2*hidden_size, 2000)\n",
        "        self.label = nn.Linear(2000, output_size)\n",
        "\n",
        "    def attention_net(self, lstm_output):\n",
        "\n",
        "        attn_weight_matrix = self.W_s2(torch.tanh(self.W_s1(lstm_output)))\n",
        "        attn_weight_matrix = attn_weight_matrix.permute(0, 2, 1)\n",
        "        attn_weight_matrix = F.softmax(attn_weight_matrix, dim=2)\n",
        "\n",
        "        return attn_weight_matrix\n",
        "\n",
        "\n",
        "    def forward(self, input_sentences, batch_size=None):\n",
        "\n",
        "        \"\"\" \n",
        "        Parameters\n",
        "        ----------\n",
        "        input_sentence: input_sentence taille = (batch_size, num_sequences)\n",
        "        batch_size : default = None. Utilisé uniquement pour la prédiction sur une seule phrase après l'entraînement (batch_size = 1)\n",
        "        \n",
        "        Returns\n",
        "        -------\n",
        "        Sortie de la couche linéaire contenant les logits pour la classe positive & négative.\n",
        "        \n",
        "        \"\"\"\n",
        "        input_ = self.dropout(self.word_embeddings(input_sentences))\n",
        "        input_ = input_.permute(1, 0, 2)\n",
        "        if batch_size is None:\n",
        "            h_0 = Variable(torch.zeros(2, self.batch_size, self.hidden_size).to(device))\n",
        "            c_0 = Variable(torch.zeros(2, self.batch_size, self.hidden_size).to(device))\n",
        "        else:\n",
        "            h_0 = Variable(torch.zeros(2, batch_size, self.hidden_size).to(device))\n",
        "            c_0 = Variable(torch.zeros(2, batch_size, self.hidden_size).to(device))\n",
        "\n",
        "        output, (h_n, c_n) = self.lstm(input_, (h_0, c_0))\n",
        "        output = output.permute(1, 0, 2)    \n",
        "        attn_weight_matrix = self.attention_net(output)\n",
        "        hidden_matrix = torch.bmm(attn_weight_matrix, output)\n",
        "        fc_out = self.fc_layer(hidden_matrix.view(-1, hidden_matrix.size()[1]*hidden_matrix.size()[2]))\n",
        "        fc_out1 = self.dropout(fc_out)\n",
        "        logits = self.label(fc_out1)\n",
        "\n",
        "        return logits"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KKT9cOVngV63",
        "outputId": "832d2999-525b-4078-d613-4702b97e6ed7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import torch.nn as nn\n",
        "from torch.autograd import Variable\n",
        "batch_size = 32\n",
        "output_size = 2 # 2 labels \n",
        "hidden_size = 100 \n",
        "embedding_length = 100 # dimension de GloVe word embeddings\n",
        "word_embeddings = TEXT.vocab.vectors\n",
        "vocab_size = len(TEXT.vocab)\n",
        "dropout = 0.4\n",
        "\n",
        "model = AttentionModel(batch_size, output_size, hidden_size, vocab_size, embedding_length, word_embeddings, dropout)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py:61: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.4 and num_layers=1\n",
            "  \"num_layers={}\".format(dropout, num_layers))\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ikFmSEAygryx",
        "outputId": "9bf61264-26e2-4e60-e154-3818bf7f9f6b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "def count_parameters(model):\n",
        "    return sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
        "\n",
        "print(f'Le modèle a {count_parameters(model):,} paramètres à entraîner')"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Le modèle a 17,248,682 paramètres à entraîner\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q-TdlrungvVo",
        "outputId": "5bf8cafe-2743-4e64-ffc0-970c93a03d93",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import torch.optim as optim\n",
        "\n",
        "def optimiseur(model, lr=1e-4, eps=1e-6, weight_decay_rate=0.001, second_weight_decay_rate=0.0):\n",
        "    param_optimizer = list(model.named_parameters())\n",
        "    no_decay = ['bias', 'gamma', 'beta']\n",
        "    optimizer_grouped_parameters = [\n",
        "        {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],\n",
        "            'weight_decay_rate': weight_decay_rate},\n",
        "        {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)],\n",
        "            'weight_decay_rate': second_weight_decay_rate}]\n",
        "    return optim.Adam(\n",
        "        optimizer_grouped_parameters,\n",
        "        lr=lr,\n",
        "        eps=eps\n",
        "    )\n",
        "\n",
        "optimizer = optimiseur(model, lr=1e-4, eps=1e-6, weight_decay_rate=0.001, second_weight_decay_rate=0.0)\n",
        "scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=10, gamma=0.5)\n",
        "criterion =  F.cross_entropy\n",
        "\n",
        "model = model.to(device)\n",
        "\n",
        "\n",
        "def binary_accuracy(preds, y):\n",
        "    \"\"\"\n",
        "    Retourne l'accuracy par batch\n",
        "    \"\"\"\n",
        "    #arrondi la prédiction à l'entier le plus proche\n",
        "    rounded_preds = torch.round(torch.sigmoid(preds))\n",
        "    correct = (rounded_preds == y).float() \n",
        "    acc = correct.sum() / len(correct)\n",
        "    return acc\n"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r100%|█████████▉| 399226/400000 [00:30<00:00, 22717.05it/s]"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zpow3OfQhDO7",
        "outputId": "b484407e-6f61-4807-dbfd-857bcd1826f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "def train(model, iterator, optimizer, criterion):\n",
        "    \n",
        "    epoch_loss = 0\n",
        "    epoch_acc = 0\n",
        "    \n",
        "    model.train()\n",
        "    \n",
        "    for batch in iterator:\n",
        "        \n",
        "        optimizer.zero_grad()\n",
        "        text = batch.text[0]\n",
        "        target = batch.label \n",
        "\n",
        "        if (text.size()[0] is not batch_size):\n",
        "            continue\n",
        "        predictions = model(text)\n",
        "        target = torch.autograd.Variable(target).long()\n",
        "        loss = criterion(predictions, target)\n",
        "        pred = torch.max(predictions, 1)[1].view(target.size()).data\n",
        "        acc = binary_accuracy(pred.float(), target)\n",
        "        \n",
        "        loss.backward()\n",
        "        \n",
        "        optimizer.step()\n",
        "        \n",
        "        epoch_loss += loss.item()\n",
        "        epoch_acc += acc.item()       \n",
        "        \n",
        "    return epoch_loss / len(iterator), epoch_acc / len(iterator)\n",
        "\n",
        "def evaluate(model, iterator, criterion):\n",
        "      \n",
        "    epoch_loss = 0\n",
        "    epoch_acc = 0\n",
        "      \n",
        "    model.eval()\n",
        "      \n",
        "    with torch.no_grad():\n",
        "      \n",
        "        for batch in iterator:\n",
        "\n",
        "            text = batch.text[0]\n",
        "            \n",
        "            if (text.size()[0] is not batch_size):\n",
        "                continue\n",
        "            target = batch.label \n",
        "            predictions = model(text)\n",
        "            target = torch.autograd.Variable(target).long()\n",
        "            loss = criterion(predictions, target )\n",
        "            pred = torch.max(predictions, 1)[1].view(target.size()).data\n",
        "            acc = binary_accuracy(pred.float(), target)\n",
        "              \n",
        "            epoch_loss += loss.item()\n",
        "            epoch_acc += acc.item()\n",
        "            \n",
        "    return epoch_loss / len(iterator), epoch_acc / len(iterator)\n",
        "import time\n",
        "\n",
        "def epoch_time(start_time, end_time):\n",
        "    elapsed_time = end_time - start_time\n",
        "    elapsed_mins = int(elapsed_time / 60)\n",
        "    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))\n",
        "    return elapsed_mins, elapsed_secs\n",
        "\n",
        "tloss = []\n",
        "tacc = []\n",
        "vloss = []\n",
        "vacc = []\n",
        "\n",
        "N_EPOCHS = 10\n",
        "\n",
        "best_valid_loss = float('inf')\n",
        "\n",
        "for epoch in range(N_EPOCHS):\n",
        "\n",
        "    start_time = time.time()\n",
        "    \n",
        "    train_loss, train_acc = train(model, train_iterator, optimizer, criterion)\n",
        "    tloss.append(train_loss)\n",
        "    tacc.append(train_acc) \n",
        "    scheduler.step()\n",
        "    valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)\n",
        "    vloss.append(valid_loss)\n",
        "    vacc.append(valid_acc)   \n",
        "    \n",
        "    end_time = time.time()\n",
        "\n",
        "    epoch_mins, epoch_secs = epoch_time(start_time, end_time)\n",
        "    \n",
        "    if valid_loss < best_valid_loss:\n",
        "        best_valid_loss = valid_loss\n",
        "        torch.save(model.state_dict(), 'tut5-model.pt')\n",
        "    \n",
        "    print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')\n",
        "    print(f'\\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')\n",
        "    print(f'\\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc*100:.2f}%')\n",
        "\n"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 01 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.628 | Train Acc: 63.93%\n",
            "\t Val. Loss: 0.559 |  Val. Acc: 70.78%\n",
            "Epoch: 02 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.544 | Train Acc: 71.84%\n",
            "\t Val. Loss: 0.499 |  Val. Acc: 75.19%\n",
            "Epoch: 03 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.496 | Train Acc: 75.62%\n",
            "\t Val. Loss: 0.456 |  Val. Acc: 77.41%\n",
            "Epoch: 04 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.462 | Train Acc: 78.12%\n",
            "\t Val. Loss: 0.417 |  Val. Acc: 80.24%\n",
            "Epoch: 05 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.434 | Train Acc: 79.72%\n",
            "\t Val. Loss: 0.432 |  Val. Acc: 79.48%\n",
            "Epoch: 06 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.415 | Train Acc: 80.93%\n",
            "\t Val. Loss: 0.401 |  Val. Acc: 81.22%\n",
            "Epoch: 07 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.397 | Train Acc: 81.94%\n",
            "\t Val. Loss: 0.386 |  Val. Acc: 82.06%\n",
            "Epoch: 08 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.379 | Train Acc: 82.87%\n",
            "\t Val. Loss: 0.386 |  Val. Acc: 82.63%\n",
            "Epoch: 09 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.373 | Train Acc: 83.38%\n",
            "\t Val. Loss: 0.362 |  Val. Acc: 83.67%\n",
            "Epoch: 10 | Epoch Time: 0m 20s\n",
            "\tTrain Loss: 0.367 | Train Acc: 83.32%\n",
            "\t Val. Loss: 0.362 |  Val. Acc: 83.88%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wjYP2vcELCBm"
      },
      "source": [
        "On trouve une valeur d'accuracy d'environ 80% pour les données de validation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2GkJAB5BLCBn",
        "outputId": "e4be7c0f-ebdc-4220-8597-12ac80c26830",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 301
        }
      },
      "source": [
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt \n",
        "import numpy as np \n",
        "sns.set()\n",
        "\n",
        "x = np.linspace(0, N_EPOCHS,N_EPOCHS)\n",
        "\n",
        "plt.plot(x,tloss)\n",
        "plt.plot(x,vloss)\n",
        "plt.title(\"Loss\")\n",
        "plt.legend([\"Train loss\", \"Valid loss\"])"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f65622fea58>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAELCAYAAAAx94awAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xV9f348dc5d2SHDJKQhJGEGSAQhuwNCkJCcNBQ3ANqwaLWWqitoqK11NrxRaHFKkK11qr8GAEtomwBAZlJEMhgZJNBEjLuOr8/AhEMkHlzk9z38/HIw5vcc8/n/fZc3ufcz/2cz0fRNE1DCCGEU1EdHYAQQojmJ8VfCCGckBR/IYRwQlL8hRDCCUnxF0IIJyTFXwghnJAUfyGEcEJS/IW4xoQJE/jmm28cHYYQdifFXwghnJAUfyFqYTKZeO211xg1ahSjRo3itddew2QyAVBQUMDPfvYzBg8ezJAhQ5g9ezY2mw2AlStXMnr0aAYMGMDkyZPZu3evI9MQ4jp6RwcgREu3YsUKjh49yvr161EUhXnz5rF8+XKefvppVq1aRVBQUHVhP3r0KIqikJqayocffsinn35KUFAQFy5cqD4pCNESyJW/ELXYuHEj8+fPx9/fHz8/P+bPn8+GDRsA0Ov15OXlkZmZicFgYPDgwSiKgk6nw2QykZKSgtlspmPHjnTu3NnBmQjxAyn+QtQiNzeXkJCQ6t9DQkLIzc0F4LHHHqNLly48+uijTJw4kZUrVwLQpUsXnn/+eZYtW8aIESN45plnyMnJcUj8QtyIFH8hahEYGEhmZmb171lZWQQGBgLg6enJokWL+Oqrr1ixYgWrVq2q7gKKjY3lo48+Ytu2bSiKwp/+9CeHxC/EjUjxF+JHzGYzlZWV1T/Tpk1jxYoVFBQUUFBQwNtvv01sbCwA27Zt4+zZs2iahpeXFzqdrrrPf+/evZhMJoxGIy4uLqiq/HMTLYd84SvEj8ydO/e63x999FH69u3L9OnTAZgyZQrz5s0D4OzZsyxZsoSCggK8vb356U9/yrBhwzh58iRvvvkmKSkpGAwGBgwYwCuvvNLsuQhxM4os5iKEEM5HPocKIYQTkuIvhBBOSIq/EEI4ISn+QgjhhKT4CyGEE5LiL4QQTqjVjPMvLLyMzVb/Uan+/p7k55faIaKWS3J2DpKzc2hozqqq4OvrcdPnW03xt9m0BhX/q691NpKzc5CcnYM9cpZuHyGEcEJS/IUQwgm1mm4fIUTromkahYV5mEwVQNN0W+Tmqk63KM6tc1YwGl3x9Q1AUZR67VeKvxDCLkpLL6EoCkFBHVGUpulk0OtVLBbnKv63ylnTbBQVXaS09BJeXj712q90+wgh7KK8vBQvL58mK/yiJkVR8fLypby8AaOB7BBPiyKTlgrhGDabFZ1OOhfsTafTY7NZ6/26Nl38E9MLmP/G15SWmx0dihBOqb790KL+Gvr/uE2fln08XcjILSXhm3RmTezu6HCEEA4yZ85DmM1mLBYz58+fIzy8KwA9evTk+ecX12kf69Z9SmVlJfHx99Wr7XvvjeWPf/wLERHd6h23PbXp4h/a3oOJt3Xm60PnmTSoI+193BwdkhDCAd55ZzUAWVmZPP74A7z//r9rbGOxWNDrb14SZ8y4127xOUKbLv4Asyf3Yvt3F/h/u1KZE9vH0eEIIVqQe++NZeLEO/juuwNERHRj7tx5vPTSb7l8+TImk4kRI0Yyb95TALz77j8oLy/nySefZvPmjXz55Rd4eXmTmpqCl5cnr776R/z929+yvQsXzvPGG7+nqKgQnU7H3LnzGTZsBBUVFbz66mLS01PR6fR07tyFJUv+wLlz6fz+9y9TXl6BzWblzjtjmT37gSbJvc0X//Y+bkwa3JEv9p1j8pDOdA7ycnRIQjilPcez2H0sq1H7UBS40RiOUf2CGRkV3KB9Xr58mXfeWQNAZWUlS5f+BXd3dywWC7/85ZPs2/cNw4aNqPG65OQkVq/+iKCgDixd+iqffvoxP/vZ/Fu29fLLvyMu7i5iYmaQlpbKk0/O4YMPPuXYsSOUlV3mgw8+AaC4uBiAtWs/ZfTosdx338PX/b0ptOkvfK+aOqwL7q56Pt2e4uhQhBAtzJQp06of22w2li//Gw899FMee+x+UlNTOH361A1f169ff4KCOgDQp09fMjMv3LKdsrLLnDlziqlTpwMQHh5Bt249SUw8Trdu3UlPT+PNN5fy9ddbMRqNAERHD2DDhnW8884KDh06gJdX0128tvkrfwAPVwPThofx321nSEovoHeYn6NDEsLpjIxq+NX5Vfa4ycvd/YfvAj/++ENKSopZufJ9XFxcWLr0NUymyhu+7mqBBlBVHVZr/YdbXhUa2pEPPvgvBw8eYN++Paxc+TarV/+HceMm0r9/NHv3fsMHH7zPpk0bePHFJQ1u51pOceUPMHFQKP7eLnyyPQWbjP0XQtxASUkJ/v7tcXFxIS8vl927dzTZvt3dPejWrQeff54AQHp6Gikpp+jTJ4rc3BxUVceYMeNYsOBZiooKKSkp5sKF8/j7+zN1aiyPPDKHpKTEJovHKa78AQx6HTNGR/DupmQOnsxlSGSQo0MSQrQwM2fO4oUXFvLAAz8hICCIQYNua9L9L178Km+88Xv++99/o9Pp+N3vXsHX15e9e/fw97+/BVTdHHf//Q/Tvn0Aa9a8x5dffoFer0dRFJ566tkmi0XRWsktsPn5pQ2a0zogwIu8vBKgak7sl1Z9i8ls49U5Q9Hr2uYHn2tzdhaSc8uTnX2WDh26NOk+ZW6fG7vR/2tVVfD397zpa9pm9bsJVVW4d1xXcovK2XEk09HhCCGEwzhV8QeIivCnV2cfNuxJo7zS4uhwhBDCIZyu+CuKwr3julFSZuZ/355zdDhCCOEQTlf8ASJCvBncK5D/fXueS6U3HsYlhBBtmVMWf4B7xkRgsdrY8E26o0MRQohm57TFP8jPnTHRIew8kklOQZmjwxFCiGbltMUfYPrIcPQ6lc92pjo6FCGEaFZOXfzbeRiZPKQTB0/mkprZdBMmCSFalmefXcC6dZ9e9zdN05g5M47Dhw/d9HWvvfYSn332MVA1n//HH394w+02b97I73736xs+d++9saSmnmlg5PZTp+KflpZGfHw8kydPJj4+nvT09Btut3nzZmJjY4mJiSE2NpaLFy8CYLVaefnll5k0aRK33347n3zySZMl0FiTh3TG293AJ9vOyJKPQrRR06ZNZ/PmhOv+dvjwIVRVITp6YJ32MWPGvfVeyKUlq9P0DosXL2b27NnExcWxfv16XnzxRdasWXPdNsePH+ett95i9erVBAQEUFJSUj3x0caNGzl37hxbtmyhqKiIGTNmMHz4cDp27Nj0GdWTm4ue2JHhfPjlKY6n5tOv663n4xZCNIz51B7M3+9s1D4URbnhRZqh5xgMPUbe9HWjR4/lzTdfJz09jbCwcAA2bdrA1KmxpKam8Oabf6CiohyTycT06Xfxk5/MrrGPa+fzN5vN/OUvf+S77w7Srp0P3bv3rFP8rWo+//z8fJKSkli1ahUAMTExLFmyhIKCAvz8fpgd8/333+fRRx8lICAA4LqpRzdv3szMmTNRVRU/Pz8mTZrEF198weOPP94kSTTW2OgQvjxwnk+3p9A33B9VlXVHhWhLDAYDt99+J5s3b2DevKcoK7vMrl07+OCD/+Lp6clf/7oco9FIWVkZc+c+xJAhw6tPEjeyfv1nZGVl8sEHn2CxWJg/fw7BwbXPWNqS5vOvtfhnZWURFBSETqcDQKfTERgYSFZW1nXFPyUlhY4dO3LfffdRVlbG7bffzs9//nMURSErK4uQkJDqbYODg8nOzq5XoLeao6I2AQG1z4H9cEwf/vjBQRLPFzFhcOcGt9VS1CXntkZybllyc1X0+h96lvW9R+PWe7TD4omLm8HTTz/J/PkL2LZtK/369SckJJj8/HzefPMPnDlzGkVRuHgxj7S0M3Tr1hVFUVBVBb1eRVV/eHz48CGmTYvF1dUIGLnzzqkcPXrkunyvpdOpVFaWc+bMKaZPn4GqqnTv3o3u3Xty8uQJevXqyf/9Xzp/+ctSBg4czMiRo9DrVQYOHMTbb/+NiooKBg0azKBBt91wwXZVVev9XmiyWT2tVivff/89q1atwmQy8fjjjxMSEsKMGTOaZP9NMbHbrfQI9SKsgxdrNiXRK9Qbg17XkDBbhJY+4Zc9SM4tj81ma/JJ2BozsVt4eDf8/duze/duEhLWM3PmbCwWG8uXL8PX1593312MXq/nmWfmU15egcViQ9M0bDYNi8WGzfbDY02j+jFUPdY07aaxWa02rNaq5ywWG+qVc4SmgdWqERQUwr/+9XH1fP4rVrzF6tX/YcyY8URF9WPv3m9YvXoVGzasv+F8/jabrcZ7odETuwUHB5OTk1O9UIHVaiU3N7fGR5yQkBCmTJmC0WjE09OTiRMncuzYsep9ZGb+MJFaVlYWHTp0qK3pZqUqCjPHdSW/uJKvDmU4OhwhhB1Mmzad995byfnz5xg9eiwApaUlBAYGodfrSU09w9GjR2rdz6BBg/nii81YLBYqKyv48ssvan1Nq5vP39/fn8jISBISEoiLiyMhIYHIyMjrunyg6ruAHTt2EBcXh8ViYd++fUyePBmAKVOm8Mknn3DHHXdQVFTE1q1b+fDDGw+ZcqTIMD/6RvixaW86o/sH4+FqcHRIQogmdPvtU3j77b8xffpdGAxV/74feugxlix5kU2b1tOpU2eiowfUup/p0+/mzJkz3H//TNq186FXrz4UFubX+rpWN59/SkoKixYtori4GG9vb5YuXUpERARz5sxhwYIFREVFYbPZWLp0KTt37kRVVUaNGsXChQtRVRWr1corr7zCnj17AJgzZw7x8fH1CtTe3T5Xncsp4eVVB5gyrDMzx3Wrd3stQUvvDrAHybnlkfn8m4a95vN3qsVc6uqdjUkc/D6X1+cOw8/btd5tOlpLLwr2IDm3PFL8m4Ys5tKM7hodjqZprNud5uhQhBDCLqT430B7HzcmDOzInuNZZOSVOjocIVqtVtKx0Ko19P+xFP+biBkRhqtRx2c7ZNI3IRpCVXVYrbJanr1ZrRZUtf5D06X434Snm4Gpw7pw5MxFTp0vcnQ4QrQ6bm6elJQUoWnO1UffnDTNRklJIW5u9b8Jtslu8mqLJg3uxFeHLvDJtjM8/8CgG95ZJ4S4MU/PdhQW5pGTcwFomu4fVVWx2ZzrZHLrnBWMRlc8PdvVe79S/G/BxaBjxugI3v/8JN+dusigngGODkmIVkNRFPz8Apt0ny19hJM92Ctn6fapxcioDgT7u/PZjhSsTnbFIYRou6T410Knqtw7tivZBWXsOpbl6HCEEKJJSPGvg+ju7enWsR3rd6VRabI6OhwhhGg0Kf51oCgKPxnXjUuXTWw5eN7R4QghRKNJ8a+jbh3bMaB7ez7fd5aSMpOjwxFCiEaR4l8P94ztSqXZysZv0h0dihBCNIoU/3oIae/B6H4hbPsug7yickeHI4QQDSbFv57iRoWjUxX+306Z9kEI0XpJ8a8nXy8Xbr+tE/uScjib7Vw3mwgh2g4p/g1w59AueLjq+XT7GUeHIoQQDSLFvwHcXfXEjggjMb2QxLQCR4cjhBD1JsW/gcYP7Ii/tyufbk/BJnOWCyFaGSn+DWTQq9w9JoKzOSV8m5zj6HCEEKJepPg3wtA+QXQK9GTtjlQsVpn0TQjRekjxbwRVUZg5risXL1Ww7XCGo8MRQog6k+LfSH3C/Yjs4svGPemUV8qSdUKI1kGKfyMpisK947pSWm7m8/3nHB2OEELUiRT/JhAe7M2QyEC2HDhHUWmlo8MRQohatenib8n6noxVi7BdLrR7W3ePicBq1diwJ93ubQkhRGO16eKvurfDlHeOiq9WoNnsuwhLoK8746JD2Xkkk6z8y3ZtSwghGqttF/92HWg/9Qms2acwHfjM7u3FjgzDYFBZK5O+CSFauDZd/AG8+o7BEDke09HNWNIP27Utbw8jdw7pzKHv80jJuGTXtoQQojHqVPzT0tKIj49n8uTJxMfHk56eXmObZcuWMXz4cOLi4oiLi+Pll1+ufm7RokWMGTOm+rkVK1Y0WQJ14TL8p6jtu1C+/R1sxbl2beuOIZ3w9jDyybYzaDLtgxCihdLXZaPFixcze/Zs4uLiWL9+PS+++CJr1qypsd2MGTNYuHDhDfcxd+5c7r///sZF20CK3ojbpCe5vHYx5Vvfxn36b1H0Rru05WrUEzcyjH9tOcXRlHyiu7W3SztCCNEYtV755+fnk5SURExMDAAxMTEkJSVRUNC6ZrNUvQNwGzcH28WzVO79t13bGt0/hCBfNz7bnoLNJlf/QoiWp9Yr/6ysLIKCgtDpdADodDoCAwPJysrCz8/vum03bdrE7t27CQgI4Be/+AUDBgyofm7VqlV8/PHHdOrUiWeffZauXbvWK1B/f896bX+tgACvKw/GkF9ylkt71+HTPQqvqHEN3mdtHontyx/WHOD42SImDelst3ZupjpnJyI5OwfJuWnUqdunLmbNmsUTTzyBwWBgz549zJs3j82bN+Pr68szzzxDQEAAqqqybt06Hn/8cbZu3Vp9QqmL/PzSBl1FBwR4kZf3w4pbWp9YdOnJ5G3+B5eNHdD5hdZ7n3XRPdiT8GBv1mxOIrKjN0ZD3XNtrB/n7AwkZ+cgOdedqiq3vGiutdsnODiYnJwcrNaqcfJWq5Xc3FyCg4N/FGAABoMBgJEjRxIcHMzp06cBCAoKQlWrmpoxYwZlZWVkZ2fXO5mmoKg6XCf+HMXgSsWXy9BM9lmIXVEUfjK+K4UllXz13QW7tCGEEA1Va/H39/cnMjKShIQEABISEoiMjKzR5ZOT88Oc9snJyWRkZBAeHl7juV27dqGqKkFBQU2SQEOo7j64Tvw5tuIcKna9b7dROT07+9Kvqz+bvjlLabnZLm0IIURD1Knb56WXXmLRokUsX74cb29vli5dCsCcOXNYsGABUVFR/PnPfyYxMRFVVTEYDPzxj38kICAAgIULF5Kfn4+iKHh6erJixQr0+ibrcWoQfUgkxsH3YDrwKeYO3TH2mWSXdu4d25XF733Lpr3pxE/obpc2hBCivhStlQxGb6o+/2tpmo3yL/6KNSMR9+m/RRcY0dgwb2jV5mR2HcvisWmRjIwKrv0FjST9os5BcnYODuvzb8sURcVt/FwUdx/Kt76NVlFql3buv6MHvcN8eW9zsiz5KIRoEZy6+AMorp64TZqPVlZE+baVaFrTL8do0Ov4xd396B7ajpUbkvjuVF6TtyGEEPXh9MUfQBcYgcvw2VjPH8N0ZJNd2nAx6nhqZn/Cgr34+/oTHE/Nt0s7QghRF1L8rzD0noC+61BMB9diyUy2SxtuLnp++ZP+hLT34K21x0lOb113SQsh2g4p/lcoioLr6IdR23Wg4qsVdlsAxt3VwLPx0QT6uvG3z45x+kKRXdoRQohbkeJ/DcXohuukJ9HMFXZdAMbL3civ4qPx9XLlL/89SlpWsV3aEUKIm5Hi/yM6v1BcRz9s9wVg2nm68NysaDzdDPz54yOcy3Gu4WtCCMeS4n8Dhu4jMESOs/sCMH7ervz6pwMwGnS8+fERMi7K8o9CiOYhxf8mXIbPbpYFYNr7uPHrnw5AVRT+9J/D5BSW2a0tIYS4Sor/TVxdAAaougHMYrJbW0F+7vzqpwOwWjXe+OgwF4vsM9mcEEJcJcX/FppzAZjQ9h78alY0FZVW3vjPYQpLKu3anhDCuUnxr4U+bADG/lMxJ2/HfGqPXdvqHOTFL+OjKSkz88ZHh7l02X6fNoQQzk2Kfx0Yb7sHXXBPKnavxlqQYde2IkK8eXpmfwpKKnjzP4dlKmghhF1I8a+D5loA5qoenXxYcE8/sgvKefM/RyirkBOAEKJpSfGvo+ZaAOaq3mF+PHl3FBfySvnLf49SXmmxa3tCCOcixb8eri4AY0nZjznpK7u316+rP0/E9SUtq4T/+/QYlWb73HEshHA+UvzryRg9FV2nflTu/Qhrbqrd2xvUM4DHYyM5db6It9Yex2yRE4AQovGk+NdTcy0Ac61hvTvwyNRIEtMKWLEuEYu16dccEEI4Fyn+DdAcC8D82Kh+wTxwRw+OnLnIyg2JWG1yAhBCNJwU/wZqjgVgfmz8wI7MmtCNg9/n8d6m5AataSyEEAB6RwfQmhl6T6ia/fPgWnRB3dCHRNq9zTuGdMZksbF2ZyoGvcpDU3qhKIrd2xVCtC1y5d8IzbUAzI/FjAgjZkQXdh7N4t9bT9t92KkQou2R4t9IzbUAzI/dNTqCO27rxFeHLvDp9hQ5AQgh6kWKfxNorgVgrqUoCvETujF+YCif7z/H+t1pzdKuEKJtkD7/JmLoPqKq+B/djC6oO/qwAXZvU1EU7ru9B2azjQ170jEadEwd1sXu7QohWj+58m9CzbUAzLVUReHhO3sxtHcQn25P4cuD55ulXSFE6ybFvwk15wIw11JVhcemRTKwRwAfbT3NF3vTm6VdIUTrJcW/iTXnAjDX0utUnojrQ7+u/iz/7CjfnMhqtraFEK1PnYp/Wloa8fHxTJ48mfj4eNLT02tss2zZMoYPH05cXBxxcXG8/PLL1c+Vl5fz9NNPc/vttzNlyhS2bdvWZAm0RM25AMx17epU5t/Vl37d2vPupmS+Tc5ptraFEK1Lnb7wXbx4MbNnzyYuLo7169fz4osvsmbNmhrbzZgxg4ULF9b4+7vvvounpydffvkl6enp3HfffWzZsgUPD4/GZ9BCGW+7B2tuChW7V6O2D0PnF9os7Rr0On73yFCeX76bdzYmYdCpDOgR0CxtCyFaj1qv/PPz80lKSiImJgaAmJgYkpKSKCgoqHMjn3/+OfHx8QCEhYXRt29fdu7c2cCQW4fmXgDmWq4uep6e2Z/OQV6sWH+CE6n5zda2EKJ1qLX4Z2VlERQUhE6nA0Cn0xEYGEhWVs0+5U2bNhEbG8ujjz7K4cOHq/+emZlJaOgPV77BwcFkZ2c3Rfwt2nULwOx4t9luAANwc9Hzy/j+hPh7sGztcU6ebZ67j4UQrUOTjfOfNWsWTzzxBAaDgT179jBv3jw2b96Mr69vk+zf39+zwa8NCPBqkhga1vgQiioeomDr+9h2rSRoxjMoeoP9mw3wIgD4/fxR/Gb5Hv7vs2O8MncEkeF+dm/bURx6nB1EcnYO9si51uIfHBxMTk4OVqsVnU6H1WolNzeX4ODgHwX3Q7/yyJEjCQ4O5vTp0wwZMoSQkBAyMjLw86sqPFlZWQwdOrRegebnlzZoFsuAAC/y8krq/bomFTEOl+GVlO39iHP/fg23259E0bvYrbkf5/zLmf34w4ffsfidb3g2fgARId52a9tRWsRxbmaSs3NoaM6qqtzyornWbh9/f38iIyNJSEgAICEhgcjIyOpCflVOzg8jS5KTk8nIyCA8PByAKVOm8PHHHwOQnp7O8ePHGT16dL2Tac2MUZNxGfMI1vMnKP/8z836HUA7Txee++kAPFwNLP33d+w4kiFzAQnh5BStDlUgJSWFRYsWUVxcjLe3N0uXLiUiIoI5c+awYMECoqKiWLhwIYmJiaiqisFgYMGCBYwdOxaAsrIyFi1aRHJyMqqq8txzzzFp0qR6Bdqqr/yvYT6zj4ptK1Hbd8H9zmdRXBvenXUzN8u5+LKJdzYmkpheyLDeQTw4pSeuxrYxw0dLO87NQXJ2Dva68q9T8W8J2krxBzCnf0fF1uWoPh1wm/ocqnu7Jt3/rXK2aRqbvkln3e40gnzd+fmMvnQKbPoTUHNricfZ3iRn5+Cwbh/R9AxhA3Gb8jS24lzKNr6OrbT5hmKqikLsyHCemzWA8koLr645yM6jmdINJISTkeLvIPqOfXGb+iu0skuUbfh9s00Ed1WvLr689OgQuoW24/3PT/LPhCQqTJZmjUEI4ThS/B1I36EH7jELwVxJ2YbfYy3MaNb223kYeTY+mhmjwtmXmMOS1Qe5kFfarDEIIRxDir+D6QLCcIv9DWga5Rtex3oxvVnbV1WF6aPC+dWsaC5XWHh19UF2HZNuICHaOin+LYDOLxT36c+DwYWyjUuxZp9u9hgiw/x4+ZHb6BrajlWbT/LupmQqTc13R7IQonlJ8W8h1HZBuE9/HsXdm7LNb2DJSGr2GNp5uvBsfDTTR4ax90Q2r6w+QIZ0AwnRJknxb0FUT3/cY3+D6hVI+Rd/xnL2cO0vauoYVIUZoyN4dlY0l8vNLFl9kN3HZG0AIdoaKf4tjOrug3vsIlS/TpRveQtzyn6HxNE7zI+XHh1CRIg3721O5t1NSVSapRtIiLZCin8LpLh64j7t1+iCulLx1d8xn3TM9Nc+ni78atYApo8M45vj2by6+iAZFy87JBYhRNOS4t9CKUY33KY+i65jHyp2vofpxJcOieNqN9Av46MpLjOxZPUBWSJSiDZAin8LpuhdcJv8FPqwgVR+8yGVhzc6LJY+4X689MgQwjt488+EZN7bnCzdQEK0YlL8WzhFZ8B10jz03YZjOvAZld9+4rAx+L5eLvzqp9HEjOjCnmNZvLrmIFn50g0kRGskxb8VUFQ9ruPnYOg1DtORTVR+8wGaZnNILDpV5e4xXXnmJ/25VGrilfcPsjex7a/KJkRbI8W/lVAUFZfRD2GImow58SsqdryHZnPMCQCgb4Q/Lz86hC5BnryzMYn3P0/GJN1AQrQaUvxbEUVRcBk2C+PAOCyndlPx9Qo0q+MmY/P1cuG52QOYNrwLO49KN5AQrYkU/1ZGURRcBt+Fy9B4LKkHKP9yGZrF5LB4dKrKPWOruoGKrnQD7ZNuICFaPCn+rZSx/524jHoQ67ljlH/xFzRzhUPjiYrw56VHbqNzkCcrNyax+ouT0g0kRAsmxb8VM/aegOu4x7FmnaRs85/QKh3b5eLn7cqvZw9g6rAu7DiSyWv/OkR2QZlDYxJC3JgU/1bO0GMkrpPmY8tLoyxhKbbyYofGo1NV7h3Xladn9qOwpJKX3z/A/qQch8YkhKhJin8bYAgfjNvkp7AVZVG+8VDCZFoAAB+tSURBVHUsxc23LOTN9OvanpceuY1OAZ78Y0Mia/73PWaLdAMJ0VJI8W8j9J364Xbns9guF5L5r99hK85zdEjV3UB3Du3M9sMZvLbmEDnSDSREiyDFvw3Rh/TCfdpz2CrKKNv4e6xFmY4OCb1OZeb4biy4tx/5xRW8/P4Bvk2WbiAhHE2KfxujC+xK8P0vg81atSxk/jlHhwRAdLf2vPTIEEIDPPj7+kTeTUiipMxxQ1SFcHZS/Nsgl6Aw3GN/AzoDZRv/gDXnjKNDAsC/nSsLZw8kZkQX9iXl8Nt39rPneJasFyyEA0jxb6NUn2Dcp/8GxcWDsk1vYMlMdnRIQFU30N1jurL4kdvo4OfOu5uSeeOjwzIkVIhmJsW/DVO9AnCf/jyqpz/ln/8Zy7mjjg6pWscATxbdP5AHJ/fkbE4pL777LRv2pGG2OG6+IiGciRT/Nk718MUtdhGqTwjlW/4Pc8q3jg6pmqoojBsQymtzhjKge3vW7UrjpVXfcup8kaNDE6LNk+LvBFQ3b9xjfo0uIIKKr5ZTeWRzi+pn9/F04ecz+vL0zH6YzDb+8OF3vP95MqXlZkeHJkSbVafin5aWRnx8PJMnTyY+Pp709PSbbpuamkr//v1ZunRp9d8WLVrEmDFjiIuLIy4ujhUrVjQ6cFE/iosHbtOeQx8xBNO3/6Vy1yo0m+NmBL2Rfl3b8+rjQ5kytDO7j2Xzu3f2sS8xu0WdqIRoK/R12Wjx4sXMnj2buLg41q9fz4svvsiaNWtqbGe1Wlm8eDGTJk2q8dzcuXO5//77Gx+xaDBFb8R14hOYvAMxHUnAVpKP2+3zUYzujg6tmotRx0/Gd2NY7yBWf/E9KzcmsedENg/c0YNA35YTpxCtXa1X/vn5+SQlJRETEwNATEwMSUlJFBQU1Nh25cqVjBs3jrCwsCYPVDQNRVFxGXIvrmMexZp5krL1r2IrcfzdwD/WOciL3z4wiPtu70FKxiVeePdbNu1Nx2KVL4SFaAq1Fv+srCyCgoLQ6XQA6HQ6AgMDycrKum67kydPsnv3bh5++OEb7mfVqlXExsYyb948UlJSGh+5aBRDrzG4Ta2aDqJs3RKsuamODqkGVVWYOKgjr80ZRr8Ifz7bkcrL7x/gTMYlR4cmRKtXp26f2pjNZl544QVef/316pPEtZ555hkCAgJQVZV169bx+OOPs3Xr1htuezP+/p4Nji8gwKvBr22t6pRzwFBMHUPJ/vg1yhP+QEDcAjx7Dbd/cPUUEODFSz9rz/4TWfx97TFe/+AQU4aH8eDU3ni6Ga7bztlIzs7BHjkrWi3fpuXn5zN58mT279+PTqfDarUydOhQtmzZgp+fHwCZmZncddddeHh4AFBcXIymaUydOpUlS5bU2OfQoUNZu3YtoaGhdQ40P78Um63+X/wFBHiRl1dS79e1ZvXN2VZeTPn//oYtNwXjkJ9g7H8niqLYMcKGK6+0sG5XGlsPncfb3cjs23swuGcAgYHecpydgORcd6qq3PKiudYrf39/fyIjI0lISCAuLo6EhAQiIyOrCz9ASEgI+/fvr/592bJllJWVsXDhQgBycnIICgoCYNeuXaiqWv27cLyqoaALqdj+T0zf/hetOAeXUQ+gqE3ywbBJubno+emk7gzvG8Tqz79nxboT9Ovqz4JZA2XcshD1UKd/3S+99BKLFi1i+fLleHt7Vw/jnDNnDgsWLCAqKuqWr1+4cCH5+fkoioKnpycrVqxAr295hcWZ1RwJdLHFjQS6VlgHb3730CC+OniB/7crjflvfE3cyHBuv60jOlVOA0LUptZun5ZCun3qrrE5m0/upGLXalSfINymPIPqFdCE0TW9/EsVfLIjlW+Tsukc6MlDd/YiPNjb0WHZnby3nYO9un3kEknU0BpGAl3Lv50rv3t0CPPv6ktxmYlXVx/kwy9PUV7Zsm5iE6IlkeIvbkgf2hv3uBdA70LZxtcxpx5wdEi3pCgKg3oG8urjw5gwsCNfH7rA7/65n+9Otbx7GIRoCaT4i5vS+YbgPuMFVP/OVGx9G9PRljUn0I24u+q5744ePP/gIDxcDby19jjLPjtGQXGFo0MTokWR4i9u6epIIH3EECr3/5fKXe+3uDmBbqRrSDtefHgwM8d3JTGtgN/+cz9fHjjfoO+NhGiLZMiNqFVrGwl0lV6ncufQLgzuGci/tnzPR1+dZm9iNg9N6UWXDs53o5AQ15Irf1EnrWVOoBsJ8HHjmZn9eSKuDwUllbyy+gD/+eo0FaaW/wlGCHuR4i/qpbWNBLpKURSGRAbx2pyhjO0fwpYD53nhn/vZevA8ZRVyEhDOR4q/qLfWNhLoWh6uBh6c0ovf3tuVaNcMPtuaxC/f3s2qzcmkZRU7Ojwhmo30+YsGuToSqPx/f6Ni69toQ3+CoV/LnRMIwFaciyXtEOb0QwTmpDAdjWkd/dntcQcJyTnsOpZFWAcvxg0IZWhkEC7Guk88KERrI3f4tkHNmbNmMVGx/Z9YUr/F0Gusw+YEulHOmqZhK8zAknYIS/pBbPnnAVD9u6APH4TqG0Ll/k/QinNQIidxwHUEXx/NI+PiZdxc9Izo24FxA0IJbe/R7PnUhby3nYPDJnYT4lZa2kggTbNhy0vHknYQc/ohtEs5gIIuqBsuw2ahDxuE6v3DdBX6jlFUfvsJ5sSt3NbuOKNjHifV3JPthzPYcSSDrw5doEcnH8YPCGVgjwAMeukpFW2DXPm3QY7K+Yc5gTrgNuXpZpsTSLNZ8So/z8Uju7CkH0a7XACKDl1oJPqwQejDBqC6+9xyH5bMZCq2/xPtcgHGfndiHDSDEhPsOZbF9iMZ5BVV4OVuYHS/EMZGhxDg49Ysud2KvLedg72u/KX4t0GOzNmSkUT5l8tQdAbcJj+NLjDCLu1oVjPWjMQrXTqH0SpLQWdA3ymqquB3iUZxqV93jWYqp3Lfx5hPbkf1DcF13Bx0AeHYNI2ktAK2Hc7gyJmLoEHfCH/GDQihX1d/h80iKu9t5yDFX4p/nTk6Z2thJuVf/AWt7BKu4+dgiLitSfarmcqxnD+OJf0QlnNHwVwBBjf0XaLx7z+Ky97dUAwujW7Hcv44FTvfQyu7hDF6GsaBcSi6qh7SguIKdh7NZOfRTIpKTfh6uTA2OoTR/ULw9Wp82/Xh6OPsCJJz3UnxlzeLQ1y7OphLI0YCaRWlWM4expx2CGvGCbBaUFy90IcNRB8+CF1IbxSdvslz1iovU7H331hO7UH171T1KcC/c/XzFquNo2fy2X74AonphaiKwoAe7Rk3IJTILr6ozTDqqSUc5+YmOdedFH95szjM9SOBxuEy6v46jQSyXS7Ekv4dlvRDWDNPgmZD8fSv6s4JH4QuqDvKj7pa7JWzJf0wFbtWoVVexjgwDmP0NBT1+iGgOYVl7Dicye7jWZSWmwnydWNsdCij+gVft8ZwU2spx7k5Sc51J8Vf3iwOpWk2TAfWYjqSgC60z01HAlWNwT+IOe0QttwUANR2HdCHD0YfPhi1fZdbfnKwZ85aRSkVe/6FJWU/akB41acA35Aa25ktVg6ezGPbkQzOXLiEXqdyW69Axg8IpWuod5PfA9GSjnNzkZzrToq/vFlahOtHAj2D4umPrfBC1Re2aYewFVwZg98+7EqXzuAbFtibaY6czanfUrn7X2jmclwG34MhanKNTyBXnc8tZfuRDPaeyKbCZKVjgCfjB4QwrE8H3FyaZoR1SzzO9iY5150Uf3mztBjXjgTC4IZWfGUMfofu6MMHoQ8b2ODhoc2Vs63sEpW712BJP4Qa1A23cY+jtutw0+0rTBb2JeWw/bsMzuWW4mLUMbx3EOMGhNI5qHEzi7bU42xPknPdSfGXN0uLYi3MpGLneygG16ounS4DUN3bNXq/zXpXs6ZhObOXij0fgNWCy9CZGPpMRFFuPuRT0zRSs4rZfjiDb5NzMVtsdA3xZtyAUG7rFYjRUP+pJFrycbYXybnupPjLm8UpOCJn2+VCKnauwnr+GLrgXriOfey6u4dvprTczDcnstl+OIPsgjI8XPWM6BvMuAEhBPvX/d4EOc7OQYq/FP86k5ybj6ZpmL/fSeXejwBwGTYLQ6+xdfpyV9M0Tp4rYtvhDA6fysNq0+jZyYdxdZxKQo6zc5C5fYRogRRFwdhrLPqOfanY8R6Vu97HknYQ1zGPonr61frayC6+RHbx5dJlE7uPZbLjSCb/2JCIp5uB0f2CGRsdQqBvy14xTbROcuXfBknOjqFpGubkbVTu+w+oOlyHz0bfY1S9hnhenUpi+5FMjpy+iE3T6B3my7joUKK7t0ev++HTQEvIublJznUnV/5CNBNFUTD2nlD1KWD7P6nY8S66tEO4jnm41onlrlIVhb4R/vSN8KewpJJdx6qmkli+7gTtPIyM7h/MmP4htG/n+InlROsmV/5tkOTseJpmw3z8SyoPfAp6I64jH0DfdWiDbvSy2TSOpeaz/XAGx1PyAYjq6s/0MV3pEuDusInlHKGlHefmIF/4SvGvM8m55bAVZVG+/R1suanowwfjMupBVDfvBu8v/9KVieWOZXLpysRyY/qHMKZ/808s5wgt9TjbkxR/Kf51Jjm3LJrNiunY55gOrkMxuuEy+iEM4YMbtU+L1UZ63mXW70ghMa0AVVHo382fsdGh9A33Q1Vb7nKajdGSj7O9OLTPPy0tjUWLFlFUVISPjw9Lly4lLCzshtumpqZy1113MXv2bBYuXAhAeXk5v/nNb0hMTESn07Fw4ULGjx9f72SEaI0UVYdLdAz6ztFUbHuHii/fwtJtGK4j7kdxvfk/zlvR61SGR4XQrYMXuYVl7Diaye5jWRw+fZH27VwZ0z+E0f2CaefZ9j8NiIap05X/gw8+yD333ENcXBzr16/ns88+Y82aNTW2s1qtPPzwwwQGBhIYGFhd/N966y2ys7N59dVXSU9P57777mPLli14eNT9hha58q87ybnl0mwWTIcTMH23EcXNC9cxD6PvHN2gff04Z4vVxnen8th+OIOT54rQqQoDuldNM92rmaaZtrfWcpybkr2u/Gv9pig/P5+kpCRiYmIAiImJISkpiYKCghrbrly5knHjxtX4VPD5558THx8PQFhYGH379mXnzp31yUOINkFR9bgMmoH7XS+guHhS/sVfKd+2EmtuCo3tgdXrVIZEBvHr2QN5bc5QJg7qSPLZQv70nyM8v3Ifn+8/S3GZqYkyEa1drd0+WVlZBAUFodNVzT2i0+kIDAwkKysLP78fbmI5efIku3fvZs2aNSxfvvy6fWRmZhIaGlr9e3BwMNnZ2U2VgxCtjq59GO53L8Z0aD2mY59jOf0Niodf1QR34YNvuGZBfQT7ezBrYnfuGRvBwZN5bD+SwSfbUvh/O1MZ1DOQcdEh9Ojk0+TTTIvWo0nG+ZvNZl544QVef/316pNEU7vVx5faBAQ0bvbE1khybiWmPYJ14k8oO32Ayyf3UZ68HfOJL9F5+ODecwgevYbh1qVvjQVkrqpLztODfZg+vjtns4v5Ym862w6eZ39SDp2CPJkyLIwJgzvh6W5s4sTsp1Ue50ayR861Fv/g4GBycnKwWq3odDqsViu5ubkEBwdXb5OXl8e5c+eYO3cuAMXFxWiaRmlpKUuWLCEkJISMjIzqTwpZWVkMHTq0XoFKn3/dSc6tUIdB6DoMwmNEOZbzx7CkHaTk2HZKvtuC4uKJPmwA+vDb0IX2rl5PuL45u+sU7h4VzrShnfk2OYcdRzJ5Z/0J3t+UxG29AhkXbZ9FZ5pSqz/ODeCw0T7+/v5ERkaSkJBAXFwcCQkJREZGXtflExISwv79+6t/X7ZsGWVlZdVf+E6ZMoWPP/6YqKgo0tPTOX78OG+++Wa9kxGirVOMbhi6DsXQdSiapRLL+RNVK5ylHsD8/S4wuqHvHI0+YjA2n+ENasPFoGN0v6pF58/llLD9SCZ7E7P55kQ2HfzcGdUvmOF9OjjFfQPOrE6jfVJSUli0aBHFxcV4e3uzdOlSIiIimDNnDgsWLCAqKuq67X9c/MvKyli0aBHJycmoqspzzz3HpEmT6hWoXPnXneTc9mhWM9aMRMyph7Cc/Q4qL6MYXNF16le1LkLnfigG1wbvv7zSwsGTuew6nsWZC5dQFIiK8GdUVDD9u7WvdYbR5tLWj/ONyE1eUvzrTHJu2zSbBWvmSfRZxyg9uQ+tvBh0BvSdoq4skBN9w3WS6yq7oIw9x7P45kQ2hSWVeLjqGdanA6OigunSwbH97c50nK+S4i/Fv84kZ+cQEOBFbs4lrNmnsKQdxJJ2EK2sCFQdutA+GMIHow8b2OAbyWw2jcT0gis3j+VhsWp0CvRkVFQww/oE4eWAL4md9ThL8ZfiXyeSs3P4cc6aZsOWm4r56omg5CIoKrqQyCtrJA9q8JKZpeVm9iflsPt4FmezS9CpCtHd2jOyXzBREX7NNrmcHOe6k+IvbxanIDlfT9M0bBfPVn1ZnHYQ7VI2oKAL7lHVNRQ2qNbFZm7mQm4pu49nsTcxm5IyM+08jIzo24FR/YLrtQxlQ8hxrjsp/vJmcQqS881pmoatMANL6gEsaYewFV4AQA3sWtU1FD64TmsP/5jFauNYSj67j2VxLCUfm6bRNcSbkf2CGdIrCHfXpl8uRI5z3UnxlzeLU5Cc685WlFXdNWS7eBYAtX0XdCGRKKoeVBUUHSgKqCqKogNFvfJ3tfqxcs3jMpPGqfOXSDp3iYvFJlSdSteOPvQJD6BzsDeKqrtu+6r96EBVrvxdV/13xdWr+l6Gpsq5NZOVvIQQTUL1CcZlQCwuA2KxFedVdw2ZE7eCzQaarf77BHpd+eHqgKCCqp+Keu5LcfPGOCAWQ6+xKPrWc+dxayPFXwgnpnoHYOx/J8b+d1b/TdM00LSqk4BmA5v1ymMNrfrx1eeq/qtd8/jqaywWC6fOFXI89SLnsy6hKBqdA9zp08WHbiFe6HX86DU2NJsFS+oBKr/5ENPRz6tOAj1H3/STgGg4+T8qhLiOoihVXT43mPS3PhM/6ICoUIgaDgXFFew5kc2eY1ls2VuOq7Gc23oFMqpfMN1C2103pYSx9wQsGUlUHlxL5e7VmI5uwmVgHPruIxqdm/iB9Pm3QZKzc2iNOWuaxukLl9h9LIsDJ3OpNFsJ8nNnVFQHRvQNvm5KCU3TsJ4/TuXBtdgupqO0CyJg3E8pC+jXqBlPWxv5wleKf51Jzs6htedcYbJw8GQeu49ncep8EYoCfcP9ie7mT4CvGwE+bvh7u6JTFSxnD2M6uBZbwQVU31CMg2agDx9U9WVxGydf+Aoh2hRXo55R/YIZ1S+YnMIy9hzP5psTWRxPza/eRlHAz8uVAB9XAtrNZkC383TM2Ipt69vg2wnX2+6ums6iBc9E2lLJlX8bJDk7h7aYs03TuFRqIq+o/Ec/FeQVlXPpsgkFG4OMaUxxO0aAroRMgkj0HoU5oCcBvu4E+PzwqaGlTEjXGHLlL4Ro81RFwdfLBV8vF3p08qnxvJe3GydT8sgriia18HYyL3xL14Jd3F78GakFQSSURZNiCQKqvpz28XK5cjJwrT4pXP3xdjc49ScGKf5CiFbD1UVPaIAnoQGeQHsYEoZmvQvzyR10PZzAAv3/qPTvwfmQiZyzBlZ/ckhMK6Co9Pr1i40GtepE0M6txgmifTtXjAb7rErYUkjxF0K0aorOgLHPJAw9x2BO2oZyJIFux1fQs3N/XIbdha59bwBMZisXL1XU6ErKu1RO0tkCTObrb27z8TQS6ONGoK87Ab5uVx5X/Xi4GhyRapOS4i+EaBMUvRFjv8kYIsdiStyK6ejnlK19CX3YIIyDZ2D060RIew9C2tecfE7TNIrLzOQVlXOxqJzconLyCqv+ezwtn0vHr//U4OGqJ/DKiKSr/w268n2Dj6exVXQnSfEXQrQpisEVl+gYjL0nYDq+BdOx/2FJ/w591yG4DJqB6hNc8zWKQjsPI+08jHQLrTntdaXJSt6Vk0Ju4dWTQxlpWcUcPJmH7ZpxM0a9Wv1Joeqk4Fb9u38712ab/ro2UvyFEG2SYnTHZdAMjH0mYTr2BaYTX2JJ/RZ99xG4DIxD9Q6s875cjDo6BnrSMbDm6BmL1UZ+cUX1J4Xcwh9+TqQVYLb80J2kUxX8vV2v70bycau+r8GlGb9nkOIvhGjTFFdPXIbciyHqDkxHN2NO/ArL6X0Yeo7GODAW1dO/UfvX61SCfN0J8q25dObVoau5hWXVJ4a8K//dn1lMWaXluu2vfs8Q4Fv1XUOwnzt33GK4ZqPitstehRCihVHdvHEdNgtj1GRMRxIwJ2/HfGo3hsixGAfEorrXHFra6DavGbras7NvjedLy83VJ4OrJ4i8wqrRSXuOZwMQFOBFJ3+3Jo9Nir8QwqmoHr64jnwAY/+pmL7bgDlpG+aTOzH0mYix/1RUN+9mi8XTzYCnm4Hw4JptVpqtlJSZiOwWaJeb+aT4CyGckurpj+uYRzBGT6Py0HrMx/+HOWkbxr63Y+x/J4qLfZekrI2LQYdLu6a/4r9Kir8Qwqmp3oG4jZ+DdcA0TAfXYTqSgCnxK3SB4T+sXKaoVTOJKtf+KNevaFZjpTIFRdXV/zXXrpamd0XzH2mXvKX4CyEEoPMJwW3SPKwFsZiObMJWcrF6ERtsVxasufKjXbsIzY8XtbnmNddt00Dl/u3AK6IJM60ixV8IIa6h8+uE24QnmnSfNVZHu3b1smtPGD9eEU3V4R7RlcvS5y+EEK1PU62O1pRaxq1mQgghmpUUfyGEcEJ16vZJS0tj0aJFFBUV4ePjw9KlSwkLC7tum88++4z3338fVVWx2WzMnDmTBx98EIBly5bx73//m8DAqtupBw4cyOLFi5s2EyGEEHVWp+K/ePFiZs+eTVxcHOvXr+fFF19kzZo1120zefJk7r77bhRFobS0lNjYWIYMGUKvXr0AmDFjBgsXLmz6DIQQQtRbrd0++fn5JCUlERMTA0BMTAxJSUkUFBRct52np2f1NKYVFRWYzeZWMa2pEEI4o1qLf1ZWFkFBQeh0VbPN6XQ6AgMDycrKqrHtV199xbRp0xg/fjyPP/44PXv2rH5u06ZNxMbG8uijj3L48OEmTEEIIUR9NelQz4kTJzJx4kQyMzOZP38+Y8aMISIiglmzZvHEE09gMBjYs2cP8+bNY/Pmzfj61pzo6GZutRBxbQICvBr82tZKcnYOkrNzsEfOtV75BwcHk5OTg9VqBcBqtZKbm0twcM0FEa4KCQkhKiqK7du3AxAQEIDBULXs2ciRIwkODub06dNNEL4QQoiGqLX4+/v7ExkZSUJCAgAJCQlERkbi5+d33XYpKSnVjwsKCti/fz89evQAICcnp/q55ORkMjIyCA8Pb5IEhBBC1J+iadesP3YTKSkpLFq0iOLiYry9vVm6dCkRERHMmTOHBQsWEBUVxe9//3v27NmDXq9H0zRmzpzJAw88AMDChQtJTExEVVUMBgMLFixg7Nixdk9OCCHEjdWp+AshhGhb5A5fIYRwQlL8hRDCCUnxF0IIJyTFXwghnJAUfyGEcEJS/IUQwgm16eKflpZGfHw8kydPJj4+nvT0dEeHZFeFhYXMmTOHyZMnExsby5NPPlljAr626q233qJnz56cOnXK0aHYXWVlJYsXL+aOO+4gNjaWF154wdEh2d22bduYMWMGcXFxTJ8+nS1btjg6pCa3dOlSJkyYUON9bLc6prVhDzzwgLZu3TpN0zRt3bp12gMPPODgiOyrsLBQ27dvX/Xvf/jDH7Tf/OY3DoyoeZw4cUJ77LHHtPHjx2vff/+9o8OxuyVLlmivvfaaZrPZNE3TtLy8PAdHZF82m00bPHhw9bFNTk7WoqOjNavV6uDImtaBAwe0zMzMGu9je9WxNnvlX9epqNsSHx8fhg4dWv17dHQ0mZmZDozI/kwmE6+88govvfSSo0NpFpcvX2bdunU89dRT1VOmt2/f3sFR2Z+qqpSUVC1iXlJSQmBgIKratsrX4MGDa8yZZs861mYXcL/VVNQ/npeoLbLZbHz00UdMmDDB0aHY1d/+9jemT59Ox44dHR1Kszh//jw+Pj689dZb7N+/Hw8PD5566ikGDx7s6NDsRlEU/vrXvzJv3jzc3d25fPkyK1eudHRYzcKedaxtnTpFtSVLluDu7s7999/v6FDs5vDhw5w4cYLZs2c7OpRmY7VaOX/+PL1792bt2rX86le/4he/+AWlpaWODs1uLBYL//jHP1i+fDnbtm1jxYoVPP3001y+fNnRobVqbbb4N2Qq6rZi6dKlnD17lr/+9a9t7qPxtQ4cOEBKSgoTJ05kwoQJZGdn89hjj7F7925Hh2Y3wcHB6PX66m6A/v374+vrS1pamoMjs5/k5GRyc3MZNGgQAIMGDcLNze26mYTbKnvWsTZbGeo6FXVb8+c//5kTJ07w9ttvYzQaHR2OXc2dO5fdu3fz9ddf8/XXX9OhQwfeffddRo0a5ejQ7MbPz4+hQ4eyZ88eoGokSH5+Pl26dHFwZPbToUMHsrOzSU1NBapmGc7Pz6dz584Ojsz+7FnH2vSsnjebirqtOn36NDExMYSFheHq6gpAx44defvttx0cWfOYMGECf//736vXkWirzp8/z/PPP09RURF6vZ6nn366zU+RvmHDBt55553qL7kXLFjApEmTHBxV03r11VfZsmULFy9exNfXFx8fHzZt2mS3Otami78QQogba7PdPkIIIW5Oir8QQjghKf5CCOGEpPgLIYQTkuIvhBBOSIq/EEI4ISn+QgjhhKT4CyGEE/r/IxtaoOftgI4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mjhiFLl8LCBq",
        "outputId": "f162b3ea-73b3-49e0-bcd4-86c3c5c53204",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 301
        }
      },
      "source": [
        "x = np.linspace(0, N_EPOCHS,N_EPOCHS)\n",
        "\n",
        "plt.plot(x,tacc)\n",
        "plt.plot(x,vacc)\n",
        "plt.title(\"Accuracy\")\n",
        "plt.legend([\"Train acc\", \"Valid acc\"])"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f656228f860>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAELCAYAAADdriHjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXhUVZ74/3fdWrKQfaXCFggQAoR9UxZlE8QEoog4oDJDiwsqPUxPQ9RuQNtuO/ym7VYEbR1B+DrOKNCAhIgYtWVREJA9ELZASKgkZN9TVbfu749gNIIkQFW2+ryeJ89Tde+595xTSe6n7jnnnqPTNE1DCCGEuEpp7gIIIYRoWSQwCCGEqEcCgxBCiHokMAghhKhHAoMQQoh6JDAIIYSoRwKDEEKIeiQwCLf16KOPMnToUKxWa3MXRYgWRQKDcEtZWVkcOHAAnU7HF1980WT52u32JstLiFslgUG4pc2bN9O/f3/uv/9+Nm/eXLfdYrHw7LPPMmLECIYPH87LL79ct+/jjz/m3nvvZeDAgUyZMoUTJ04AEB0dzcWLF+vSJSYm8te//hWAffv2MWbMGN555x1GjhzJ888/T0lJCU8++SQjRoxg6NChPPnkk+Tk5NQdX1xczPPPP8+oUaMYOnQo8+fPByAuLo4vv/yyLp3NZmP48OGkpaW55kMSbsvQ3AUQojls2bKFf/3Xf6V///7MnDmT/Px8AgMD6y7YX375JXq9nmPHjgHw6aefsmLFClauXElsbCyZmZkYDI3798nPz6ekpISvvvoKh8NBVVUVDzzwAH/7299QVZUXXniBl19+mVWrVgGwaNEivL292bZtG97e3hw6dAiAadOm8cknnzBu3DgAvv76a8LCwujdu7cLPiHhziQwCLdz4MABLl++zL333ktQUBCdOnUiOTmZ/v37k5eXx6JFi+ou+kOGDAFgw4YNPP744/Tr1w+ALl26NDo/RVFYsGABJpMJAE9PTyZNmlS3/+mnn+axxx4DIC8vj507d7Jv3z78/f0BGDZsGABTp05l1apVlJeX4+PjwyeffMLUqVNv89MQ4loSGITb2bx5MyNHjiQoKAiobaLZtGkTYWFhREREXPdOwGKx0Llz51vKLzAwEA8Pj7r3VVVVvPrqq+zatYuSkhIAKioqUFWVnJwc/P3964LCT4WHhzNo0CA+++wzJk6cyM6dO3nxxRdvqUxC3IgEBuFWqqur+fTTT3E4HIwcORIAq9VKaWkpISEhWCwW7Hb7NcHBbDaTmZl53XN6eXlRVVVV9/7KlSuEh4fXvdfpdPXSr169moyMDD7++GNCQ0M5efIkCQkJaJpG+/btKSkpobS0FD8/v2vyuv/++1m/fj2qqjJgwIB6+QjhLNL5LNxKamoqer2ebdu2sXnzZjZv3kxKSgpDhgwhNTWV0NBQ/vKXv1BZWUlNTQ0HDx4E4MEHH2T16tUcP34cTdO4ePEi2dnZAPTq1Yvk5GRUVWXnzp3s37//hmWoqKjAw8MDPz8/iouLefPNN+v2hYWFMWbMGF566SVKSkqw2Wz1zjdhwgTS0tJYt24dCQkJLviEhJDAINzMpk2beOCBB4iIiCA0NLTuZ/bs2Wzbto23336bixcvMnbsWMaMGcOnn34KwL333stTTz3Fb37zGwYNGsQzzzxT1wz04osv8tVXXzFkyBC2bt3KhAkTbliGOXPmUFNTw4gRI5g5cyajR4+ut3/58uUYDAbuvfde7rzzTtauXVu3z9PTk3vuuYesrCwmTpzo5E9HiFo6WahHiNblzTff5MKFC/zXf/1XcxdFtFFyxyBEK1JcXMzGjRuZOXNmcxdFtGESGIRoJT7++GPuvvtuRo8ezdChQ5u7OKINk6YkIYQQ9cgdgxBCiHokMAghhKhHAoMQQoh62sSTz0VFFTgcN99VEhzsQ0FBuQtK1HJJnd2D1Nk93GqdFUVHYGC7X9zfJgKDw6HdUmD44Vh3I3V2D1Jn9+CKOktTkhBCiHokMAghhKinTTQlXY+maRQVXcFqrQauf6uVl6fgcDiatmDN7PbrrMNk8iQwMPSaWUOFEG1Dmw0M5eUl6HQ6wsM7otNd/8bIYFCw290rMNxunTXNQXFxPuXlJfj6BjixZEKIlqLNNiVVVZXj6xvwi0FB3BqdTsHXN5CqKvca/SGEO2mzV02HQ0Wvb7M3RM1KrzfgcKjNXQwh3JorZzNq01dOaQN3Dflchbh9msMBtiq0mko0648//PC+5jrbfrIdh4rf7GXgEeH0srXpwNBSzJs3B5vNht1u49KlTLp2jQKgZ89oXnhhaaPOsXnzBmpqapg5c7YriyqEaCTN4QDrzy/YVbXbfnYh52cXda2mEmxVDWdi8kJn8kbn4Y3O5I3iEwJBV997+mIM6Qhlzu8nlcDQBN59t3YFLovlMo8//ijvv//hNWmut87wTyUkPOiy8gkhbkxzOHDkncOeeRh75hEcZflgq27gKB2YPOtf2H1D4Orrn26/7jajFzrlxq39es92UFbmvIpeJYGhGT34YDzjx9/D99/vp1u37jzxxHyWLXuRiooKrFYrd945kvnzfw3Ae+/9naqqKp599t9JSdnK559vx9fXj/Pnz+Hr68MrrywnODjkmjzefPNvHD78PTabjYCAAH73u6WEhrYHYM+eXaxe/Q52ux1F0fHiiy/RvXsPjh8/ysqVr1NZWQnAM8/8mmHDRjTdByNEC6DZarBnH8d+4TBq5mG06jLQ6dGbe2KM6F13Add5eMPPLuo6D28werbawS9uERj2HLOw+6jlmu06Hdxu/82ofmZGxppv+fiKigrefXcdADU1NSQl/RVvb2/sdjv/8R/PsnfvN4wYcec1x508mcbatf9LeHh7kpJeYcOGj3jyyWeuSffII//Ks8/+OwBbt25m5co3WLbsT2RmXiQp6RVWrnyXTp06Y7VasdttlJaW8MILv+WPf1xObGx/VFWloqLilusnRGviqCjCfvEw9ouHUC+ngWoHkzeGTv0wdBmAoVMsdr0XdtWBougw6BUUpe31ublFYGjJJk++r+61w+Fg1arXOXbsKKBRUFDAmTOnrxsY+vXrT3h47Tf/Pn36sn//vuuef+/ePfzjH+upqqpEVdW6juP9+/cxYsSddOrUGQCTyYTJZOKbb3YTGdmV2Nj+AOj1evz8/JxZZSFaDE3TcBRk1jYRXTyM40oGADrfUNSoMeT7RpOptcdSVEPOwUpyPz9CQUl1vUdmdYBer0OvKOgV3dXXte8Neh16/dXtdft+TGdQlHrpa7f9+PradD8e62nSc+8ob5d8Lm4RGEbGXv9bfUt4wM3b26vu9Ucf/Q9lZaW88877eHh4kJT0R6zWmuseZzKZ6l4rih5VvXb4aE6OhRUrXuPdd9cREdGBY8eO8NJLv3N+JYRoRTTVhmpJx37hELYLh6CyEA0dJV4dyGg3mkNVHTiZ7YH1ogZUAOfwMOlpH+hNVAd/7uzbHk+TAdXhQFU17A6t7rWq1r62O358rda9vrpP1bDaVdSaH7ZpqKrjmtf2H16r2i/M3QBdOwVi9vd0+mfkFoGhtSgrKyM4OAQPDw+uXMlj9+6vSUiYfsvnq6iowGAwEhwcjMPhYPPmjXX7hg0bwdq173HpUma9pqS+fWO5cCGD48eP0rdvv7qmJLlrEK2VXXVwJfcKFecOob98FP/Ssxg1K1ZNzylbBMet0aTZOlJe5EWovxftg725u6s34UHetL/6E+BjatZh2o6fBJUfAohOB1FdgrlyRTqf27QZMx7m979fzKOPPkRoaDiDB9/egu9RUd0ZO3YCjzzyEP7+Adxxx0iOHDkEQKdOnVm06EWWLn0eVXWg1yu8+OJLREV1549/XM6KFX+luroKnU7hmWd+zdChw51RRSFcQtM0CkurSc8swlJYSU5BJVVXsgkoOUWkPYNuhjx8dBolDi8OO7qS064H1pCehAf7MzTIm/ggb8ICvTDoW2ZnsaLoUBQ9xia6Yus0Vz4+10QKCsqvmZM8J+ci7dt3ueFxLaEpqak5q86N+XxbitBQX5d8q2rJ2mqda6wquUWV5Fy9+OcU1v7kFlVSXWOjq+EKfY1ZxJouEaYvBaDUI5yqkN4Yuw4kuGs0Pl4ezVwL57nV37Oi6AgO9vnF/XLHIIRosapq7KRfKuZERiFpFwqxFFTW22/20zHY9wrTwy4RYT2PwVaBpujRm3th7DIQQ5cB+PpeO4xb3FijAkNGRgaJiYkUFxcTEBBAUlISkZGR9dIUFBTw/PPPY7FYsNvtDB8+nN/97ncYDAZWrlxJSkoKiqJgNBpZuHAho0ePBiAxMZFvvvmGwMBAACZPnszTTz/t3FoKIVoFh0PjQk4ZJzIKOHGhiHPZJagODZNBoWfnAEb0Dqejj42I6nO0yz+BZjkFVXbwaIdP9GDs4X0xdOpb+4CYuGWNCgxLly5l1qxZTJs2jS1btrBkyRLWrVtXL83bb79NVFQU77zzDjabjVmzZrFjxw6mTJlCv379mDt3Ll5eXpw6dYpHHnmE3bt34+lZ25v+xBNP8Mgjjzi/dkKIFi+/uIrjFwpJyyjk5MUiKqrtAHQJ92XKkPbEBlfTwViKrvQU6uWTOE5crD3QLxxjn/EYugxE374HYeEBbbL5rDk0GBgKCgpIS0tjzZo1AMTFxfGHP/yBwsJCgoKC6tLpdDoqKipwOBxYrVZsNhvh4eEAdXcHANHR0WiaRnFxMe3bt3d2fYQQLVxVjZ1TF4vqgkFuURUmbPTwqyIhwkq3duUEa4UopRa0swVwFuwAegP6kK6Yhj2EIXIAir9ZJnR0kQYDg8ViITw8HL1eD9Q+8BQWFobFYqkXGObPn89zzz3HqFGjqKqqYvbs2QwePPia823evJnOnTvXCwpr1qzho48+olOnTvzmN78hKirqpipxvU6UvDwFg6HhEQaNSdPWOKPOiqIQGurrhNI0jdZUVmdpKXVWVQdnLhVzKD2PY+mXKbl8kXBdER1NJTzmU0mYuQhTTVFt4mKgzIApuAPGLjGYQjphCq39MQSEo1P0N8yrpdS5Kbmizk7rfN6+fTvR0dGsXbuWiooK5s2bx/bt25k8eXJdmu+++47XX3+d1atX121buHAhoaGhKIrC5s2befzxx0lNTa0LRI1xvVFJDoejwdE3Mirp1jkcjlZz295WR+jcSHPXOe9KMRmnTlNwKQO1MIsQiojSFzNMX47uh+uYYkDxaY8S2BMlsANKYAT6wA7o/MLqAoD96k+lCvys4/nnmrvOzaHZRiWZzWZyc3NRVRW9vvYJ27y8PMzm+k8Sf/DBB/zpT39CURR8fX0ZN24c+/btqwsMhw4d4re//S2rVq2iW7dudcf90NwEkJCQwKuvvkpOTg4dOnS46coKIZqWZrfiKLZQlZdJ/sXzWPMv4VGZRwCl9L7ayuMwKdi8Q/EI7YUppBNKYARKUAcUv4bvAETzaDAwBAcHExMTQ3JyMtOmTSM5OZmYmJh6zUgAHTt2ZOfOnfTr1w+r1cq3337LxIkTATh69CgLFy7kjTfeoE+fPvWOy83NrQsOu3btQlGUesGiLfjNbxYwevSYelNna5rGQw8l8MILSxg48NomN4A//nEZvXrFMH36zBuux5CSspVvvtnFK68sd1kdhHvT7FYcJTk4irJxFGZjL8zGmn8JfWUBuqsTNgRoOvId/pR4tacyeAghXboR1LErSkA4OkVGxrcmjfptLVu2jMTERFatWoWfnx9JSUkAzJs3jwULFhAbG8sLL7zA0qVLiY+PR1VVhg8fzkMPPQTASy+9RHV1NUuWLKk75/Lly4mOjmbx4sUUFBSg0+nw8fHhrbfeuuG6BK3RffdN5f/+74N6geHQoYMoio4BAwY16hyyHoNoSlp1OXZLOurVH0dhZt1UxA4Urjh8uWwPIFeNwOEXQWDHSCJ7dqdbx6AW+/SwaLxGXYGjoqJYv379NdvffffdutedO3euG7n0cxs3brzudoD333+/MUW4LbbTe7Cl77xmu06nu+11U43RYzD2HHnDNKNH38Vf/vIqFy5kEBnZFYBt2z5hypR4zp8/x1/+8meqq6uwWq1MnXo/Dz0065pz/HQ9BpvNxl//upzvvz+Av38APXpE/2Lesh6DaAxHZQlqTjrq5auBoCirdrtioMAYwUl1AOcrfchRA1B9wujdNZQ+XYOY0iWQdp7GZi69cLa29dW8hTIajUyceC8pKZ8wf/6vqaysYNeur/ngg4/x8fHhb39bhclkorKykieemMOwYXfUBZDr2bJlIxbLZT74YD12u51nnpl3TZ/PD2Q9BnE9jooiVMupq4HgFI6SHAA0gwel3p047TmSfQV+nK8JwmAy0btLIH27BjGzaxBhgfLwWFvnFoHB2HPkdb/VN+WopPvum8p//udzPPnks3zxxefExvYnLCycwsIC3nzzz5w9exqdTiE//wpnz56+YWD4/vuD3HtvHAaDAYPBwKRJ93L06OHrppX1GASAo+xKXbOQ3ZKOVppXu8PoRXVAVy6E9GF/oT/f53njQCEswIt+scFMjQqhZ6cAjG44rNuduUVgaAl69OhJcHAoe/d+Q0rKJ8yYUdtc9Pe/ryQoKJjVq/8Hg8HAwoXPYLVanZKnrMfgnjRNw1ZowXrqYF0w0MoLand6tIPQHuQED+VwaTC7LimU5zrQKzp6dPTnwbEh9O8eTPsgb3l4zI1JYGhC9903ldWr3yE318Lo0XcBUF5eRlRUDwwGA+fPn+XIkcNMnDj5hucZPHgI27enMG7cRFTVzuefb69bze2nZD0G96BpGo5iS23T0NVAUF5ZDIDO0xfFHE15t3GkVYWyN0vhzLFSNA18vIz06x5M/+4h9IkMwttTLgeilvwlNKGJEyezcuXrTJ16P0ZjbYfdnDm/4g9/WMK2bVvo1KkzAwYMbPA8U6c+wNmzZ3nkkRn4+wfQq1cfiooKrkkn6zG0TZrmwFGYfTUInELNOY1WVTvFtM47AL25Fz5RsZwoC+TAZYUjZwopKK0Gyukc5sN9d0TSPyqYrma/Nrlesbh9sh6DPPl8S2Q9hqajORw4CjPrOortOaehpnZAgM4nGL05Gr05mgq/bhzJ1XH0XCEnM4uosaqYDAq9I4Po1z2Yft2CCfJz/jKQLUVr/z3fClmPQQg3omkaauYRrCe/Qs05DdYqAHR+YRgjB6E390IX3pOL5SaOnCvg6Df5ZOadASDYz5MJQzvTI8KPXp0DMBnl6WJxcyQwCNGCaJqGmnWcmgObcFw5j84nGGPU8Kt3Bb2o1vty7EIhR0/nc3T7Kcoqbeh00KODPw/eHUX/qGAiQtoRFubndt+ehfO06cCgaZqMrHCBNtD62CLZs9OwHtiEmnsGnU8wHmP+DUOPO8kttnL0XAFHvsvgTFbtwjXtPA3EdgumX/dg+nYNxsdLHjITztNmA4Oi6FFVOwaD/MM4m6raUWTyM6ex55zGuv8fqJZT6NoF4jHqMa6EDGbvyXz2f32AvKLaZqQOoe2YNKwz/aKCiergh16RZwuEa7TZwODl5UNZWTEBAcHodPIP5Cya5qCsrAgvr1/uuBKNo+ado+bAJtSs4+i8/FAHzWSfrSff7isgM/cgOh30jgzinqGd6BcVTIi/V3MXWbiJNhsYfHz8KSq6Qm5uFnD9pg9FUXA43GtU0u3XWYfJ5ImPj7/TyuRu1PwLtQEh8wh4+JDd+V4+LezK8dRyNC7S1ezLv0zowbCYcPzbmZq7uMINtdnAoNPpCAoKu2EaGd4mmpJacAnrwU3YL3yPavDiaLvRrLd0psKiJyxAJX5kJCP6tKd9kMxFJJpXmw0MQrQUalE2NQc2o2bsx6bz4GvrQD4visbo1Y4R/cIZ0TecbmY/GSghWgwJDEK4iKMkh6JvNmK8tB+rZuSf1bHssfclpkdHnuoTTu9IWbtAtEwSGIRwsiJLNgW7NxBadBhN0/NlTR+yw0YxaGRXXu0ZgqdJ/u1Ey9aov9CMjAwSExMpLi4mICCApKQkIiMj66UpKCjg+eefx2KxYLfbGT58OL/73e8wGAyoqsorr7zCrl270Ol0PPHEE8yYMQPghvuEe1GLL1Oz833Q6dCHRaGER6EP647i3fI7uiur7Rw9dhrd8RSirScIQsdhQz+03pMZ2y9KOpFFq9KowLB06VJmzZrFtGnT2LJlC0uWLGHdunX10rz99ttERUXxzjvvYLPZmDVrFjt27GDKlCls3bqVzMxMduzYQXFxMQkJCdxxxx107NjxhvuE+7BfOkbVF6vQKQZ0vqFYj30GR1QAdL6h6K8GCX14d5Tgji1iDWG76uDY+QIOHz2H2fJPhhtPA3A5aAjBd97PXR0imrmEQtyaBv+7CgoKSEtLq1u2My4ujj/84Q8UFhYSFBRUl06n01FRUYHD4cBqtWKz2QgPDwcgJSWFGTNmoCgKQUFBTJgwge3bt/P444/fcJ9o+zRNw3b8c2r2/i9KUEe87vk1im9I7eLz+RdR886i5p5DvXwK+9m9tQfpTehDI2uDRBPfVTg0jbNZJexNy+XkyQvcoRxmmsdp9CaNms4jCB75AIG+IU1SFiFcpcHAYLFYCA8PR6+vfdJVr9cTFhaGxWKpFxjmz5/Pc889x6hRo6iqqmL27NkMHjy47hwRET9+ezKbzeTk5DS4r7FuNEtgQ0JDfW/52NaqpdRZU23kb/9vag6n4t1zGGHTFqCYfvIQlzkYGFSbVtNQS/Opzj5NdVY6NdmnqTm2A47YATAEhOHZIRqPDj3x7NATU3gkOv2Pf963W+fMnFL++X0WX3+fRUVxERPbpbGoXTp6VHz63kXQ6AcxBl67JkZzaim/56YkdXYOp92Pb9++nejoaNauXUtFRQXz5s1j+/btTJ5840VnnOF60243hjuO6W8pdXZUl1H9+ZuolnRMA+NRhtxPQYkduFHZPCG0H4T2wzQQjD+7q6jIOEb5iV21SX9yVxHYoy/lnh1u+q6iqKyGfWm57E3LITO3HG+lhofCM+gXfBi9w4YhagQeg6eh+Len2A60gM/1By3l99yUpM6Nd9vTbpvNZnJzc1FVFb1ej6qq5OXlXbP4/AcffMCf/vQnFEXB19eXcePGsW/fPiZPnozZbOby5cv069cPqH+XcKN9om1SC7Op+uxvaJVFeI57EmP3O27pPDqDCX37Hujb9wBq7yq0ikLU3LO1P3nnsB77jNwjKbXp6/oqoq72VXS6pq9C0zSOnivg8wOXOHmhCA2Ibm/iP/tk0TH/W3Q1VRi6DcM0OAF9oPydirapwcAQHBxMTEwMycnJTJs2jeTkZGJiYuo1IwF07NiRnTt30q9fP6xWK99++y0TJ04EYPLkyaxfv5577rmH4uJiUlNT+Z//+Z8G94m2x555mKov3kZn8MA7/gX0Yd2cdm6dTofOJxjl6lTVAJrdip96hYL0ozfsqyCkG0dK/Nh6qIjsKxUE+XmQcEcEdxrTMJ35AiwVGCIH1waE4E5OK7MQLVGjVnA7d+4ciYmJlJaW4ufnR1JSEt26dWPevHksWLCA2NhYMjMzWbp0Kfn5+aiqyvDhw3nxxRfrhqu+/PLL7NmzB4B58+Yxc+ZMgBvuayxpSmq85qqzpmnYjm6nZt/HKCGdazuZfYIaPtAJflrnn99V2HPOoRZcRNFqR0AV4wsh3QiJiEA9vRutugx95/54DLkffUhkk5TXGeRv2z24qimpzS7t2Rjyh9Q0NNVG9a612E/vxtBtKJ53P47O4NFk+V+vzmWVVr44mMWX32dTXVXNHeYaxnaqJkzNwZF7Fq2yGH3HvrUBISyqycrqLPK37R5kaU/RKjmqSqna8QaO3LOYBidgGjStWecEyi+pYsd3l9h59DJWm4MB3UOYMqIf3Tv+2DGtaRrYqtGZZJpr4Z4kMAiXUQsu1XYyV5XhOWE+xm7Dmq0sWXnlfLrvIvvS8tDpYETvcCYP70yH0Gu/Nel0OpCgINyYBAbhErYLB6n+8h10Ht54T30BfWhks5Tj9KViVm05wYGTuXgY9Ywf3JFJwzoR5OfZLOURojWQwCCcStM0rIe3Yd2/ASW0K16Tfo3iHdCkZXBoGkfO5vPp3kzOZpfg185EwuiujBvUUdZGFqIRJDAIp9HsVqp3rsF+9lsM3UfgOWYuOkPTTR5nVx3sS8vl032ZXM6vINjPk9kTe5IwrgdlJVVNVg4hWjsJDMIpHJXFVH32Bo4r5zENnY5pQFyTdTJXW+3sPHyZz/Zfoqisho6h7ZgX35uhvcIw6BU8TYYbPk8thKhPAoO4bWr+Bao+ewOtpgLPic9h7Dq4SfItrbTyxYEsvvw+i4pqO9GdApgzuRex3YJkNTQhboMEBnFbbOf3U/3Vu+g8ffCe9iL64M4uz/NKcRWffZfJ7qMWrHYHA3uEMGVEF6I6tPx1G4RoDSQwiFuiaRrW7z/BenATSnh3vCY+5/Kpry/llfPp3ot8d7J2yOkdfdozeXhnIkLauTRfIdyNBAZx0zR7DdX/fA/7+e8w9BiJ55h/Rad3zWgfTdM4famYlL2ZHDtfgIdJz8ShHZk4RIacCuEqEhjETXFUFFH12es48i/iMXwmxn6TXdKe79A0Dp3O59N9Fzl/uRRfbyP3j+nGuEEdaOcpQ06FcCUJDKLR1LzzVO14A81WjdekX2PoMsDpedhVB98ez2H7d5lYCioJ8ffkkXt6MirWjMmod3p+QohrSWAQjWI7u5fqr99D5+2P95TfoQ9y/prcpy8V8+7WNApKq+kU5sOTU/swpFcoekVxel5CiF8mgUHckKY5sB7YhPXQVvTmaDwnPIPi5efUPBwOjeRvL7Bldwah/l4sfKg/fbvKkFMhmosEBvGLNFs11V+9i/3CQYzRY/AY9Vi9dZSdoaishne3nuBUZjEjeofz6KRovDzkz1KI5tSo/8CMjAwSExMpLi4mICCApKQkIiMj66VZtGgR6enpde/T09NZuXIl48ePv+G+FStW8OGHHxIWFgbAoEGDWLp0qROqJm6Ho7yAqs/+hqMwC487ZmHsO9Hp3+CPnM3nvW0nsdpV5k6JYWRse7lLEKIFaNRCPY899hjTp09n2rRpbNmyhRryLVMAACAASURBVI0bN7Ju3bpfTH/q1CnmzJnDrl27MJlMN9y3YsUKKisrWbx48S1XQhbqabzG1FnNPVvbyWy34TXhaQyd+jm1DDa7g41fn2PH/kt0DPXh6YQ+mINd9yyC/J7dg9S58RpaqKfBXr2CggLS0tKIi4sDIC4ujrS0NAoLC3/xmA0bNhAfH39NUGhon2h+ttN7qNz6ZzB64Z3we6cHhdyiSv70wUF27L/E+EEd+f2cwS4NCkKIm9dgU5LFYiE8PBy9vnaooF6vJywsDIvFQlDQtWv2Wq1Wtm7dyvvvv9/ofdu2bWP37t2Ehoby3HPPMXDgwFurjbhlmsOBdf8GrEdS0EfE4DXhGXSev/yN4lZ8eyKHdZ+lY1B0PPtALIN6hjr1/EII53B6L19qaioRERHExMQ0at/DDz/MU089hdFoZM+ePcyfP5+UlBQCAwMbneeNbokaEhrqe8vHtlY/r7O9tIAr297Cev4QfoMmEXzPXKd2MlfV2Pn7pqN8sf8SvbsG8ZvZgwkL9Hba+RtDfs/uQersHA3+95vNZnJzc1FVFb1ej6qq5OXlYTabr5t+48aNTJ8+vdH7QkN//NY4cuRIzGYzZ86cYdiwxi8DKX0MjffTOmuaA9upndTs/QgcKh6jHkPrPY78QuetXZCZW8bbW06QW1jJ1JGRxI+MRGdXm/Rzd/ffs7uQOjfebfcxBAcHExMTQ3JyMgDJycnExMRctxkpJyeHgwcPEh8f3+h9ubm5da9PnjxJdnY2Xbt2bahY4jY5SnKpSl5Oza730YdG0m7GK5h6j3Pa+TVN44uDWbyy7gDVVjv/+S8DSRjdTR5WE6IVaFR7wbJly0hMTGTVqlX4+fmRlJQEwLx581iwYAGxsbEAbNq0ibFjx+Lvf+0sm7+077XXXuPEiRMoioLRaGT58uX17iKEc2kOFevRT6nZvwkUPR5j/g1j9BinDhMtr7KxJuUkh87k0y8qmLn3xeDnLYMNhGgtGjVctaWTpqTGUQuzUL9ZS83lM+g7D8Bz9ByUdo3vy2mM05eK+fsnJyitsDLj7igmDu3U7M8muNvvGaTO7sJVTUnyiKkb0FQ71sPJWA9tRfFsh+e4pzBEDXfqBdvh0Ej+5gJb9mQQGuDFi48NJrK9c6fOEEI0DQkMbZyad57qr1fjKMrC0H0EHeKeoLDSue38RWU1vPPJCdIvFTOiTziP3iPTWgjRmsl/bxul2WuoObAJ27HP0HkHXJ0meyD6dr5Q6bzb7cNn81m97SQ2u4Nf3RfDnX1lWgshWjsJDG2Q/fJJqne+j1aai7HX3XiMeAidybnPDdjsDtb/8yypB7LoHObDk9NcO62FEKLpSGBoQzRrFTX7PsJ28p/o/MLwiluMIeLaBw1vV25hJW9tOU5mbjnjB3fkobFRGA2yiI4QbYUEhjbCnnmE6l1r0SqLMMZOwmPoA+gMHk7P59vjOazbUTutxXMPxDJQprUQos2RwNDKOarLqPnmQ+xnv0UJ7IDXxGfQh0U5PZ9qq50Pdpzmm+M59OzozxNT+xDk5+n0fIQQzU8CQyulaRr2899Rs+cDtJpKTIOmYRoY7/SFdAAu5pTx9icnyCv6cVoLeYJZiLZLAkMr5Kgoomb3OuwXD6GEdsUrbi76oE5Oz0fTNFIPZrH+q7P4eBn57cMD6dXFuQ/ECSFaHgkMrYimadjSd1Kz9/9AteMxfCbG2HvQKc7v+C2vsrF620kOn82n/9VpLXxlWgsh3IIEhlbCUZpH9a73UbPT0Juj8RwzF8U/3CV5pWcW8c7WNMoqrfzLhB5MGNxRnk0Qwo1IYGjhNIcD24nPqdm/EXQKHqPmYIy5C53O+W38DofG1m8u8MmeDMICvHjx0SF0ae9+89sL4e4kMLRgatFlqneuxpF7Fn2nfrWT3vkEuySvwtJq3tmaxulLxdzRpz2P3NNTprUQwk3Jf34LpDnsWA+nYP3+E3RGTzzHPoGh+x0ua84pKKnmTx8cpLLazq/ui2Fk7PUXYRJCuAcJDC2Mmn+B6q/fw1FwCUO3YXiMfATFy3WzlJZX2Xjt48NUW1Wef2QQncOl6UgIdyeBoYXQ7FasBzdjPbodnZcfnvcswBg5yKV51thUXl9/hCvF1fxmZn8JCkIIoJGBISMjg8TERIqLiwkICCApKYnIyMh6aRYtWkR6enrd+/T0dFauXMn48eNZsWIFH374IWFhYQAMGjSIpUuXAlBVVcXzzz/PiRMn0Ov1LF68mLFjxzqpeq2D3ZJO9c41aCU5GKPH4DFiJjoP105IpzocvL35OOcvlzL//r5Ed5bnE4QQtRoVGJYuXcqsWbOYNm0aW7ZsYcmSJaxbt65emuXLl9e9PnXqFHPmzGH06NF12xISEli8ePE1537vvffw8fHh888/58KFC8yePZsdO3bQrp17zNRZczgZ63cb0PmG4nXfIgwders8T03TWLs9nSPnCnh0UjSDo8NcnqcQovVocMxjQUEBaWlpxMXFARAXF0daWhqFhYW/eMyGDRuIj4/HZGr4gahPP/2UmTNnAhAZGUnfvn3ZuXNnY8vfqtkvHsb63QYM3YbR7sFXmiQoAPxj53l2H7UwdWQkYwd2aJI8hRCtR4OBwWKxEB4ejl5f+3StXq8nLCwMi8Vy3fRWq5WtW7cyffr0etu3bdtGfHw8c+fO5dChQ3XbL1++TIcOP16czGYzOTk5t1SZ1sRRmkfVV++ghHTB8+7H0RmdPxPq9WzddZ5t317krgERTBvVtUnyFEK0Lk7vfE5NTSUiIoKYmB/XAXj44Yd56qmnMBqN7Nmzh/nz55OSkkJgoHPatW+0qHVDQkObvsPVYavh8pa3UBSFDjMXYwxwzbMJP7frcDbvbjnG8D7tWThrMHq9+0yE1xy/5+YmdXYPrqhzg4HBbDaTm5uLqqro9XpUVSUvLw+z+fpj3Tdu3HjN3UJo6I9z9o8cORKz2cyZM2cYNmwYERERZGdnExQUBNTeoQwfPvymKlFQUI7Dod3UMbXl8uXKFectc9kYmqZR/fVq7LkZeE1eSLHNG5qgDCcvFvHXjw8TExnEv02OprCwwuV5thTN8XtublJn93CrdVYU3Q2/UDf4lTE4OJiYmBiSk5MBSE5OJiYmpu5C/lM5OTkcPHiQ+Pj4ettzc3PrXp88eZLs7Gy6dq1txpg8eTIfffQRABcuXODYsWP1Oq3bGlv6Tuynd2EaNBVD5/5NkufFnDJWbDxKeKA3v587HJNRVlsTQvyyRjUlLVu2jMTERFatWoWfnx9JSUkAzJs3jwULFhAbGwvApk2bGDt2LP7+/vWOf+211zhx4gSKomA0Glm+fHndXcSvfvUrEhMTmThxIoqi8PLLL+Pjc+tNQy2ZeuUCNXv+H/qOfTENSmiSPPOKq/jr+iN4expY+FB/fLxNVFXUNEneQojWSadp2s23wbQwraEpSasup+IfS0HT8J7+Eoqn69tCSyus/OmDg1RU2Xj+kcFEhLST2203IXV2D83WlCRun6Y5qPrqHbTKErwmPtskQaHaaudv649QXFbDrx/sT0SIezwXIoS4fRIYmoD1+62ol47icecs9GHdXJ6fXXWwctNxMnPLeWpaX7p39G/4ICGEuEoCg4vZLx3FenAzhh53Yoxx/VQfDk1jdcpJTmQUMufeaAb0CHF5nkKItkUCgws5yvKp+vLvKEEd8Bw9p0lWQVv/1Vn2nsjlgTHdGN0vwuX5CSHaHgkMLqLZrVSlrgSHA6+Jz6IzuP7J5u37Mvnsu0uMH9SR++7o4vL8hBBtkwQGF6n59kMcVzLwHDsPxb+9y/P79ngOH391liG9wviXCT1kjWYhxC2TwOACttO7sZ38J6b+U1y+pgLA8fMFrE45Sa/OAcyL642iSFAQQtw6CQxOphZkUr1rLfqIGExDpzd8wG3KsJSyctNxIkLa8ewD/TAa5FcqhLg9chVxIq2mgqrP30Tn0Q7PcU+hU1w79UROYSV//fgIvt5GFj7UH29PWZBPCHH7JDA4iaY5qP7nf6OVFeA14RkUb9c+O1BcXsNrHx0G4D9mDiDAp2mm7RZCtH0SGJzEeiQF+8VDeNzxMPr2PVyaV2W1nb9+fISyShsLH+pP+yBvl+YnhHAvEhicwJ6dhnX/RgzdhmHsM8GlednsDt78x1Eu51fwzP196Wr2c2l+Qgj3I4HhNjnKC6n+4i0UfzOed8116TBRh0Pj3eQ0TmUWM/e+GPp2a5oFfoQQ7kUCw23QVDtVqSvRVBue9zyLzujpurw0jf9NPcOBU3k8NLY7d/Rx/bMRQgj3JIHhNtTs/T8ceefwvGsu+gDXTj+x7duLfPF9FpOGdWLy8M4uzUsI4d4kMNwi29m92E6kYoydhLHbMJfmtevIZf6x8zwj+oQzY2x3l+YlhBCNGviekZFBYmIixcXFBAQEkJSURGRkZL00ixYtIj09ve59eno6K1euZPz48axcuZKUlJS6FdwWLlxYt3xnYmIi33zzDYGBgUDtUp9PP/20k6rnGmphNtU7V6Nv3xOP4TNcmtfhs/ms3Z5On65BzJ0SgyJTXQghXKxRgWHp0qXMmjWLadOmsWXLFpYsWcK6devqpVm+fHnd61OnTjFnzpy6i3+/fv2YO3cuXl5enDp1ikceeYTdu3fj6VnbJv/EE0/wyCOPOKtOLqVZq6j6fAU6oyeeE+ajU1z3UNnZ7BLe3nyczuE+PHN/Xwx6ucETQrheg1eagoIC0tLSiIuLAyAuLo60tDQKCwt/8ZgNGzYQHx+PyWQCYPTo0Xh5eQEQHR2NpmkUFxc7o/xNStM0qr9+D600D88Jz6B4B7gsr+z8Cl5ff4QAXw/+fUZ/PE3yVLMQomk0GBgsFgvh4eHo9bXTO+j1esLCwrBYLNdNb7Va2bp1K9OnX3+eoM2bN9O5c2fat/9xVM2aNWuIj49n/vz5nDt37lbq0SRsx7ZjzziAx7AZGMzRLsunsLSa1z46jF6v8B8zB+DXzuSyvIQQ4uec/jU0NTWViIgIYmJirtn33Xff8frrr7N69eq6bQsXLiQ0NBRFUdi8eTOPP/44qampdYGoMW60qHVDQkMbt/5yVeYJLPvW4x09nPDxM1z2vEJ5pZU33t9PtVXl1fkjiero/LuSxta5LZE6uweps3M0GBjMZjO5ubmoqoper0dVVfLy8jCbzddNv3HjxuveLRw6dIjf/va3rFq1im7dflz3ODw8vO51QkICr776Kjk5OXTo0KHRlSgoKMfh0Bqd/gehob5cuVLWYDpHZTGVG/+Czi8M5Y5/JT+//KbzagyrTeUvHx3m8pVyFs7oj5+HvlHluxmNrXNbInV2D1LnxlMU3Q2/UDfYlBQcHExMTAzJyckAJCcnExMTQ1BQ0DVpc3JyOHjwIPHx8fW2Hz16lIULF/LGG2/Qp0+fevtyc3PrXu/atQtFUeoFi+amOexUp65Cs1XVrsRm8nJJPqrDwd8/OcHZrBLmxfchJvLaz1cIIZpCo5qSli1bRmJiIqtWrcLPz4+kpCQA5s2bx4IFC4iNjQVg06ZNjB07Fn//+jOLvvTSS1RXV7NkyZK6bcuXLyc6OprFixdTUFCATqfDx8eHt956C4Oh5XS01uxbj5pzGs+xT6AP6uiSPDRN4/99dppDZ/KZPbEnQ3uFuSQfIYRoDJ2maTffBtPCuKopyXZ+P9WpKzH2Ho/nqEdvp4g3tHnXeT7Zc4H77ujC9LuiXJYPyO22u5A6u4dma0pyV2rxZaq/fg8lrBsed/yLy/LZfdTCJ3suMKqfmQfGdGv4ACGEcDEJDNeh2aqp/vxNdHojXhOeQad3TdOWXXWwadd5unfwZ87kaJfOzCqEEI0lgeFnNE2jeucaHMUWPMc/jeLjuqmtvz99haKyGqaM6IJekV+FEKJlkKvRz9hOpGI/tw/TkAcwdOjt0rxSD2YRFuBFv+6yroIQouWQwPATas4Zar79P/SdB2AacJ9L87qQU8rZrBLGDe4oE+MJIVoUCQxXOapKqUpdic4nCK+x89DpXPvRpB7IwsOkZ1Ts9R8UFEKI5iKBAdAcKtVfvIVWU1H7EJtHO5fmV1Jh5buTuYzqa8bbs+U8syGEECCBAQDrgX+gXj6J56jH0Id0cXl+Xx/Kxq5qjBvc+Gk/hBCiqbh9YLBd+B7r4W0Ye92NMXq0y/Ozqw6+OpRNbLdgzMGuvTMRQohb4daBwVZoofqrd1FCIvG4c1aT5Ln/VB4lFVYmDHHN9BpCCHG73LaBW7PXkLv5/wNFwWviM+gMTbPmQeqBLNoHedOnq0ySJ4Romdz2jsF+8QjWvEy8xj2J4hvaJHmeyy4hw1LKeBmiKoRowdz2jsHQdQidnl5BsXrri/zcrNSDWXh56BkZ277hxEII0Uzc9o5BpygYg5ruGYKishoOnMpjdL8IWb9ZCNGiuW1gaGpfHcrG4dAYN1g6nYUQLZsEhiZgs6t8fTib/t1DCAtwzQpwQgjhLI1q08jIyCAxMZHi4mICAgJISkoiMjKyXppFixaRnp5e9z49PZ2VK1cyfvx4VFXllVdeYdeuXeh0Op544glmzJgBcMN9bcW+tDzKKm1MlCGqQohWoFGBYenSpcyaNYtp06axZcsWlixZwrp16+qlWb58ed3rU6dOMWfOHEaPrn1gbOvWrWRmZrJjxw6Ki4tJSEjgjjvuoGPHjjfc1xZomkbqgUt0CG1Hry6BzV0cIYRoUINNSQUFBaSlpREXFwdAXFwcaWlpFBYW/uIxGzZsID4+HpOp9tmAlJQUZsyYgaIoBAUFMWHCBLZv397gvrbgTFYJmXnlTBjcURbiEUK0Cg3eMVgsFsLDw9Hr9QDo9XrCwsKwWCwEBV37kJbVamXr1q28//779c4RERFR995sNpOTk9Pgvsa60dqlDQkN9b3lYxvjv1NO4uttJO6u7i1mNJKr69wSSZ3dg9TZOZx+pUpNTSUiIoKYmBhnn/oXFRSU43BoN32cqxcPLyip5ttjFiYP70xZSRUtYZlyWTDdPUid3cOt1llRdDf8Qt1gU5LZbCY3NxdVVYHazuK8vDzM5us/A7Bx40amT59+zTkuX75c995isdC+ffsG97V2Xx7KQoeOcQPbRn+JEMI9NBgYgoODiYmJITk5GYDk5GRiYmKu24yUk5PDwYMHiY+Pr7d98uTJrF+/HofDQWFhIampqUyaNKnBfa1ZjU1l5+HLDOoZQrC/Z3MXRwghGq1RTUnLli0jMTGRVatW4efnR1JSEgDz5s1jwYIFxMbGArBp0ybGjh2Lv79/veOnTZvGkSNHuOeeewB45pln6NSpU4P7WrO9J3KoqLYzYUjrr4sQwr3oNE27+cb5Fqal9TFomsaS975Dr+hY+m9DW9RoJGmHdQ9SZ/fQbH0M4uadulhEdn4FE4Z0alFBQQghGkMCgwt8fiALX28jw3uHNXdRhBDipklgcLK84iqOnM3nrgEdMBr0zV0cIYS4aRIYnOzLg1koio6xAzs0d1GEEOKWSGBwomqrnV1HLzOkVxiBvh7NXRwhhLglEhicaM+xHKpqVCbImgtCiFZMAoOTODSNLw5m0dXsR1QH/4YPEEKIFkoCg5OcyCgkp7CSCbLmghCilZPA4CSpB7Lwb2diaC8ZoiqEaN0kMDiBpaCCY+cLGDuwAwa9fKRCiNZNrmJO8OXBbAx6HXfJEFUhRBsggeE2VVbb2X3cwrCYcPzbmZq7OEIIcdskMNym3ccs1FhV6XQWQrQZEhhug8Oh8cXBS3Tv6E9ke7/mLo4QQjiFBIbbcPRcAVeKq5koay4IIdqQRi3Uk5GRQWJiIsXFxQQEBJCUlERkZOQ16VJSUnjrrbfQNA2dTseaNWsICQlh0aJFpKen16VLT09n5cqVjB8/nhUrVvDhhx8SFlY7zHPQoEEsXbrUObVzsdSDlwj09WBgj5DmLooQQjhNowLD0qVLmTVrFtOmTWPLli0sWbKEdevW1Utz7Ngx3nzzTdauXUtoaChlZWWYTLWdscuXL69Ld+rUKebMmcPo0aPrtiUkJLB48WJn1KfJZF8pJ+1CEdPv6iZDVIUQbUqDV7SCggLS0tKIi4sDIC4ujrS0NAoLC+ule//995k7dy6hoaEA+Pr64uFx7URyGzZsID4+vi5otFZfHMzCaFAY0z+iuYsihBBO1WBgsFgshIeHo9fXri2g1+sJCwvDYrHUS3fu3DkuXbrE7Nmzuf/++1m1ahU/XzXUarWydetWpk+fXm/7tm3biI+PZ+7cuRw6dOh26+Ry5VU2vjmew4je4fh6t+4AJ4QQP9eopqTGUFWV9PR01qxZg9Vq5fHHHyciIoKEhIS6NKmpqURERBATE1O37eGHH+app57CaDSyZ88e5s+fT0pKCoGBgY3O+0ZrlzYkNNT3po/Z9dUZrHYHMyZG39Lxza01lvl2SZ3dg9TZORoMDGazmdzcXFRVRa/Xo6oqeXl5mM3meukiIiKYPHkyJpMJk8nE+PHjOXr0aL3AsHHjxmvuFn5oegIYOXIkZrOZM2fOMGzYsEZXoqCgHIdDazjhz9zKQtqqw8EnO8/Rq3MAPkal1S0+Lgumuweps3u41Toriu6GX6gbbEoKDg4mJiaG5ORkAJKTk4mJiSEoKKheuri4OHbv3o2madhsNvbu3UuvXr3q9ufk5HDw4EHi4+PrHZebm1v3+uTJk2RnZ9O1a9fG1a4ZHD6TT0FpDRNkiKoQoo1qVFPSsmXLSExMZNWqVfj5+ZGUlATAvHnzWLBgAbGxsdx3330cP36cKVOmoCgKo0aN4sEHH6w7x6ZNmxg7diz+/vXXKnjttdc4ceIEiqJgNBpZvnx5vbuIlubzA1mE+HsyoLsMURVCtE067ec9xK1QUzUlZeaWsWzNfh4a253JwzvfdH4tgdxuuweps3totqYk8aPUA1mYjAqj+5sbTiyEEK2UBIZGKq20sjctl5F9zbTzNDZ3cYQQwmUkMDTS14cvY1cdjB8ss6gKIdo2CQyNYFcdfPV9Fn26BhER0q65iyOEEC4lgaERvj99heJyKxPkbkEI4QYkMDTC5wcuERboRWxUcHMXRQghXE4CQwMyLKWcyy5l/OCOKDpdcxdHCCFcTgJDA1IPXMLTpGdUrAxRFUK4BwkMN1BSXsN3J/MYFWvGy8Np8w0KIUSLJoHhBr46lI3DoTF+iHQ6CyHchwSGX2CzO/jn4cvERgUTHujd3MURQogmI4HhF+w/lUtphZWJMouqEMLNSGC4Dk3T+PxAFuZgb3pHNn7BICGEaAskMFzHuexSLuaUMWFwR3QyRFUI4WYkMFxH6sFLeHkYuLOvDFEVQrgfCQw/U1hazYFTVxjT34yHSd/cxRFCiCbXqMH5GRkZJCYmUlxcTEBAAElJSURGRl6TLiUlhbfeegtN09DpdKxZs4aQkBBWrFjBhx9+SFhYGACDBg1i6dKlAFRVVfH8889z4sQJ9Ho9ixcvZuzYsc6r4U366lA2GhrjB8kQVSGEe2pUYFi6dCmzZs1i2rRpbNmyhSVLlrBu3bp6aY4dO8abb77J2rVrCQ0NpaysDJPJVLc/ISGBxYsXX3Pu9957Dx8fHz7//HMuXLjA7Nmz2bFjB+3aNf0splabyteHLzOgewghAV5Nnr8QQrQEDTYlFRQUkJaWRlxcHABxcXGkpaVRWFhYL93777/P3Llz69Zr9vX1xcPDo8ECfPrpp8ycOROAyMhI+vbty86dO2+6Is6wLy2X8iqbDFEVQri1BgODxWIhPDwcvb62vV2v1xMWFobFYqmX7ty5c1y6dInZs2dz//33s2rVKn66nPS2bduIj49n7ty5HDp0qG775cuX6dChQ917s9lMTk7ObVfsZmmaRurBLDqG+hDdOaDJ8xdCiJbCaRMAqapKeno6a9aswWq18vjjjxMREUFCQgIPP/wwTz31FEajkT179jB//nxSUlIIDHTOMwI3WtS6IaGhvgAcO5fPpbxynntoAGFhfk4pV0v1Q53didTZPUidnaPBwGA2m8nNzUVVVfR6PaqqkpeXh9lcfyhnREQEkydPxmQyYTKZGD9+PEePHiUhIaGueQlg5MiRmM1mzpw5w7Bhw4iIiCA7O5ugoCCg9g5l+PDhN1WJgoJyHA6t4YQ/Exrqy5UrZQBsTD2Nj5eRPp3867a1RT+ts7uQOrsHqXPjKYruhl+oG2xKCg4OJiYmhuTkZACSk5OJiYmpu5D/IC4ujt27d6NpGjabjb1799KrVy8AcnNz69KdPHmS7OxsunbtCsDkyZP56KOPALhw4QLHjh1j9OjRN1nN25NfXMX3Z65w14AITEYZoiqEcG+NakpatmwZiYmJrFq1Cj8/P5KSkgCYN28eCxYsIDY2lvvuu4/jx48zZcoUFEVh1KhRPPjggwC89tprnDhxAkVRMBqNLF++vO4u4le/+hWJiYlMnDgRRVF4+eWX8fG59aahW/HloWx06Bg7sEPDiYUQoo3TaT/tIW6lbqcpKSu7mN+s3EOfrkE8ndDXBaVrWeR22z1Ind1DszUltXXfnsihssbOBFlzQQghADcPDD8MUe3S3pfuHfybuzhCCNEiuHVgOHLmCpfzK2QWVSGE+Am3Dgyf7DqPn7eRYTHhzV0UIYRoMdw2MOQWVXLgZC53D+yA0eC2H4MQQlzDba+Il3LL8fIwcLcMURVCiHqcNiVGazOkVxh3De1MRVl1cxdFCCFaFLe9YwDw9jQ2dxGEEKLFcevAIIQQ4loSGIQQQtQjgUEIIUQ9EhiEEELUI4FBCCFEPRIYhBBC1NMmnmNQlFuf5+h2jm2tpM7uQersHm6lzg0d0ybWYxBCCOE80pQkhBCiHgkMQggh6pHAIIQQoh4JDEIIIeqRwCCEEKIeCQxCCCHqkcAghBCiHgkMQggh6pHAIIQQoh63DQwZGRnMnDmTSZMmMXPmTC5cuNDcRXKpSSvNKAAAA/ZJREFUoqIi5s2bx6RJk4iPj+fZZ5+lsLCwuYvVJN58802io6M5ffp0cxfF5Wpqali6dCn33HMP8fHx/P73v2/uIrncV199RUJCAtOmTWPq1Kns2LGjuYvkdElJSYwbN+6av2OXXcc0N/Xoo49qmzdv1jRN0zZv3qw9+uijzVwi1yoqKtL27t1b9/7Pf/6z9vzzzzdjiZrG8ePHtV/96v9v7/5d0tvjOI6//EGUQRwtzANS0RDYkuABlyA6NR76A6ItaAhKh4ZoiqzBJQyynzS3RYQtDbbYEA4tokScIg6oWUJkZ4nOOXcI4ivfH9wL348frr4fm05PUN9vz1HOmbXGx8et29tb3jnMxWIxa2NjwzJN07Isy3p+fuZcxJZpmpYkSd+vbaFQsILBoGUYBueyvyubzVrFYvGn9zGrOdaSRwzVahX5fB6KogAAFEVBPp9v6m/QgiAgHA5/Pw4GgygWixyL2Pv4+MDa2hpWV1d5pzSErus4PT1FJBKBzfZ1kbSenh7OVezZ7XbUajUAQK1Wg9frhd3eXKNNkiSIolj3HMs51hRXV/2vSqUSent74XA4AAAOhwNerxelUgkej4dzHXumaeL4+BiyLPNOYWprawtTU1Pw+/28UxpC0zQIgoDt7W1cX1+js7MTkUgEkiTxTmPGZrMhkUhgfn4eLpcLuq7j4OCAd1ZDsJxjzbVWyb8Si8XgcrkwMzPDO4WZm5sb5HI5TE9P805pGMMwoGkahoeHcXJygqWlJSwsLOD9/Z13GjOfn5/Y39/Hzs4OLi8vsbu7i2g0Cl3Xeaf9r7XkYhBFEU9PTzAMA8DXB6pSqfx0qNaM4vE4Hh8fkUgkmu5w+0fZbBaqqmJiYgKyLKNcLmN2dhaZTIZ3GjOiKMLpdH6fWhgZGYHb7cbDwwPnMnYKhQIqlQpCoRAAIBQKoaOjA6qqci5jj+Uca97J8Afd3d0IBAJIpVIAgFQqhUAg0PSnkTY3N5HL5ZBMJtHW1sY7h6m5uTlkMhmk02mk02n4fD4cHR1hdHSUdxozHo8H4XAYV1dXAL7+sVKtVtHf38+5jB2fz4dyuYz7+3sAgKqqqFar6Ovr41zGHss51rI36lFVFcvLy3h7e0NXVxfi8TgGBwd5ZzFzd3cHRVEwMDCA9vZ2AIDf70cymeRc1hiyLGNvbw9DQ0O8U5jSNA0rKyt4fX2F0+lENBrF2NgY7yymzs7OcHh4+P2D++LiIiYnJzlX/V3r6+u4uLjAy8sL3G43BEHA+fk5sznWsouBEELIr7XkqSRCCCG/R4uBEEJIHVoMhBBC6tBiIIQQUocWAyGEkDq0GAghhNShxUAIIaQOLQZCCCF1/gFrrL/BMRgO0wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yve_Vm7kLCBu"
      },
      "source": [
        "## Test\n",
        "\n",
        "On teste le modèle sur notre jeu de données test."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3-H80W8bhQ2O",
        "outputId": "455ceca3-8010-425b-c057-b39ec388e359",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "model.load_state_dict(torch.load('tut5-model.pt'))\n",
        "\n",
        "test_loss, test_acc = evaluate(model, test_iterator, criterion)\n",
        "\n",
        "print(f'Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:.2f}%')"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test Loss: 0.366 | Test Acc: 83.91%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lAjRJYz4hWgR",
        "outputId": "bb5a9ef0-d921-4bf8-c2a5-6bf5fba15068",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "import spacy\n",
        "nlp = spacy.load('en')\n",
        "\n",
        "def predict_sentiment(model, sentence):\n",
        "    model.eval()\n",
        "    tokenized = [tok.text for tok in nlp.tokenizer(sentence)]\n",
        "    indexed = [TEXT.vocab.stoi[t] for t in tokenized]\n",
        "    length = [len(indexed)]\n",
        "    tensor = torch.LongTensor(indexed).to(device)\n",
        "    tensor = tensor.unsqueeze(0)\n",
        "    length_tensor = torch.LongTensor(length)\n",
        "    #prediction = model(tensor, 1)\n",
        "    \n",
        "    prediction = model(tensor,1)\n",
        "    if (prediction[0][1].item() > 0.5):\n",
        "        print (\"Sentiment: Positive\")\n",
        "    else:\n",
        "        print (\"Sentiment: Negative\")\n",
        "\n",
        "\n",
        "predict_sentiment(model, \"this film is not good at all \")"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentiment: Negative\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "favtpTGNLCB2",
        "outputId": "7e9b7424-4966-41a6-f8c0-9274ea2d382f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "predict_sentiment(model, \"This film is amazing good \")"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentiment: Positive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yd4vDOoxLCB6",
        "outputId": "90f39b3b-9eed-4f98-8dd5-93192f8069dc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "predict_sentiment(model, \"This film is terrible \")"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentiment: Negative\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "idWgqJd0LCB-"
      },
      "source": [
        "On remarque que contrairement aux modèles précédents ce modèle prend en compte la négation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2HvPytKULCB_"
      },
      "source": [
        "## Références\n",
        "\n",
        " - [Text-Classification-Pytorch](https://github.com/prakashpandey9/Text-Classification-Pytorch/blob/master/models/LSTM_Attn.py)\n",
        " - [NLP at IEST 2018: BiLSTM-Attention and LSTM-Attention via Soft\n",
        "Voting in Emotion Classification](https://www.aclweb.org/anthology/W18-6226/)"
      ]
    }
  ]
}