c2q_transfer_cifar.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6XphVd0X2Brw"
   },
   "source": [
    "# Hybrid transfer learning for image classification (CIFAR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pY_vKbQB2Br0"
   },
   "source": [
    "This is an example of a hybrid network for image classification, developed according to the *classical-to-quantum transfer learning* scheme presented in [1].\n",
    "\n",
    "This notebook is inspired by the official PyTorch  [tutorial on transfer learning](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html) by Sasank Chilamkurthy [2].\n",
    "\n",
    "The starting point is a convolutional neural network—ResNet18, introduced by Microsoft in 2016 [3]—which is pre-trained on the public ImageNet dataset. We replace the last fully connected layer with a *dressed quantum circuit* [1] that we train on a different dataset. \n",
    "\n",
    "Specifically, we focus on the following sub-classes of the CIFAR 10 dataset:\n",
    "- Ref. [1], Example 3, dataset (a): *dogs* and *cats*.\n",
    "- Ref. [1], Example 3, dataset (b): *planes* and *cars*.\n",
    "\n",
    "The code can be easily modified to deal with different and/or more CIFAR classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 17289,
     "status": "ok",
     "timestamp": 1613631306725,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "3w-he0f52fAH",
    "outputId": "a56f40ea-7164-4035-d9b2-fdb4131d708f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\r\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vbzBnhDL2eWr"
   },
   "outputs": [],
   "source": [
    "!pip install torch==1.3.1\r\n",
    "!pip install torchvision==0.4.2\r\n",
    "!pip install Pillow==6.2.1\r\n",
    "!pip install pennylane==0.7.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y_mAyhoj2Br1"
   },
   "source": [
    "## General setup\n",
    "**Important:** this code makes use of the machine learning framework *PyTorch*, which is assumed to be correctly installed.\n",
    "\n",
    "The main imported modules are: some common PyTorch libraries, the quantum \n",
    "software framework `pennylane` [4] and the python plotting library `matplotlib`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1814,
     "status": "ok",
     "timestamp": 1613631471351,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "Y5h89BNC2Br1",
    "outputId": "4e65dde5-0624-4e60-cdb8-329523c8fa94"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: OMP_NUM_THREADS=1\n"
     ]
    }
   ],
   "source": [
    "# OpenMP: number of parallel threads.\n",
    "%env OMP_NUM_THREADS=1\n",
    "\n",
    "# Plotting\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# PyTorch\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torch.optim import lr_scheduler\n",
    "import torchvision\n",
    "from torchvision import datasets, models, transforms\n",
    "\n",
    "# Pennylane\n",
    "import pennylane as qml\n",
    "from pennylane import numpy as np\n",
    "\n",
    "# Other tools\n",
    "import time\n",
    "import copy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MphjIvaS2Br2"
   },
   "source": [
    "### Please choose and run only one of the following two cells, depending on the dataset of interest."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dlWsOWxz2Br2"
   },
   "source": [
    "Settings to reproduce the results of ***Ref. [1], Example 3, dataset (a)***, i.e., _dogs_ vs. _cats_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "executionInfo": {
     "elapsed": 605,
     "status": "ok",
     "timestamp": 1613631471352,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "Gp8YWyzw2Br3"
   },
   "outputs": [],
   "source": [
    "filtered_classes = ['cat', 'dog']  # Subset of CIFAR ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n",
    "n_qubits = 4                       # Number of qubits\n",
    "quantum = False                    # If set to \"False\", the dressed quantum circuit is replaced by \n",
    "                                   # An enterily classical net (defined by the next parameter). \n",
    "classical_model = '512_nq_n'          # Possible choices: '512_n','512_nq_n','551_512_n'. [nq=n_qubits, n=num_filtered_classes]\n",
    "step = 0.001                       # Learning rate\n",
    "batch_size = 8                     # Number of samples for each training step\n",
    "num_epochs = 3                     # Number of training epochs\n",
    "q_depth = 5                        # Depth of the quantum circuit (number of variational layers)\n",
    "gamma_lr_scheduler = 1             # Learning rate reduction applied every 10 epochs.                       \n",
    "max_layers = 15                    # Keep 15 even if not all are used.\n",
    "q_delta = 0.01                     # Initial spread of random quantum weights\n",
    "rng_seed = 0                       # Seed for random number generator\n",
    "start_time = time.time()           # start of the computation timer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PoP4dRmT2Br3"
   },
   "source": [
    "Settings to reproduce the results of ***Ref. [1], Example 3, dataset (b)***, i.e., _planes_ vs. _cars_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 586,
     "status": "ok",
     "timestamp": 1613631472702,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "Z3cKe-2k2Br3",
    "outputId": "76392e7f-3506-4982-d071-1d54cd798a76"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'\\nfiltered_classes = [\\'plane\\', \\'car\\'] # Subset of CIFAR (\\'plane\\', \\'car\\', \\'bird\\', \\'cat\\',\\'deer\\', \\'dog\\', \\'frog\\', \\'horse\\', \\'ship\\', \\'truck\\')\\nn_qubits = 4                        # Number of qubits\\nquantum = True                      # If set to \"False\", the dressed quantum circuit is replaced by \\n                                    # An enterily classical net (defined by the next parameter). \\nclassical_model = \\'512_n\\'           # Possible choices: \\'512_n\\',\\'512_nq_n\\',\\'551_512_n\\'. [nq=n_qubits, n=num_filtered_classes]\\nstep = 0.0007                       # Learning rate\\nbatch_size = 8                      # Number of samples for each training step\\nnum_epochs = 3                      # Number of training epochs\\nq_depth = 4                         # Depth of the quantum circuit (number of variational layers)\\ngamma_lr_scheduler = 0.1            # Learning rate reduction applied every 3 epochs.                       \\nmax_layers = 15                     # Keep 15 even if not all are used.\\nq_delta = 0.01                      # Initial spread of random quantum weights\\nrng_seed = 0                        # Seed for random number generator\\nstart_time = time.time()            # Start of the computation timer\\n'"
      ]
     },
     "execution_count": 3,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'''\n",
    "filtered_classes = ['plane', 'car'] # Subset of CIFAR ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n",
    "n_qubits = 4                        # Number of qubits\n",
    "quantum = True                      # If set to \"False\", the dressed quantum circuit is replaced by \n",
    "                                    # An enterily classical net (defined by the next parameter). \n",
    "classical_model = '512_n'           # Possible choices: '512_n','512_nq_n','551_512_n'. [nq=n_qubits, n=num_filtered_classes]\n",
    "step = 0.0007                       # Learning rate\n",
    "batch_size = 8                      # Number of samples for each training step\n",
    "num_epochs = 3                      # Number of training epochs\n",
    "q_depth = 4                         # Depth of the quantum circuit (number of variational layers)\n",
    "gamma_lr_scheduler = 0.1            # Learning rate reduction applied every 3 epochs.                       \n",
    "max_layers = 15                     # Keep 15 even if not all are used.\n",
    "q_delta = 0.01                      # Initial spread of random quantum weights\n",
    "rng_seed = 0                        # Seed for random number generator\n",
    "start_time = time.time()            # Start of the computation timer\n",
    "'''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4VvmvHVO2Br3"
   },
   "source": [
    "Let us initialize a PennyLane with the default simulator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "executionInfo": {
     "elapsed": 572,
     "status": "ok",
     "timestamp": 1613631473700,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "r0mjr3bq2Br4"
   },
   "outputs": [],
   "source": [
    "dev = qml.device('default.qubit', wires=n_qubits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-i2fetxl2Br4"
   },
   "source": [
    "Configure PyTorch to use CUDA, only if available. Otherwise simply use the CPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "executionInfo": {
     "elapsed": 721,
     "status": "ok",
     "timestamp": 1613631474708,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "FAYWr4UV2Br4"
   },
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q4qWpbZy2Br4"
   },
   "source": [
    "## Dataset loading\n",
    "\n",
    "**Remark:** It may take several minutes to download the CIFAR dataset (only the first time).\n",
    "The PyTorch packages `torchvision` and `torch.utils.data` are used for loading the dataset and performing standard preliminary image operations: resize, center, crop, normalize, *etc.* ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 5078,
     "status": "ok",
     "timestamp": 1613631479391,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "2mMBEVE82Br4",
    "outputId": "f3552d44-4f35-4147-b33f-19c47333b605"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 8192/170498071 [00:00<37:46, 75233.82it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "170500096it [00:01, 99091218.48it/s]                               \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data/cifar-10-python.tar.gz to ./data\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "# Fixed pre-processing operations\n",
    "data_transforms = {\n",
    "    'train': transforms.Compose([\n",
    "        #transforms.RandomResizedCrop(224),     # uncomment for data augmentation\n",
    "        #transforms.RandomHorizontalFlip(),     # uncomment for data augmentation\n",
    "        transforms.Resize(256),\n",
    "        transforms.CenterCrop(224),\n",
    "        transforms.ToTensor(),\n",
    "        # Normalize input channels using mean values and standard deviations of ImageNet.\n",
    "        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
    "    ]),\n",
    "    'val': transforms.Compose([\n",
    "        transforms.Resize(256),\n",
    "        transforms.CenterCrop(224),\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
    "    ]),\n",
    "}\n",
    "\n",
    "\n",
    "# =================== begin CIFAR dataset loading ===================\n",
    "trainset_full = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=data_transforms['train'])\n",
    "testset_full = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=data_transforms['val'])\n",
    "image_datasets_full={'train': trainset_full, 'val': testset_full}\n",
    "\n",
    "# CIFAR classes\n",
    "class_names = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n",
    "\n",
    "# Get indices of samples associated to filtered_classes\n",
    "filtered_labels=[class_names.index(cl) for cl in filtered_classes]\n",
    "sub_indices={'train': [], 'val': []}\n",
    "for phase in ['train', 'val']:\n",
    "    for idx, label in enumerate(image_datasets_full[phase].targets):  \n",
    "        if label in filtered_labels:\n",
    "            sub_indices[phase].append(idx)\n",
    "            \n",
    "# Initialize sub-datasets according to filtered indices\n",
    "image_datasets = {x: torch.utils.data.Subset(image_datasets_full[x], sub_indices[x])\n",
    "                for x in ['train', 'val']}\n",
    "\n",
    "def labels_to_filtered(labels):\n",
    "    \"\"\"Maps CIFAR labels (0,1,2,3,4,5,6,7,8,9) to the index of filtered_labels\"\"\"\n",
    "    return [filtered_labels.index(label) for label in labels]\n",
    "# =================== end CIFAR dataset loading ==========================\n",
    "\n",
    "# Number of samples\n",
    "dataset_sizes = {x: len(image_datasets[x]) for x in ['train', 'val']}\n",
    "\n",
    "# Initialize dataloader\n",
    "dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], \n",
    "                  batch_size=batch_size, shuffle=True, num_workers=0) for x in ['train', 'val']}\n",
    "\n",
    "# Function to plot images from tensors\n",
    "def imshow(inp, title=None):\n",
    "    \"\"\"Imshow for Tensor.\"\"\"\n",
    "    inp = inp.numpy().transpose((1, 2, 0))\n",
    "    # We apply the inverse of the initial normalization operation.\n",
    "    mean = np.array([0.485, 0.456, 0.406])\n",
    "    std = np.array([0.229, 0.224, 0.225])\n",
    "    inp = std * inp + mean\n",
    "    inp = np.clip(inp, 0, 1)\n",
    "    plt.imshow(inp)\n",
    "    if title is not None:\n",
    "        plt.title(title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FfKnzALm2Br7"
   },
   "source": [
    "Let us show a batch of the test data, just to have an idea of the classification problem.<br>\n",
    "*Hint:* re-run the next cell to see more samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 106
    },
    "executionInfo": {
     "elapsed": 4323,
     "status": "ok",
     "timestamp": 1613631479801,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "rKAEzX8I2Br9",
    "outputId": "1fec8f90-2f8f-4b35-a00f-2a214b598c5e"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAABZCAYAAAA0Gj+BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9ebBv2XXX91l7OOf8pnvf0INa3a1ZSDa2UTxAgRlcBBJTKYcMpDAp4iiJoUjKISEhLpOYYBIcTBUUhnI5AQJx4iQ2OHGcEJOEchyDjSd5lqVWSz2/12+68286wx5W/tjn3nff05N4dner29JdXbfv/f32Ofvss4fvXuu71tpPVJULuZALuZAL+fwT82Y34EIu5EIu5ELeGLkA+Au5kAu5kM9TuQD4C7mQC7mQz1O5APgLuZALuZDPU7kA+Au5kAu5kM9TuQD4C7mQC7mQz1P5ggV4EVER2YjIt7/ZbXlYEZEfE5FvfLPb8VYSEfkaEbn+ZrfjrSZvtbkiIu8a15x7s9vyeouIfI+ItG/FefgFC/Cj/BZV/c/gbAK+9ForFJGXROT3nfv80PWOYPVjr7UND/GcbxORb3vIa79HRD78Gp/3YRH5iV9vvSNYfc1racNDPuehkkIu5spnvPY1z5WHfM5LIvKuh7z2NSf6PGizPF+vqn4Y+AOv9TlvhHyhA/yFXMiFXMjnrVwA/GcQEXlaRH5QRPZE5EBEvmv8/r0i8qPjd/si8j+JyKWx7HuBdwB/X0TWIvLNr7ENv19EPiEiJ+Pz5VyZEZFvFZGXReSOiPwPIrJ7rvwbxrIDEfmz92uLr6FNf0xEnhGRlYh8XES+fPz+W0Tk+XPf/8vj918E/DfAbx/75Pg1Pn8yaopHIvJx4KvuK/+iUeM6FpGPici/eK7sqoj8fRFZishHROQv3G9Z/DrbdDFXPr09VkT+8vjeLwD/wn3lbxeR/0NEDkXkORH5Y+fKJiLy349j/IyIfLO8DvSHiFwRkf9ORG6Mdf/Q+P1lEfk/x/E7Gv9+aiz7duB3Ad81jtN3vdZ2fE5FVb8gfwAF3vcZyizwy8BfBWZAA/zOsex9wO8HauBR4B8D33nu3peA3/c6tO8RYAX8IcADfwqIwDeO5f828BzwHmAO/CDwvWPZFwNr4HcCFfCXgfBa2wX8a8CrFFCVsS/eea7s7RSl4Q8DG+CJsezDwE+8TuP2HcCPA1eAp4FfBa6PZX7sk/90fO/fO/bhB8by7x9/pmMfXXut7bqYK5+xTX8C+MQ4RleA/4+y5txY/o+B7x7760PAHvB7z43xPwIuA08Bv3I6xq+xTT8M/N2xXg/8nvH7q8C/Os6LBfADwA+du+/HTvvys9T9Na9HG1/vnze9AW/ai392gP/t44RzD1HPvwT84rnPr9ei/Qbgp899FuD6uUX7/wL/3rnyD4wL0wH/OfB958qmwPA6LNr/B/gPHvLaXwL+4Pj3h18rkJ6r9wXga899/uPcBfjfBdwCzLny7wO+jQLEgRHsx7K/8FrbdTFXPmObfhT4E+c+/3PjmnMU0E/A4lz5XwS+59wY//Pnyr7xtYIn8ASQgcsPce2HgKNzn3+M36AA/3nn0X6d5GngZVWN9xeIyOPAX6OAyYKisR69AW14O0XDBEBVVUSu3Vf+8rnPL1MWz+MPuHcrIgevQ5ueBp5/UIGIfAPwHwHvGr+aUzTL11vueTfu7YO3A9dUNd9X/iRFg3b33Xv+71+vXMyVh2gTnz5Oh6q6uq/8Kz/Dva/XOB2q6qf1v4hMKRbY11K0e4CFiFhVTa/Ds980ueDgHyzXgHfIg0O6/iuKJvKlqroD/FHO8Z1j2eshNymTEgARkfOfgRvAO899fgfFLL893vvUuXsnFDP0tco14L33fyki7wT+FvBNwFVVvUShTk775fU8svSefqG896ncAJ4WEXNf+asULTtyrl/uq+fXKxdz5SHaxKeP0xURWdxX/uq5e9+Icbpy6gO5T/5jilXz28Zx+t3j92/E/P2cygXAP1h+ljLJvkNEZiLSiMhXj2ULCmd5IiJPAv/JfffepnCdD5TRQfg9D9GGHwZ+s4j8KyN4/EngbefKvw/4UyLybhGZU8Dk746a5P8CfJ2I/A4RqSgUhfAZREp88tc8RJv+W+BPi8hXSJH3jeA+oyyCvbG+fwv4knP33QaeGtvyoOefxki/6yHa8PeAPzM6xp4C/v1zZT8DbIFvFhE/vtPXAd8/amI/CHybiExF5IMUauOBIiU88Mceoj0Xc+XB8veAPykiT4nIZeBbTgtU9Rrwk8BfHPvry4B/B/gfz917OsZPUhSHz9SeD8tDhJaq6k3g/wK+e6zXi8gpkC+AFjgWkSvAn7vv9s86Tm9luQD4B8gIBl9HcZK9QuEz//BY/OeBLwdOKAvrB++7/S8C3yoliuNPP6D6p4F/8hBt2Kc4Lr8DOADef999fwf4Xoqz6kWgYwQ7Vf3Y+Pf3U8BnDdwB+vufIyJPUxx0H32INv0A8O3A/zze80PAFVX9OPBXgJ+iLIYvva+tPwp8DLglIvsPqPppion+6gPK7pc/P177IvAPKX1w2r6BMm5/ANinOPG+QVU/MV7yTcAuhaf/XgrwfVqfnGvTw4zTxVx5sPwtis/ml4Ff4NPf/Y9Q6LwbwP8G/DlV/ZGx7L+g9OOLwI9QNqHXNE6j/BsU38MnKO/4H47ffycwocyZnwb+7/vu+2vAHxojbP76Qz7rLSEyOgi+4EREOsqk+euq+mc/R8+sKBP+y1Q1fC6eOT53DhwD71fVF+8r+6PAb1bVP/O5as/9IiLfCuyp6t/4HD/3LwFvU9V/8wFlvwT8s6r6evDRv2a5mCv3PPffBb5eVX/PA8r+IcXx/8znsk33teFvUzbYO6r6vjerHQ+SL1iA/3wXEfk6SvSEULTr3wZ8uX4BD/hIy1QUDfSrgH9AiY74oTe1YW+yvNXmiog8QaFEfopijfww8F2q+p1vRnt+I8sbQtGIyNeKyLNSEhi+5Z9+x4W8AfIHKebvDcoi+fovZHAfZUGhCjaUeOi/Avzvb2qL3hryVpsrFfA3KHTQj1LG6LvfxPb8hpXXXYMXEQt8kpLgcR34CPBHRp72Qi7kQi7kQj5H8kZo8L8VeE5VXxidXt9P0RAu5EIu5EIu5HMobwTAP8m9iQnXx+8u5EIu5EIu5HMob1omq4j8cUqaOZOm/ooPvO/dKGMArpTfqnqWYWCMQTCoQs6JnDNZMyiICGIMIpBSJHRbuvWSFALcR0EZEVSVmJWQFCMGZwURkPuif8vHcr+KZZUdKeW7VQoIgp62dTwBQe/e9gA5e8vP0jnjFaoYGd9v/LmfUhNAjGDEYIzgrGU2dZjTjhzr0fO/0bN3ECn/E+79PdZc2nt6pEX54566ynhkcsqknIkxoeUyUk6oKoJinT1ro4jgfENT76AmkiSRMuRO0SSo9SSroAETB5q6Yjqb0UymCEoMgdB1pCGSUgYU5xzWWgyQUyKnSMqJTMlPR8AagwEkZyRljAjOGVzjEC931R0FskBWlm081+8ZVBERjJQ5o6fDOV6TtfRFzkrKGRFD3TRUTYN3HoAUAiEMDMNwt49EELEIkFXH+SgYA5oz5HLNPeNyOlQoJ9uevus/fRKfXqlyd8KcfXlurqHjbzn33dkDyrvlTB5f2FiDtQ5rHTknQgjknDBnffPp7TAizGcTrJFzK+DcdWf3jG04V0eZR3Lv5ffULme3y7nFfG8zhPtbpShdP7Dtetp2QFWp64qn3/EufPXAtI23jGw3a575xLP7qvroZ7rmjQD4V7k38+wpHhDfrKp/E/ibAB/8Te/RH/nBvw0IzjmMgayRGHsQcK7GV1NEPDEmNpsNXbclDgHNYH1NNZ0gkjm8/TIvf/TneOXn/wmbvVsYVdBUFghC7TztELiz6bm+TMzqhsd3J8wmAqYsYAMY4WyiGiBUC358fYXDkw0xRlQVay3OeazzxJRp+56uH4gpoyoFWPQMUu8B59PJem6dIcjZ5DQoNZlF47i8M8d5T0yJ7bYlpYgxhqquaJqGpq7wzmONYTGv+eoPPUrl5GzhntavgOayUFPKWGOxzmGdGbclBSOoWDJCShBjoms7NusN7bZFUbyvSRm2bc96vYWUCwCkSIyZy5cf4dLlq9y8dZtXrr/Cnb09UNhZzHnibY/xrnc9STN7irdd+R300z3a6RGtS6w+FVm+2rDhEfJlh003mBw9y9PvfJwv/me+kg986VeSQ88rz32SZ37hF3j5E6+wOl4y3ZnyZR/6Lfym97+fK9MpR6+8zPErL7PZPyQmQ2wcultRX5lRe7BdB4crLiE8ennO1XfvUj1aITOLYDHJQK/EZc8P/OQzPPfSi4SYkQxeLNOmYj6bMJ01+MrjmxrfeLIofQj0fU/uB/rNFmM8b3/3e3nH+z/A4088hbOe7cmSg9u3uHnzGvtHe6zbDaqGSTPH+4qUIyENOJNZNIaZBCaS8eJAhZwV1YyxirEgRvlff+YGz714/UzZyKokLUrMEJUYFc0GEYsxBkQRyVireGuonMEbUzbBcRM+rSvGSNu2tF3PkDP1ZMrVx97Gk0+/k0cefYyTk2NeeP45br96HdHIvJngrcWKQcypcgK+qvndX/UlXJ66u+A9bo6IGX8sjPPSOVdWTs7knLHW3t1sc9mQztboadutRazDWIdIWcfWgIhBxJQ5bwQRRQXaEPnos8/z8x99lmefv4YTwwfe/17+y7/6X/PEk+dg7OyInwfI6dHwn1Gpu7shf3oV/xRl77PIL37kJ/mK3/rVL3+2a94IgP8I8H4ReTcF2L8e+Nc/6x2qQMIYVzpelZQiWTNmHPicyzUpRVKO5FzAGBFUIOWMNZk4BIa2JYVYwEwzoqealymT9nRyacKSMJpICcpliijYcVKWwROSVebTKTkLwzAw9AM5g1WYGMF6x8wqKxLbXumjghoyMmq0RQPUUw1+1JY/XacwZeIiTCrLpfmEK4s5MSU2KeKsUPmapmmYzafMphOcc2jO9F0ALVq8d0UdzTlxHuTzqIUbY7DWYs3pYlNSTsQhsm03LE82HB2tWG861uuO9aZlGEKxEnzZ0IZY6r5yaYf5tMYZoWo8jz+yy9PvfIpLl2Yggc16xeHhCa0rG3RWIatiph6dV8TGE4MyZEvfC6EbqF3Euo5Ax9HmmJdfvUavnoP9fV745LM8/8yzHNw8JMfE1XCFx9Zr3lF7mre/jVlOHC/XbPbXDH0gkZAmk0MijVqnqTxtiLR9YLMMaFNT4THGoRkYAton+q5j//CwjLVx1K4mIWTjSSZRY2hsJDmDtQWgrPVYLySfSDGxOjnh5OCA2XTOdDZDLDSzhsXOgm5Yk0JP30c0RnAe7ytUQAhgYDaZcGXimdY1KKQ8HiYlBdwziqv2EGfK91lJKTOkRB8y7ZAIIWOMZ9J46ukU1USMPUMayJpQLKaqsNaixpC1KAEhDPT9QNt2hBAxzuN9w2K+4PKlS1y+tIsRZXexYFk3xKHHSLG2yywWBIMRwRp3DtBPrea7So2Oa1kMGCMYgZwKwJ/O3zMd/3T9nLNuz7R2FNE0ri1Fs5SNECnvmoSYMl0IHK623LxzxNHJGs3KZFY27wfD7kNY3+iZLXTa5l9fPa+PvO4Ar6pRRL6JksVmgb8zZst9Vsk5YYwt9IsmYoyAlkE/pQg0k3MipVTMcM1gDJlEzAE0E4aOvmtJo5atqqAZURBR0qkmrYpVxZryI5rRXKgLyWWQ1JQJpxSzf/dS0bCKRtvS9wGy4jUztQ7rKyoyVkFToM+KjnZ/GU5D0ZLPTUzu17SLKe4FFk3F5WnDTuPZtJEepfaOqmlYLObs7i6YTRpUYbst7VEKeBtrxzrzqPHpWT/nlEDlbCFrUPoY2HYdm/WW5dEJB/tH3Ll9yMmyZbnuWG8DOQvOW3xliCmTDUymDZOmYTapcM7SVI5JY5k2lkeuzHnbo5c52Nslx0hdVxgrbNst+A5plDyzROsY2sR2m+nXigmJZkioDKzywHBwwKp7ludfuMHNG7d49dp19m/vE7tIU9XY6ZSb+4fsr1Y8XVXMnnyKRRc5bDMnN/fYDlu065l1homtqQSsNdRR2aZIvY3IMpMTeJMwmmEYGFY9OZbxsNZQVZ6maaibKbaegKsw3oO1ZXStxRtXgMwlwDD0A/3Qszw+pGlqQr/AGEtMPdZC4yydtUQN5BBQ5/F1hXENqMU6xTee2c6c3cW8UCJFgT2jnmJOWP98UVfzqYKUGIaBdkgF4GOmqS31pObRxx8BMpvNmtXqmGHoyTGiYojjnAwxnoH70PcMQ0AVanWkBCPmFktChNp7mromqOKMP7MEyo/BGFs08nOcyV1qVQoojqBvBOz4O2nR3u+nJ88D+1k9Y10F68u1SSGkTB7K+4QQCUnZDpHVpmP/aMlL1+9wvNyiGKazGfPFnLsArHe19zPu9VzZKU13n/p+epWOa3pcffcq8g+gse6p4Fydenr5rzHo8Q3h4FX1H1CSSB72DlJOEPsCSTmPvGQZNM1lxEUYOc5UuEvJY08lNIFqoO+2dO22TIxTLTwXOiQbIaEkzUUrEEGsQ7zFSsLkwueKaNGjFVQyKkqWgcV8RtNM2foWg8VIS+wHSAmrhllV4ewMVWEIEc2ZpKPFYO7SPWLOlJiyACgmo4ztEhRvhMsTw6WJpbGJZJVYWTyOetKwmE9ZzOfUlRutiUSMiZTsGV8Kp3RMOuvT089QmCtyph8iB4cn3Nk74GD/kPZkSb/d0rc9bRfotpF2k4hqqWuPEU9WRTPkkIghkFPGNhXNpCHlyNHRATklJpXhHW9/jKu7CzAOW9X0/YDte7L2JCJRDWEwdOtM7DLzyjKtoXeWbRc5OTqhb28SA6zXLe22IwwRI5aMpesyN67f4rlPvsDVRx/nPe95H098yZeS5pdY/spHWV57nr4/RgOYBGIN2SRSA7kSghW2m4GhjVSieE0wBLp1h6owmUwxxtA0E+azOU0zw1cV3juaicWONEnROi2qDnWKrSrqFAkhsNkuuXUzcHI8wVlXlJGuJQwdohlRZRh6xFia6ZSmqRHjMTmQcERTY5od6skEjC2bthgUIaSIcRXWWFCIGkkxMQyBYUiEIRGzYqaw2Jny9NNvp6o8y+UJN2969vb3CgXXb0clK9P3PSEEYoyk0VoWFYYBMiua6SHT+ZycM6HviUPEGY/xhfKxltFKLOBujMF7Wzj6MyAu6nrGkEfr2pxSpKOleRcLhJzz2b0YOQfopwAvI2YIWQwxC0M0bPvA8fGKw4M7nCyPaYfEqh1YbTs2247VZkvbB7x1zBc7LObzM1y6F1Hv5VRPfVA6YgZaDP67ni7uUl1lNQKFVjqr4CEU+dNH6vnN4SHlLXJccNmaQuhHB5PBWY91VTGnjS30iio5ZWKMhBjIZLIoRj3eenLsx0XTl062cte5pOOfFhAhqbCNQr9NtNJTmQFDRlSwCNaYQtOY4vAyDpzziIUQYnHqGUsSg1C0mMo7mokHBM2JdTeQctGUEXPGuxctHZwxVN7inOAMGDJo0SArJzx+acojOzOqyrOZ1jTbgW1IiHNYhBwj3ciPbtZbural8jI6OzOM2o9qvmvNwNifQsrKar3m1q19Xrl+h9u3j1gt1+jQ40h4B5iMrzNNzsQEVWWpK1BxJBRrIMcEClVVAB6U7WYJqtROePJtV/C+Jotj00VuHxydbXaml7Iwsy0T2SjGKSEGlts1B3urAvBdIEUIoWxkmsFXnspXiGYObt/ikx//KDs7DVcfucKjjz/Jo+98ip39W7D3Etv1hkfEM5tMWUw8ZpK5VE3ZbWpqUxE2PcOyIw6BKmc0JtZ9cRjPZjNEhMlkwmw+K4BvLcZA3Xi8N8X5mxJDCOQ0zj2jqC3Wo0qi77fEoR2JizIPSPlM2019YBgGcs54X2EdpEEZInRRSFIh1QzrKrIWt6iKoLFYZCnl8uwx8MAai5Vc/Eko5ISg1LVnd/cSdV3TdQPHyzV9WLFp2zM0OQVWxBROWwSSMsRMWm/It+7Q58zxao13ln7bEnMuiosVrCsObGsNxhbHq/N3CUkRQcWA9YifgAohBNLQsbAj+J+PZhhFR1p2rIQxCqEAvrWIMSQsfTQcb5STjXK0yuwdDBwetKyWK7qhpeuHcawiIRY613hD09TUTf0AFB219TN7e6TIABkp5aHri/M/FAXKWov3Fb7yWO8Rd+rF/7XSM/er7Q9//1sE4EuTS0QBOFd4TO9qjLGomBGQCgcfYyDGQJZMEsVJ4cw1DIQYSaokDHmMShBDIdhHni8ZIYqlR2n7xLEq1maMlKgVSwFcazhbjBMrPJpH52nOZ5E8iiLGYqzFuaLhFtW2YVYZopaJnDFn3LPmElninWFae5ra0lSGygmVVZyBysLl+ZTd+RTvazZ9plm3HCxbugQ5DLSbshC7tqNvO3IIcKapF+5StVgEpR/kzNmkatgsC7h/8lMvc+3aPkdHW0JIOFWaSpjODNXEUDswXkANzlvq2oA4QqLsmiMF5pzDVx5jDUoBlqa21NWES5evksWzf7LheN2ixmDUlGOvKJyDqYA60uY1m+Mlh+t9jvZXbNYdmhXEFmpAixZYV55JZWkkEjcrTq51XNuxvPrOJ5hPK5q6YT6r8FYxsWMikcuN4fJOhVHDpemEnabBBIU0kFY9w2ZLHhIpKUsVsrEsFgtUlaqqqOuKpqkwRlASzkFVFT9G3LbFPzNExApqit9IRKiswxglhISmsb+sLYqEtXjvsUNxJFZ1xXQyQSx0OZI1krKQxaFSgalAT+kJg9FEzoxBB4VmtGLw1uFtJppU2IEU6duOzWbLfLYovgJXoSp0Q2C96UAEPzo4Tx2k1lrqukIUQjewbVsOjk/YDAPLTct8OsEZQWOgMgXgnTd4Z7CuvKMYwbpRy6ZECmURrGuo5pdGR/CK7bApSo+n+M64i+dncD9GdBWN2NzV4K0F48jq6bLlYJu4fZI4WCaONw2buEOnkT7CkEZLdqSaZFz7xpxj+u/B1VETl9OACUgxEruOfrtmvVqyPDpifbwktIWJcJVnOpkyX8yZ7ixodnaYTKdUVY1x7l7K5X7MFr1blst6PoukMw8P8W8ZgEcFIw6xpux6rsZZP5pwY1RAzsQciTmSNRVtWMcBz4E+9gTNRFsxmJpkIkYyljg6XWAQoTcQKsVOijMoWEtvFB0dqgaK9pCVnIrWMO+FWduSs9L1XTFhYyBnxfjCLzprsCQmrjgQd2pbNhoMIUGflCFkQkyghYapnGFWO3Zmnp1ZzWJaMW0ctYOJMzTeYV3FZijUQj8Ewrqn6yMMsVg0IWByZuotE++Qs/+K3A1SEKxziBjaLnJwcMzLL73KtZdvcXzcMwxKTqZskArZCova0UyEegLeO/wYjpiyYQiQ8il1dmodGKq6pmnq4uhNESNQTWqSOpztz65LEVLMBB0IucdNMtQDh/uHrPt9VuubtOuO0GWU4ihLY3hjXVU0tWPiMgsG6mpgEnvSjU/x8s/9I6YSeOSp92DaNU3omQ6BRYjs5Mxla3F1TTOtcJXDVmDyDBsim1VHHwN9MixthXrDYmoLdTbSC84ZnD9lVhOqEWNcAQdVYojkPhO1gLM1Bm0mUFVFzxh9MTlEKl8s1LpuUKmopjOuXr3KfHeHmAZCv8VmChiLIyclSS6atTEjb22KQ7S/+2+OCAZnLJW1JGshKRoTm9WKWzduFkeyWI6OTlitNrRtzxAidd1Q1xOquiKl4gvLKkymM6ZNQwqB/f19Vpstq/WGISTW04ZZUzHxhmpSYX0Bc+tOQd4XusY5jLWoCP0QwDlqP2G+uMK67cnrln7kx60VvJW7ESqUdX4Gv5pQzaAG1KMIGYuxFYmaYBxdTvRktG6YPzaluXyZYbOmXR6yWd5mu95j6I6AWKLtSHR9T9t1nxY1cxZuiyJaxm57csLeq9e5+dKL3Lx2jb2btzk5OKbvepImnHfszGbsXrnM5bc9zqNPPc2T73w3j779CWY7C6zzo6/gPrrmXOQdCmEIDF2PcRZfVbjKPzSsvjUAftRqjXElbM86xCpZe1QFxWHElxA2cSPPCWkIxBzociSmwHaz4mh9TG8gL3bQugJJ5UdLmIwqmKBMJ4lHdzI7WhZAMnd5s1PHSs565lCqmynL1YohRMIQ2LYtYYgIgmYppl4PRHCizCqDsxVYR8IwJGiD0g6Jri80g5FinntrqKyldobZxHNpPmE28VSSMaNTWb3l0ryiD3Nihrhq2bQtQ4iFHjFSwvcajzW2xPYbHekgBaMjyBvaNnDz5h4vv/Iqt+/sMYShaGoYVFPZUCnUknU1Ve3wlVBVBm/HeHbxDFEZgmKsFLoMcHXD7tVHuXz1Ct4ZQt+WyArf0G4H+iHQeEcyluPjDb2NqMuoHbB1grqj0yOGvCRrwGIxOGIuDjMRi688zWSC8w4h4HTgSiXs1orvl9z++C8hCk8cLWk3PbRrTLclLFf0yxnDbIJv6hK1ZSyQyA6SF2LlGbxhYywnriLGLTNvilUyRmWFNBSKxkGMAyK5aIBGxo0+EXPEZoPBYdTgbUVTT4tje3R0S04lBFCLg3YyrZnM50wmE7x3dN2Gk5MTJiLsNBOGYUCMwbpYQNOOgJl1VHTudTwaY8s2r0DWEsq73bK/t0+MxSLaPzws83oIhb6pHIvFlNlsRtt2bDYbUkxYI8ymNd5OYPzHq5brDUPfITlic8VkMcEajz11rorFGF9CiUcrV0xRILZthzhhssglpNKYEv3lKxKZIRW+3chdUBdkDF+GEuYpIJaojqE3hN4gdcWQheU2sB0CVW14fNdRzSxDX7M+mXNc7yBuQkIIsQONyBjff7JccXB0cg6bTjX3wqJLVoZtx9Ht27zwsV/llU98nFsvvsDy4IjtekvbDrT9wJAjxgqLpqaZNlQ7O8weeYwn3/1e3vuBD/LeL/ogVx57tPhUzsIs5RwCFSshDAO3Xn2Vay++zM6lS7ztySe4+thjWGvuT/F5oLwlAP70lYyMca45FT4+B7ICNFjXkJISho5haOnaNV2/YRg6+qGl67Zst3GgbR8AACAASURBVGs2mzXJKGYxQ1KDkQjEsx1ftTjaJkkxQUlatKDx17iZFoBXhZQgZTC25tbemu22I6dMGCIkqKzFktEQCETUgtSO2hct1nlPFkNQpRqUakh4l+mGwv9Zc47VO02WElM0QhKaImkYEOOYVparuw0hRfqhZ7Ue6LpQtLXG451Q+eLUsrboiXYMpVPJJFXaNnD79iGf/NQrvPTyDY6OVyBSeFILTgwYxVWGZlLRTKdMZjXOUX6MYEWofEOjhj6kQh8AUQHrqWYL5pcfpa4cod+yWS1pty3LVRmfSe1o1XK0bnGzjBolpp4hdAx5TZI1mFDoD++JgbPQWOMN3jvqymGNwJjwJgJWBe0Gbh9eZxU9y61S1ROWR4esliv2a2G2M6W+tIPfWVDrmJyEEiWTvEEWM1LObHvDUhwx9yAZX3kUiCGQUvFHOAMhDuQcUE2gFoPirEGzHUGuROA0VU1dT3GThiyU62MgtFtCF0oon3M478rcy8owBFarDVpXtCHTDpHIgHOK94qYhLG2WE8KzhbrzIyUBaKFHgHImW03EPqek6Nj+j6SVFiuVwXEU8AZYVJ7duZTdhZznIEUOtoUyLGHHKiahvmspmsrhr4r0TUpQBKcaagcYyLT6CpVg2Zz5i8o7cqEFCEPhCFALtq5ESnvQ0lETFnLPBbOLFIjZf6pARVLVMO2h8NNZDWANhOCQtsN9G3PfGa5uqvsXLEMseZoWoG7RDSeIQ0Mw4rUJkzqMcaw2bQcn5yg96rUIx5kQttx5/qrvPDRj/Kxn/4pbn7qGbYHe0jM5Hxq2QaGXNb20HniEsKd24RXrnPz2qsc3L5D17W854s+yGNvf4LJdIq17gzozyLeUuLk6Jhnn/kEv/AzH+GpdzyNGOHyI1dLSO5DhNS8JQBeAFPmAzEOJT43rImpHaNBKoypSDGxbdcsT45ZrZe07Za+7xnCQAgdIfSkXHhRSzWGB0ZUIzlH0qhl5qwQFScZMwbiiOQx/nZE+pHHzmb0lIshxJ5tW/hgC9TOMXWWiRWcZvK4YSRnCOqwOMSURKJKM8YkrDVYK1iTGAbFkEtiSzQMUWkHxWwibZ+pVLEpQApYH7GVY1EL/VxYbwyHx6WNMWckRJpgmGu6GyY2OryMMWQ1hD5w69YRzz77Ms984iWOTk5IOeOcI+TCiTbzmqoSqsYynTbs7s6ZTGuMKWNkyFiUuq6xrmaiQtsFjLUMMdN2kZNVh5+u2VnM0QRtHzg4OORgb49u27KYzxiiYUlmItD1idVyxdGtI9ZHW2IKBTTJGGuxNiIxozkgajBqMRrxYsfnOu5sW040QI4cd4GJO6FzN/BVzf7tO6wOj4go1dUrzPrIXJWJJnxWLIr1hmo+QajpTGJYJtpBSJQwxEoEX1VY70kxlFDeUDJ3I5EUewyWFJQcQYzDOId3ltoanLOINUwXO9TzKc4JaejYHJ+wPFqx3bQkzfQx0PY9tvKgFjENUs9J9YKN1rhkMAouJ6DQjiVkWHHGlogVW0BeRcsmSPHVhJiIfaDdbmj7QFLYdmMGrGYq75k2nmntaCpHrBxd5YgD9O2W1QlobBi6HtFI7YsfxRlh4i3T2lM7W4ITtPhscoIhhZJk5ErmuPeGuq7pQ/G7Zc2krPRDYL3tqCxIFiwl5NJZi0EwRs8yZVWUhNAPyq39Fdf2AwdbhzZCslOyCrUYrA2kPmByxXw2wzS7BGfpNdKHq0h6gu4EtF3iyOTUjf0B5+z5EkM/RPZv3OTjP/dz/PJP/Dh7L3wKNifMJeNqyxCFHDJGcvGjOcOiLlnlXcqs2pbD69dp12tOjg/ZP7jDF3/oQ7zrfe9ntliUHAQ99Z0pXdvy8ksv8cu/+Iv8/M9+hOXJCY+//Qne/8UfxPuHo2neEgBftOZETkrf92zbNW23JMZuTFQyoEJMkWFoads1cejRVLgzN3r6rTA6/EpURs5lcaYcy29NZClcfgqRZCKqMqb16z2eizRGHJQ6i6PUmRLmhQhT75nXNTPvsZqQFIkpEYHQZ1oCLmTqOjKZ1sxnDXUt2JhRExjCQN+XqAtVh3Eelx1pmzlYr4l9j0NpjDD1MJ9aprOMbxzOKJUTvCsp46pjvK+WsEznDMaeOlYNMWc225bbd4547oVXeP7F6+zvLwkx4SozhnMVrX2+qJjNK6ralc3IncaAO6rK462MAN9gfU1Wg3VdefcQWa+3+MMjMsJ2syXHjs3yiG69pK4tO/NLzGcTNvuGG7fvYI96hu6Yk5M7bJcb0pAxmDFvwWJ8Rkmk2JM1YEyFdzDxhtoK3njECattYm87MHSRLIZLA9TbDtkMnCw7Vl3GbyNXNgPrtkRQpDhG0FooaaqgFegEJAguj9FUtlCHzvsSghccmiOaAzY5YowMIUAeSFHJ2YCHqqqpZzNmVaGDXDWhmSy4dOUqk9mE2HfU9SFiDsEtUYWmmWBcRdtHtl1ETY2dXEKml4l+TjbF3ReghPpqJheWrgD7WSbqaQjfqd+g+E9CiPQxEEJgSEoXyrpw1lDXFtHI0G3pDGhOo7/BI2JBSqYvgBHFu+KYdALeQWUN3vqzXACRoh3lMe9Ex7ATay11VZE0k3Jivd1wcLTk9t4++/t7TBrPrLKsrbCYNcynE2aThsq4EjAxOljbtmfvcMOLrxxwfa9n2de4aYudPYZrdlFfsd4KR8cZW3t2fY2vLXXdM530XJpbZvkS67RlmyI5bIkpE8Poy9B7tenN8TEv/uqv8twv/Dx3nvsUujlhbhPzyp7RSTFnaBxqhaYy7FjPECMhJ7xEyEpaHXPjuU/Q91tWqyVdP/Ce972f3Z0FfdexWi5ZrlYcHR7wsV/+FV565hOs9/Y4vnyJ1ckxKcViYXy2OPpR3hIAX9Khh6IBth3bbUvXlfhbHTNRNRdNN6WEEUtTN1ReyPk0qqU4tDRnNGXSkBmGiIbhjL/NKjhrqKyAV1KOnJ6pksd06JQK522swThzlouX1VHZSDU602ZNw6KZMHGWOJQU7kGVGIt3PXeJLFDVjp0hoa5mPq3BKVkiIRaHTt8HfAb1ivQQtj2bzZbNeo1RmHrH7tTxyKWaR1AWImNWpeCdpfJKFnDeYb0vyTbe4t1pgpiw2ay4dXufl16+wSvXbnBweEIYYgFuZ0s4WyNUzrO7UzNbTLCuZA+XRBWo6pLh11QeZwTvKsR6QlJiUvp+KCbs0LFdrVCFdr0ip57Ub2m8Yffygt3FDOcMrxysufbKi6R+RWrXDN0WKyWZyNYOsb7wxhIJcShcqSjgca7QNI6SsKQYenEso9AGmEzrcp5N1qIEOE89XaC+ZkiZoe/RGMnREM+tEbGAV2ytNI2yE5XQGeqmLs4t7zBiSFDOzMmg2pQY9pgZ+o4US1ao9xVVM2G2e5lpPUW0cNLOOLxpqOsdfLUAqRHT4Js5WSmmugonJyuWm45satx0FzvdJVfTwgVrsfxUioMojeaVWCkAeE+MdlFOjLFU3hGSJXSJHCMhFs1ZBLwv5TEMbDcrhEQzmTGfTQsVkpSqnjKbzc+OEMAkchKMKpUXrFjs+I7GlI0gU4Ytj0mKQKGRrEEM9H3P7Tt73Lizz/Ubtzg5OWY6qZlWDi8wm9bs7i64cmmX2bTGWUG1nH2zd3jCS6/e4cVre+yfBEJqqLYbmmFgshvR+S5D8Kw2FnPoMVWiWSyp0jELe4SbdGT12G1D3no2Q9mM8hi9w1kEO/Rdy8Gtm7z40V/h9qc+iS6PmXthp3JMfIm7L5lyQuMq/NQzrS1NFo42mW3INFaZWoOK0q1P2HvxOYZhICYhD4nHH32E9fERt27e4Nbt29zZu8P151/k4NoruBBJbVuy9HPZgB4mkuYtA/Btt2HbDgxDJkbBmCmYBqGYZWU3TYW7lJLEo4ze7Xw3w7Vk8PUsV0tW3THHmzWbdluyThF2J3N2Fwum8wbrhJgHQhyIIRKHSNKBFBN1M2E6nVLZwokOA1hpi5MRgzceK3e15yELbYI+ZboUGVIm5oy1huNtoI3ClUsLmsrRbkp42cm6JYREpY7sIp22bNqO1XpD1w2AUBlhtoJtmKAGXFUhSAFDa2gqi/GWqq6ZzyfUkwmuKhx1TsJ603PrzhHPv3CNW7fu0G7bEp7ZOJwzNI0btfWSibjYnSDOkRmBvfLUTcVkUjOZ1Ezrmsp5xNiRe49nmZ7OOSrnMChp6Olij7fKYlJz+dKc6cSXqAqjEAP7L77A0K4QDXjvmS/mJZfAe/CwzYlV39F3HTFEjLdEhaBCQogUkzjngMaIN2AnFTs7O0wnDQK42vLo/OpoiSjT2iMp4SlWXowlac57h/MWEVeiV2JAc2LbGWazpmSXWlsiuVI5VsAYqPwEJxZRw9AlUoyIyeXQt/mCxaVHsb5GQ0CGgdiuWR04hihUswW2XrD7SM1kcZmcSo7F8mTFyWafZRuop3P8dAfTzMnOj87GTEbHMEJFTEScA29K4KAwas9AykgE6x2VViRNhJToc8LkEhJspCgL3go5BkKvmGnD1cs7uHrC0XLN8cmaybTh6iOPYERZruccHu0ThpYcIk5PnbolDBfhLPdCOf27+MBOz5bJKbLdnrC6c8i12/scHC9H/1agdRZr4OBkzexky9GqYz5vcFaIKbDZDrx6Z59Xbt7meN0xREC3hGFN6A7R/jYz806q2Ttxbs52FTjmDrubDZVdcsn0TOpMH6B1mZWhhF2extVzCu8F4DcnJ9x88UXuvPwC8eSIHSMsnGFqBC9CFIM48EbJjWd+acLOtMKHklXc9gMTioJZHMPCUHvakxN+6Wd+lu3Rkkd3d1jt3+HmjVe5vb/HarMhtj25DzRisapoTiVk+GE8rLxFAB6UfujZbDeEAbI6tDBiGKMlqqbkgJIpSSViBCn8CXEY2PY9XdfRdsXs2dvf4/DokOVqTdf3xBgRDHt+wuXdXS5fWTCZVigB1UhlbQkjtIX2WOzucOXqIwXMVNhsAmJulzWjmSEGegE1hpASQ0p0MbIZIm2I9LFkGjoHiUDSJds2MG08mgPdkEniwFmyGLoh0YYNbdfTdj19iCiWXsqin/jMpZnj6ryh9hY3AryzmZiLc8pVFd7XeFfCNtchcHPviFdv7nFwcEwcAhNnsFNHJRFjBF+5kqDUTGimDc2kKlmAWqCkaWpmkynz6YzalQgmkEJtWUttLL4axpMTS98hxbz33rIzb7g8b5hPa6yR0SIzqGa6zQlDtx4PhBK8M9SuUG0ppWIZdVtSHEqopS/x5kMIbLY9s6qiEsXkRG0UXxXn9O68oa49zhvqpuKRxx7h0pVL5NijcYOmTAwJsCAl9DTlQNMIrqpppkLMoEao15aqqgrnaQwaAqoF5CUq3nmcm2AnvnDvuRyUV1lL4z113UA1JemKtN3QbY7pN8fYbYvffQxTudG3URKV2m3P0bolime6e5VLV64y3bmMcXVJmEPvRltIWQvYEkIrvmjNZxmeaVSOjMHmknxknClHK7gO6QNDLNaxM4Izhma01K5c2uHK7oJqMsX7kuhUVTXNpKauK6qmHKmwWZ3QbjboEMbYeQsio8ZeNOLTw8+MmuJMhTFRL9C1HatNT4jDWeLapKqovcfY0idRldvHG17dOyTnEjAxROV4tWG1GUghFeMuJ1QHQurZas+Jj0wXnsWupzYWl7bk9SFUJ9TWYCpHTyTElj72xJzHHJoip3HqOSVODg+48dKLbI6OIAxURopfBjA5j+GvpWtFEo5IJY7aWRa1J0wbyp4rJUlNwO3ucBgtz968zTPrlluTGmk3rJfHdJsNQ4jEVM7j8X6k+Zwv1tFD0DPwFgF4BYYY6PqWvs+EaIrZIhljBe8dIGcnIZZzLTzWW7qh52R5xNHRQYmkaTecLE/Y29tnuVzRdS0hRmA8Upc1m37LalgynXqcg0ntuDLfwbuykI337F66zOWrj4xp5ZBkW5KkpGRu9jJgUdSackhXjgwp0IeBbkiEBGI9Yh0xwXrTMfQDTe2ofXGG+WZanE9lVhBCQDThbYljTpizc3NUM2PgOMYKDgqQm4SM4XtZS5SCseWMmdVmy/Wbt7izf0jfD9TO4l3hmbumuOhUXDmEatIwnU6xVdEuTg868b6iric0VYPm4lgULNYZnKvAWOo6MIRATGk8TjljrbBYTLmyO+HSaFrnlEmJcpCKKllbMmHMrFWsSInSoQB8Dv8/de+xZFmWped9Wx55lbuHSlWqq6sbZiAneAo+AMd8JUw55IwvwTknnNAwQDcbXVWdGZEhXFx11FYcrOMe1TADqoeJa+aVmVZmEe7Xz1177bX+//sX4jJjSFSVoelqshIH7zAOeKVpKkPtZKmnjaHyju2mwlQVxYqz9vXhwLtvv0HpwsPn98TpxHWY2e0KlfXMYZWbKkWjPUobjHO4mtVqL2wf2VWsO/hnNHIC7Q3eNmw30gmHMNFWHm9Eh453hCWTwpF4/omoKvI8U0JkypoQ00vHHaOMFl3Vstts2R9usHUj6hOk4jyTSfX6/RQUyhqMl9GJWo2BRRchTmZQGHwx+MrhvEdbi7Yz4xxJKa5LQcfNYcftYc9+v2Xbd2jvEed3Edha7df9iyWWwjwtoGYy6XkLv/pW8ooVETQIRZy9su4qL/sDZWSM2fQ1NkFTt2zbltZXKK2YY+J4Hvj88Mj9wxPLPL7cUGLMGAzOyO+jKPlzU5qZxsjjY6Q7NNzeVrS7Ay4L7yfmiKksJUXh8QwXxnki5PSi3pH+WCSSyySyyJ///CeW6xUZCiphWT1LUEMkrwozkyMqLKQZQjGYkumcxVSOkIvUIwO3rw74aPnnD/c8vX9PUIW91VSrZ0drw2mF+iUiWRuM8aI2/DdanX4RBZ7CupzKjPOV83XkOs6gDdZZrFXEdTGUU8E6T9ducK7m54+feP/hX3h8+oTzCqUy87KQ0iKyPydXIq013jkBYtUAE5fLhbar6ds9rmqw2gKGqu1o+x3aekIpLCFyGieK1mIN15CVIqqCZoWglbRKMUVvL/rj/CL/MlqKdQhJlkzOydLSWSrvMQqWeWSeNCmKWWlOcr1zynC3q9j3DbWzaJXXJZdownVJhBhYlkDKUIpmGCceHx55uL9nngacKdQ601pFZWuK7hkjXJfMMst7aqsG4y0pBNlp5MyKIiemwjIvWGOpW2GzP2OSjRWjyTQvWDvgK0vfbrndb9n1Fd5Ih551RqmvILmcJ5RKgqNQkFOEnPCVh2JxRuM0KKvoOs9u1zKEyDQH1Kou8lqzr0QyWWvLpmvY3vSoekPEgXZUWLzyHO5uWOaJj5cTD8cLh9stm31DoxvGUXgkIU7EqJmWwhgU8hvmRQOfYpQOdIXcTeNMmjOlbdluNmw2HSkvUAqV84LgdaDNQuZETF/QVKTFcT1X/PQw8vHzE8Mw4qua7WbH4XDDbbvFVq04oUNEZ2l29PqBUeVF/iXAr/U2lpMU1gIYqyhFDg6RzBrIBevsatxRFBZyluV5U1f0mx37wy2H/QbrPVmBqxzb7QbnKjYbccCer1eGceb+8cTl6YShUNsK7zJaS7FNpRBX3j0Iiz/nzLwkhnGgaOj3GyrjWT4/cL3OeK9pG0fjrIwiEEd5Kc8QPYcqK1+p5Bc3MDzv8gJFzYQ8sUyK6/0nrtuG2y6xFEtaKhrTU8LM5XLm/tM959OFJS5fR0l/MQIpKXM5nrj/8J6H9+9J04QrACIG0EpqQs4istBF0xpNHTN5XHhYApcxkpWlbQUpHXLCOsPmZkttO9799Jkvw4hfFnrn8dawGE1eEkOURiyVsoooyr+xtMvrl1HglcL5mrrdMIZIGQeWPGN0g1GKkBPDdOZ8fuJ8PgPQNB1N3XM+jVwujyxhWFUOoJA5vTFKlnYr28I5Q1VZqspijCInuQmgjEj9VME6g1eOcYqMy5HT+cL9wyNPjxcqY2mtJxThyFAgpq8hEM4YamcpZEJWKG1lBOL0Cx3PGhmfFBBXapSZWttU9H3PYdOhSqSkxBwCKUacybze1uz2DVXjyTmhUsFWliZpgkosAp+XRXEsnI5XPn+653w6s0wTOi9EnTDPs/q2ZwgKc104n2e08xQtpqyiHcZpnDK4qkYbeY+0sWjn0NaBNi/StmlZWEIkFwn26NuGw7Zj28k4SZX0whAxK3nRGUPvPclkrKvwTpzAgjoWGJe3Bmc01msaC15FljjhUsQZR+9hU2l6n7HArnLcHVraXUN2NVN2hGQgJMI442yFMp4hZL6cruxPZ7pdJd4B64hz5nQeGGfFvCgChphEIhdDkKCRRQBhyzQRlkCOCmUyYTHE3FD1G7rakUpB25qwzHCZKNcvlHCkqFnksOOF8/TE48PA+w+f+PTpC8Y6NpstN4cbbm8f2B8O9F1H07Y0bUPTNPhKmhSz8s+l+K3GJoUsYbMUeK2djG6KelGECexO433EzYuMw3JhSYWcJuzjSQ5ca9nuHXXX0tQ1e2PXG64nhcw4L8RYGK4z58uI17C0LZksmIayMueNXsNK5GYJYt45Pj0xxYKqGqhaVEg01rKtK2pAh5mUM85YvFVo9XyTLSsMTQB7z7eVZ0yGdZY8ZXE/B7g+Xfj00884A66+weka6ys0gbIkdCnUVhO9YYpStJ35WkJTSpyfHjl9+cJ8OqJiFJwVrN9PWumegmjOFHxxVEkTyFymmcsSQSfU7JhzZsmR2inmMOFsw7ubLfrxkXxaaC0Yo0iRF92/shqsIpCZw8I8z9Rt828qrb+IAq9Q1FVHVpY5ZenSUsa5Du8bMpGiYJhGxuULSxi4zmcq10DWGFfodI330iVHk8UmXQzGaFKS2aVdAUh2TaPRtsLbmpwVp9MAxWB8ZAqgHi9M08Lnz/d8/PiJGDJ/883foWNhGK7klF9YLyCGEu+sAJR0wsQCelV8GIVCCpb3HmUN87wwzQJXGypHTD2vb/YcDhtab4jTwHC9EMOMd7DftXSbGldJqpRThrpYOpXAZOZYMFZ+zutl5NPHB37+8JnhPDCNEyotJJNpK0cxjqppUZUhsrBEQzGrFn5JFKUw1uOdx1cVZkXiOl3jnIydQkzMS+RyHXh8OjGMI84a+q7jZr/ldtfTVWt4S5KQClF2KZyTWetNd8MUJ9QanCK2/X9987FGU4xGkSjLiA0TlkLrDLeNZluDVxnvFLuu4nbfU20aom3w1IRswVQvXU8qiinAcln4+Hih6Sv2XY0qihAUj8eF8whzMjJuyAqlDXERk9AyjszzwDyPpCVijUcZQynivuysx20OOKNFIx9m9DyQB5n9F9cQi+W6ZB6HE0+PA8enM1++PJJyxvsHfv7wkc1mw2azYbvbsdlu2e52bLdb+r6jaRvqqqKqKrz36DVDQa14jRgCKWeqWsZ/yih0WaWTee2sk+jiQ0hCnIyJnBPDNDGME3OMfO8c/eGG27vXNH2PtY6cMsN14jxMcijmwrxEksosMcp4Q8mnWvYx6mVxaazcpEsqTNPE03Eg5hOm7dFVTd91dL4mzxPzPJJLpt5sqZ2hqZzgg2N8QSiIlFmtpEp5pgqwhCjKOwzjMPPp8wMJze5guT14bOPxuoKq4XYf8d7TjjOXcSLHwLarXyQqKUVOj49cnx4p04TJSW5Rz1r8lIkI+yqxuq1XxnhOsORCpKByZhlnppyJquCc4fHLA1UT6RWE2hBmUfhls9KjrKbEhHEa3dUop5jmgePjI23X/htsTr+UAq/AuhpbDG1b2BeFdjVaVVjbYpzHWMN+/wVf3bCEKynNxDAzXs7rKV6xLMLTyDlT157kZJmWkrj5jFaYolF51QZrgy6KHBLzFJmXJPrjcSFE0dken06EZWG73fLm9Q3bruHhUTNcB+ZlJqXCC/sla3FlFrXyctYrfcmyQPSeqq6YQ2AcB87XKwUYJ0MKC04VXt9s2e12RG8I05UwzhhrcUYKo608VimULyif0HbBGOmgiwJD4v2Hn/njP/+ZDz/+zOUi3JycwkqHXFB+wbZFgFDO4X1iyYplCUwhoJ2losLaIvI7JT4E/RdwqnGaeDqeeXh44uHpSCFzd7Nnu+m5PezYda2kSqlC1mpFGa+4VKPxruLV4TvuL5+JBLy3aKOJMXCNkZJXRYpSRA2hyCdmV1karegaw21b8C5DzpJu1fe0ux2+30J7Q1/vUb5FGYOr1k59lbyOc+Hz44KzZ5Z2pvMV81IYrnBNlkU5nLJYndDGo7TcQkKMzIs4MEsuWCPSRl/X62FYoZ1He4OKYtayNChzwNWQ7AEbIR4Tn48nfv78wOPTkRAiMQkGYxonzuezqJK8p6oqmq6l73sp+tstu/2O/W7/ohjKIVOiIsyJ03lkCYHNTtN0spfQVhRfy7LwdLry8HjifBmYl0iIUqhSWqW7y0Ki0O12vP3mW7abLYebG6xzovryV87DSNN1aOfICuJq0prCgg1WHLTqL9DAPMf8GSpT0zU9Tw9XrpcBEw13397Qbfbkknm8XLg8PWAovHKetul5fdjz9HjkepY4RmstpoiqCcBYi/fVuscyqGLIJJacSbNCPQ409UD7JvHqbkutLWG29LsN0xK4jhPX60iIgaZpXmbcKSbOD49cj0eIIqtGaTCKohIJoYimoshak40mGC2dOlkk3VajkkYtIhk1VqMifPnxZ8ifZIQYJpF9m0KpHc54qipBeMJYRXPY0m5axunKjz/+ie1h929S0vwiCjwojK2wxVAV6FCgvCBvlcE6R93UaP2KlIR9UkrkcnrkX/70j1wuDxKtBKvJw8hOxxjhwWSJJ9MUwQtYu45uVpY3GZziep15fDrxeLySomRrTvMoXbruubndE/oGYwqPFoYBmSdnccfGVMipYKKMbUJYQCmUc7RNR91IBzzNiyCPQyAj10CrYBgrYozyfdW1yA3DQrIC2RI9v4CpbO2wdaGogVKuTGpkpRLz+PDA99smCAAAIABJREFU4+MD18uV4SIL0FwywWqKzrgmslugd+IudE4zT0FkojGuTkg5EJ+53KnIz5amwDjNXC5Xvnx55OOnB47HM31fU7255bBt2W0afCW2fZRCF9B6dQavLj3rLLd3b7iEK8v4RMqSHiRzeJkzSoHXNP2O17d7fnh9R1sCjGdYBtpaRm0Ux6bt2d29YvfmG5rbt7j9W0x/g/KV+BJCQBehLBplKCFSZg2TIaMpQaGiQi8WVUTdhJLIwsswskwD83VkvI6M47iqhBzGeaqmoe16+u2Ouu0x1oHKWKvQTY13G5Lp0NUBlUfxDIyPnMd7nk5nLpervC95ZSSufgwJ3JgYBsP5dOTJOaq6oqkbuq6jX4v9drsRXPR15uly4f54YlxmhhDZzDNd10oYR4icT1fuH45crqN03kkMSGvzT06SDzCMs3D3Q8RqLUvjqmKykWEaKWRSSesYRmb7U0hcpwVrHbX364L6L9k48vmsbMVmu6V5OnNdEvKArKTXDMv6ZRCQWl97KquZX++pdeZ4HSXEJD0/lzImmdaDN6VnN3de9yeCReh84raDXWdpK08pDSllAYyNE+M4kWIAsyZPlUKOkfPxyPl0ZgwREwvRSpBPg8GWjC1gVKFoiQssyIGXimj+LPL+TksiIu9FXhLzMJMX2cn1BpzX5KbBf/MtZX+gmxPn//xPxMsVt8qjj09H/vzHP/Lt978ipvTfLqnr65dR4JV0hxZFpeVKbLRnniZSTmgjWIG29rx59Za2ET738fjA6fGJcbwS07DO4syLlNEah6sqrNZolSg5ktY31Dkj1mctM6+6cpzOF5YwMy8zGkEK1I2VP8truq4hV46UFgFhGVG+xBCJIVGWjC5r+gyIXrrwIluUvNk14WnFxD5H45R1hvccsuDVc8i4ImVFSIqQDBFP5Xp81WCzYomOZcnkGPBWkYzm/nFgHkc5HFZWuFGWgiJlvWIRMkEkLSJFVRmrC8XKvsAq9TIDlwVoEqXBIqqLsCwcj2fuPz9xuVxpa8u2qzlsG7raCU43l5eitY7gV3VXwTjL67dv+PjwnsfjipIwQdjoSlj8KSeMdRxu7/ibv/9b/ue/+z11mjl/+pGHD3+kxBlVwFnPze2Bm3ffcvju1/TvfsDfvMN0e4qC+XpmeHpiPJ1E7bAkCJla1Wz9nt47Gi06402diEkcFoVIWDXM8zQzjRPjODJNs8g6q4aqbmm6nq7fstnucV2HcoZUZgFoWY91Lcp0ZNti0oguV3IZGOfEOEpqknwMhI3zNWz6ZU5ASlJYx4vipIX17ryjaVr6zYbOBJ4eT3w+Hnk8nxljkPnvMLDpO7qmZZoXjscLp/NVlvHolaItBilVNGRZKoeYuA4jl/OFaRgpKaGUUFvDMjIOZ8brWfKBjRb/QJTDYQlJmDRm7dyVMP9FaAAojXGidDLVTNKClxbYmWO37XGqQMq0dU1XWWxtcN++YtN43n858tOXI3EOqAJaScJYGCdSCCJh1SubHkFvb1vN7dZxs3XUrtBUIm8FxTRPNFVF7FpyXJjjc30vxLAICvhy5RISKiU8kLWmU5qmGCyZaqW1Wq1RuUgWABm97mtDzowhk43GJuH855BhSSidsJWmbhvsfsP219/h3n3LZsncz4Hxz+9ZoiiWnu4f+OArjo8PxLD81dL6yyjwyC+pciJTrF1FWzekKJiBnL7KhkwvnHhQ1Abu375jCSceHgPzEiTeS2usFhRp1/S0VYVWiRCndfwSiSWRYqCupStyWm4JomwxWCNLv3lZQaSrlFEOEUfbdiilmeeZaZwpeQbCOpeXn0cbs86dRUq5LIvMRLWiqqpVnSLRgs7JARBjZBonilXiwKx7jHcU25FdD9UeVW8xVYMuCj8b3BjQ04xZRyolr2qCHOkbWYxpY4g503Qtm74hLAvnc0ApmUs3bUPdGhlNLZGSwVtPWzcYoxnGmYcv4ivQWrPZ9hQ00xQIMVNVFTf7LZu2xjkN5dkVHNcEKTFDOWcpJeO94933d/yXP/4DOYgzMWhZite+4jnRq2k63rz9jt/87t/xt//+f2Lr4eH9H/nzf24433+kLIGmbrh7947b737F7rvf0L75HrU5kG1FjgEVIkt+5OOnL3z8+SOnxydKUThXsT28Yt+3VMaQi8In8OPE/XDiMh7JxrLZ7rBaE+ZJblFZlofGOuqmo+02NF33spDOSlEwFG1kDhwX5mlkWQZiWZjGK9N1JIwLOWZe5I/Pyo3nq3dZKZGrlkfOxgxpVaPEwDJNDNcLd9uKz1++8HC5cA0LgcKcAtM0cTld8d4TQ2QJAhmza5hOTkX4NUqRdFq5TZIx8OXLFz68f8+7t685HLYUEpfrmdPxnuPTPafTI2GZxVC2mr1UVvL1XNifPzNa46wlpch1XHg4n3m4ThyniHEi+9u1Nbe7Hm465vHKNM1UzlLZIhr9u1u0r7lE+On+xDhNpFxw3gNyE04xymFhlYxDtKJvPW9f73jzaidu8hVVIiIkkTY23lGcJQUDU5C3PyfCMnMZLlynkTFFVEyEnCAntlVFrzWVMlQqC6cpR2zW6CyHWs5KsnGTeAFUKaKbL0US1RBpaSILq2jb0XQ1uvXUteHNDz/wcJ35059+FIJoLnTeMTx9IZaveOj/1uuXU+D1CkiikJUoBIop5JyEVleyYHBthdBQIHhP09SglCw+c3zpkEsBjBImtjWyiMpqXYquZpAs2ZPTPFFsBp3x3uLcM75UE7OoEJ4Dql1TY6yn3+yZl5lxGLleLlwuV+xlQF1Hkprl2q81KQlqNafEMs8YbYTxoRW1t4Qk0JDaW7wT7vg0T5AMpmrY3jU0/YbN/sDmcKDb7mjaDuecGHZUzZJkDkgahdcTFkpOVF541E3T4r3HrolLxltSjsxhJueCqxrqpkMbz7QE5ngBJMDDO0flrJSgGBguF4w17Hb9GuydaGrLphfKoNbrdqlAToGwyDhKDjGHs5Zc5MB89+07dtsdRjuWsKzuR14OaWMM3tc0VYtzNcY37F4faPuOqml4+vSBNM/UVc3N69cc3r6je/UGu7tdbf3yOyvqzPky8k///Cf+5ccfeXh8RGnDl6cHvpzvaPsWtwYx7JzDpUQ37DifN3z+/DOvXr9h7FoUmXkaiHF5kbyO00w9zbhpIZkry2VgDDNzmKgqT1PVqFyYhzPLfEapzDQkWGaq1als9RqkjSyi/5WTcvVAKArOKJmnr89VyWVFU2jxKMQorlRnZay2AsZymJiGeSVnym25WJBF6FqI14KslSaTiTFwOp346aef6Fc42qtXN6QSuJweGc5HxsuZZRyI0wzK4IoYvczqtH6J3lZrupQ2hBi5DGd++viRD5+PXKdI1xmsMWwbz6F12AKphiXKTD1nBPaHYgiZ0zAzLkH2TkClxatQSiEUeR9lTZnZtC3fvr3jb37zPW/f3NG1DQZx0aaSXkxhdgW0YRSpaEqOhHlkOB+5no8sy4xVGufUusjOXElsK0vrLV4HbMkYJZAztGIpyNhqjqSUV/e5whsBoCwlE3Mh5EIsKzdrWSjTiE2RXd/z7fff8vHLE//wj/+F4/0DJszMjeX88SeK0/y11y+kwKuX6+kKUHlBJGulROr1bFRSBVadr6BZDTklhmGQpaH2qKIoCVIQel40oFRgmSfCNKGMo6ocWUNKC8s0kbVov53TOG9eclytMSJ50xrna/p+I1xrpQkxMgwDl9OJ8/lEezpRHc/4y0B1HakGIdOVnEUDHEUupTXUTuGtJRcjGu7Ks9901LWXYATv6bstdbdhe7iTwt5vxbruZQSSQiDbikgmEpnOD+TrRVgrJVNXju2moeuE772/OeAqT8yZ8ziQjpklJJyvqJsWZTyxTBg/r5hbhTHQNp6+q3g4bJjGgVwKfVORl0DfOuq64nbfUXsDKcoSzJgXaaiikFOUuWRYWJaFFHtef3Ngf7iha3uW8ywYA/O1eDkry7p5GHn4/IUPHz6xv9mxv3tLs9ly982vSGHBWku721H1G2zTgWskZQsZPaSkOF0G3n/4mU/395yvFwqaH3/+QLtp6TYdvu/wjaduaxrv2acD03DHPA/sthsa71jmkePTA9fLhbDMjOPMoz6S0SwpY68DY4w8XS6crxe89zIeqR0pXonLRTADwaDCwra23PQVOc0Mc2AOcR0PiYtYZBKifvFGsWs9rw5bau8JQcJevLNUTcWn04W2qTGVp0qJKUXmEIhLIIW0LiOffyESfal1Rin9MkJ7VjkpLU7UcRz5+OkTlESKC9//8I7truN6HZiHK2ES1n8OgaKzAPOUSIjNOh5RRVy6zxTIEBZOpxNfvjysKhpL10BjHb0z1FoMfcYUvLWMqjDGzBjg4Try/v7Ix4cjwyQKtOclrl4bAq0VKckZWTnD27s9v//Vt/zuh2/Y7zY4BUWAUeSUUVm/xG/KjsDiHYTrkYHM6ctHhuMTzDOt0bTGSt7tHBmWxNVqtspQa0FwGKWwVhaupMIcM+OSUDlTe02twSk5fp59AUsqhKKlYXg64j5+wnYbus2Bd2/e8PabB9qmZj6dyKPGx8D08JFU/XWi5C+kwK+d+l+S57JwusWZt1IhVHlJclpHedIlNQ2V96CF/WK1XI8pME4jRgdymRjHgbhA03ga6zBrvFcIC8M0wToqcc6RooyFjDUSsaY0rmnpdgeM8+g13b4NC/3hwuZyYnM6sj2eOJ8vXM5XLpcr1/OF8XmGiTgjfaWxTr/ovquqoutaNtsdm92efrOlbnt8t6XpdzSbPc7XGFutIQ8KyGAXvMk07AhlIpeZ6zS8KAuUUaQi4eS2srR9i68rwQs7yxITTAFfNxjnKQi0qm1btAGlE8YkNr3j9uaGtvG8fX3D0/HMfr/jqXLM19ds+oZv3uzpao1RoreuvHT+dVVJzGKSw/D4dOTh/pGLesM7Zdjvbri7ec0cJ7QVyek0jnKNdg6N5uP7n8TIlRZc6/nbP/yem/0r6u2dGH6MsNSVOIpAG9F7Z7kGhyWuMXoWZS2RwrJMPB0fuf/8icfXrzns92zKBq0yzoCpGvpmizWOZRq/7lqiUE9TyCxhpqzZANdhxLctGENYFi5PT1yuV7QxvH51Q+01JS3EZQJBwXPYWKy95bBv+PRw5tPDicuwkGUg+VK4Kmd4ta35/bsdf/jhLbu+ZQmJFBPOSpj1//Wf/szT3S1zyIxh4TJLoPSQJEovlfRSxPMq70OJe1ehX7p4tXbxIIfA5XIhxYUYF8bxzPfffSNqnGmhxERlLK7W1M7RN5Ls5O3KwVlHWRrpelOMxCD5BiZneucwruX1tmffeCojV7gpSOay0oasLUtJPFxH/vzxgZ8+P3AaJlL5euuPIbzAApWSQls7w5vbLX/49Q/84Tc/8GrT44wWDo6SWlLKGtGp1MuBVHIix8D5w59IvuL444/EpyfcEnBKs7GWpAyXBFMInGNgm6CvLH1tsUaxhMSyBpIoLR4LnSXgp0QxL8rvWDwDEQhZsUwR7o8scyAk8Nsbbv7wHb/9za/4/W9/zZd/+TM3jeftvsdnoaL+tdcvo8CXss7OeLFhPzvKSklE4fe9zOFXqAM5BTZdx6ubWz7t9tw/fqLkhO9lRk6GZZkZ55EYZ8KS8Lah8hXWGlIYoURJjVm57c5DVTlmJcoYjcYoWcxqKwVCGYMyQs6rV0dq0zT0fc9uu2O4DgzDwHC5cj6duJ6l40tJEMTWKZwXN2tTV7RdR9dv6bZ72u2Bpt9TdRtc3WKrBusqlLIre0QMJJBEgudrqrannUfCeMUcj6tj10vcGmp1wiliAaftyt5PGFOhlISApDwJ8ycX2tpTV4ZN57ndd9zsO273Yly6228FCewM1+uBb15t6bqKm5ueu7sN+92WtmmF3VLKv1JyaWO4XAZRiRS5Xe03N7x7/R2X64k5XFjitM4+vxpZxmmSGWiYaDYdrvK4v63YbXc471a7vn5uB17+PqUApanrls3hlu2rN7QPj5xWBck0zDw+nPj48TOHw4HNtqduBEamjMgj52Xhw4czOUXOpyM5ZWpfyRghShe4LDNKixW/aVs2bYUxB5w1oikfJwwVzljAEaKwkZzTvNnccLi7o+1PLOlHxuWRnNZOekVU32w7fvftDf/+N6/4zZstfVvJCFApUW+UxP/9z5/oux4fEma25AyLChi1jie1hLGXVRWVs8zyy9pLaqVXMQJrAyEmKsHfFM7nM/dfHujqVtK0tOPmcEvXbmXRrRWOQmXFea5f0BN6DR2RIu+so287bndbGp+pqo5XtzsapwnLzLkIVyrmgrYGZS2XkLk/j/z8+YH7h0fGcRT20XPXvnLUc5ay2dZy0/nDb7/jtz98y6v9Fm8UKovjXL1MClj/XbKWSxZZ87LMLKd7lLZMx3vyNKJzxCAiirK+RzHBNSeuRWGqnq6T3IQ4jsKKKoYqKoKKEKCUsDaxohjyTlOsZsmFa87kKdEuATXPuO09eRhoneHVYct3b++o5jNbq9l3Hp0CMf4PoqIpReSAIpVaZ4/PxLksBoxcvpoalH7m0iT6pub17Ss+373h8ekLISwYo9htehRwPheu1zMxZIzx9P2GrmvQKjMOCzlLWpJxNTYnrMk45wirK00rOVCsdYCkp+ecMSZJN60lxs7VDbVzdHVD2C4sy8w8jQzXK8NwZZlnQlhIMYKS0VLlHW3X0fcbmn5L3e+o+j2+3WHr7mvkGiDXFvXyfhXEUGGUw9uG3GxYuoGqfqBtO3xViVJC23WEULhOAeWSGJVCISaY58QwDTRNYtN1tE1F18lIZrepeHWz5bDraLymrysOm14yxUtagzm+w1eaqrFUjcdVNXalHpYXl69o6cd5wTqPcR6CIobEfnvDt+9+4OnxC58fAnEZMRTmHAhBkMY5K8J45TJf2R52HG52HA57qroShO+6k3n++lrjhU/TbXe8+fZ7vv31J56OZy6nK/MwE0Lm6XTh/c8f2R/27PYb+q7BWUPWkqd7vl54+Phe0rWi7IKapiEZS1gEsmacwTmBpFkNbV2x3+/YbjY8nS7M04izDu9lFJDWfFRtLZv9Hut7iuv56csJ/XCGtRPVSmGV4tB1fP/qjt++e81tp4Tb7iuUc0AizCPWmpWbJIoSZ+zqYBVUsEyk1yxjBLsgHj1ppoxeuS7eUFmHdzLv98+YZG1wrkasDIam7bnTXtRROVNiIE4jKsvOySr5XNj1M7LKZ6jqmt7fklJhvM5Y49luWkqKPJ7OJFUIGbKxaJvJBJ7OA58eTnx5ECd7CGl16kpxd9bKBCAnnLe8vtnyux/e8Xe/+xVv7w6y23rJcGUN7X4ZhMl7kNKKOogi7phHkjLEeaKkCMjhMOdFlG1kosqMZCZlsXWNqypQCVOyPOPa0euZxEDIgRKLhM0rkeBap/G+YrI1w3lgGAcJElonaSYn1DLjcmDfevRhQ6szlYUc5Pn9a69fRoGnsCyjbNPXJSlFHklVMpS02q/Li7Tr5cRWmm2345u333N/f89lONO4jsZXOOswRTNdJ4x1bLcdh90GXzlCCIxVyzSL+9Q4jwrzKueT7omVK2eNfVlq5igbdnIkx+e9gHp5WKxWmMpTVw42LemwIyYBcYUQiTGQknQP1hjRNbedzNabDbrqUK6mGPfiTtR/0ZfKEjK9wJo0Cqc9VD2xn6jqnq7vqeqGcZIknaI0MRWuwwTa4zyECMMgUsdxnlZI14Fv39yy27R0jaNtHX3jqbxBl4QuwoF3jcc4i3i216xXq9BObjlarR1bWdn8MTEvM5ermLuu00xUhaqr2L/SLOEtx9N3LPFCWC4ko7CsyTYhrGlakTAkTqcnnh4eOD4+8OrulqZeKX1FhI2idHpWPMnvpKprbl+95g9///dM1yuXhycuTyfG65Xz5cqnz1+4udlxe7sTpHNToVUhkTieL/ynf/j/6GtP31T0TY3ThoL8/FVtVmepJ+Uo+4V0pMkJ7zy3hy3zUsu1PGeKLRLjiML6Glt1ZO2YkyzaitJfR5J5DYIPCypGLFBZg/eW4i3JrNy2ddmW13xX1nCOoiQmTa36c61Whou07i85rnlV5hRdqL3jbtsJamK7oes31F2PchXOV1S1R2vDME3MuRCWwLzMLNcrZVmoraGqG2l61u5dPuPChKrqmpt+T9v3PHx+YLyOZCJPl4U5SaaCqRq0q4lF3v/7xyc+f3ng8ekkozYtucDC4ZFRjVGFSjsO+57f//o7/vDb73lzd6ByUvxVXrXxaxvwHNzzVZIqL6012q5hQWrlw6tCUusGuRRyhiln5hxBK7IxWF+TtCEAdneg2W1pjOf0+SjxiNer7AycwRlHKRHjFO2rA92vfsf7n+85/vwBtYzsDhtu3r3DOcvpy0eOxzPMA5vGUmtJqZvnmfDXRTS/jAIvt8aFkiTs4BnTKaSrjHr+p1Jr0s/zKEcodbWveH37mt/++m/48vCZkCJTyeKws44JWdbpqqOqa5wTQ8JGOdzKg8klo7Voer33LEnCEEop6JV1rrXQDhUFsihD0jrTe9bN5rx2F8bSNA1N+5UEKKoG+YpBMlllt6DXkVSCHCCZl86qIDiE8nWAJTCkzBqyLG5cazxV3VPVLW3b4Jxbb0BiUIoxMc8BpSd8EnaGUhrrLL1r+eabO37763e8ud3SemGDO6vxXmFNkSu3Aeekg9TOoJyMqyRwS0AnSn+9ZVCUJE2FyHgV9MIwjlzHhdLC3Td3lKDoNg0xDQzTkXk6kfNEcgLOMkqRjGjUM4oYI8NwFfPJ8UjlhG4IUs2VMevI7RmpKre+pm14/eYt33zzLe9f/4nP798zjwNhWTidjnz5/InPH3fcHTY0zZpcrzXDOPJPf/5A13g2bcO2a+nrVphCWtNqR2U82jfEMBHmiSUuYAZsEoppXIIQBJXCGIurGmKZGabA9fMDw5J5//Ge4/lKzPJzqoL87rPgkZfVFIeRZxdr0U4Cw5UuX/dSq1jh2SxVSpZu0DzffoVeKSH0ShhD62GcV3VX3zW8vj2w3++omhZcTdEOVzc0TU2mMC4LwyCh4NfTmThONMbhNr34GFaTk3y4198NSkCBfU9V12hjGAeRnp6HmfkyMg0zcZ5Y0sh1mnk4HjmezlwGGas95xWzKme81dTeUhtLVzu+/+YNv/vhG969vhP2/18EjTzPC/VzUX8xYOmX901uM5qUiszir8OKkxaCZClqhahlQinorAhZEY2D3Z5619Pc3dLeHEjKsPz4M9OcmE5nSOuo10BeEcfdpuNXf/s31K9e83HXEh8/sznsqO9uhMH18wfO5wsmzWvUZ1njIiHl/2FUNIVlGaTzU1/jxp4XJ89vvFLPG2+Zt7IS5VQpbLqWX333Pb7y/PzlE9cYGIqwRIJ2wlpRjqgMVotcb996xjmsCVIX0c8bS10rQsqEkF7mXErpl7GKKpkYFmIMgvDVmUJeSXkLMRes9S+KGOcbtJYQjZghrKlCyzSwzAMxRBQjqhRUTmi3uumAQiLmRC5ik1ZGrbhQA8VAXkMW0IJ78BW+EuiauGUjdkkoHRH2/URKRXC43rLd9lSV5Yfv3vCr71+zbS06J1iNLVqLc9is123rLcYbuUNqLTcNLc5hcQ7KYVxyoaRCDIllDoyXq6iVYiLFjNGau29ucKpis+1JS+Dp4Z7hekQRqJwhZwGRqaZlKRBSoqorwhK4ni+cno7UzlO65x2JQSOxbqLeMS9PmNaatm04HA7c3t7SdR3n4xNzWAjTyPV85vj4wNPDFw67nrptMMawhMinxxP2JKiJtq7ZdT1NVcmtZ44syrAUvZJF5RmOGfIiI7lxmgkxoa2jaXucrwgJhtOTSDWfrnx+PHG6TCu8bh01IejaYQmcloVrzETtqFwNzmO8lfAUrVH66+03Zwm9iWGBLIvYqnYoZVmiZhgzIWVQAocrSNA9RbJdQRoUlGJeFqbrTCiadrMRnoyReflwOfH45TOXpyMlJtzuBq2M7HK0XieKfzkMEW+H8RUYw+ZwoN0J5tqfB4o7M5cj18cLT8cTT+cL5+HKtCykVHDeotcoQDErWhpv6SrDvqt5c7PjNz98y7vXd2yaRuBrWWbuRUmdYP1OeEYo/Fdfz3UmLIHxPPDp0z2X68gSVnrkiuLOWZOLYEmCMqRuQ/Or37D99a/p376h2u2IMRK7P/Ll4xfuf3oPMYmLlUROAYOmakTia/cHlIXje4Vra5K3XMaBcZqZphmvxDy1HvvPve9fff0iCnxMkfc//4RVX7XwX2fwsgABIcZVVSXMdi1b6JwlSjuvS4tt33G+NpwfRlJJOFOz6/Zo7TDaM0WFryu6rsdoS1hOpFWrTSkYa6mtYwoRPc7ifo2JGIOEBTsj/52j6PNzxBq1YonlIQmxrPpiMTwpbcF4lDJivy5gXIX3jjg5clzQJWNVRqUZVWQskUoWJksOxCSmJOcdzlco7SEbSrGUlW9ntHy4Uopic1/mNRvTkopiCVns3bngVi1Z09TUtUMDKQR0MVTWoqwRMqCV8ZW1FVp7cI5iZaxVlGSnPg++c5HdiQiXi1z/k1zjh2EkhYS3lrauydZiK7mutpueu9s3fP/db5mniX7bMi0XtFH02y13777F+JrT+czx6QmjNdMwcj2dGZoGpw2+8ijlV3qi4JrVGrhYlBITjlLUTc12u6HvO7w1BIoQKzVQxNiyzBMpzNi6hlJYIpJfGgOXqfB0CRJbqDVN7dltOw77DTf7Lfvdhu1GJJdWZ+IixX1aAtMwkjJstqKUGkPhw/2Jx6cjp/MVQdI/d73S1CQFp2ni56cTPz6euHt7oGt6tDVgtGR3KIOxEtmYQhA39jiQlhmtoKta7m52OG+Z54kvj48M00JaR5MFJSKHIHTQh6cTlELzWFPQzCESMmz3e+ZpxnnDMFzQJdNYQ1jjHds927mcAAAgAElEQVS6pm0a4c3r5yPq6/+ihNujjWUJaW2oZJTUZUVIRQBoy8I4aiqrcLsNGYi5vHTPJRdJ/vLyVVnNN2/u+O333/L29S1dU2ONPJtFlZfQ+WcZ9ktB/4si/6//PyTD+OMn/vmPP/Ll8UKaEsEoBBhgpaHJmWw1pe/pfv87Xv2H/8Dht3+DbXuMq1iGK9X5imobspHEq5Qyef3s1W2L7TdkY4hEirGYpmaMkXA+o9Y8CFWKQPfWQ+WZZvx8GP/3Xr+MAh8j//SnP2K1Wbv3REqBGMV1miReB2cddV3T1DXWWZSR8wzWq7mxov2NC7oUjDI0lafzEhrsrKVtHJt+R1O3svxcJuZlZFmmF0OORNAZGXMkKRbyZkpiTIoTKc1Q0lrEZUNfSkSXJFmhqgj2N6/OOqVAi37esPLRVRFNbPKQk0i1lIw3YoqkZSSEkRiFUmSdQ1sF0QjkKCIBz8qirFslD5JzOU0z87hQkgGkIOuUpdgp0UIbZ6TbDInLeeR8HNjVFd2mwnkj899VgmhtLX+PsWTzdfyRS4EY15GAIHUhY7W83yUXwhIZh4lxmImLBJS8LLeQTrtyPYftG755N9LuO0xdaLYth5sbXr/7Bmst95+/8I//8I8M5wvD9cqnn3/GGUNbydhNljNZdjdr8hdfb+MYY+n7nru7O27vbrn/+IFlvMqyu66onIOUmMaB8XrBVvV6e5SfNWXpjkOICH5XcZ2FQvh4unB/vHDY77i92XG337DrJPRDuxpfKVBBMltTou4cb1+/5nge+XJ/5PHpDElUW8/jx4TA+K/zwk/3T/y///wTrq9ZrOPuZketFTqJFFTkeJoS1mcnhTU3ACpT2Laebd8RU4vX8Hi5cpkDYR19WWtQSmiM52FiXgLOrulMuWC8R63URkUmxwWTEzebFl8S0zCzXdHCznkK6UX6XNbnWq23vlzkuSmsYSpKU9eKfelXbn0R3f9GFGTWe5TSLEFiMCnCiCdHUpgpKbLvO25vD/T9BrsqZhQF9BrU8jzkLPJM/Ndd+7+KF0RxOV/4/PmBT18eOF9ncixMOmGiwq2+HIyj3XRs3rymevOG6vUb6tvXaNuglSElpCkqal1oC7JAGai7js27b9i8e4duOyyGbrfHlEi4nChxwa7iBIVMM0KcCaGQYnop9H/t9VcLvFLqe+D/AN7It8j/Xkr5j0qpG+D/BH4N/BH4X0spj0pakP8I/C/AAPxvpZT/57/3d8SU+JcPP6+qkUIqgRBHpvnCPE2EKZKTkvFJU9PUwsVWQgpDafWSuqPWBV8piqat6ZtKuNgYvHMctlt22z0Fw+PjE+MyEfLCEmdCDKRYKGvyjHMS15dzXudyC2ERVGyMUa5sWlyDYV5IcSbnJEufYiAtkBeZqyuP0halVhOXEhVA8RVkhyavYQ7issspkeNCmC6EMOOcxzqHUQqSpAlNS5BFi7ZYX2GcJ5fENM1M48I0RpZlEs6Pd3it1tlrRGHkz0KTohT449PAzaZn0xtc1WK8uBrRCmM8pWiy0qQsZ4miQEori2eRsdMiJqnKV+i6Is4L8zgzDYFpDCyTSAvJhRwzIQXCkMkzNG7Dzf4d7c2G/vX/3965xkqWXXf9t/be51GPe2/37e7p6fE8HWwk84UYK1giiZCCHNuCmIcUGSHihEgRUpCIAIHBEsrHBBQ+IBCREQiMAg4IIvwBREzEQ3xwgm0cP0jGHg92Jj397ntv3ao6r/3gw9pVt7o9PTM96unbPaollaruvqeq1tm1zzprr8f/P2H/yX3OXdhnd3eX5AM74x3mBzMuNx3HB4csjmZYEfb39xlNRjksw8muQnIpXL5snbPs7u5x8dJTPP3MM9y4+irN8pjRqFKo2tEISLTLJfOjGbaoiT7kmK2sK5pC7oaOIviYmLcdi7bjcN5w/daM3Ru3efL8GS6e22V/d0JVlpTVhKKM+L5XWA5XcO7MWXqvGPS3bx8SB8VeghXrag5PDonrB8f4qPXVTYj8ofc8x6W9CYUzxJxtS1ar0IyBojCM6wJSoDARlzzjwlCMK4pVaOZozuGyJZG0S9yWpBBoB0/T9WT2RYwxTE1G+pwvGPoGR2BnVLI7HTMywsI17E4mjEc1RVngcwmmElhrs+IKD16NuzZUrfDcS1fgJkr8XVrDtC5o2o6y0iKEshzRD1GLOq0lxIHjowNu37pJs1xQlY6qrNbkMawSqqIMZORSWi3MiGsjD7yGgYfj2TGHBzOaRcvgE0NQ4DGtgDNUhaGuC85euMClp5+lnEzwUdE8q1VzZjKavM60e8SETwq0V+2dZf/ZZzn37POMzu4TKiU4ipMxi9vXCMt5Lq6QdXVPXCpHhM+MU+EBefAe+OsppS+LyA7wJRH5PPCTwG+klH5BRD4JfBL4W8BHgPfkxx8F/kl+vqekfGfToFIixoG2V8TC5WLO0HmImjk38+M1cJdYjf2mpAZZDb163tPpLrYomUrCFibjqijrTlmWtN1AiEHjhMZkqjnVR2FtNSEmpS6KonD0bUPjlLBAOCGv6PuB0LfE0K9bnm2KmOgxyeNEKaJTsmSMPJIoryyi2DSKIZ1jhEkvhlVSzBhNhlrrlLszamXK0Pe0bU9IgikryvFIG4LagbbxtE0gkSjLnuk0MhqVjMcVdV1SVY6yKjFW2ZOcLVg2HTdvHVHVJUXtmJQj3WmkRBhaQiSDVKGUiSJKX7hsGLouE2IMWGdxISq2+GLJ4mhG8kHB3woohkgwOq99M2d50NIdDhTeslueoSpGTCdT9nbOsLO7p41noaUUSxkhHC+5fe0affAURcnFS+/izJl9rCkxKCbKKulKTrApnguabH3yIu9+73u5ev0Ks/kRtTVMpzvsTKY4sfRNx/HhMTE5hn5YrVI2MWJWMdzVnyFGQlSo3cViwfHhATevjzl/do8nLpzniQvnObO7g9sx1LUmGqfTHZ56ynE0b3n1yg367pomoFml02Pu6BbaPnHzaEl86ZWMMzRg3/M8T5/fwxVWGZpyjqYeVeztTfG1Iw4dziSGdkG/cJTTKdNRRR+mtCFqNYhffU9ae4Yms6lJSgoEF0Jm9OqQpACAlkRpoRjVVLZgd2fKqB4hVtmuRH8NjHEae3fFCRTFeidp1s1VzlicseBzSWroKIvEtBLG41IRZwt1ZLqh44ZNdN0CoqcsFMLBGu2YXUGSgEJ4r8KIIrrDU07KE8NOXiXk6ppm0dAvW0zQRigv4NHkaoqeHstob8rFS5d47rnnqF1Je3jEvLiJu1hga3UWpLAM1tCK3igToo2Sowm7T1zi/NPPMr5wkXS0UDju3SmFSTQHEPsWQQhRaQ99TFqkEbQFNCPTv668oYFPKV0BruTXxyLyO8C7gI8Bfzwf9i+B/44a+I8Bn0k6a18QkTMicil/zut9T55ojRHHCCFom3lQVFL1arxXHkijMbRELo/LjQsIlKUCKa3wT6qqpCpq6pEmIcUaogSCBHwI2hUYwNoCBPpWcd6zm01VlIxGY5aLpaLckfI2lDWpsBhLYcrcbm9zlYWi5kgKmKRATpp40m1XQrt0Twq1csVMUsOMCNYViFHGK+McIfM/+n4g9J22/odE8j0DXrsro8ZlnXW07UCz7OiaFvbG1JVjZ1pROpMTZhaRghVr0fHxMbNZzWRaU9UFxmrWPoZA8CnDyyYQqwQHIZAGj43aFWicQUTrdNumpW1bou8zkQEMaBhFS9UMMbT4viEGT1mUmHJCUYwYlyMqN1ZCDVGE0dKVjIua2ikm/mCgG3pmi3lGxsysRilmujdRBobVBZzhF8bTMecvXeTSM8+wWM6ZOMtTFy9yZm+H6Duljlw2+GjWBv5uIDDhzoqVuJHx0o7NnrZVjPFlOyDWMRpPqCcTqnFNPR5Tj0ecqyc8vex55pmrzI4bmu5WJo3Qb1EvWLsdYx+5ebhAvntFywKtwdnnuHjuzHr9OCdMJjWVM0Q/wveNhjFItH2HbQtspV72zu4Ofg1frXfvoR8IXjBExnWBs+r8lPWYaqRUjaVJVDYxclo5aAXKUcnupKCaOAKKtoi3GoZxBcYWGOfW88UduyJt29fdV8IaoSoMoXQUhWAlQlRgtihK+0j0eq6jERYYj2qcydg3aFXXnf6tJn0VNXOV+GUdkl2jeOabju8GGAKlKA9zMNAnvWZ9iJgIRVVx7sIFnnr6KYpamB3e5ODgkPnRTerJlOA9s8PbLLuWZYzYqMneEBNtAqnGVNM9qskO46S806ERXFVjXEHom9wXFHX3lsHihuyMBvOA6+BF5Hng+4HfBC5uGO2raAgH1Pi/svG2389jr2Pg0zr2rBdnVMcrOYyU2OzhamIhEGJQRiWJ2bjrZ4RcvRHzwzlNvo3qMWVR4YpSa6pjoPMt3dCy7FqabiAlk5sycslm1NhjAsqqZDwes5wvqCzUdaXYGrlpwdgaKxXOKA7HCvfa2EITOeq2aDnbqtQxrV7n4Fz2KjSpqvC5WsWmJMlFqWS7SjihMfa+6/D9gPcRLzAkz9C1jOuaM3s7zA6X+OEI33csFwuaRY3s7yqwmZOcyNPSM5IwDIn5fODgsKQelRSloa6tBgxiUORDr7XtQ9RYYAiBsiwpXAHO4pJiyXRdx3J5rFj3zmIwxACdjTgHySgao3MlbuRIuwkTCyIlFkWydDhF5XN6bDUes7e/z9nz5zhazkhdQzKJpmsYwgC54znFSBgys1CO2UJmGDIWsTCajnny6acwkjgzGrFf15QSWS6OGYaOkCJNo3mZu+O0m1t7OKn2OonrqqfWdIFhWNANkbKqGU+mVOMRI2Og0K7oUVFz7vw5nnzySb77yhVu3j5U9E05CQllX14BdT3cPGxIL7+qcM5lQVHWWmMtyupVmhqpKyRFhmFM3zX4rgenVWQJg6tLdsdjyjOJtmvp2x4/eIa2o29b0tCyN1bnKJkCW+1Qj6eMR2NGTigIFKGH0GGI1KVQV4myDHgRhiQkqwUAYgrEFhhnT+aQHAcHhde1Qop6Y0xxoHQWNx1jrCjkRujwfUNq5sgKiiR6ppMRk7piZzrOsfHsMG1U461+K5FV7CbvmGENUrb+LfOOInqPTYnKaD38YCI2ZW8+A8MZY/XarAqidNy88SoH125x8MqU8c4OGMfx4TGzg0Pa3kPmZXACVdPR+EBAeWbridXqsKPbdP2QG64Ut9hnGHEtl/UnIZq7bmGvJW/awIvIFPj3wM+llGabDQIppSQib/xtd37ezwA/A7C7M8YVBcEPdJ2nbRXIqe8DMQhgc4oErfs2aANNDGsPWm1owhihLErqutYuPJfhgF2FNQUxCfP5jGu3rnDt5jWOjo/xPlKI0yx91I5XEz3qN6knbUQY2o40HVE4jfeVVa1bRudyLFEB0IxkNDtjwBREWf1/nUUFySiBucZ9tW1cddxpdRCIddhCt7gkxRBpGyUMb5ZL9bhiIhqDHQZC1/GuSxcJOWQSQk/TKujZ7OiY5f4ufm9KaYqczO3pY6+Jr5AgRJZNS9t2hOA5f26XUe20Rdt7Qq94IjEEhV0OgeBLhrLKkMRac9/3PX4VyrDa8DKyalgme4mZ7GClYJzhn/1eT+gN7SLh516NWzy5EMVZqumEJ557mqP2mMtHNzj4zjWGmfBMaBHxwED0iZA8njx/xmpuR0xuBNJHWRVceuopzu7sMXUO2oZ+cUxNpIgVPni6zm+s1/Wr7zHwd5fakQ18wjLExPFy4MqN29STMfVkRDWqqMc1y66lSgrvvDJ2a8C9VYmhKIG1mi2FH269cHDsefHlaxhxhEH7AwpncSLrxL+1BpgSYyAMAWscriyxRaGVGwI9inkehoE4eJplQ3N8hF8adqqC8XiMq6eEYgcpRpRFQSkJF/0aysCZgCuFwTfELqhhNxYjauBTrmRKSC4xPOlpWTlnRM09+aGD6BnVJaNqR1m+YmTwyvm74gAWgboqGdW1JmmL7AjmuHW28uvfz+SQoAbjT67D9W+2UgW91mMIpExoYiyYKDj1b7AJTIR+ueTa5ct88xsjxA4cXL9Kc+s2Mq5oq5qAYXbc0ty4SewGum5giBETEsXxkqOZFgtMhoHCVZgEi6MjDm/exB/PKEWbs1IIJK+7aB9yPgC0+eoN5E0ZeBEpUOP+Kyml/5CHr61CLyJyCbiexy8Dz2y8/ek8doeklD4NfBrg0pP7SQkvAsPQ0XUtXd9pBUtuDllVXKglV7osE2TdYrwy8uPRhPP7F5iMJphkSD5ixOFsBRjavuX20W2u3rjOrYMDmm7QSKEzxMz3KLkEM2Tex85AX/Zq4DIRdgzaDWisVZo2p1UGubYkh14ykrc4kihxtWQgpxMP/mQtrvIIYjRJY6Myy7iiABH6vmOxXDI7Pub46IhmuWDotUt21RlprOG9Tz/JqBScg7IUrt+4zXLR0jYtRwfH7Ewm2L0pxgi9Dyz6QRNJfWToBmbHC7pBoZfjMHBmb6zlocNAGDwp5B0XGjqJaENTEgPOYG2hGD2j0fri2lyLMQlxPtJdlnWU4xo3coRWCHhMkwg+0rcDQ6dkKcYJpirYPb/PhcVTXDy4wUE/Z7ozZf/8WcrKqQfoe0IYCAI4q+EBp632KYfxEmBdwf7+OdJ0DzsMtEcKyiVOwyJKZNFirc1e4KYRZ+1QnPTMyOqXzNFCNSQRIYXI4WzB5SvXqUYVRVVST8ZUY2AIzBdLDo8U3zxmPoF1Y5vkG78okbUg+Gho+sT12w2Gq8QAsSgoXKFhCiGTjWj55Ko+3ppCryWj/KEDiTJFUl1B8CQfWBaOxgS8DUwry2hUYcqK3hZE65TTWKBIgk2CTRYnEWsjPkWGriPaQLBl7qSNa57SxEk4Swlhsr+dNCSbYsAQSUYTmWVZ4azDx4i1Ps9zq+s9KahdUSgmVOkydo7kRsS0suMbRl7099Auazbu2huAILl8MknEOHClxQKFV25V5QJwWKN9DjcuX+bbqaMsE/Qtzvek0NI3c/oIy+OO0DXa0SuGmLR15njRcuXyZb770ov4FKlHE+aHByyuX6Y9vAXdUlFtszObgtdGyaj8yxGI9gEY+FwV88+A30kp/YONf30O+ATwC/n5P26M/xUR+SyaXD164/g7dF3PMGgVhlK3DevqmNUWm5RrP2P2kiSQki4iclxvOtnliQsXGdcV0QfaZcMw7ilMSQTmyxm3Dm5y6/YB83nDEBPOaGXLCuCqqmsgEHzHMHhS8NRuBDHgu5Z2kat1JLswxq0JFMRo4lVVXkUWM5celpwVXhV4aEZ/ZXXUSdPbQozKDoNWB/m+p2l6ZrMFR4czZmsD3ytZRlD8i+nOlCcuXuLMXsloUjCalEy/c4WrV25wdLRgMW84uD1Tw1oVDDHSdgNt7+lbT9/2LMUQQ6KwTss9SexMxzn8ZTLJsaNwVukTRRDnMEWppATWrHHAyTdek3c4oBfyrSFTGqJQB8mJkmuLxSRL3/W0tqWcJEaVNoohUIwqzl28wPcN76XeGzMajXj2mae1wzIGfNcR/IAUSvVYuApbVoisAKm0acxYRRU1LtDNZpqCNZqPcM5gQyJ6m28MrHdgq1Diqk79xEs84SHQEMTK64aIsGx7rt88ACO4oqKsJ4ipibHhlVevcvnVq5lpKWOortb8KvG3EWpISRNsTR+5cvMY7z3v/gPPUmQvNvc+abK7KNYJemsUsC4kJYFOMWo3dCIn/g1SlTg/IsnApDCUZUHIEMPBJmzGMy+Mw0mBQfNLJE8IPT5EfAikIoLRvEhKyhq2ctm12infLFPORiVlYCqcJcVsjKPy+aqDppALISaqIjD4AUvErq79qPSSCr6WiyJX9m+1K+Kkwkqj9Sk7VidzvMLER8CVjnJU4cUTjSJBQiRaxeYvTGR5eJtrw4KdyrEzKhiPCgiCjyHX9Of+laKkiDn3B/jec+WVVxiNC5rZAZPJmNC2LG5dI85n2BSIaOlziFo7v6IF7XwgJUEehIEH/hjwF4GvichX8tjfQQ37vxWRnwa+C/x4/t9/QkskX0LLJH/qjb4gpZTjnR3DqhZ9xeWIljyuriWTTrZfqw487WxT76AoC+22rGBIS47mHucG/DBFxHB0PGd2cIPmeEYKAYeSJSyXPT4oH2olorAEzkCy6sV3ndKVNUuWKSdDo3bSapu+dhNiLGIcadVtuk4j6baOk7/yasseGpFI0HMyYAq9UEEyQ1NP2w4slz3zRct83rBcLhm6nm7o6b02R0WBqnLsTM7k+GzJZDxhZ2fClSvXWS5a5osFVV0xZoJYS4yiXnkKVFbhdRezBYd1xcULZzBlxejMGcrRWJNlotXahqCt4CmtjbzushReQnI+hZSwNs8PCZ90+zv4DlB4iBQsy6PA8iDhG/BDIIqnWXiqiTZeIeq5jHemPP/CC1x615MYEepSk66haRk63cqWxlFIibUVxlRarZFDHjY5UrTYqL/jMHiapuN4vqD1jSbjcfTtKhe04ZkLJ+sxmbUNEU6Miq5V/a6VbxgjNI3n6pUDhuFlbt5asH/2LMPguXXrFq/8/mXmy4aY1lsCXSdRdD7Rxi2NGGqoIQKdF24vBp5HKAvF0NECaYXxUIA+TaZrDbqWeaYcxkkxKQhWzl9ZcdRlhZEJY6e/m0fJpK012FI7egurnLsx9AQPySdisoRkiCLEvNtYdRUrFk6iKksmk/H62iaRbzQRYiAlm69pyZ3fMYc6FWSwKEpGo8TIu3XfSNcPpKEjGCHZjDYb0/pmu7IxacPQbyZ51zfuPC6iDlmR0V5TEowJOJcoSz2usMK4tNSFoTBJoYCDp+siYdAmxT4Elm3Ae73jWmcz6iYUpSU0Cw5f/T1sd8S41twTvsemRLIGL5FkLCGHjmNckbho86d5E4Xwb6aK5n+xGcy6U37kNY5PwM++4TdviCCUbgdragobiGXcKF1afS55ca86XU+SqSnH8VJMTCYTQigZvMLIBg8HqaNZqhfZdD3Jl4zLM4xKWceMPYHKaTNMIQ5b1lR2Ckm9hrIoaaMQeu02LYZEsRwoZw1FPaMoR7iyXmOSq4HPiyhpid6qVVr9u/WZ5e34ZtnWxvknSDHRtS2LxZz5bMaiaRWcCkuwbu2RJGDwiW/+3gFlYQkhsPAVdnqes09VmPE+8/mSYQhIUTDYQg1fWePMgKsjToS+95CE3tbc7gzlbc8sNhS1bk81MRYzmXHGGRdRNNCV/uFOA2+sY8VVG4Pn6lGk++oXFUpWhJQsi8Oe7jiSekNMinxZXhN2dieUdQmyblchY03p7AXND/hGCV1IkaIqKccTTDXKsA8Z3yVFUgj4YUBCIPY97XLOcn7McjmnG1rNq0hJ6ANJEs89/cRmEH69IE/Ca3fko07W9cb4RgifojA0ixk3hm7N9LW3U7MzLteOzJ0L/+4L5sT5EUE7qV1JBikEc1I2jHFErN6Mcm16ymWkQsKkpNDD2Ri7woF1SFmT8ASBKE7zUjaT0WQsmEgippJoLTF5DUNmL3gFnSDmhLqPJNw+Xqxj3av1Td7xqEVe4S6BSLOxc9JHTEoeIzGs39N75T42pqEoFyCrZC53Xkfrb9703E+eNawjLLuBoapIe7uKmdN76hAJKRHyDdiaDJHstKQ5WKExQp9vHDFFvE14m4hVothRp6dKGtaz1mBrxzIJab6kyDy/JrPZGZsQHzREg9qphXH4smLV3EnxxoQfcrchPQ0RkWPgxdPW4y3IeeDmaSvxFuVx1X2r98OVx1VveHx1vx+9n0spXbjXPx8JqALgxZTSB05bifsVEfni46g3PL66b/V+uPK46g2Pr+4PUu83boXayla2spWtPJayNfBb2cpWtvIOlUfFwH/6tBV4i/K46g2Pr+5bvR+uPK56w+Or+wPT+5FIsm5lK1vZylYevDwqHvxWtrKVrWzlAcupG3gR+bCIvCgiL2XY4UdGROQZEflvIvJ/ReQbIvJX8/jPi8hlEflKfnx04z1/O5/LiyLyo6eo+3dE5GtZvy/msX0R+byIfCs/n83jIiL/MOv9VRF5/ynp/Ac35vQrIjITkZ97VOdbRP65iFwXka9vjN33HIvIJ/Lx3xKRT5yS3n9fRH436/ZrInImjz8vIs3G3P/yxnv+SF5jL+Vzu1e/zNup932vjYdtc+6h969u6PwdyU2kD3y+7wC7f8gPwALfBt4NlMBvA+87TZ3u0u8S8P78egf4JvA+4OeBv/Eax78vn0MFvJDPzZ6S7t8Bzt819veAT+bXnwR+Mb/+KPCf0XaPDwK/+QjMvUVRSp97VOcb+GHg/cDX3+ocA/vAy/n5bH599hT0/hDg8utf3ND7+c3j7vqc38rnIvncPnIKet/X2jgNm/Naet/1/18C/u7bMd+n7cH/APBSSunllFIPfBbFk38kJKV0JWU2qpTSMbDCwr+XfAz4bEqpSyn9PxSu4Qfefk3ftHwMxe4nP//pjfHPJJUvAGdEAeROU34E+HZK6buvc8ypzndK6X8Ct19Dp/uZ4x8FPp9Sup1SOgA+D3z4YeudUvr1lNIKPvMLKEjgPSXrvptS+kJS6/MZTs71bZF7zPe95F5r46HbnNfTO3vhPw78m9f7jLc636dt4O+FHf/IiYg8zwkWPiig2lfz9utsHnuUzicBvy4iXxKFZob7x/A/Tfk4dy76R32+V3K/c/wonsNfQj3ElbwgIv9HRP6HiPxQHnsXqutKTlPv+1kbj9p8/xBwLaX0rY2xBzbfp23gHwuRu7DwURrC7wP+MEpk8kunqN695AdTSu9HKRR/VkR+ePOf2Qt4JEuoRKQEfgz4d3nocZjv75FHeY7vJSLyKZSm81fy0BXg2ZTS9wN/DfjXIrJ7Wvq9hjyWa2ND/jx3OjIPdL5P28C/Kez40xR5DSz8lNK1lFJISvz4TzkJCzwy55NSupyfrwO/hup4bRV6kbeA4f8Q5SPAl6yi9oAAAAHxSURBVFNK1+DxmO8Nud85fmTOQUR+EviTwF/INydyiONWfv0lNH793qzjZhjnVPR+C2vjUZpvB/xZ4FdXYw96vk/bwP9v4D0i8kL22j6O4sk/EpLjY9+DhX9XfPrPAKvs+OeAj4tIJSIvoMTjv/Ww9N3QbyJKkI6ITNAE2tc5wfCH78Xw/4lc6fFB3gSG/9ssd3g1j/p83yX3O8f/BfiQiJzN4YUP5bGHKiLyYeBvAj+WUlpujF8QEZtfvxud45ez7jMR+WC+Tn6Ck3N9mHrf79p4lGzOnwB+N6W0Dr088Pl+O7PHbzLD/FG0OuXbwKdOW5+7dPtBdIv9VeAr+fFR4F8BX8vjnwMubbznU/lcXuRtrip4Hb3fjVYH/DbwjdW8AueA3wC+BfxXYD+PC/CPs95fAz5winM+AW4Bextjj+R8ozehK8CAxkR/+q3MMRrzfik/fuqU9H4JjU2v1vkv52P/XF5DXwG+DPypjc/5AGpQvw38I3Lj5EPW+77XxsO2Oa+ldx7/F8BfvuvYBzrf207WrWxlK1t5h8pph2i2spWtbGUrb5NsDfxWtrKVrbxDZWvgt7KVrWzlHSpbA7+VrWxlK+9Q2Rr4rWxlK1t5h8rWwG9lK1vZyjtUtgZ+K1vZylbeobI18FvZyla28g6V/w/HZ1d6Sz7QuQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get a batch of training data\n",
    "inputs, classes = next(iter(dataloaders['val']))\n",
    "\n",
    "# Make a grid from batch\n",
    "out = torchvision.utils.make_grid(inputs)\n",
    "imshow(out, title=[class_names[x] for x in classes])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SYS8GcLY2Br9"
   },
   "source": [
    "In order to get reproducible results, we set a manual seed for the random number generator and re-initialize the dataloaders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "executionInfo": {
     "elapsed": 692,
     "status": "ok",
     "timestamp": 1613631483212,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "QRXadcB82Br9"
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(rng_seed)\n",
    "dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], \n",
    "                  batch_size=batch_size, shuffle=True, num_workers=0) for x in ['train', 'val']}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HKdzGz2E2Br9"
   },
   "source": [
    "## Hybrid transfer learning model (classical-to-quantum).\n",
    "\n",
    "We first define some quantum layers that will compose the quantum circuit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "executionInfo": {
     "elapsed": 392,
     "status": "ok",
     "timestamp": 1613631483559,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "cMVoNy782Br-"
   },
   "outputs": [],
   "source": [
    "def H_layer(nqubits):\n",
    "    \"\"\"Layer of single-qubit Hadamard gates. \n",
    "    \"\"\"\n",
    "    for idx in range(nqubits):\n",
    "        qml.Hadamard(wires=idx)\n",
    "        \n",
    "def RY_layer(w):\n",
    "    \"\"\"Layer of parametrized qubit rotations around the y axis. \n",
    "    \"\"\"\n",
    "    for idx, element in enumerate(w):\n",
    "        qml.RY(element, wires=idx)\n",
    "\n",
    "def entangling_layer(nqubits):\n",
    "    \"\"\"Layer of CNOTs followed by another shifted layer of CNOT.\n",
    "    \"\"\"\n",
    "    # In other words it should apply something like :\n",
    "    # CNOT  CNOT  CNOT  CNOT...  CNOT\n",
    "    #   CNOT  CNOT  CNOT...  CNOT  \n",
    "    for i in range(0, nqubits - 1, 2): # Loop over even indices: i=0,2,...N-2  \n",
    "        qml.CNOT(wires=[i, i + 1])\n",
    "    for i in range(1, nqubits - 1,2): # Loop over odd indices:  i=1,3,...N-3\n",
    "        qml.CNOT(wires=[i, i + 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "02BkJPVk2Br-"
   },
   "source": [
    "Let us define the quantum circuit by using the PennyLane `qnode` decorator . The structure is that of a typical variational quantum circuit:\n",
    "1. All qubits are first initialized in a balanced superposition of *up* and *down* states, then they are rotated according to the input parameters (local embedding);\n",
    "2. Successively a sequence of trainable rotation layers and constant entangling layers is applied. This block is responsible for the main computation necessary to solve the classification problem.\n",
    "3. Eventually, for each qubit, the local expectation value of the Z operator is measured. This produces a classical output vector, suitable for additional post-processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "executionInfo": {
     "elapsed": 577,
     "status": "ok",
     "timestamp": 1613631484963,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "GsmDV--k2Br-"
   },
   "outputs": [],
   "source": [
    "@qml.qnode(dev, interface='torch')\n",
    "def q_net(q_in, q_weights_flat):\n",
    "        \n",
    "        # Reshape weights\n",
    "        q_weights = q_weights_flat.reshape(max_layers, n_qubits)\n",
    "        \n",
    "        # Start from state |+> , unbiased w.r.t. |0> and |1>\n",
    "        H_layer(n_qubits)\n",
    "        \n",
    "        # Embed features in the quantum node\n",
    "        RY_layer(q_in)\n",
    "       \n",
    "        # Sequence of trainable variational layers\n",
    "        for k in range(q_depth):\n",
    "            entangling_layer(n_qubits)\n",
    "            RY_layer(q_weights[k+1])\n",
    "\n",
    "        # Expectation values in the Z basis\n",
    "        return [qml.expval(qml.PauliZ(j)) for j in range(n_qubits)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SfqHs3IQ2Br-"
   },
   "source": [
    "We can now define a custom `torch.nn.Module` representing a *dressed* quantum circuit.<br>\n",
    "This is a concatenation of:\n",
    "1. A classical pre-processing layer (`nn.Linear`)\n",
    "2. A classical activation function (`F.tanh`)\n",
    "3. A constant `np.pi/2.0` scaling factor.\n",
    "2. The previously defined quantum circuit (`q_net`)\n",
    "2. A classical post-processing layer (`nn.Linear`)\n",
    "\n",
    "The input of the module is a batch of vectors with 512 real parameters (features) and the output is a batch of vectors with *n* real outputs (one for each class of images)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "executionInfo": {
     "elapsed": 569,
     "status": "ok",
     "timestamp": 1613631486835,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "WnPJ5eCX2Br-"
   },
   "outputs": [],
   "source": [
    "class Quantumnet(nn.Module):\n",
    "        def __init__(self):\n",
    "            super().__init__()\n",
    "            self.pre_net = nn.Linear(512, n_qubits)\n",
    "            self.q_params = nn.Parameter(q_delta * torch.randn(max_layers * n_qubits))\n",
    "            self.post_net = nn.Linear(n_qubits, len(filtered_classes))\n",
    "\n",
    "        def forward(self, input_features):\n",
    "            pre_out = self.pre_net(input_features) \n",
    "            q_in = torch.tanh(pre_out) * np.pi / 2.0   \n",
    "            \n",
    "            # Apply the quantum circuit to each element of the batch, and append to q_out\n",
    "            q_out = torch.Tensor(0, n_qubits)\n",
    "            q_out = q_out.to(device)\n",
    "            for elem in q_in:\n",
    "                q_out_elem = q_net(elem,self.q_params).float().unsqueeze(0)\n",
    "                q_out = torch.cat((q_out, q_out_elem))\n",
    "            return self.post_net(q_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YYl3lmWY2Br_"
   },
   "source": [
    "We are finally ready to build our full hybrid classical-quantum network. We follow the *transfer learning* approach:\n",
    "1. First load the classical pre-trained network *ResNet18* from the `torchvision.models` zoo.<br> **Important:** the model is downloaded from Internet and it may take a long time (only the first time). \n",
    "2. Freeze all the weights since they should not be trained.\n",
    "3. Replace the last fully connected layer with our trainable dressed quantum circuit (`Quantumnet`). Alternatively, if `quantum==False`, an entirely classical analogue is used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 3573,
     "status": "ok",
     "timestamp": 1613631493382,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "a4VBxkDz2Br_",
    "outputId": "36056507-d14d-4ba2-cbb4-564445163c98"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading: \"https://download.pytorch.org/models/resnet18-5c106cde.pth\" to /root/.cache/torch/checkpoints/resnet18-5c106cde.pth\n",
      "100%|██████████| 44.7M/44.7M [00:00<00:00, 315MB/s]\n"
     ]
    }
   ],
   "source": [
    "model_hybrid = torchvision.models.resnet18(pretrained=True)\n",
    "\n",
    "for param in model_hybrid.parameters():\n",
    "    param.requires_grad = False\n",
    "\n",
    "if quantum:\n",
    "    model_hybrid.fc = Quantumnet()\n",
    "\n",
    "elif classical_model == '512_n':\n",
    "    model_hybrid.fc = nn.Linear(512,len(filtered_classes))\n",
    "\n",
    "elif classical_model == '512_nq_n':\n",
    "    model_hybrid.fc = nn.Sequential(nn.Linear(512, n_qubits),torch.nn.ReLU(),nn.Linear(n_qubits, len(filtered_classes))) \n",
    "\n",
    "elif classical_model == '551_512_n':\n",
    "    model_hybrid.fc = nn.Sequential(nn.Linear(512, 512), torch.nn.ReLU(), nn.Linear(512, len(filtered_classes)))\n",
    "\n",
    "# Use CUDA or CPU according to the \"device\" object.\n",
    "model_hybrid = model_hybrid.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MBviMfg42Br_"
   },
   "source": [
    "## Training and results\n",
    "Before training the network we need to specify the *loss* function. We use the *relative entropy* as objective function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "executionInfo": {
     "elapsed": 681,
     "status": "ok",
     "timestamp": 1613631497173,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "c5BztmJ72Br_"
   },
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3EoLE2L72Br_"
   },
   "source": [
    "We also initialize the *optimizer* which is called at each training step in order to update the weights of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "executionInfo": {
     "elapsed": 364,
     "status": "ok",
     "timestamp": 1613631497534,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "8pTLIqYh2Br_"
   },
   "outputs": [],
   "source": [
    "optimizer_hybrid = optim.Adam(model_hybrid.fc.parameters(), lr=step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mqvrtR9G2Br_"
   },
   "source": [
    "We schedule to reduce the learning rate by a factor of `gamma_lr_scheduler` every epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "executionInfo": {
     "elapsed": 512,
     "status": "ok",
     "timestamp": 1613631498224,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "N7E-uY852BsA"
   },
   "outputs": [],
   "source": [
    "exp_lr_scheduler = lr_scheduler.StepLR(optimizer_hybrid, step_size=3, gamma=gamma_lr_scheduler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sso2D_mC2BsA"
   },
   "source": [
    "What follows is a training function that will be called later. This function should return a trained model that can be used to make predictions (classifications). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "executionInfo": {
     "elapsed": 687,
     "status": "ok",
     "timestamp": 1613631499215,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "3a32gGl52BsA"
   },
   "outputs": [],
   "source": [
    "def train_model(model, criterion, optimizer, scheduler, num_epochs):\n",
    "        since = time.time()\n",
    "        best_model_wts = copy.deepcopy(model.state_dict())\n",
    "        best_acc = 0.0\n",
    "        best_loss = 10000.0   # Large arbitrary number\n",
    "        best_acc_train = 0.0\n",
    "        best_loss_train = 10000.0  # Large arbitrary number\n",
    "        print('Training started:')\n",
    "        for epoch in range(num_epochs):\n",
    "            \n",
    "            # Each epoch has a training and validation phase\n",
    "            for phase in ['train', 'val']:\n",
    "                if phase == 'train':\n",
    "                    # Set model to training mode\n",
    "                    scheduler.step()\n",
    "                    model.train() \n",
    "                else:\n",
    "                    # Set model to evaluate mode\n",
    "                    model.eval()   \n",
    "                \n",
    "                # Iteration loop\n",
    "                running_loss = 0.0\n",
    "                running_corrects = 0\n",
    "                n_batches = dataset_sizes[phase] // batch_size\n",
    "                it = 0\n",
    "                for inputs, cifar_labels in dataloaders[phase]:\n",
    "                    since_batch = time.time()\n",
    "                    batch_size_ = len(inputs)\n",
    "                    inputs = inputs.to(device)\n",
    "                    labels = torch.tensor(labels_to_filtered(cifar_labels))\n",
    "                    labels = labels.to(device)\n",
    "                    optimizer.zero_grad()\n",
    "                    \n",
    "                    # Track/compute gradient and make an optimization step only when training\n",
    "                    with torch.set_grad_enabled(phase == 'train'):\n",
    "                        outputs = model(inputs)\n",
    "                        _, preds = torch.max(outputs, 1)\n",
    "                        loss = criterion(outputs, labels)\n",
    "                        if phase == 'train':\n",
    "                            loss.backward()\n",
    "                            optimizer.step()\n",
    "                            \n",
    "                    # Print iteration results\n",
    "                    running_loss += loss.item() * batch_size_\n",
    "                    batch_corrects = torch.sum(preds == labels.data).item()\n",
    "                    running_corrects += batch_corrects\n",
    "                    print('Phase: {} Epoch: {}/{} Iter: {}/{} Batch time: {:.4f}'.format(phase, epoch + 1, num_epochs, it + 1, n_batches + 1, time.time() - since_batch), end='\\r', flush=True)\n",
    "                    it += 1\n",
    "                \n",
    "                # Print epoch results\n",
    "                epoch_loss = running_loss / dataset_sizes[phase]\n",
    "                epoch_acc = running_corrects / dataset_sizes[phase]\n",
    "                print('Phase: {} Epoch: {}/{} Loss: {:.4f} Acc: {:.4f}             '.format('train' if phase == 'train' else 'val  ', epoch + 1, num_epochs, epoch_loss, epoch_acc))\n",
    "                \n",
    "                # Check if this is the best model wrt previous epochs\n",
    "                if phase == 'val' and epoch_acc > best_acc:\n",
    "                    best_acc = epoch_acc\n",
    "                    best_model_wts = copy.deepcopy(model.state_dict())\n",
    "                if phase == 'val' and epoch_loss < best_loss:\n",
    "                    best_loss = epoch_loss\n",
    "                if phase == 'train' and epoch_acc > best_acc_train:\n",
    "                    best_acc_train = epoch_acc\n",
    "                if phase == 'train' and epoch_loss < best_loss_train:\n",
    "                    best_loss_train = epoch_loss\n",
    "        \n",
    "        # Print final results             \n",
    "        model.load_state_dict(best_model_wts)\n",
    "        time_elapsed = time.time() - since\n",
    "        print('Training completed in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))\n",
    "        print('Best test loss: {:.4f} | Best test accuracy: {:.4f}'.format(best_loss, best_acc))\n",
    "        return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A27mgHjL2BsA"
   },
   "source": [
    "We are ready perform the actual training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 111000,
     "status": "ok",
     "timestamp": 1613631611465,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "owpmbUXJ2BsA",
    "outputId": "4a9767b5-c713-4efb-cb89-2a6f5701759a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training started:\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/torch/optim/lr_scheduler.py:100: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n",
      "  \"https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\", UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Phase: train Epoch: 1/3 Loss: 0.4966 Acc: 0.7501             \n",
      "Phase: val   Epoch: 1/3 Loss: 0.4122 Acc: 0.8125             \n",
      "Phase: train Epoch: 2/3 Loss: 0.4540 Acc: 0.7860             \n",
      "Phase: val   Epoch: 2/3 Loss: 0.3881 Acc: 0.8210             \n",
      "Phase: train Epoch: 3/3 Loss: 0.4463 Acc: 0.7893             \n",
      "Phase: val   Epoch: 3/3 Loss: 0.3790 Acc: 0.8300             \n",
      "Training completed in 1m 51s\n",
      "Best test loss: 0.3790 | Best test accuracy: 0.8300\n"
     ]
    }
   ],
   "source": [
    "model_hybrid = train_model(model_hybrid, criterion, optimizer_hybrid, exp_lr_scheduler, num_epochs=num_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VD7ym33s2BsA"
   },
   "source": [
    "We save the optimal weights that achieved the best accuracy into a PyTorch file that will be created in the current working directory. In this way, the saved weights can be loaded in the future without training again.\n",
    "\n",
    "**Important: Be aware that running next cell will overwrite a previously saved file. So, we suggest to run the next cell only after a good training process.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 855,
     "status": "ok",
     "timestamp": 1613631664596,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "3exXqOJ_2BsA",
    "outputId": "d726c3bd-e751-4b25-a98f-28ec6a494911",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model state_dict saved.\n"
     ]
    }
   ],
   "source": [
    "path = '/content/drive/MyDrive/Qiskit-Hackathon-Korea/qnn-visualization/'\n",
    "if quantum:\n",
    "        torch.save(model_hybrid.state_dict(), \n",
    "            path+'quantum_' + filtered_classes[0] + '_' + filtered_classes[1] + '.pt'\n",
    "        )\n",
    "else:\n",
    "        torch.save(model_hybrid.state_dict(), \n",
    "            path+'classical_' + filtered_classes[0] + '_' + filtered_classes[1] + '.pt'\n",
    "        )\n",
    "print(\"Model state_dict saved.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ybqXssPX2BsB"
   },
   "source": [
    "## Load model from file\n",
    "To load the pre-trained weights it is necessary to first define the model. So, before this cell, one should have run all the cells above the *[Training and results](##Training_and_results)* section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 110093,
     "status": "aborted",
     "timestamp": 1613631611748,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "F05jFHN82BsB"
   },
   "outputs": [],
   "source": [
    "if quantum:\n",
    "    model_hybrid.load_state_dict(torch.load(\n",
    "        path+'quantum_' + filtered_classes[0] + '_' + filtered_classes[1] + '.pt'\n",
    "        )\n",
    "    )\n",
    "                                 \n",
    "else:\n",
    "    model_hybrid.load_state_dict(torch.load(\n",
    "        path+'classical_' + filtered_classes[0] + '_' + filtered_classes[1] + '.pt'\n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Vme6Msx22BsB"
   },
   "source": [
    "We apply the model to the test dataset to compute the associated *loss* and *accuracy*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 109816,
     "status": "aborted",
     "timestamp": 1613631611749,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "MDh7uWfJ2BsB"
   },
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "running_loss = 0.0\n",
    "running_corrects = 0\n",
    "n_batches = dataset_sizes['val'] // batch_size\n",
    "it = 0\n",
    "model_hybrid.eval()\n",
    "\n",
    "# Testing loop\n",
    "for inputs, cifar_labels in dataloaders['val']:\n",
    "    inputs = inputs.to(device)\n",
    "    labels = torch.tensor(labels_to_filtered(cifar_labels))\n",
    "    labels = labels.to(device)\n",
    "    batch_size_ = len(inputs)\n",
    "    with torch.set_grad_enabled(False):\n",
    "        outputs = model_hybrid(inputs)\n",
    "        _, preds = torch.max(outputs, 1)\n",
    "        loss = criterion(outputs, labels)\n",
    "    running_loss += loss.item() * batch_size_\n",
    "    batch_corrects = torch.sum(preds == labels.data).item()\n",
    "    running_corrects += batch_corrects\n",
    "    print('Iter: {}/{}'.format(it+1,n_batches+1), end='\\r', flush=True)\n",
    "    it += 1\n",
    "                    \n",
    "# Print final results\n",
    "epoch_loss = running_loss / dataset_sizes['val']\n",
    "epoch_acc = running_corrects / dataset_sizes['val']\n",
    "print('\\nTest Loss: {:.4f} Test Acc: {:.4f}        '.format(epoch_loss, epoch_acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LxdaBJtz2BsB"
   },
   "source": [
    "## Visualize the model\n",
    "Let us compute and the visualize the predictions for a batch of test data.\n",
    "*Hint:* re-run the next cell to see more samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 109545,
     "status": "aborted",
     "timestamp": 1613631611749,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "b8d19XZn2BsB"
   },
   "outputs": [],
   "source": [
    "def visualize_model(model, num_images=6, fig_name='Predictions'):\n",
    "    images_so_far = 0\n",
    "    fig = plt.figure(fig_name)\n",
    "    model.eval()\n",
    "    with torch.no_grad():\n",
    "        for i, (inputs, cifar_labels) in enumerate(dataloaders['val']):\n",
    "            inputs = inputs.to(device)\n",
    "            labels = torch.tensor(labels_to_filtered(cifar_labels))\n",
    "            labels = labels.to(device)\n",
    "            outputs = model(inputs)\n",
    "            _, preds = torch.max(outputs, 1)\n",
    "            for j in range(inputs.size()[0]):\n",
    "                images_so_far += 1\n",
    "                ax = plt.subplot(num_images // 2, 2, images_so_far)\n",
    "                ax.axis('off')\n",
    "                ax.set_title('[{}]'.format(filtered_classes[preds[j]]))\n",
    "                imshow(inputs.cpu().data[j])\n",
    "                if images_so_far == num_images:\n",
    "                    return\n",
    "        \n",
    "visualize_model(model_hybrid, num_images=4)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JxhZvjs42BsC"
   },
   "source": [
    "## References\n",
    "\n",
    "[1] Andrea Mari, Thomas R. Bromley, Josh Izaac, Maria Schuld, and Nathan Killoran. _Transfer learning in hybrid classical-quantum neural networks_. [arXiv:1912.08278](https://arxiv.org/abs/1912.08278), (2019).\n",
    "\n",
    "[2] Sasank Chilamkurthy. PyTorch transfer learning tutorial. https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html.\n",
    "\n",
    "[3] Kaiming He, Xiangyu Zhang, Shaoqing ren and Jian Sun. _Deep residual learning for image recognition_.\n",
    "  Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770-778 (2016). [DOI: 10.1109/CVPR.2016.90]( https://doi.org/10.1109/CVPR.2016.90).\n",
    "  \n",
    "[4] Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, Carsten Blank, Keri McKiernan, and Nathan Killoran. PennyLane: Automatic differentiation of hybrid quantum-classical computations. [arXiv:1811.04968](https://arxiv.org/abs/1811.04968), (2018)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 109292,
     "status": "aborted",
     "timestamp": 1613631611750,
     "user": {
      "displayName": "JOON SUK HUH",
      "photoUrl": "",
      "userId": "03235780383723321017"
     },
     "user_tz": -540
    },
    "id": "5k3xFpPG4LCW"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "c2q_transfer_cifar.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "file_extension": ".py",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "mimetype": "text/x-python",
  "name": "python",
  "npconvert_exporter": "python",
  "pygments_lexer": "ipython3",
  "version": 3
 },
 "nbformat": 4,
 "nbformat_minor": 4
}