01_pytorch.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vn322/Synergy_DBM_301_CS/blob/main/01_pytorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ufPqJXHN1DLB"
      },
      "source": [
        "<center>\n",
        "<img src=\"https://camo.githubusercontent.com/1faf678af2925dc98a2fb877b80461f1c992b9e7e229412e5d6b729c49246ffa/68747470733a2f2f7079746f7263682e6f72672f7475746f7269616c732f5f7374617469632f7079746f7263682d6c6f676f2d6461726b2e737667\" height=\"400\" width=\"800\"> \n",
        "\n",
        "# Семинар 1: введение в PyTorch\n",
        "</center>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W7iGGLLX1DLJ"
      },
      "source": [
        "Если кому-то интересно,  можно подробнее почитать про [раличия между Tensorflow и Pytorch.](https://towardsdatascience.com/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kWNroJQF1DLJ"
      },
      "source": [
        "Устанавливаем библиотеку\n",
        "\n",
        "```\n",
        "pip install torch torchvision\n",
        "```\n",
        "\n",
        "Дополнительно с торчом выходят `torchvision` и `torchaudio`. В них куча всяких методов дял работы с изображениями и звуком. Обе эти библиотеки тоже можно поставить виесте с торчом. Первая из них уже сегодня нам понадобится. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KW_GO_EA1DLK",
        "outputId": "436733ff-e47b-4a46-c7de-eabeb813ba25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'1.22.4'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 1
        }
      ],
      "source": [
        "import numpy as np\n",
        "np.__version__"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vVUKtn1P1DLN",
        "outputId": "65fb5dbd-a6fe-4b4d-862b-bb228d701480",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'2.0.0+cu118'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "import torch\n",
        "torch.__version__ "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X6uhN7G61DLO"
      },
      "source": [
        "# 1. Тензоры и базовые операции над ними\n",
        "\n",
        "Основной объект в PyTorch - это **тензор.** Или многомерный массив чисел. Чтобы не нужно было учить кучу новых команд, PyTorch косплеит numpy. \n",
        "\n",
        "```\n",
        "    np.zeros -> torch.zeros\n",
        "    np.sin -> torch.sin\n",
        "    np.cumsum -> torch.cumsum\n",
        "    \n",
        "    x.reshape([1,2,8]) -> x.view(1,2,8)\n",
        "    x.sum(axis=-1) -> x.sum(dim=-1)\n",
        "    x.astype('int64') -> x.type(torch.LongTensor)\n",
        "```\n",
        "\n",
        "Правда говоря, не совсем косплеит. Но чаще всего оказывается довольно близок. Для помощи вам есть [таблица](https://github.com/torch/torch7/wiki/Torch-for-Numpy-users), которая поможет вам найти аналог операции в numpy."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n2Er0jHm1DLP",
        "outputId": "47b0fc64-790b-47eb-9c57-c44ae066ccf2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3, 4])"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "# как в нумпае можно объявить вектор\n",
        "x = torch.tensor([1, 2, 3, 4])\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4V6yf7-n1DLP",
        "outputId": "b8a97d24-4749-43e3-b0ce-635016abf115",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0.]])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "# можно задать тензор из нулей\n",
        "torch.zeros([3, 4])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u_MA7Nin1DLQ",
        "outputId": "86e4c9da-7bd0-4b89-900d-13c198105b8e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.]],\n",
              "\n",
              "        [[1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.]],\n",
              "\n",
              "        [[1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.],\n",
              "         [1., 1.]]])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "# или из единиц, но уже более сложной размерности\n",
        "torch.ones([3, 4, 2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "thDUwbVj1DLQ",
        "outputId": "2f1d2c99-99d5-4af1-a20b-36a5e689e809",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.6616,  0.0057],\n",
              "        [-0.3199,  0.3626]])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "# можно сгенерировать тензор из стандартного нормального распределения\n",
        "torch.randn([2,2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Qy48ckMW1DLR",
        "outputId": "2dfeae93-9dff-4291-faa1-4e5fca3ab7c3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 8.4059e-35,  0.0000e+00,  9.6575e-36],\n",
              "        [ 0.0000e+00, -2.4189e+02,  4.5765e-41],\n",
              "        [ 0.0000e+00,  0.0000e+00,  0.0000e+00],\n",
              "        [ 0.0000e+00,  0.0000e+00,  0.0000e+00],\n",
              "        [ 0.0000e+00,  0.0000e+00,  0.0000e+00]])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "x = torch.empty(5, 3) # пустой тензор\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RdNSq7Dc1DLR",
        "outputId": "e6667381-d924-4b63-bcfd-de75c75c4354",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0., 0., 0.],\n",
            "        [0., 0., 0.],\n",
            "        [0., 0., 0.],\n",
            "        [0., 0., 0.],\n",
            "        [0., 0., 0.]], dtype=torch.float64)\n"
          ]
        }
      ],
      "source": [
        "# тензор с нулями и указанием типов чисел\n",
        "x = torch.zeros(5, 3, dtype=torch.double) \n",
        "print(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JJ71Okek1DLR",
        "outputId": "7fdd7ffe-5a8a-46c2-b0c4-3caad632c652",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[1., 1., 1.],\n",
            "        [1., 1., 1.],\n",
            "        [1., 1., 1.],\n",
            "        [1., 1., 1.],\n",
            "        [1., 1., 1.]], dtype=torch.float64)\n"
          ]
        }
      ],
      "source": [
        " # используем уже созданный тензор для создания тензора из единичек\n",
        "x = x.new_ones(5, 3, dtype=torch.double)\n",
        "print(x) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3fmOJoqd1DLS",
        "outputId": "8e362bba-a055-4b1d-af2d-61f6b835eab6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[-0.3248,  1.0169,  0.8309],\n",
            "        [-0.6893, -0.0356,  1.8037],\n",
            "        [ 1.1152,  1.2825, -0.3707],\n",
            "        [-1.6880,  0.5909,  0.6446],\n",
            "        [ 0.5001,  0.6603,  1.6465]])\n"
          ]
        }
      ],
      "source": [
        "# создаем матрицу с размерами как у x\n",
        "x = torch.randn_like(x, dtype=torch.float)\n",
        "print(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GBDDsLwu1DLS",
        "outputId": "6b42a351-a0b0-4c19-9c37-faf46fc8884b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.3248,  1.0169,  0.8309],\n",
              "        [-0.6893, -0.0356,  1.8037],\n",
              "        [ 1.1152,  1.2825, -0.3707],\n",
              "        [-1.6880,  0.5909,  0.6446],\n",
              "        [ 0.5001,  0.6603,  1.6465]], dtype=torch.float64)"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "x.double() # можно поменяить тип данных"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4bNr5K7a1DLS",
        "outputId": "2c353470-d043-437a-f2cf-a8db8c3a2161",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 0,  1,  0],\n",
              "        [ 0,  0,  1],\n",
              "        [ 1,  1,  0],\n",
              "        [-1,  0,  0],\n",
              "        [ 0,  0,  1]], dtype=torch.int32)"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "x.int()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yCtLMVlI1DLT",
        "outputId": "784d5c43-59d5-46f3-9f8d-d815285a48b2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.3248,  1.0169,  0.8309],\n",
              "        [-0.6893, -0.0356,  1.8037],\n",
              "        [ 1.1152,  1.2825, -0.3707],\n",
              "        [-1.6880,  0.5909,  0.6446],\n",
              "        [ 0.5001,  0.6603,  1.6465]])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "# как думаете, почему не пишет dtype? \n",
        "x.float()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O8ZqY9yE1DLT"
      },
      "source": [
        "Все срезы, операции, размерности работают как в numpy."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Vt7o_bci1DLT",
        "outputId": "c6c14049-229d-4bf8-975a-ec0bf61fb59c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([5, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "x.size()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ODKQRhbl1DLU",
        "outputId": "9762602a-8b64-4384-8bd0-fb651ec035c6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.float32"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "x.dtype"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WbXWkbkf1DLU",
        "outputId": "58c9e3b7-e06e-4aaf-9838-404ad6c17ede",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([5, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "x.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1koOxsSH1DLU",
        "outputId": "78e13b95-8798-457b-de4a-186e3b767c48",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([-0.3248,  1.0169,  0.8309])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "x[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WSYzH-ry1DLU",
        "outputId": "68b4f3c6-7c8f-4b84-8fac-486ba00e43db",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 1.0169, -0.0356,  1.2825,  0.5909,  0.6603])"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "x[:,1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0oCr2mZ21DLV",
        "outputId": "657a642e-a14b-41e9-ab3c-539ebf975b63",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 9.6752, 11.0169, 10.8309],\n",
              "        [ 9.3108,  9.9644, 11.8037],\n",
              "        [11.1152, 11.2825,  9.6293],\n",
              "        [ 8.3120, 10.5909, 10.6446],\n",
              "        [10.5001, 10.6603, 11.6465]])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "x + 10"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SynJOlGi1DLV",
        "outputId": "bf7e2680-2542-4f2e-d341-cef18e2dca4d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.7227, 2.7647, 2.2953],\n",
              "        [0.5020, 0.9651, 6.0721],\n",
              "        [3.0502, 3.6058, 0.6903],\n",
              "        [0.1849, 1.8057, 1.9052],\n",
              "        [1.6489, 1.9353, 5.1886]])"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "torch.exp(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gtQsn37T1DLV",
        "outputId": "86f81253-23a5-499b-ded4-f3c27838b01c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1.0169, 0.8309, 1.8037, 1.1152, 1.2825, 0.5909, 0.6446, 0.5001, 0.6603,\n",
              "        1.6465])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "x[x > 0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dCqo_n6e1DLV",
        "outputId": "bfc8fc0c-301e-4cf6-fbae-63802cd4da40",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.0389, 0.4549, 0.8718],\n",
              "        [0.2286, 0.1297, 0.5867],\n",
              "        [0.8427, 0.0651, 0.1727],\n",
              "        [0.5814, 0.2242, 0.9706],\n",
              "        [0.7439, 0.9451, 0.2814]])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "y = torch.rand(5, 3)\n",
        "y"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P5eYT2I01DLW",
        "outputId": "a8e998e5-cf9d-4983-f8a5-5669cf598940",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[-0.2859,  1.4718,  1.7027],\n",
            "        [-0.4606,  0.0941,  2.3904],\n",
            "        [ 1.9579,  1.3476, -0.1979],\n",
            "        [-1.1066,  0.8151,  1.6151],\n",
            "        [ 1.2439,  1.6054,  1.9279]])\n"
          ]
        }
      ],
      "source": [
        "print(x + y) # операция сложение"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mdA8zSY31DLW",
        "outputId": "6eea87db-a625-4b13-d507-dd081b1efa16",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[-0.2859,  1.4718,  1.7027],\n",
            "        [-0.4606,  0.0941,  2.3904],\n",
            "        [ 1.9579,  1.3476, -0.1979],\n",
            "        [-1.1066,  0.8151,  1.6151],\n",
            "        [ 1.2439,  1.6054,  1.9279]])\n"
          ]
        }
      ],
      "source": [
        "z = torch.add(x, y) # очередная операция сложения\n",
        "print(z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vkVBWO251DLW",
        "outputId": "b7fe1e4c-1b83-45c9-b31e-318f64fce003",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[-0.2859,  1.4718,  1.7027],\n",
            "        [-0.4606,  0.0941,  2.3904],\n",
            "        [ 1.9579,  1.3476, -0.1979],\n",
            "        [-1.1066,  0.8151,  1.6151],\n",
            "        [ 1.2439,  1.6054,  1.9279]])\n"
          ]
        }
      ],
      "source": [
        "torch.add(x, y, out=z) # и наконец последний вид\n",
        "print(z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "N2tnfMPE1DLX",
        "outputId": "ccda4a01-4754-4c08-a9e1-3b41abe810da",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[-0.0126,  0.4625,  0.7244],\n",
            "        [-0.1576, -0.0046,  1.0582],\n",
            "        [ 0.9397,  0.0835, -0.0640],\n",
            "        [-0.9814,  0.1325,  0.6256],\n",
            "        [ 0.3720,  0.6240,  0.4634]])\n"
          ]
        }
      ],
      "source": [
        "print(x * y) # поэлементное умножение"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lL_gdwY31DLX",
        "outputId": "152723b9-31bb-44e7-9e36-37828b77cbd4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[ 1.1743,  0.5451, -0.0640,  0.8456,  0.9534],\n",
            "        [ 1.5295,  0.8960, -0.2716,  1.3419, -0.0387],\n",
            "        [ 0.3036,  0.2038,  0.9592,  0.5762,  1.9374],\n",
            "        [ 0.7650,  0.0689, -1.2726, -0.2233, -0.5157],\n",
            "        [ 1.7552,  1.1659,  0.7488,  2.0368,  1.4594]])\n"
          ]
        }
      ],
      "source": [
        "print(x @ y.t()) # матричное умножение"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0m5UtVOh1DLX",
        "outputId": "c64f5b4b-2076-4040-ca45-be039554703c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[ 1.1743,  0.5451, -0.0640,  0.8456,  0.9534],\n",
            "        [ 1.5295,  0.8960, -0.2716,  1.3419, -0.0387],\n",
            "        [ 0.3036,  0.2038,  0.9592,  0.5762,  1.9374],\n",
            "        [ 0.7650,  0.0689, -1.2726, -0.2233, -0.5157],\n",
            "        [ 1.7552,  1.1659,  0.7488,  2.0368,  1.4594]])\n"
          ]
        }
      ],
      "source": [
        "# и опять матричное умножение\n",
        "print(x.mm(y.t()))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YRpKSEP_1DLX",
        "outputId": "a5ea3085-5486-4597-ec65-ce648ad38d5a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([1, 5, 3])\n"
          ]
        }
      ],
      "source": [
        "# добавили измерение в начало, аналог броадкастинга \n",
        "print(x.unsqueeze(0).shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tqqBgoR71DLY",
        "outputId": "fe8c3b57-a54a-4e36-a4a9-40e77d836926",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[-0.3248,  1.0169,  0.8309],\n",
              "         [-0.6893, -0.0356,  1.8037],\n",
              "         [ 1.1152,  1.2825, -0.3707],\n",
              "         [-1.6880,  0.5909,  0.6446],\n",
              "         [ 0.5001,  0.6603,  1.6465]]])"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "x.unsqueeze(0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OLHowmqm1DLY",
        "outputId": "212811fe-5848-4f31-d16e-ffae331d40c1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([5, 3])\n"
          ]
        }
      ],
      "source": [
        "# убрали измерение в начале, аналог броадкастинга \n",
        "print(x.unsqueeze(0).squeeze(0).shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VtVdjXRP1DLY",
        "outputId": "1e1310a0-de21-427b-df07-bde2f88f0a5d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-0.3248,  1.0169,  0.8309],\n",
              "        [-0.6893, -0.0356,  1.8037],\n",
              "        [ 1.1152,  1.2825, -0.3707],\n",
              "        [-1.6880,  0.5909,  0.6446],\n",
              "        [ 0.5001,  0.6603,  1.6465]])"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "x.unsqueeze(0).squeeze(0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Vpu5L0J1DLb"
      },
      "source": [
        "Мы также можем переводить матрицы назад в numpy:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "t64Jv1gW1DLc",
        "outputId": "d9f43fc9-0137-497e-a4f6-8e172914d037",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.32478788,  1.0169196 ,  0.8308774 ],\n",
              "       [-0.68925005, -0.03556003,  1.8037102 ],\n",
              "       [ 1.115194  ,  1.2825487 , -0.3706749 ],\n",
              "       [-1.688005  ,  0.59092957,  0.6445874 ],\n",
              "       [ 0.50009596,  0.66025287,  1.6464661 ]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "x.numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5pQWqsI51DLc"
      },
      "source": [
        "### Задание 1: \n",
        "\n",
        "- При помощи numpy посчитайте сумму квадратов чисел от 1 до 10000\n",
        "- Сделайте то же самое с помощью pytorch"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZJcbHgOa1DLe",
        "outputId": "ac7c879c-9705-4677-96fc-f3704f715fe3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "333383335000"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "np.square(np.arange(1, 10_000+1)).sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1Tsvx2t91DLe",
        "outputId": "4e88c8cd-fee3-49a9-f087-7ad31d2811bb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "333383335000"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "# .item() возвращает число из тензора. \n",
        "# Без этого будет тензор размерности 1x1\n",
        "\n",
        "torch.square(torch.arange(1, 10_000+1)).sum().item()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Mbmzqdl1DLf"
      },
      "source": [
        "### Задание 2:\n",
        "\n",
        "Реализуйте на PyTorch сигмоиду \n",
        "\n",
        "$$ \\sigma(x) = \\frac{1}{1 + e^{-x}} $$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rr_yKl2f1DLf",
        "outputId": "12eaf402-f1ef-4e0a-92f3-e7a1fa7f563d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzeklEQVR4nO3de3RU5b3/8c/MJJkkhCRAroRAuCgXUVCQnHhptaZEammt1kXVn1COl5+KVo09VVTASzUcL8ipxaZekJ51joXaX7WtIlajaJVUlIsXFJRrIpCEAMmEhEySmef3R5iBSIBMyGTP5f1aa1bIzt6Z7+40Mx+f/X2ebTPGGAEAAFjEbnUBAAAguhFGAACApQgjAADAUoQRAABgKcIIAACwFGEEAABYijACAAAsRRgBAACWirG6gK7wer3atWuX+vbtK5vNZnU5AACgC4wxamho0MCBA2W3H3v8IyzCyK5du5Sbm2t1GQAAoBsqKys1aNCgY/48LMJI3759JbWfTHJyssXVAACArnC5XMrNzfV/jh9LWIQR36WZ5ORkwggAAGHmRC0WNLACAABLEUYAAIClCCMAAMBShBEAAGApwggAALAUYQQAAFiKMAIAACxFGAEAAJYijAAAAEsFHEbee+89TZ06VQMHDpTNZtMrr7xywmNWrlyps846S06nUyNGjNCSJUu6USoAAIhEAYeRxsZGjRs3TosWLerS/tu2bdMll1yiCy+8UOvXr9ftt9+u6667Tm+88UbAxQIAgMgT8L1ppkyZoilTpnR5/9LSUg0dOlRPPPGEJGn06NF6//339eSTT6qoqCjQpwcAABEm6DfKKy8vV2FhYYdtRUVFuv322495jNvtltvt9n/vcrmCVR4AhDRjjFo8XjW6PWp0t8nd5lFzq1fuNq9a2rxyt3nU6jFq83jV4vGqzWPU5vXK45U8Xq88XqM2r5ExktcYecyhf3uNvEYyOvT10HYj31f5v28vRL5/+evSoX061NvpOXThPDs9Er3p388dqtz+iZY8d9DDSFVVlTIzMztsy8zMlMvl0sGDB5WQkHDUMSUlJXrggQeCXRoAWKKppU2V+w6qytWsalezquubVd3QrP1NrapvalXdwRbVNbXKdbBVTS0etXn5oEbwTR03MHLDSHfMnj1bxcXF/u9dLpdyc3MtrAgAAufxGm2scmldRZ021xzQlj0HtKXmgHbVN3fr9zlj7EqIc8gZY5czpv1rXIxdsQ67Yh02xdjtinHYFOuwy2G3yWGzyeE49NVuk80m2W3t39tskt1uk90m2XT4Z5Lku9u7b7vt0DabzaYON4I/Yr8Omzu5W/zxbyB/bCe48zx6UGZyvGXPHfQwkpWVperq6g7bqqurlZyc3OmoiCQ5nU45nc5glwYAPcrrNVpXWaf3v67Vxzv2aV1FnQ642zrdNyUhVtkp8cpMjldmslOZyfEa0CdOqYlxSkmMVWpCrJITYpXkjFFinEOJcTFy2PlkRmQKehgpKCjQ8uXLO2x78803VVBQEOynBoCgM8Zowy6X/v7pLr36yW7trDvY4edJzhidOThVY7KTNTw9ScMz+mhYWpL69YmzqGIg9AQcRg4cOKDNmzf7v9+2bZvWr1+v/v37a/DgwZo9e7Z27typ//7v/5Yk3Xjjjfrtb3+rX/3qV/r3f/93vf322/rTn/6k1157refOAgB6WUubVy+tqdTz/9ymrbWN/u1JzhhdMDJd+UP7a8KQ/hqZ1ZcRDeAEAg4jH3/8sS688EL/977ejhkzZmjJkiXavXu3Kioq/D8fOnSoXnvtNd1xxx36r//6Lw0aNEjPPfcc03oBhKVWj1cvr92p37z9tb7Z3z4K4oyxq3B0pqaOy9YFIzMUH+uwuEogvNiM6cqkK2u5XC6lpKSovr5eycnJVpcDIEr9/ZNdevwfm7Rjb5MkKb2vUzdfMFxXTMxVkjMk5wMAlurq5zd/PQBwAk0tbZrzygb9v7XfSJIG9InTTRcM19X5Q5QQxygIcLIIIwBwHF9VN+jm/12rzTUHZLdJt3zvFP3f7wxTH0ZCgB7DXxMAHMOfPq7U3L9+ruZWrzKTnfrNz85U/rABVpcFRBzCCAB0Yv7rG1X67hZJ0vmnpOnJaeOVlsT6R0AwEEYA4Ft+/+4WfxC58/unataFI2Rnei4QNIQRADjCSx9XquT1jZKke34wSjd8Z7jFFQGRz251AQAQKt76olp3/+UzSdL//c4wggjQSwgjACDpo+37NOvFtfJ4jX46YZDunjLK6pKAqEEYARD1dtUd1LVLPpK7zauLRmVo/mWny8btYoFeQxgBENWMMZr718/lam7TuNxU/faqsxTj4K0R6E38xQGIaq9/XqW3vqxRrMOmx396BiuqAhYgjACIWvUHWzXvbxskSTdfMEKnZPa1uCIgOhFGAESt/1yxUXsa3BqW3kc3X8jMGcAqhBEAUWn1tn168cMKSVLJT06XM4bLM4BVCCMAoo67zaPZf/lUknTlpFzuNwNYjDACIOo88+5WbdnTqLQkp+6+eLTV5QBRjzACIKo0utv07D+3SpLm/HC0UhJjLa4IAGEEQFT585pv5GpuU96ARE09Y6DV5QAQYQRAFPF4jZ5/f5sk6drzhnInXiBEEEYARI03v6hWxb4mpSbG6vIJg6wuB8AhhBEAUeO5Q70iV+cPVmJcjMXVAPAhjACICusq9uvjHfsV67BpRkGe1eUAOAJhBEBUeO5Qr8iPxuUoIzne4moAHIkwAiDiVe5r0uuf7ZYkXXf+UIurAfBthBEAEW/Jqu3yGum8EWkanZ1sdTkAvoUwAiCiuZpbteyjSkmMigChijACIKKt+KxKB9xtGp7eR989Nd3qcgB0gjACIKK9/nl7r8iPx+fIZmORMyAUEUYARCxXc6s+2LxXkjRlbJbF1QA4FsIIgIj1zsYatXi8Gp7eR6dk9rW6HADHQBgBELFe/6xKkjRlbLbFlQA4HsIIgIjU1NKmlV/VSJIu5hINENIIIwAi0rub9qi51atB/RJ02kDWFgFCGWEEQER6/XPfJZosZtEAIY4wAiDiuNs8enuj7xIN/SJAqCOMAIg4H2yu1QF3mzKTnTozN9XqcgCcAGEEQMTxzaK5+LQs2e1cogFCHWEEQERp9Xj15pfVkqQiZtEAYYEwAiCifLh1n+qaWtW/T5wm5fW3uhwAXUAYARBRfPeimTwmUzEO3uKAcMBfKoCIsnLTHklcogHCCWEEQMT4Zn+TdtYdVIzdpvyhXKIBwgVhBEDEWL1tnyRpbE6KEuNiLK4GQFcRRgBEDF8YYVQECC+EEQARwxdGJhFGgLBCGAEQEWpczdpa2yibTZo4hDAChBPCCICIsHp7+6jIqKxkpSTGWlwNgEAQRgBEBPpFgPBFGAEQEegXAcIXYQRA2KtratHGqgZJ0tksAQ+EHcIIgLD30fb9kqRh6X2U3tdpcTUAAkUYARD2Vm/bK0nKHzrA4koAdAdhBEDYo3kVCG+EEQBh7YC7TZ/vckmieRUIV4QRAGFt7Y798niNBvVL0MDUBKvLAdANhBEAYY0pvUD4I4wACGv0iwDhr1thZNGiRcrLy1N8fLzy8/O1evXq4+6/cOFCjRw5UgkJCcrNzdUdd9yh5ubmbhUMAD7NrR6tr6yTJE1iJg0QtgIOI8uWLVNxcbHmzZuntWvXaty4cSoqKlJNTU2n+7/44ou6++67NW/ePH355Zd6/vnntWzZMt1zzz0nXTyA6PZJZZ1aPF6l93Uqb0Ci1eUA6KaAw8iCBQt0/fXXa+bMmRozZoxKS0uVmJioxYsXd7r/qlWrdO655+qqq65SXl6eJk+erCuvvPKEoykAcCK+UZGJQ/rJZrNZWwyAbgsojLS0tGjNmjUqLCw8/AvsdhUWFqq8vLzTY8455xytWbPGHz62bt2q5cuX6wc/+MExn8ftdsvlcnV4AMC3bTg0pXdsTorFlQA4GTGB7FxbWyuPx6PMzMwO2zMzM7Vx48ZOj7nqqqtUW1ur8847T8YYtbW16cYbbzzuZZqSkhI98MADgZQGIApt2FUvSRozMNniSgCcjKDPplm5cqUeeeQRPf3001q7dq3+8pe/6LXXXtNDDz10zGNmz56t+vp6/6OysjLYZQIIM00tbdpa2yhJOi2bMAKEs4BGRtLS0uRwOFRdXd1he3V1tbKysjo9Zs6cObrmmmt03XXXSZJOP/10NTY26oYbbtC9994ru/3oPOR0OuV0crMrAMe2sapBxkhpSU5lJMdbXQ6AkxDQyEhcXJwmTJigsrIy/zav16uysjIVFBR0ekxTU9NRgcPhcEiSjDGB1gsAkg73i3CJBgh/AY2MSFJxcbFmzJihiRMnatKkSVq4cKEaGxs1c+ZMSdL06dOVk5OjkpISSdLUqVO1YMECnXnmmcrPz9fmzZs1Z84cTZ061R9KACBQXxwKI6cRRoCwF3AYmTZtmvbs2aO5c+eqqqpK48eP14oVK/xNrRUVFR1GQu677z7ZbDbdd9992rlzp9LT0zV16lQ9/PDDPXcWAKLOF4eaVwkjQPizmTC4VuJyuZSSkqL6+nolJ/PGA0S7No9Xp817Q+42r96+87salp5kdUkAOtHVz2/uTQMg7GytbZS7zas+cQ7lDehjdTkAThJhBEDY8a0vMjo7WXY7K68C4Y4wAiDsfMFMGiCiEEYAhJ0NzKQBIgphBEBYMcboi92HRkayuScNEAkIIwDCyq76ZtU1tSrGbtOpWcyiASIBYQRAWNmws715dURGkpwxLJwIRALCCICw4r9EQ78IEDEIIwDCyuHmVfpFgEhBGAEQVrgnDRB5CCMAwkZdU4t21h2U1L7gGYDIQBgBEDZ8oyK5/ROUkhBrcTUAegphBEDYOLy+CKMiQCQhjAAIGzSvApGJMAIgbPhukEfzKhBZCCMAwkKrx6utexolSaO4TANEFMIIgLCwY2+T2rxGiXEODUyJt7ocAD2IMAIgLGyuOSBJGp6eJJvNZnE1AHoSYQRAWNiyxxdG+lhcCYCeRhgBEBZ8YWREBnfqBSINYQRAWNhyxGUaAJGFMAIg5BljtOXQTJrhjIwAEYcwAiDk1TS4dcDdJofdpiEDEq0uB0API4wACHm+mTSD+yfKGeOwuBoAPY0wAiDkMZMGiGyEEQAhz7/GCP0iQEQijAAIeYdHRggjQCQijAAIeVtqDs2kIYwAEYkwAiCkNTS3qsrVLEkaQRgBIhJhBEBI892pNy3JqZTEWIurARAMhBEAIY2ZNEDkI4wACGm+mTTckwaIXIQRACGNmTRA5COMAAhp3JMGiHyEEQAhq9Xj1fba9jDCZRogchFGAISsin1NavMaJcQ6lJ0cb3U5AIKEMAIgZG3xLwPfR3a7zeJqAAQLYQRAyNpM8yoQFQgjAEIWy8AD0YEwAiBk+ab10rwKRDbCCICQZIw53DPCyAgQ0QgjAELSnga3GtxtstukvLREq8sBEESEEQAhyde8Orh/opwxDourARBMhBEAIYlLNED0IIwACEnbapskSUPTuFsvEOkIIwBC0o697dN68wgjQMQjjAAISdt9YWQAYQSIdIQRACHH4zWq3HdQkjRkADNpgEhHGAEQcnbXH1SLx6tYh00DUxOsLgdAkBFGAIScHXvbm1dz+yfKwQ3ygIhHGAEQcugXAaILYQRAyPGNjNAvAkQHwgiAkOOb1jukP2EEiAaEEQAhxz8ywhojQFQgjAAIKcYYekaAKEMYARBSahrcam71ymG3KYdpvUBU6FYYWbRokfLy8hQfH6/8/HytXr36uPvX1dVp1qxZys7OltPp1Kmnnqrly5d3q2AAkW17bfuoSE5qguJi+O8lIBrEBHrAsmXLVFxcrNLSUuXn52vhwoUqKirSpk2blJGRcdT+LS0t+v73v6+MjAz9+c9/Vk5Ojnbs2KHU1NSeqB9AhGEmDRB9Ag4jCxYs0PXXX6+ZM2dKkkpLS/Xaa69p8eLFuvvuu4/af/Hixdq3b59WrVql2NhYSVJeXt7JVQ0gYtEvAkSfgMZAW1patGbNGhUWFh7+BXa7CgsLVV5e3ukxf/vb31RQUKBZs2YpMzNTY8eO1SOPPCKPx3PM53G73XK5XB0eAKIDIyNA9AkojNTW1srj8SgzM7PD9szMTFVVVXV6zNatW/XnP/9ZHo9Hy5cv15w5c/TEE0/o17/+9TGfp6SkRCkpKf5Hbm5uIGUCCGM79jEyAkSboHeHeb1eZWRk6JlnntGECRM0bdo03XvvvSotLT3mMbNnz1Z9fb3/UVlZGewyAYQAY4x21LaPjOSlMTICRIuAekbS0tLkcDhUXV3dYXt1dbWysrI6PSY7O1uxsbFyOBz+baNHj1ZVVZVaWloUFxd31DFOp1NOpzOQ0gBEgH2NLWpwt8lmkwb1I4wA0SKgkZG4uDhNmDBBZWVl/m1er1dlZWUqKCjo9Jhzzz1Xmzdvltfr9W/76quvlJ2d3WkQARC9th/qF8lOjld8rOMEewOIFAFfpikuLtazzz6rP/zhD/ryyy910003qbGx0T+7Zvr06Zo9e7Z//5tuukn79u3Tbbfdpq+++kqvvfaaHnnkEc2aNavnzgJARPDfk4Z+ESCqBDy1d9q0adqzZ4/mzp2rqqoqjR8/XitWrPA3tVZUVMhuP5xxcnNz9cYbb+iOO+7QGWecoZycHN1222266667eu4sAEQE38gI/SJAdLEZY4zVRZyIy+VSSkqK6uvrlZycbHU5AILktqXr9Nf1u3T3lFG68bvDrS4HwEnq6uc3ay0DCBn+kRHWGAGiCmEEQMiooGcEiEqEEQAhob6pVfubWiWx+ioQbQgjAEKCb+XVjL5OJcYF3FsPIIwRRgCEhMP9IlyiAaINYQRASNhR2z4yMphLNEDUIYwACAnMpAGiF2EEQEhg9VUgehFGAISEHfvoGQGiFWEEgOUa3W3a0+CWRM8IEI0IIwAsV3FoVKRfYqxSEmItrgZAbyOMALCcL4wM7s+oCBCNCCMALFdxaCbNYPpFgKhEGAFgucMjIwkWVwLACoQRAJbbwWUaIKoRRgBYrtIfRrhMA0QjwggAS3m8Rt/s9/WMMDICRCPCCABL7a4/qFaPUZzDrqzkeKvLAWABwggAS/lm0gzqlyCH3WZxNQCsQBgBYCn/TBou0QBRizACwFLMpAFAGAFgKVZfBUAYAWAp/+qrhBEgahFGAFjKNzIyhKXggahFGAFgmfqmVtUfbJUk5bIUPBC1CCMALOMbFUnv61RiXIzF1QCwCmEEgGVoXgUgEUYAWGjHvkZJ0hDCCBDVCCMALOO7QV4uYQSIaoQRAJbZsdc3k4YwAkQzwggAy9AzAkAijACwSEubV7vqDkrivjRAtCOMALDErrqD8hopPtau9CSn1eUAsBBhBIAljrxBns1ms7gaAFYijACwxOF+EZaBB6IdYQSAJSr2tq8xQvMqAMIIAEscvkEeYQSIdoQRAJbwrTHCyAgAwgiAXmeM8a++yrReAIQRAL1uX2OLGls8stmkQf0SrC4HgMUIIwB6nW9ab3ZyvJwxDourAWA1wgiAXscN8gAciTACoNdxgzwARyKMAOh1zKQBcCTCCIBeV7GvfcGzIQNYfRUAYQSABbYfGhnJI4wAEGEEQC9rdLdpT4NbEmuMAGhHGAHQq3z9Iv37xCklIdbiagCEAsIIgF61Y6+vX4RREQDtCCMAehX9IgC+jTACoFf5RkaY1gvAhzACoFdtPxRG8tIIIwDaEUYA9KrDq69ymQZAO8IIgF7T3OrR7vpmSfSMADiMMAKg1/hukNc3Pkb9EpnWC6AdYQRArzlyJo3NZrO4GgCholthZNGiRcrLy1N8fLzy8/O1evXqLh23dOlS2Ww2XXrppd15WgBhjjVGAHQm4DCybNkyFRcXa968eVq7dq3GjRunoqIi1dTUHPe47du365e//KXOP//8bhcLILxtJ4wA6ETAYWTBggW6/vrrNXPmTI0ZM0alpaVKTEzU4sWLj3mMx+PR1VdfrQceeEDDhg07qYIBhC9m0gDoTEBhpKWlRWvWrFFhYeHhX2C3q7CwUOXl5cc87sEHH1RGRoauvfbaLj2P2+2Wy+Xq8AAQ/vxrjBBGABwhoDBSW1srj8ejzMzMDtszMzNVVVXV6THvv/++nn/+eT377LNdfp6SkhKlpKT4H7m5uYGUCSAEtbR5tXP/QUlSHpdpABwhqLNpGhoadM011+jZZ59VWlpal4+bPXu26uvr/Y/KysogVgmgN3yzv0leIyXEOpTe12l1OQBCSEwgO6elpcnhcKi6urrD9urqamVlZR21/5YtW7R9+3ZNnTrVv83r9bY/cUyMNm3apOHDhx91nNPplNPJmxUQSQ73iyQyrRdABwGNjMTFxWnChAkqKyvzb/N6vSorK1NBQcFR+48aNUqfffaZ1q9f73/86Ec/0oUXXqj169dz+QWIIjvoFwFwDAGNjEhScXGxZsyYoYkTJ2rSpElauHChGhsbNXPmTEnS9OnTlZOTo5KSEsXHx2vs2LEdjk9NTZWko7YDiGy+Bc+GcIM8AN8ScBiZNm2a9uzZo7lz56qqqkrjx4/XihUr/E2tFRUVsttZ2BVAR/4Fz/ozMgKgI5sxxlhdxIm4XC6lpKSovr5eycnJVpcDoBu+9/hKba1t1IvX5eucEV1vaAcQvrr6+c0QBoCga/N4Vbnfd5mGkREAHRFGAATd7vpmtXqM4mLsyk6Ot7ocACGGMAIg6Hwrrw7unyi7nWm9ADoijAAIOt9MGlZeBdAZwgiAoNtR67tbL/0iAI5GGAEQdNuPWH0VAL6NMAIg6Cr2MTIC4NgIIwCCyus1/vvS0DMCoDOEEQBBVd3QLHebVzF2m3JSE6wuB0AIIowACKotNe2XaHL7JyrGwVsOgKPxzgAgqLbsOSBJGp6eZHElAEIVYQRAUPnCyIgMwgiAzhFGAATV5hrfyAgzaQB0jjACIKj8l2kYGQFwDIQRAEHT0NyqapdbEj0jAI6NMAIgaLbsaZ9Jk97XqZSEWIurARCqCCMAgmYL/SIAuoAwAiBoNjOTBkAXEEYABM3hkRHCCIBjI4wACBoWPAPQFYQRAEHR6vH6b5DHZRoAx0MYARAUO/Y2qc1rlBjnUHZKvNXlAAhhhBEAQXHkJRqbzWZxNQBCGWEEQFCwDDyAriKMAAgKmlcBdBVhBEBQ+FZfpXkVwIkQRgD0OGPM4TVGCCMAToAwAqDH1TS4dcDdJrtNGjIg0epyAIQ4wgiAHucbFRkyoI+cMQ6LqwEQ6ggjAHrc5j3MpAHQdYQRAD2Oe9IACARhBECP882koXkVQFcQRgD0uM2MjAAIAGEEQI864G5TlatZkjSCMAKgCwgjAHqUr18kLcmplMRYi6sBEA4IIwB61BZm0gAIEGEEQI/yhRGWgQfQVYQRAD3K17w6jH4RAF1EGAHQo77c3SBJGp3V1+JKAIQLwgiAHuNqblXFviZJ0piByRZXAyBcEEYA9JgvdrkkSTmpCUpNjLO4GgDhgjACoMf4wsjobEZFAHQdYQRAj9lwKIycxiUaAAEgjADoMRt21UsijAAIDGEEQI9wt3n803ppXgUQCMIIgB7xdfUBtXmNUhJilZOaYHU5AMIIYQRAj/jiiH4Rm81mcTUAwglhBECP8PWLjGEmDYAAEUYA9Aj/TJocwgiAwBBGAJw0r9foy93tYWRMdorF1QAIN4QRACdtx74mNbZ45Iyxa3h6H6vLARBmCCMATpqvX2RUVl/FOHhbARAY3jUAnDTfTBrWFwHQHYQRACdtgz+M0C8CIHCEEQAn7Qt/8yojIwACRxgBcFJqGpq1p8Etm00and3X6nIAhKFuhZFFixYpLy9P8fHxys/P1+rVq4+577PPPqvzzz9f/fr1U79+/VRYWHjc/QGEF98lmmFpfZQYF2NxNQDCUcBhZNmyZSouLta8efO0du1ajRs3TkVFRaqpqel0/5UrV+rKK6/UO++8o/LycuXm5mry5MnauXPnSRcPwHpf0C8C4CQFHEYWLFig66+/XjNnztSYMWNUWlqqxMRELV68uNP9//d//1c333yzxo8fr1GjRum5556T1+tVWVnZSRcPwHpH3pMGALojoDDS0tKiNWvWqLCw8PAvsNtVWFio8vLyLv2OpqYmtba2qn///sfcx+12y+VydXgACE2+5lXCCIDuCiiM1NbWyuPxKDMzs8P2zMxMVVVVdel33HXXXRo4cGCHQPNtJSUlSklJ8T9yc3MDKRNALzngbtO22kZJzKQB0H29Optm/vz5Wrp0qV5++WXFx8cfc7/Zs2ervr7e/6isrOzFKgF0le8STWayUwOSnBZXAyBcBdT6npaWJofDoerq6g7bq6urlZWVddxjH3/8cc2fP19vvfWWzjjjjOPu63Q65XTyxgaEuo+275MknTW4n8WVAAhnAY2MxMXFacKECR2aT33NqAUFBcc87tFHH9VDDz2kFStWaOLEid2vFkBIWb2tPYxMGnrsHjAAOJGAFwUoLi7WjBkzNHHiRE2aNEkLFy5UY2OjZs6cKUmaPn26cnJyVFJSIkn6z//8T82dO1cvvvii8vLy/L0lSUlJSkpK6sFTAdCb2jxerdmxXxJhBMDJCTiMTJs2TXv27NHcuXNVVVWl8ePHa8WKFf6m1oqKCtnthwdcfve736mlpUU//elPO/yeefPm6f777z+56gFY5svdDTrgblPf+BiNyqJ5FUD3dWu5xFtuuUW33HJLpz9buXJlh++3b9/enacAEOI+3LZXknR2Xn857DaLqwEQzrg3DYBuoV8EQE8hjAAImNdr/DNpCCMAThZhBEDANu85oP1NrUqIdWgs96QBcJIIIwAC9uGhSzRnDUlVXAxvIwBODu8iAALm7xfJG2BxJQAiAWEEQECMMVp9aCYN/SIAegJhBEBAKvY1qdrlVqzDpjMHp1pdDoAIQBgBEJAPt7Zfohk3KFXxsQ6LqwEQCQgjAALyIeuLAOhhhBEAAVm9nX4RAD2LMAKgy3bVHVTlvoOy26QJQ/pZXQ6ACEEYAdBlvlVXTxuYor7xsRZXAyBSEEYAdBn9IgCCgTACoEuMMfpgc60kwgiAnkUYAdAlG6satGNvk+Ji7Dp3RJrV5QCIIIQRAF3y+udVkqTvnJKuJGeMxdUAiCSEEQBd8sahMDJlbJbFlQCINIQRACe0dc8BbapuUIzdpsLRmVaXAyDCEEYAnJDvEk3B8AFKSWRKL4CeRRgBcEIr/Jdosi2uBEAkIowAOK7KfU36bGe97DZp8mlcogHQ8wgjAI7rjQ3toyJn5/VXWpLT4moARCLCCIDjWsEsGgBBRhgBcEw1rmatqdgvSbqYfhEAQUIYAXBMb2yokjHSmYNTlZUSb3U5ACIUYQTAMfmm9F58GpdoAAQPYQRAp/Y1tvjv0suUXgDBRBgB0KlXP90lj9doTHayBg9ItLocABGMMALgKB6v0eL3t0mSpp2da3E1ACIdYQTAUcq+rNb2vU1KSYjVFRMHWV0OgAhHGAFwlOf+2T4qcnX+YCXGxVhcDYBIRxgB0MEnlXVavX2fYh02zTgnz+pyAEQBwgiADp471CsyddxAZSaztgiA4COMAPDbWXdQyz/bLUm67rxhFlcDIFoQRgD4Lflgmzxeo3NHDNCYgclWlwMgShBGAEiSGppbtXR1pSRGRQD0LsIIAEnSso8q1eBu04iMJH331HSrywEQRQgjANTobtPzhxpXrz1vqOx2m8UVAYgmhBEAeuIfX2l3fbMG9UvQT87MsbocAFGGMAJEuU8q67RkVfuoyK8vHav4WIfFFQGINoQRIIq1ery6+y+fyWukH48fqAtGZlhdEoAoRBgBotji97fpy90upSbGas4Px1hdDoAoRRgBolTF3iY9+dZXkqR7fzBaaUlOiysCEK0II0AUMsbo3lc+U3OrVwXDBuinE7gzLwDrEEaAKPQ//9qhf35dq7gYux657HTZbEzlBWAdwggQZV7/bLfm/W2DJOnO75+qoWl9LK4IQLQjjABRZNWWWt22dL28Rrpy0mDd8B2WfQdgPcIIECU+31mvG/57jVo8Xl18WpZ+felYLs8ACAmEESAKbKtt1IzFq3XA3aaCYQO08Gfj5WDJdwAhgjACRLh1Fft11bP/0t7GFp02MFnPTJ/AKqsAQkqM1QUACA5jjJ5/f5vmv75RbV6jYel9tGTmJPWNj7W6NADogDACRKC6phb98qVP9daX1ZKkS07PVsnlpyuZIAIgBBFGgAhijNE7m2o055UN2ll3UHEOu+ZMHaP/kz+YZlUAIYswAkQAY4z++XWtFrz5ldZX1kmShgxI1KKrztLYnBRriwOAEyCMAGHM4zX6YHOtnnr7a320fb8kKT7WrukFebrleyO4LAMgLBBGgDBjjNG6yjr9/ZNdeu3T3appcEuS4mLs+j/5Q3TjBcOU0Tfe4ioBoOu6NbV30aJFysvLU3x8vPLz87V69erj7v/SSy9p1KhRio+P1+mnn67ly5d3q1ggGhljVLmvSX9Z+43uefkznf/oO7rs6VV64YPtqmlwKzk+Rj8/J0///NWFmjt1DEEEQNgJeGRk2bJlKi4uVmlpqfLz87Vw4UIVFRVp06ZNysjIOGr/VatW6corr1RJSYl++MMf6sUXX9Sll16qtWvXauzYsT1yEkCkaGhu1dY9jdqy54C27mnU1zUNWldR5x/98EmMc2jymExNHTdQ55+SrrgYlgwCEL5sxhgTyAH5+fk6++yz9dvf/laS5PV6lZubq1tvvVV33333UftPmzZNjY2NevXVV/3b/u3f/k3jx49XaWlpl57T5XIpJSVF9fX1Sk5ODqRcwBLGGLnbvGp0t6nR7VFjS5sOuNtU19SquqYW1R9sVV1Tq2oamlXlcqu6vllVrmbVH2zt9PfFOmw6bWCKJg7pp7OH9td3TklXQhwLlwEIbV39/A5oZKSlpUVr1qzR7Nmz/dvsdrsKCwtVXl7e6THl5eUqLi7usK2oqEivvPLKMZ/H7XbL7T78X4IulyuQMrvs+fe36Zv9TUH53ZEksLgajOfvWIDxbz9ym+mwreM+h39mTPu+3iP+LSN5Tftv8BrJ6zXymvaHx6tDXzs+Wr1etXq8avMYtXq8avF45W49/NXd5pG3m/+7pfd1alhaHw3PSNKwtD46PSdF43JTWTUVQMQKKIzU1tbK4/EoMzOzw/bMzExt3Lix02Oqqqo63b+qquqYz1NSUqIHHnggkNK65bVPd2ltRV3QnwfRLSHWoT5Oh/o4Y5SaGKfUhFilJsYqJSFWGX2dykiOV1ZyvDKT45WdGs8MGABRJyRn08yePbvDaIrL5VJubm6PP8/lEwapYPiAHv+9aGdT9xbZ6mxtrqM2dbKT7Vs/sskmm63jdpvNt+3wz+y+bbb2ih12m+w2yW63yW6zyWGzyWHv+Iix2xQbY1es3a4Yh02xDrviY+1yxjjkjLHLGWtXn7gYJcQ6ZOeGdABwXAGFkbS0NDkcDlVXV3fYXl1draysrE6PycrKCmh/SXI6nXI6nYGU1i1X5w8J+nMAAIDjC6gFPy4uThMmTFBZWZl/m9frVVlZmQoKCjo9pqCgoMP+kvTmm28ec38AABBdAr5MU1xcrBkzZmjixImaNGmSFi5cqMbGRs2cOVOSNH36dOXk5KikpESSdNttt+m73/2unnjiCV1yySVaunSpPv74Yz3zzDM9eyYAACAsBRxGpk2bpj179mju3LmqqqrS+PHjtWLFCn+TakVFhez2wwMu55xzjl588UXdd999uueee3TKKafolVdeYY0RAAAgqRvrjFiBdUYAAAg/Xf38ZtlGAABgKcIIAACwFGEEAABYijACAAAsRRgBAACWIowAAABLEUYAAIClCCMAAMBShBEAAGCpgJeDt4JvkViXy2VxJQAAoKt8n9snWuw9LMJIQ0ODJCk3N9fiSgAAQKAaGhqUkpJyzJ+Hxb1pvF6vdu3apb59+8pms/XY73W5XMrNzVVlZWXE3vMm0s+R8wt/kX6OnF/4i/RzDOb5GWPU0NCggQMHdriJ7reFxciI3W7XoEGDgvb7k5OTI/L/YEeK9HPk/MJfpJ8j5xf+Iv0cg3V+xxsR8aGBFQAAWIowAgAALBXVYcTpdGrevHlyOp1WlxI0kX6OnF/4i/Rz5PzCX6SfYyicX1g0sAIAgMgV1SMjAADAeoQRAABgKcIIAACwFGEEAABYKirDyPbt23Xttddq6NChSkhI0PDhwzVv3jy1tLR02O/TTz/V+eefr/j4eOXm5urRRx+1qOLAPfzwwzrnnHOUmJio1NTUTvex2WxHPZYuXdq7hZ6ErpxjRUWFLrnkEiUmJiojI0P/8R//oba2tt4ttAfl5eUd9ZrNnz/f6rK6bdGiRcrLy1N8fLzy8/O1evVqq0vqMffff/9Rr9WoUaOsLqvb3nvvPU2dOlUDBw6UzWbTK6+80uHnxhjNnTtX2dnZSkhIUGFhob7++mtriu2mE53jz3/+86Ne04svvtiaYgNUUlKis88+W3379lVGRoYuvfRSbdq0qcM+zc3NmjVrlgYMGKCkpCRdfvnlqq6u7pX6ojKMbNy4UV6vV7///e+1YcMGPfnkkyotLdU999zj38flcmny5MkaMmSI1qxZo8cee0z333+/nnnmGQsr77qWlhZdccUVuummm4673wsvvKDdu3f7H5deemnvFNgDTnSOHo9Hl1xyiVpaWrRq1Sr94Q9/0JIlSzR37txerrRnPfjggx1es1tvvdXqkrpl2bJlKi4u1rx587R27VqNGzdORUVFqqmpsbq0HnPaaad1eK3ef/99q0vqtsbGRo0bN06LFi3q9OePPvqofvOb36i0tFQffvih+vTpo6KiIjU3N/dypd13onOUpIsvvrjDa/rHP/6xFyvsvnfffVezZs3Sv/71L7355ptqbW3V5MmT1djY6N/njjvu0N///ne99NJLevfdd7Vr1y5ddtllvVOggTHGmEcffdQMHTrU//3TTz9t+vXrZ9xut3/bXXfdZUaOHGlFed32wgsvmJSUlE5/Jsm8/PLLvVpPMBzrHJcvX27sdrupqqryb/vd735nkpOTO7yu4WTIkCHmySeftLqMHjFp0iQza9Ys//cej8cMHDjQlJSUWFhVz5k3b54ZN26c1WUExbffO7xer8nKyjKPPfaYf1tdXZ1xOp3mj3/8owUVnrzO3h9nzJhhfvzjH1tST0+rqakxksy7775rjGl/vWJjY81LL73k3+fLL780kkx5eXnQ64nKkZHO1NfXq3///v7vy8vL9Z3vfEdxcXH+bUVFRdq0aZP2799vRYlBMWvWLKWlpWnSpElavHjxCW/zHE7Ky8t1+umnKzMz07+tqKhILpdLGzZssLCykzN//nwNGDBAZ555ph577LGwvOzU0tKiNWvWqLCw0L/NbrersLBQ5eXlFlbWs77++msNHDhQw4YN09VXX62KigqrSwqKbdu2qaqqqsPrmZKSovz8/Ih6PSVp5cqVysjI0MiRI3XTTTdp7969VpfULfX19ZLk/9xbs2aNWltbO7yGo0aN0uDBg3vlNQyLG+UF2+bNm/XUU0/p8ccf92+rqqrS0KFDO+zn+1CrqqpSv379erXGYHjwwQf1ve99T4mJifrHP/6hm2++WQcOHNAvfvELq0vrEVVVVR2CiNTxNQxHv/jFL3TWWWepf//+WrVqlWbPnq3du3drwYIFVpcWkNraWnk8nk5fn40bN1pUVc/Kz8/XkiVLNHLkSO3evVsPPPCAzj//fH3++efq27ev1eX1KN/fU2evZ7j+rXXm4osv1mWXXaahQ4dqy5YtuueeezRlyhSVl5fL4XBYXV6Xeb1e3X777Tr33HM1duxYSe2vYVxc3FH9d731GkbUyMjdd9/daVPmkY9vv9Ht3LlTF198sa644gpdf/31FlXeNd05v+OZM2eOzj33XJ155pm666679Ktf/UqPPfZYEM/gxHr6HMNBIOdcXFysCy64QGeccYZuvPFGPfHEE3rqqafkdrstPgt825QpU3TFFVfojDPOUFFRkZYvX666ujr96U9/sro0dNPPfvYz/ehHP9Lpp5+uSy+9VK+++qo++ugjrVy50urSAjJr1ix9/vnnITVhIaJGRu688079/Oc/P+4+w4YN8/97165duvDCC3XOOecc1ZialZV1VBex7/usrKyeKThAgZ5foPLz8/XQQw/J7XZbdo+CnjzHrKyso2ZnWP0aduZkzjk/P19tbW3avn27Ro4cGYTqgiMtLU0Oh6PTv7FQem16Umpqqk499VRt3rzZ6lJ6nO81q66uVnZ2tn97dXW1xo8fb1FVwTds2DClpaVp8+bNuuiii6wup0tuueUWvfrqq3rvvfc0aNAg//asrCy1tLSorq6uw+hIb/1NRlQYSU9PV3p6epf23blzpy688EJNmDBBL7zwguz2joNEBQUFuvfee9Xa2qrY2FhJ0ptvvqmRI0dadokmkPPrjvXr16tfv36W3iypJ8+xoKBADz/8sGpqapSRkSGp/TVMTk7WmDFjeuQ5esLJnPP69etlt9v95xcu4uLiNGHCBJWVlflncHm9XpWVlemWW26xtrggOXDggLZs2aJrrrnG6lJ63NChQ5WVlaWysjJ/+HC5XPrwww9POKMvnH3zzTfau3dvhwAWqowxuvXWW/Xyyy9r5cqVR7UhTJgwQbGxsSorK9Pll18uSdq0aZMqKipUUFDQKwVGnW+++caMGDHCXHTRReabb74xu3fv9j986urqTGZmprnmmmvM559/bpYuXWoSExPN73//ewsr77odO3aYdevWmQceeMAkJSWZdevWmXXr1pmGhgZjjDF/+9vfzLPPPms+++wz8/XXX5unn37aJCYmmrlz51pceded6Bzb2trM2LFjzeTJk8369evNihUrTHp6upk9e7bFlXfPqlWrzJNPPmnWr19vtmzZYv7nf/7HpKenm+nTp1tdWrcsXbrUOJ1Os2TJEvPFF1+YG264waSmpnaY/RTO7rzzTrNy5Uqzbds288EHH5jCwkKTlpZmampqrC6tWxoaGvx/Y5LMggULzLp168yOHTuMMcbMnz/fpKammr/+9a/m008/NT/+8Y/N0KFDzcGDBy2uvOuOd44NDQ3ml7/8pSkvLzfbtm0zb731ljnrrLPMKaecYpqbm60u/YRuuukmk5KSYlauXNnhM6+pqcm/z4033mgGDx5s3n77bfPxxx+bgoICU1BQ0Cv1RWUYeeGFF4ykTh9H+uSTT8x5551nnE6nycnJMfPnz7eo4sDNmDGj0/N75513jDHGvP7662b8+PEmKSnJ9OnTx4wbN86UlpYaj8djbeEBONE5GmPM9u3bzZQpU0xCQoJJS0szd955p2ltbbWu6JOwZs0ak5+fb1JSUkx8fLwZPXq0eeSRR8LijfBYnnrqKTN48GATFxdnJk2aZP71r39ZXVKPmTZtmsnOzjZxcXEmJyfHTJs2zWzevNnqsrrtnXfe6fTvbcaMGcaY9um9c+bMMZmZmcbpdJqLLrrIbNq0ydqiA3S8c2xqajKTJ0826enpJjY21gwZMsRcf/31YROej/WZ98ILL/j3OXjwoLn55ptNv379TGJiovnJT37S4T/Sg8l2qEgAAABLRNRsGgAAEH4IIwAAwFKEEQAAYCnCCAAAsBRhBAAAWIowAgAALEUYAQAAliKMAAAASxFGAACApQgjAADAUoQRAABgKcIIAACw1P8Hn97TFmSu4G4AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "x = torch.linspace(-20, 20, 100)\n",
        "\n",
        "def sigm(x):\n",
        "    return 1/(1 + torch.exp(-x))\n",
        "\n",
        "y = sigm(x)\n",
        "\n",
        "plt.plot(x, y);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7bhtyQIm1DLf"
      },
      "source": [
        "### Задание 3:\n",
        "\n",
        "Реализуйте на PyTorch среднюю квадратичную ошибку. \n",
        "\n",
        "$$ \n",
        "MSE(\\hat y, y) = \\frac{1}{n} \\cdot \\sum_{i=1}^n (\\hat y - y)^2\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jqUs1uS41DLl"
      },
      "outputs": [],
      "source": [
        "y_pred = torch.tensor([1, 2, 3, 4, 5], dtype=torch.float)\n",
        "y_test = torch.tensor([-1, 3, 4, 6, -5], dtype=torch.float)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7Qzufiei1DLl",
        "outputId": "90cb5603-e3d6-4113-be19-2fb552e7094f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "22.0"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "def mse(y_true, y_predict):\n",
        "    return torch.mean((y_true - y_pred)**2)\n",
        "\n",
        "z = mse(y_test, y_pred)\n",
        "z.item()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uSmAcse31DLm"
      },
      "source": [
        "### Задание 4:\n",
        "\n",
        "Что будет в переменной `x` в результате выполнения следующего кода? Почему?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aUjoTNIH1DLm"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "y = x\n",
        "y[2] = torch.ones(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sAfWqwef1DLm",
        "outputId": "888cbf1d-d237-44d8-c865-9a21dcfc20f0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1, 2, 3],\n",
              "        [4, 5, 6],\n",
              "        [1, 1, 1]])"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ],
      "source": [
        "y"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cFJHY8_C1DLm",
        "outputId": "0e49a167-9839-4899-fca5-74729fd5c34e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1, 2, 3],\n",
              "        [4, 5, 6],\n",
              "        [1, 1, 1]])"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "albVtIP61DLn",
        "outputId": "16b6a678-b4f5-4a71-8949-ca5ef4ce86a0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1, 2, 3],\n",
              "        [4, 5, 6],\n",
              "        [7, 8, 9]])"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "y = x.clone()\n",
        "y[2] = torch.ones(3)\n",
        "\n",
        "x"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n64vWj2M1DLn"
      },
      "source": [
        "------------------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QYYXrP2r1DLn"
      },
      "source": [
        "# 2. Производные да градиенты\n",
        "\n",
        "Если в PyTorch те же самые операции, что и в numpy, то на кой чёрт он нам нужен? В отличие от numpy, в PyTorch есть возможность при создании тензора указывать нужно ли считать по нему градиент или нет, с помощью параметра `requires_grad`. \n",
        "\n",
        "Когда `requires_grad=True` мы сообщаем фреймворку, о том, что мы хотим следить за всеми тензорами, которые получаются из созданного. Иными словами, у любого тензора, у которого указан данный параметр, будет доступ к цепочке операций и преобразований совершенными с ними. Если эти функции дифференцируемые, то у тензора появляется параметр `.grad`, в котором хранится значение градиента.\n",
        "\n",
        "Он проходит по всем операциям, которые фигурируют в графе вычислений, и применяет к ним chain rule:\n",
        "\n",
        "$$ {\\partial f(g(x)) \\over \\partial x} = {\\partial f(g(x)) \\over \\partial g(x)}\\cdot {\\partial g(x) \\over \\partial x} $$\n",
        "\n",
        "Давайте попробуем!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QvNQb3f71DLo"
      },
      "outputs": [],
      "source": [
        "x = torch.tensor(\n",
        "    [0.3, 1], requires_grad=True\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jjQ7ARYS1DLo"
      },
      "outputs": [],
      "source": [
        "y = torch.sum(x)\n",
        "z = y ** 2 "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-FE8BFx71DLo"
      },
      "source": [
        "Если к результирующему тензору применить метод `.backward()`, то фреймворк посчитает по цепочке градиенты для всех тензоров, у которых `requires_grad=True.`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5AxqUB-g1DLo",
        "outputId": "61d577f2-1272-4270-bb88-2744922d8e69",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "None <- gradient\n"
          ]
        }
      ],
      "source": [
        "# пока градиента нет\n",
        "print(x.grad, '<- gradient')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nu8Muaem1DLo",
        "outputId": "f17baa23-e007-46e7-acc3-34628913c748",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([2.6000, 2.6000]) <- gradient\n"
          ]
        }
      ],
      "source": [
        "z.backward() # считаем градиент\n",
        "print(x.grad, '<- gradient')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TkuCmBgt1DLp"
      },
      "source": [
        "**Важно:** если вы вычисляете градиент дважды, градиенты будут суммироваться в тензорах, поэтому полезно обнулять градиенты между итерациями.\n",
        "\n",
        "Обычно мы обучаем модель не по одному объекту, а на большой выборке. Вся выборка обычно в память не влезает. Мы прогоняем через нейросеть данные батчами. PyTorch позволяет накапливать градиенты по батчам не помещая в память все данные. Иногда это оказывается полезным."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M3ZAy91-1DLp",
        "outputId": "801a149f-7536-4f87-a1d9-c59371841c53",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<PowBackward0 object at 0x7f93c16672e0>\n",
            "<SumBackward0 object at 0x7f93c1667790>\n",
            "<AccumulateGrad object at 0x7f93c16672e0>\n"
          ]
        }
      ],
      "source": [
        "# можно посмотреть на порядок действий\n",
        "print(z.grad_fn)\n",
        "print(z.grad_fn.next_functions[0][0])\n",
        "print(z.grad_fn.next_functions[0][0].next_functions[0][0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vO2XTPxn1DLp"
      },
      "source": [
        "### Задание 5:\n",
        "\n",
        "Реализуйте расчёт градиента для функции \n",
        "\n",
        "$$\n",
        "f(w) = \\prod_{i,j} \\ln(\\ln(w_{ij} + 7) \n",
        "$$\n",
        "\n",
        "в точке `w = [[5,10], [1,2]]`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JrAeFAEy1DLq",
        "outputId": "79b5a178-272f-446b-c9b4-d0cf1e48420e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0.0201, 0.0109],\n",
            "        [0.0449, 0.0351]])\n"
          ]
        }
      ],
      "source": [
        "# ваш код\n",
        "w =  torch.tensor([[5.,10.],[1.,2.]], requires_grad=True)\n",
        "\n",
        "function =  torch.prod(torch.log(torch.log(w + 7)))\n",
        "function.backward()\n",
        "\n",
        "print(w.grad)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LsTi1EIy1DLs"
      },
      "source": [
        "------------------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RX3XP2Mg1DLs"
      },
      "source": [
        "# 3. Оптимизация\n",
        "\n",
        "Итак, PyTorch умеет искать производные, осталось научиться применять его для оптимизации. \n",
        "\n",
        "Давайте обучим линейную регрессию."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-PJsDBi-1DLs",
        "outputId": "57a82188-5c0a-4b48-c6dc-74fa972f5448",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABd0ElEQVR4nO3dfXhU5Zk/8O8k5B0yIUQzQXmJvMcICCuQorbGUFC0VNnfr77tqmWxIvRSsC3iVaoudlG7W2pX1NZa2F0Fqq6UIpb98SYsmIACESNKIQahkkAJZBICeTFzfn/EM8xMzstzzpxz5szM93NduVqSyZlnZuI89zz3/dyPR5IkCUREREQOSYn1AIiIiCi5MPggIiIiRzH4ICIiIkcx+CAiIiJHMfggIiIiRzH4ICIiIkcx+CAiIiJHMfggIiIiR/WK9QAiBQIBnDhxAn369IHH44n1cIiIiEiAJEloaWlB//79kZKivbbhuuDjxIkTGDBgQKyHQURERCYcP34cl19+ueZtXBd89OnTB0D34HNzc2M8GiIiIhLR3NyMAQMGBOdxLa4LPuRUS25uLoMPIiKiOCNSMsGCUyIiInIUgw8iIiJyFIMPIiIichSDDyIiInIUgw8iIiJyFIMPIiIichSDDyIiInIUgw8iIiJylOuajNmlKyBhT90ZNPgv4ExrB/J7Z8CXm4kJxflITVFviNIVkPD+kdP4731/xfmOr/B3A/MxwtcHe744gxNnL6B/Xha+MaQA1xTn44O6M6j8/DQAD8qG9MOkK/r1uLbeOOSfn2ppw6V99Mdn5XMT7X3qXScWj42IiNzHI0mSJHrjJ598Ek899VTY90aMGIHPPvsMANDW1oZHH30Ua9asQXt7O6ZOnYoXX3wRhYWFwgNqbm6G1+uF3++3rMPpxpp6PLX+IOr9bT1+VuTNxBO3lmBaaZHi7y144yOc7+gydb952Wl45vargtfWG8d3xhThTx/Vh/1ca3xWUBqTmfvUu45V90NERO5kZP42HHy89dZb2Lx5c/B7vXr1QkFBAQBgzpw52LBhA1auXAmv14t58+YhJSUFu3btsmXwIjbW1GPOa/ug9SA9AF66Z1zYJLixph4PvrYv6vsHgJfvGQcAuuNQGxsUxmcFtefG6H3qXeeB64vx2x11Ud8PERG5l5H523DapVevXvD5fD2+7/f78eqrr2LVqlUoLy8HAKxYsQKjRo1CVVUVJk2aZPSuotYVkPDU+oNCE/5T6w9iSokvmPp4Yt0nlo3jqfUHIUmS4cADACR0T9Kh47OC1nNj5D5FrvPK//YMPIzeDxERJQ7DBaeHDx9G//79ccUVV+Duu+/GsWPHAAB79+5FZ2cnKioqgrcdOXIkBg4ciMrKStXrtbe3o7m5OezLKnvqziimOCJJAOr9bdhTdyb4eydb2i0bR72/DQ3N5q8XOT4r6D03ovcpcp2ARtRlx2MjIiJ3MxR8TJw4EStXrsTGjRvx0ksvoa6uDtdddx1aWlrQ0NCA9PR05OXlhf1OYWEhGhoaVK+5dOlSeL3e4NeAAQNMPRAlp1r0Aw+l2xv9PadYOS7Ra+ndzqoxufU5JyIi6xlKu9x0003B/z969GhMnDgRgwYNwhtvvIGsrCxTA1i0aBEWLFgQ/Hdzc7NlAcilfTJN3d7o7znFynGJXkvvdlaNya3PORERWS+qPh95eXkYPnw4jhw5Ap/Ph46ODjQ1NYXd5uTJk4o1IrKMjAzk5uaGfVllQnE+irz6k5oH3TsvJhTnB3+vsE+GZeMo8mbCl2v+epHjs4L83KhVWYjep8h1tEo57HhsRETkblEFH+fOnUNtbS2Kioowfvx4pKWlYcuWLcGfHzp0CMeOHUNZWVnUAzUjNcWDJ24tUZ0YQz1xa0mw4DE1xYOnZlxp2TieuLUET37nSqFxRJJ/J3R8VpCfm9D7MHOfIteZfV0xPFHeDxERJQ5DwcePfvQjbN++HUePHsX777+P2267Dampqbjzzjvh9Xoxa9YsLFiwANu2bcPevXtx//33o6ysLCY7XWTTSovw0j3jVFdAiryZils9p5UW4eV7xiE7PdX0fffNTsPLX19bZBw/uL64x899KuPrCkiorG3EuuovUVnbiC6tqk4V8ph8gvdp9jqLbi6x5H6IiCgxGOrzcccdd2DHjh1obGzEJZdcgmuvvRY///nPMWTIEAAXm4ytXr06rMmYVtolkh1NxoDE6nBqdcMudjglIqJo2dZkzAl2BR+JwqrGYERERFYyMn/zYDmLWZEO0bq2VkMvoLthl5X3SUREZLWkOVjOCUrpkLysNNw/eTDmlQ+LOsVgpDFY2ZB+Ud0XERGRXbjyYRE5HRIZHDRd6MSyzYcx/ulN2FhTH9V9WNUYjIiIKJYYfFhA5AyZpvOdmPPavqgCEKsagxEREcUSgw8LGDlDJpqaDKsagxEREcUSgw8LGElzRHOImlWNwYiIiGKJwYcFjKY5oqnJsKoxGBERUaxwt4sF5HSISOoFiL4mY1ppEaaU+Niwi4iI4hKDDwvI6ZAHX9uneTsPulcorKjJSE3xcDstERHFJaZdLCKfBZOXnab4c9ZkEBERdWPwYaFppUXY+9MpmF8xHHlZ4UEIazKIiIi68WwXm/AQNSIiSiZG5m/WfNiENRlERETKGHzYgKseRERE6hh8WEzpcLkibyaeuLWE9R5ERERgwaml1A6Xa/C3RX2uCxERUaJg8GERrcPl5O9Fc64LERFRomDwYRG9w+UkRHeuCxERUaJgzYcgvSJS0fNaojnXhYiIKBEw+BAgUkQqel5LtOe6EBERxTumXXSIFpHKh8upbaj1oDtgseJcFyIionjG4EODkSJS+XA5AD0CEJ7rQkREdBGDDw1Gi0inlRbhpXvGwecNT63wXBciIqKLWPOhwWgRaVdAgjcrHT+ZNhJnzrUjPycdPm8WO5wSERGFYPChwUgRqVZRKgMPIiKii5h20SBaRHq2tZ2dTYmIiAQx+NAgUkS6eHoJlmz4lJ1NiYiIBDH40KFXRNo3J12oKHXZpkOorG1kEEJEREnPI0mSq2bD5uZmeL1e+P1+5Obmxno4QWodTtdVf4mH11QLX4cn3BIRUSIyMn+z4FRQaooHZUP69fi+0Y6lch0It94SEVGyYtolSnpFqZFYB0JERMmOwUeUtIpS1fCEWyIiSmYMPiygVpSqhyfcEhFRMmLNhwFqRadAdwAypcSHPXVnsOvIabyw7Yju9XjCLRERJSMGH4K0OpjKhaNyUeqE4nz8976/osHfptj/w4Purbo84ZaIiJIR0y4CNtbUa3YwffdAPSprG7Gu+ktU1jYCAE+4JSIiUsE+Hzq6AhKufXarZiOxFA8QunFFXhEBoLtaQkRElAjY58NCe+rOaAYeQHjgAYT38ti5sFy1ToSIiCgZMfjQYWZHioTu9MpT6w9iSolPsTkZERFRsmLNhw6zO1LYy4OIiEgZVz50TCjOhy83Aw3N7aZ+3829PLS2DhMREdmFwYeO1BQP7pwwEMs2Hzb1+27t5SGydZiIiMgOTLsIGFyQY/h3POiezN3Yy0Nv6/DGmvoYjYyIiJIBgw8BZlcv3NjLoysg4an1BxWbn/HQOyIicgKDDwFGT64FgAeuL3Zl+kJv6zALZYmIyG4MPgQYPbnWA+BPH9W7cvVAtADWzYWyREQU3xh8CJpWWoTld41D35x03du6efVANIXk1kJZIiKKfww+BG2sqceSDQdxprVD+HfcuHqgl0Jyc6EsERElBgYfAtR2h+hx4+qBVgqJh94REZETGHzo0NodosbtqwfTSovw0j3j4POGB0c+byZeumecKwtliYgocbDJWAiljp8iB8uFipfVg2mlRZhS4mOHUyIichyDj6+pdfy8qdRn6Dq+OOoSmpri4aF3RETkOAYfuFjTEZlaafC34fe7jgpdY94NQzF5aAFXD4iIiHQkffCh1/HTA8DjAdRadnjQvdoxf8pw4aCDB7oREVEyS/rgQ6Tjp/R14OEBwoIUM/UdPNCNiIiSXdLvdhHtxTFr8uCod4fwQDciIiKufAj34qgo8eHx6SWm0yUi6Z2n1h/ElBIfUzBERJTQkj74kDt+NvjbFAMDuaZDDjTM7g4xcqAbd6AQEVEiS/q0i1MdP3mgGxERUbekDz4AZzp+8kA3IiKibkmfdpHZ3fHTSHqHiIgokTH4CGFnx085vTPntX2WbNklIiKKV0y7OIgHuhEREXHlw3E80I2IiJJdVCsfzzzzDDweDx555JHg99ra2jB37lz069cPvXv3xsyZM3Hy5Mlox5lQ5PTOjLGXoWxIPwYeRESUVEwHHx988AF+85vfYPTo0WHfnz9/PtavX48333wT27dvx4kTJ3D77bdHPVCyV1dAQmVtI9ZVf4nK2kZ0qR1mQ0REFCVTaZdz587h7rvvxiuvvIKnn346+H2/349XX30Vq1atQnl5OQBgxYoVGDVqFKqqqjBp0iRrRk2W4nkzRETkJFMrH3PnzsX06dNRUVER9v29e/eis7Mz7PsjR47EwIEDUVlZqXit9vZ2NDc3h32Rc3jeDBEROc1w8LFmzRrs27cPS5cu7fGzhoYGpKenIy8vL+z7hYWFaGhoULze0qVL4fV6g18DBgwwOiQySe+8GaD7vBmmYIiIyEqGgo/jx4/j4Ycfxuuvv47MTGs6cS5atAh+vz/4dfz4cUuuS/qMnDdDRERkFUPBx969e3Hq1CmMGzcOvXr1Qq9evbB9+3b8+te/Rq9evVBYWIiOjg40NTWF/d7Jkyfh8/kUr5mRkYHc3NywL3IGz5shIqJYMFRweuONN+Ljjz8O+97999+PkSNHYuHChRgwYADS0tKwZcsWzJw5EwBw6NAhHDt2DGVlZdaNmizB82aIiCgWDAUfffr0QWlpadj3cnJy0K9fv+D3Z82ahQULFiA/Px+5ubn44Q9/iLKyMu50cSGeN0NERLFgeYfTZcuWISUlBTNnzkR7ezumTp2KF1980eq7iStdAcmVHU153gwREcWCR5IkV21laG5uhtfrhd/vT4j6j3jooREPYyQiInczMn8z+LCR3EMj8gmW1xHcdJicW1dniIgoPhiZv3mwnE30emh40N1DY0qJzxWTvHzeDBERkd2iOliO1LGHBhERkTIGHzZhDw0iIiJlTLvYRLQ3RkFOBiprG1lrQURESYPBh01Eemh4s9Pw6JsfoaGZu0yIiCh5MO1iQFdAQmVtI9ZVf4nK2kbNA9fkHhrAxd0tMrmnRtP5zrDAA+BpskRElPi48iHITC+MaaVFeOmecT1+rzA3A21fBdB0vrPH77hxJwwREZGV2OdDgF6/juV3XY2+ORmqdRuRPTQCkoS7f7db935Xz57E7a9ERBQX2OfDQnr9OgBg3ur9CM3ARK6IRPbQWFf9pdB9R6ZkiIiIEgFrPnTo9esAgMjSD726DdGdMEve+YS1H0RElHAYfOgw04dDjkWeWn9QsShV3gmjV81xprUTD7L4lIiIEgyDDx2iqxSRtDqYau2EUfLY2x9r7qwhIiKKJww+dIiuUqhRWzmRd8L0zUnXvUbT+U68sPWIyREQERG5C4MPHUZXKSJprZxMKy3C4umjhK6z4v26mK1+GOlvQkREpIe7XXR0BSR4s9Jx/+TB+GP1CZxp7Qj+LMXTs9hU5gHg83Zvu9Xi82YJjaPpfCf21J1xfOutmf4mREREWhh8aFCaePNz0nDb2MtQUeLD2dYOzF21DwDCtuLKKyRP3Fqi2yRsQnE+8rLS0HShZ8OxSE4fQqfW30TezfPSPeMYgBARkWFMu6iQJ97IbbZnWzvx+11H4b/QgZtHd9dt+LzhqRWfN1N4Yk5N8eD+yYOFxlTQO0N4/EqMpE9E+puo7eaJ9r6JiCixscOpgq6AhGuf3ara30NOqexcWI7UFE+PDqZGT6btCkgY//QmxXbroXy5GXjyO1eaWm0wkj7pCkhYuasOSzZ8qntdkS6sTN0QESU+I/M3Vz4U6DUWi9xGK3cwnTH2MpQN6Wf4PJbUFA+euf0q3YLWk83tpg6dU1vFUWqGtrGmHtc+u1Uo8AD0U0FG7puIiJIDgw8ForUVVtZgyFtvfbnqu2PMpjtE0ydqgYIWrd08VqduiIgoMTD4UCDaWMxsAzI100qL8G//Z4zmbbSalykRXcWp+rxRNVBQ4kF36kRrN4/RFSQiIkoO3O2iQG4s1uBvU5yMRbfRqgmtESnonQFIwOnWdlzaJ9PyVRfR21XWNgqveIju5onFChIREbkfgw8FcmOxOa/tgwfmt9EqUSq+DJUv0PEUsGN1Rjz14RMsFo3VChIREbkb0y4qgjUYUWyjjSRSU3E2pImZEpF0B3Bxa2uD/wLyc9JVi1nl65VdUaA9+K8tnj4KOxeWCz1+vdb0oo+FiIgSC1c+NEwrLcKUEl9U22hlWsWXobR+Lrrqore6onS9SUP6CaWa7ptcLPz47VxBIiKi+JWUKx9GGl5Fu41Wpld8qaRvdnhsWJibobvqYmTHSugqjtYZNtEECnasIBERUXxLupWPWDW8MlNU2f5VZFCkPfGLrK7k56Rh8S1XwpfbcxVHDhQinx/RGg81Vq4gERFR/EuqDqdqZ5XIU6DVn8RDd7WcbmkXbtylRm+clbWNuPOVKt3r6HUljbZjKxERJR8j83fSrHzoNbzyoLvh1ZQSnyUTrWjdhRF647Rqa6ucaiIiIrJD0tR8ONnwykynUFFa4xTdsnr0dKvFoyIiIhKXNMGHUw2vRHe1REtpnBOK8zXbs8tW7znGluZERBQzSRN8ONXwysyuFjOUxpma4sGdEwbq/m5DcztbmhMRUcwkTfDhVMMru1uF641zcEG20HXY0pyIiGIlaYIPu/pYRLKiVXjf7DQA5sbp9pbmRnqsEBFRYkqa3S6AfX0sQukdSgcAKR4gdM715WbgzgkDMbggJ7i1ddPBBlPjtPtQvGjEqscKERG5S1L1+ZDZ3cdC3u0CKLcUX37X1eibk6F7/2bHqXf/segs6nSPFSIicpaR+Tspgw+jzAQBsfyU3xWQ8MLWw1ix6yiaLnQ6fv9K47n22a2qhbjyaszOheVsZkZEFKfYZMxCZoOIaaVFKB9ZiP+qPIovzpzHoPxs/EPZYKT3srfMRmm8eVlpuH9yMeaVD43J5G6kxwqbmxERJT4GHxrUUgUN/jbMeW2fZqpAKQj43c46y1ceugISqj5vRGVtI2r/1oI/15zscRv/hU78avNfMMLXOyapDad6rBARUXxg8KEimnbsakFLvb8ND762Dy9HBC3R1HY89vbHaDrfqXm7aNvHR1sj4/YdOERE5CwGHyrMpgpEOpz+6M2PcKEzAF9uJs62dmDJBuNpnY019Xjw66JSEWZTG1bUrrh5Bw4RETkvafp8GGU2VSDS4fRcexfm/6Ead75ShYdW9TwDRk7rbKypV/z9roCEJ/90UGh8euPVonZGjd74IjnVY4WIiOIDgw8VZlMFVtQtyKsDT60/iK6A1KMxV1VtIxqazd2P6OPSSzuFjk+E3GPF5w2/f583k9tsiYiSDNMuKsymCqyqW5DTJC9sPYI1HxwLW33ITk81fD2jqQ07dqhMKy3ClBKfrT1WiIjI/Rh8qJBTBXNe2wcPlJt1yamEytrG4GQ6flBf5GWlhfXXiMayzX/p8b3zHV2GrmEmtWHXDpXUFA+30xIRJTkGHxr02rED6NE8q8ibieuG9cP6Aw2Oj1eNmfbxBTkZQrfjDhUiIjKKwYcOtVTBpoMNqj1A1h9oQE56KloNrlBYbdbkwago8RlObWysqdctaOUOFSIiMosFpyaI9ABJ65XSY2eHqGgrIPKy0/DyPePw+PTu1Zl3DpwQPkFW3uGiVdDKHSpERBQNrnzoUOpzkZ+TjjOtHaq/IwFoOt+J+RXDexSLivB5M3HHNQOwbPNh4d/5dkkhhhf2RtkVBZg0pB82HWxQTAlppV9EepQAQGFuBp78zpXcoUJERKYw+NCg1qlUK/AINbggGzsXlgdTNkdPt2L1nmNoaG4P3qbIm4nF00f1OOW2KyDhPyq/EL6v+ycXBws5zbaFF+lRAgD/9n/HYvLQAqFxERERRWLwoUJ0FUDL6ZZ2vHPgBC7tk4lbRvdHaooH88qH6W41lVdbRAKPyNqLjq8CeHztx6bawovuXDl9rl3/RkRERCoYfKgQXQVQ4/EASzZ8Gvx3aMpDXqFQOjNFrZBV8T6+/l+59mJjTT0eX1uDM63q23y1+nPwDBYiInICgw8V0XYqlSKih8iUh1ItiS83E21fdQmvtoRuoVVLtahRenw8g4WIiJzA4EOF6Kf7/Jw0zZUGWWjKIxAAHlrV81A40Zbp824YgslDLwmmbMykiJQen2hjNe5wISKiaHCrrQp5FUBtmvWgO5Wya+GNyM9JE7qmnPL4yX8fiGpswwr7oGxIv2AQYCRFJI9bbfUi3s5giTz3RvSsGSIiih2ufKgQXQWoPt4ktPIR6lz7V1GNLdrD7PRWL+LlDBal1JXedmIiIoo9rnxoEFkFsOIUW1FqqxZGUkSiqxfyGSwzxl4WtsriFnKNS+SKj1xbs7GmPkYjIyIiPVz50KG3CuDUzg+tmgu9QlEA6JeTjspFNyK9V/zHmyIdZtW2E8ea0g4nt42RiMhuDD4EaJ3EKjLxWyE/Jx0/v61UcdVCJEX089tKEyLwAPRrXLS2E8cS00RERN0SYzaKIXniB6I/k0XLT6eP0pyg4q1QNBqiqS4nU2J6mCYiIrqIKx8WkCf+yE+1uZm90NwWXXGpzOfNEhqHk4WisUohxFsztHhOExER2YHBh0WUJv4G/wXMf+OjqK5rtLGXWopIJFAwEkzEMoUQb83Q4jVNRERkFwYfFoqc+CtrG6O6nlWNvUQCBSPBhNmD66wSb83Q4jFNRERkJ0M1Hy+99BJGjx6N3Nxc5ObmoqysDH/+85+DP29ra8PcuXPRr18/9O7dGzNnzsTJkyctH3S80GtUpseKeg2RWgMj9Qh6KQSgO4Vgd7MvrRqX5XddDW9Wumsaj8VbmoiIyG4eSYo8hUTd+vXrkZqaimHDhkGSJPzHf/wHfvGLX2D//v248sorMWfOHGzYsAErV66E1+vFvHnzkJKSgl27dgkPqLm5GV6vF36/H7m5uaYelJtsrKnHg6/1bKWuZ/H0UbhvcnFUn967AhKufXar5pJ/kTcTkiShoVn5pFo5hbFzYTlSUzyorG3Ena9U6d736tmTHEkhRKaKzrZ2YMkGd+0okV8HvTSR/BwTEcUjI/O3oZWPW2+9FTfffDOGDRuG4cOH4+c//zl69+6Nqqoq+P1+vPrqq/jlL3+J8vJyjB8/HitWrMD777+Pqir9ySpe6bX3nlZahPkVw4SvJzcSizbwAMTartf721QDDyC8HgFwXwohtBma/0IH5q6ydkeJFe3btXZEuTFNRERkN9M1H11dXXjzzTfR2tqKsrIy7N27F52dnaioqAjeZuTIkRg4cCAqKysxadIkxeu0t7ejvf3i5Nfc3Gx2SI4TrZMYXJBj6LryRBTtbhLRg+pEyMGEW1MIduwosbKoVm1HlI99PogoCRkOPj7++GOUlZWhra0NvXv3xtq1a1FSUoLq6mqkp6cjLy8v7PaFhYVoaGhQvd7SpUvx1FNPGR54rBkpujQyET9SMRzTSossmfjOnFNf0TBKfgxu3Wli9Y4SO4pq4+XMHCIiuxluMjZixAhUV1dj9+7dmDNnDu69914cPHjQ9AAWLVoEv98f/Dp+/LjpaznFaNHlhOJ85GWJnXw7uCDbsoZU+TnpQrfzZvbSPb1XDibcmkKwMh1kZ1Gt28/MISJyguHgIz09HUOHDsX48eOxdOlSjBkzBs8//zx8Ph86OjrQ1NQUdvuTJ0/C5/OpXi8jIyO4e0b+cjsjn7KB7gnn/snFQtcu6J1h2cQn0pgMAKaU+IKpiVDyvxdPH4U9dWeCdQ9TSnyu66ZqZTrI6OtLRETGRN3nIxAIoL29HePHj0daWhq2bNmCmTNnAgAOHTqEY8eOoaysLOqBuomZT9nDLu3doydFpBQP8IGF6QM5RaJXdPrWvr8iL7t7ZabpfGfw+z5vJr4zpghLNnyqmP7ZubDcNSkEK9NBbiuqJSJKNIaCj0WLFuGmm27CwIED0dLSglWrVuG9997D//zP/8Dr9WLWrFlYsGAB8vPzkZubix/+8IcoKytTLTaNV0Y/ZW+sqcfcVT3rByIFJOBXWw4LXVtk4gttxqV33/6vg475FcMxuCA7uG1VadxONRMzwsrGY24tqiUiShSG0i6nTp3CP/7jP2LEiBG48cYb8cEHH+B//ud/MGXKFADAsmXLcMstt2DmzJm4/vrr4fP58Pbbb9sy8FjSax4WWiehVT8QjdMt7WGpF7UtofIuiyKv9kQpX2nNB8dwy+j+mFCcjyUb7Kl7sGL7qhKrDtcz8voSEZFxhpqMOSFemoy9e+AEHlq1v8f35QlLnuxEm3KZIac/AOjujOkKSFi5qw5LNnyqe93Vs7tXquxoJubEmTBWHHgnF/0Cyqsoblr1ISJyAyPzN892MWFjTb3qJB7Zt8HOuoAGf5tq91Q5NfJISBolv3eG0HWNjDn0tnqTvlNnwqgdrmcE+3IQEdmHwYdBahOobPH08InJzroArSUr+WfLNv8l+L38HLHtvqdb2jHSJ7bqFFrXorWiEY/HyrMvBxGRPRh8GKBXv+EBsGTDQUwtvTiB6u3CcNKZ1k79GwFYsuFT+HIzkZedBv/5Tt3dI3orGsvvGod6/4W4PFbeilUUIiIKZ7jPRzIz0/9BqylXLOmN5WRzG5q+Djy0mokB0FzRkADMW71PqNYEsDdNZVehKxERGcOVDwPM9n9Qqx+Ipb456TjT2qH6cznoyMtOQ0avlLDD50LrHiprG3Ufk5E53q40lROFrkREJIbBhwHR9H8IrR/YdLABv9911OLRGbN4+iicae3QXJGQAJw934nX/2kiUjwexboHq1Yq7DwTxqlCVyIiEsPgw4Bou2impngwoTgfC96otmQ8oc209LqnRjrT2oEvzpwXuu3pc+2YMfYyxZ9ZtVIhAbjjmgGWXCtUPBa6JiIrtj8TUeJg8GGAFV009epGjPBp9PnQkuKBcA0GoB1gWFlQu2zzYaz54LilqRCrT7sl45jyIqJILDg1KNoumlakKf6xbBBWz56EnQvLMa20CNNKi7BzYTlWz56E5+8Yi/kVwwCoF5WK1mCIdPK0uqDW6Mm9enhOS2xZdUIzESUWrnyYEE3/ByvSFDeVFvX4lB65JXSEr0+PT5spHmOBByB2HopaQa2R+5NZnQrhOS3OiUytjB/UlykvIlLE4MMks/0fok1T5OekYfygvrq3iwyQTre0G0q1GO3kqRSQVX3eiOcFD8oLZWUqxMrTbkmdUmolPydNs7cMU15EyYtpF4dppSk8Kv8/1JnWTnzzF9uElqvlAGnG2MtQ0Eestbps8fRRhvPxofdXNqQfrrgkx9DvR7IiFSLyfIuedkvK1FIrok3tmPIiSj4MPmJAq27k5XvG4WWFn4Uyky83klbo7tT6adRNuKJNZViVCrHqtFvqyYpTm5nyIko+TLvESGiaoqG5DWfOtSM/Jx3erHRMKM5H+chCTFq6RbERmPxG//jaj3GhMwBfbs+aE6X8u2i6x6rlcLMpJjtSITynxR7R7N5iyosoeTH4iKHUFA/8Fzrw3MbPemxDvOOaAZodSIHuZe35f6gO/o5co6G2tfE7Y4rw2x11wuPbdeR0VBO01tZkNXamQnhOi/XMpkyY8iJKbh5Jklx1wEVzczO8Xi/8fj9yc8VOVo1Xap03jTYMk38HAB64vhi/3VGneE3552/u/VI3sJHp9WMQaR61saYeT/7pk7AW7Wbvz62StYlWZW0j7nylSvd2+RHt/OP1dSYidUbmbwYfMdIVkHDts1stPevFA8Cjsb1VXube+ui3MPnZLUIFgfL0qVQbYaR51K4jp3H373br3t/rsyZi8rAC3du5STI30ZL/jvV2E23/8Q3Y+8XZpAvOiJKJkfmbBacxYmWnU5kE7b4aci1H9fEm/MttV3UHKwLXBLr7MYQWoBptHnX6nP6qBwCcbhW7nVskexMt0d1E6b1SwnZCMfAgSm4MPmLESK7c6rfpUy1tqjtAlIQWoAL656UAPYOVRGz2ZeZ5SETcTURERrHgNEZEJ9n5FcOx5oNjlq6SFOR09/yQd4As23QIL2yr1f09OWAyc15KIjb74rkxF3E3EREZwZWPGJEnY7W3ZvlclXnlQ4Pntiz7v2OQn5Me/UpIyAVSUzyYPPQSoV+TAyYz56UkYrMvnhsTLrLJXDy9lkTkLAYfMWJkMpbf1G8bdzn+5bZSzd8REVl/IRoIyasSZlMoibY8n4ipJCIiJzD4iCEzk7HW78yvGC50v5GTodFVCaPBSuT4Q0/gDT2dN95E8zwQESUzbrV1AaM9IroCEqpqG1H5+WkA3asik67orikQ2fa4c2G54vWNbBmVd3kA4T1JtLbmJiI+D0RE3djnI4HpBQhqjcuA7glRbzI0EggZ7W+RqI24krnPBxGRjMFHAlCaqDcdbFANLADgplIfhlySg7rTrfhzTUNYz48UDzD7umL8ZNoo3QDASJAgett3D9Tjp+tqwrpc5uek47tj+2NKiS/uA5FEDayIiEQx+IhzSp+kfbmZaPuqC03nxY4pjyS3bM/LTgu7RuQndDs+xS999yB+o3OmDFcKiIjiG4MPh1n5qVcrbWIHeZSPVAyH/0IHfr/rqOptzNQvvHvgBB5atV/49t+fPDghVkKIiJINgw8HWblS0BWQMPmZLUIHsDlNr1hVSVdAwt89vQlnTazWcCWEiCi+8GwXh1h9rscLW4+4MvAAerZYF7Gn7oypwANw99koXQEJlbWNWFf9JSprGxO+fToRkdXYXt0kvXM9POg+12NKiU9opWBjTT2Wbf6L1cO0nGi3zq6AhF1H/mb6frSeQ700l53Fn9GsdLEolYioG4MPk6w810MOZOKBSLdOpQnaDKXnUGSrsV3bXtXqceRVGq2aGG7HJSK6iGkXk6w810MvkHGLvtlput061VJR0ZCfQ70019J3D9p2vH00J9hanZ4jIop3DD5MsvJcj3g5eCxyWo2sfej4KqA6QUfj0j6ZupO/BOCV/62z7Xh7IytdoaIJWoiIEhXTLiZZeUT80dOtlo/PDk3nO4MpEKU0Qn5OelgTsWiFPociq0Na87ccHKzcVYeCPhmGay7MrnRZmZ4jIkoUDD5Mkg9jm/PavmADL5mRI+K7C00P2zVMy51qaVOtfbA68AAuPoebDjZYct0lGz4N/n8jNRdmV7qsTM8RuR2LqkkUg48oyCfM9uhGKjCpdQUkVH3eiMf++2MnhmqZgpwM/OitjyxPrUQGcKHPYVdAwh+rT1h8j2KFojKzK11WpueI3IxF1WQEg48oTSstwpQSn6Fo36rdIE7Ly04DPLBl3BKAxdNHKaZE9tSdsXRVJfQ+RbdEm13psjI9R+RW0ewEo+TEglMLpKZ0H2s/Y+xlKBvSTzfwsHo3iFM8ADZblP5QUtAnQ/E5tDMlYaR52rTSIiy/axz65qSHfd/nzVR9c5WDFuBikCIzkp4jcisWVZMZXPlwkNZ/pPHg7PlOrHj/C9uuf7qlHeuqv0RBTgbgAU6fa8elfTK7/21AXlYami4Y66yqF+B0BSS8sPUwVuw6Gnbt/Jw0LJ4+SjHwkPPf7V8F8EjFMKzafQwnWy52sBVJz9kpmvw8c/skY1E1mcHgw0F29fPonZGKc+1dll/XSSme8GLQUN6sNOSkp6K1Q+wxLr9rHFJSPDjV0obTLe2q1w2lVXOxsaYej739seKJwmdbOzF31X68lOIJCyKUUmuRc3Msj1WKJj/P3D6FYlE1mcG0i4Ps+o+vsyte11Iu0lqR9V/oFA48+manYdKQfsE02H2Ti1HkzeyR8giVl52GQEDSbBCmFHgAysvKaqm1yMufbG6PSZOxaJqesWEaRWJRNZnB4MNBdv3H1/5VwJbrOsHqlfqz5zvx9Duf4NX//Rxr9/0Ve+rOYPF05ZoLWdP5Ttz96m5c++zWsMlTNE0WuqxsJLUWi3x4NPl55vZJiVxUrfbflwfdK2MsqqZQTLs4aEJxPvKy01Q/RScDX24mrh3aD9kZvSBJEv6r6pjl9xFZl1LkzcQD1xfjTx/Va6a96v1tePC1fZhfMQzzyocZTpOdamkz/DtO58Ojyc8zt09KrOp5RMmFwYeDNh1s0A08bhx5CfYfb8KZ1sQLUG4d7cMHR8/irX1fOnq/9f42/HZHHZbfdTW82emY+/o+zYLUZZsP4/e7juIbBifQS/tkmk6tOZUPF72fXUdO9ygmZW6f1ETT84iSE4MPh4ieXLvls+5j6PNz0tHW2YXzgrUOblbkzcR3xhThtzuUz15xgoTugtZ//fsxQjth/Bc68eca8W3FRSFt4M0QTclFu8tE9H5e2HYk+P/lYlLm9kmLmZ5HlLwYfDjE6HK8HU21YiE/Jw1bH/0Wyv/tvZhvMa73t6Hy89O2XFteVtZrKhbJSJMxK3aZGB0fcLGYdPldV7NhGmmSex4R6WHBqUOSdSn6TGsnVu3+wkVN1az9FJaXnYaXQxqMaTUVUxuJ6BlAVuwyMTI+mRxoLNnwqWrxLnP7RGQEgw+HJPNS9Bdnzsd6CEFlQ/rpbr0VkZedhvkVw7H3p1OCgUdXQEJlbePXTcWGozA3/DWPnJO1OqOGsnqXiZyf93nF/yblYtK+OemKvyv6WIiIAKZdHGNmuTtRDMrPjvUQAHSngE61tOOOawaYOkl47reGYLivj2IuWyklEhngXNI7HXdNHITBBTmG8uF27DKJzM8fPtmCF7bV6v7eqZY2zBh7GXP7RBQVBh8O0dqOlqjkGoBLc92x6nOmtRPz/1AN4OtD8iQJTRe+Ev79E00XcO2wSxQDD6VDtSL/faqlA7/afBgv3TNOOEjoCkjYdUSsTkWkRXxkwCCPo7K2USj4kFfw3JDbZ4t3ovjF4MNBatvREpEcYE29shBP/OmTWA+nB//XW55vGV2Edw6I1UusrT6BtdUnwoo8jTYVEz1FFzB++rFei3itYtUJxfnw5Wagobld8ffdVkzKFu9E8c0jxfKACQXNzc3wer3w+/3Izc2N9XBsEfqJ7ejpVizbfDgmqyHzK4Zj9Z5jaGi2PhCy6/F4PIBVf7HyhPp//24Ant9iLA3jAfDSPePgzUrHna9UGb7vxdNHoaBPhuondrXVFLWx+LyZ2LmwXDGgUbuWfMuX7hkHAKrn14Tezg0Tu8jjccM4iZKNkfmbKx8xELlkPcLXx/HVkCJvJuaVD8W88qF4YethUzUQkXpnpKIrIOFCZ8C2QEqSgMenjcSL22sNn1zb41rorpXQ+9Sv9rtPrT+In0wbaeq+Qw+7i/zEbmQ1RW+XiV6xqgfdQYf/fKfq/Xmz0/DM7Ve5YkIXeTyiK0tEFDvc7eIC00qLsHNhORZPH+XYfcqTVWqKBw9XDMfL94xDkYHdD0rOtXfhQqf958w0t3+FZ2ZeZdn1Kmsb8bNbrjT8e/X+Npw5Jx6wqIncLmukJ4zeLhORYtUmjcADALLSUjGlxCc0HrsZKb4lIvdi8OESqSkeoRNYo+UB8OJdV/eYrKIJgFI8QE56qkUj1Ff7txZMKfEhL8uahbsXth3Bkg0HccOISwz/bl5WGvJz0qO6/8jtsqI9YebdMAQ7F5ZrrkhY0V/GTZM5W7wTJQYGHy6i1QDK8/XX/IrheP6OsZhfMQy+3Iyw2+Rlp+nex/K7xuHm0f1V77+gT4biz7RkpaUIH3lvhT/XnMQLW48I7VRJSxUL5er9bdh26G+Gx9J0oRPfHav8fBoR+oldtCfM5KGX6KYWrOov45bJnC3eiRIDaz5cxsgBTfLJq6FbDTcdbFAsHMxOT8UPrr8CU0u1l8+NvGlnp6fgfEcArR32p1pCeQCseL9O6LadXfaW8R5tbMXgfjmWXe9USxtuGd3fsjbmVvWXcctkrvd43LYrh4iUcbeLS0X2MBg/qC/2fnFWqKdBV0BC1eeNeK3qC+z4y9/CViX0tiN2BSRc8/NNQqfq9s5Ixbn2+D/4zgopHkCrwWhuZi80t+mv1KyePQllQ/oFd3QAykeUG9nRoXUtCd0rZmoFp3o7aWLByueGiKxjZP5m2sWl5B0xM8ZeBv+FDnzzF9tw5ytVeHhNNe58pQrXPrtV9TyP1BQPWto6sbGmoUc6ROQskInFYs2jYh145GWl2VofY4ReZ/PvXTNAs57Hg+7AcPygviEt2oehMCK1ZqaNuVo7dZ83Ey/fMw7P3H5VcAyRYwLcd16L1uNh4EEUH7jy4XJmehp0BSRc++xW1V0Bap9mjTa1irX5FcPwq82H46JbbJE3E4unj8LcVfsBKH9if+D6Yvzpo/rwdFtuJr53zQB0BQIAugPSSVf0MxUMaHUEXfruQbzyv3VhQVSKB5h9XTEW3Vxi+L6cwA6nRO5iZP5m8OFiZoOIytpGocZX8hI/YKyplRukeIDPltyEzQcbMG/1ft2VBzeYXzFMsadLkTcT3xlThN/uqBN6/q3u5Kn12svN1LiaQER6mHZJEGZ7GhjdjmikqZVbBCRg7xdn0TcnIy4CDwBYtvkwPq1vxu3jLsNtY/tj7reG4PVZE7H9xzfgTx/VCz//IqkzUSKvvZETc+OdfDLxuuovUVnbmDSPm8hphna7LF26FG+//TY+++wzZGVl4Rvf+AaeffZZjBgxInibtrY2PProo1izZg3a29sxdepUvPjiiygsLLR88InObE8Do9sRjTS1chO3bP804vktR8L+/fqeY7j/G8WGnn+5k+eTf/oEfTLTcPpcu+m0gx0n5sYrnhdD5BxDKx/bt2/H3LlzUVVVhU2bNqGzsxPf/va30draGrzN/PnzsX79erz55pvYvn07Tpw4gdtvv93ygScDsz0N5O2IesWN8nbEeJzEge7HffR0q/4NXazpfCeWbf6L4d+TADQ0t+Pu3+0WKkJWw6Zd3eTUU2QgZuUqExFdZCj42LhxI+677z5ceeWVGDNmDFauXIljx45h7969AAC/349XX30Vv/zlL1FeXo7x48djxYoVeP/991FVZfzwrWRnNIiQ6TUrA8J3MLilh4Oo0J0hq/cci/VwXMPMRBlPTbvsSononRcDJFfqicgJUdV8+P1+AEB+fvfkt3fvXnR2dqKioiJ4m5EjR2LgwIGorKxUvEZ7ezuam5vDvqib0SAilJHtiHpBjpuEPu69X5w1dBhcopNw8cA70YnSTIAbi7qIjTX1uPbZrcLbzY3geTFEzjPd4TQQCOCRRx7B5MmTUVpaCgBoaGhAeno68vLywm5bWFiIhoYGxessXboUTz31lNlhJDwjHU+VfndKiU93O6Ic5Mx5bV+w8ZRbhT7uddVfxno4lrPi+a/3t+GFrYfxcMVw3dtqvfZKAW4s6iLUduPIKz3R7sZh6onIeaaDj7lz56KmpgY7d+6MagCLFi3CggULgv9ubm7GgAEDorpmohENIpTIzcpE7kMpyJHlZaeh46sAzjt4hkvk/S+/cxwmDekXt+kiPfMrhmHNB8ctKf5dtvkwRvj6CE3KogGu3UGAEr2UiAfdKz1TSnyme3yYTT2xzwiReaaCj3nz5uGdd97Bjh07cPnllwe/7/P50NHRgaamprDVj5MnT8LnUz5TJCMjAxkZxg8zSzaiQUQ0ppUWIRAAHlq1r8fP1NpvO+WawX0xeVhB2PesOrfEDfJz0jDnW0Mx7NLeeOjrRmTRMjIpRwa4BTkZgAc4fa4dlbWNGD+or+1BgBLRlEjV542YPLRA9XZazJwXw50xRNExVPMhSRLmzZuHtWvXYuvWrSguLg77+fjx45GWloYtW7YEv3fo0CEcO3YMZWVl1oyYbNMVkLBkw0HFn8kTTKw+2G06eArvHgjP72vVxMSbM62duP65bXj8jzWWXTO0TkGkTkMOcDN6peBHb30UtpNm0tItMamLEE11zH3d/I4Uo7VV3BlDFD1DKx9z587FqlWrsG7dOvTp0ydYx+H1epGVlQWv14tZs2ZhwYIFyM/PR25uLn74wx+irKwMkyZNsuUBkHVEPmXK/XBjURuyeF0NppaGf7LWSxe5SXZ6qmbaqqHZ+vGfamnDuwfq8dN1NTjT2hH8fne79xL0zUnvcSqyUmol9Hf17s9KoimRpgudUaV+RFNPTqSBiJKBoeDjpZdeAgB861vfCvv+ihUrcN999wEAli1bhpSUFMycOTOsyRi5n+jEMWvyYLxb0xBVi3AzGls7FJtdySmDZZv+ghe2HVH57djpnZGKf/0/YzClxIdfbT6EF7bWOha4bTp4Eu8c6PlJvN7f1iO95svNQNtXgajGZnUdjtHUWjQTv0htFZuyEVnDUPAhcgxMZmYmli9fjuXLl5seFMWG6MRRUeLD49NLFN+krx7YF4+9/TGaznfaMsZdR06r7tiZPLTAdcFHfnYaqh6vQHqvFGysqce/b6115H49ALxZvRQDDzXRbFtWqouwQuhuHD1WTPx6tVXcGUNkDZ7tQkETivORl52m+vPQng/ym/SMsZehLGQHyrTSIuz96RTcMtqeorsXth1R7e/gpn4lnq+//uX2q5DeKyW4XO8UCXBsZ5Jez5loySmRvCz1v81Qdk788dSUjcjNGHxQ0KaDDZorFhLEJ5i9X5y1cGTh6lUK++RPyW7Y+eLNSsMjFcMwpaR7l1cszs/p6HLmmVBqXGe1aaVFWH7XOKHb2jnxm+06TEThGHwQAAh9Ms9JT0WfzDTdjpZOTLShnTxDd3Icajhn6/3qyUlPBdBdALls8+HgKo2RT+M5Ganw5cbH9vP5FcOxc2G5I9tLJw3pF/OJP5quw0R0EYMPAiAWMLR2dOHu3+3WbWvtVL5b7uQZ2nZb9JA2OS3yg+uL4cuN/pOynK5qjUh1yNsvj54+L3wtDzyYeqVyXxw38QBY84FzZ+u4ZeI3cnQBESnzSCJVpA5qbm6G1+uF3+9Hbm5urIeTNNZVf4mH11QL3VZ+a1d7o62sbcSdr7j7IMHQhlC7Dp/G3a/uNnyNrLQU/Py7V6EwNxOPvvmR6lZZuRhTkiTXn0XjAeDNTkNmr1Thrb+rZ09ydGeHWxp8scMp8W8gnJH523R7dUosRvLkev0M3N55dN4NQzF/yvDguE+3mgsI/u3/jMHNo/ujsrZRc6KWd2HMrxiGZZsPm7ovJz1z+1Vfb10+hBe26e/OcXpnRzTHDVjJia7DscAJVYxbguB4xeCDABgPGLS2NRo9qC43sxea274yOXLjJg8tCL6ZdgUknG4xHnz84Ppi3Dy6PwDxyXdgvxz8/bjL8NY+dxyI5/FcbBoHdHevnX1dcfCNMy01Veg6sdjZkagTf6xxQhUTi3OOEg2DDwJg/mRbtYlXrWOkUmfNBv8FzH/jo+gfhI7IXhRKb7R6+uWkY8mMUtwcspVYdPL95/Wf4KyF/U8WTx+F/Jx0LNnwKc62dhheZYpMuEoS8Nsddbh6YF8AwK8E6mf6ZqdxZ0eC4IQqhl1urcHgg4LMtCrXmnhFl8craxsNj/WW0UWGGmhFFiSqvdGqubm0EHdPHBx2qq5MdNXIqsBDDqLum1yM1BQPstJT8aBAEy49oW+ckiQJr4BR/OOEKo5dbq3B3S4UZlppEXYuLMfrsyZqNnUS3dao1owslNHmYLOvK8bzd1xt6HdCdyJovdGqebfmJH701kfYdLChx8+cPOBOaVfHtNIizK8YZsn15TdO0cLYpvOdlh8mp0XkgLx4Y/Qx2fEcGJlQkx273FqDKx/UQ2qKB5OHFeCZmVcF21qHvr1Zva3RSAttD4B3DtTjsZtGqaaJ5H/PrxiGwQU5PVZczPYh0Vp+duqAu8iDzmSDC3Jsu089am+yVhcuJmI9gtHHZNdzwAlVHLvcWoMrH6TKyX4G8n3l52i30A79BKY1vpfvGYeHK4YrrriYfQOVAxy5uZnSY9i5sByv/5P2qpER8qi/P3kwVs+epNrQK5ZvdEr3vbGmPqz/yp2vVOn2h9GSiMfYG31Mdj4HnFDFscutNbjyQZqc3NY4rbQIFzoDmP+Hat3bygGEmfEZafgVSS+fm5riQYrHg6YL1tR3qK10RJLfEKNddQntSXKyuV03NaX0Jmt14WIi1iMYfUx2Pwd6dUt2HRwYj7SK89nlVhxXPkiXSN2GVUS7jYZ+AjMyvq6AhNV7ou/KqbV6YtXS9OLpo4Rbl4fWnYjS6hL65HeuFLrGHdcMCPu33iQJqK8cqUnEegSjj8nu58At3WPjBbvcRo/BB7mK3Uuae+rOCHfu1KK1/BzNyoosxQP8Q9lgQ2/23YWnw4VuO79iuOYbp/zmWuTVDgaXbT6Ma36+Ge9+vfPIjkkyEesRjD4mJ54DTqjGyGnW1bMn4fk7xmqmRaknpl0oKlYXFdq9pBntBKW3/GzVykpA6j4ZWGSrXuhr8HeD+6KwTzpOtnQo3lYe/7zyoZhXPlTztQtNaTU0t2HX4b8pNkg709qBh1btww/+WoyS/l6hx2fkdUjEegSjj8mp58CONGsid0xlszvzGHyQaXZV3qvtHBGtf9ASzZuzSPBj1coKIDZBK70G8iF3IsGb3hun/ObaFZDw3MbPNG/7mx11ePjGobpjBoCC3hmorG0UmpASsR7B6GNy8jmwckJNxB1KZA0GH2SK3d0QzXwCE/mEFc25MyLBj5VL/3qBktpr4P+6mZk3Ow1NIY3N8nPSMWNsf3iz0tEVkAx9+hTdnvyflV/Al5uJk83qk2RedhoefaM6rJeI1oSUiAV+Rh9TPD4H7JhKWniqLRnWFZBw7bNbVScj+VPYzoXljr0Z6n3CCg1Mjp5uxbLNhzXbyCu1gRdZLrbqRN+8rDTsXTxF9f5EX4N//fsx2PrZSayt/hJnWkMDkTQ8PaM0eD6NHiOnHs+vGB5sza7Uf0VtvID6SclAYn6KdkufD6u58T2C7MdTbclWbmsvrPcJ64Hri/Gnj+oVUxNNKi3PJUlCSop+WiKSVSf63j9Zu9hU9DX48Isz+P2uoz3Gcqa1Ew+t2o8f/LUJi27W3yVjJF01uCBbNW12obNL8TkX2S7qltNsrWT0McXLc+C29whyHwYfZJibdh+IbO38zY66Hj+TUxNqZ8ScbG43tTQsL49Hc9ZK3+w0zCvXbpcu+tyuUAg8Qv1mRx3GXN437KA8JROK85Gfk44zrcqFrKEu7ZOJsiH9ekySgYCEu1/drfp7IhNSIhb4GX1M8fAcuOk9gtyJW23JMDftPjDbKl2ekN/9WLkrpNmeFAAwpcQXXFkxygNg6e1X6X6SFX1uRZqdLV5Xo/sYU1M8eHpGqe61QrdBR/ZfOd0qdl4MJyRtRs52idVZOG56jyB34soHGeam3QfRTFQSeh4rH/lzM0vDe+rOqKZztHg8wAPXFQuttJxtbUeKp3tLbrQaWzuEHuPNo4vwg78WK64kAd2vu1bRIyek6Bmp+YhlfYib3iPInbjyQYa5qRuiExOV0QDH9NkxEvDbHXW6Z3RsrKnH3FX7LQk8ZJsVTutVsujmErx419U9zuDJz0nD8ruu1pzUrG4gl4gn3GoxcrZLrM/CcdN7BLkTgw8yxS3dEPUmNCsYDXCiDYi0Uj1aNS7ReGPvcazd3z2Jd3wV0JzUp5YW4R8nDUbvjNTg9860dmLJhk81JzUrJySrD65zOyNt6+1ocW+GW94jyJ241Zai4obuhfKnPEB9K6caj0c99WJ2O6C8zTCaHS+rZ09STINYtZVXS2Q6J3SpfmNNPR57+2PFtJLIdlkg+nSA2u4m0fuPR6Kv++rZkwBA+LZOFK664T2CnMGttuQYqyrvo3mDUuuIWuTNxHfGFKnWKAAXAw/Rxk0i49RqCCVKLXXjRDFm5Afi0C3Lms8lxE5XVdsuCkC362kinnArwo7dI04V9sbD7hxyHoMPijkrCuO0JrQ/fPhX1QJQD7o7gWb2Sg1ri67UzdTIONUCovyctLBmX2rUUjexKMaUJ/VX/lc98Ai9rUiRbuSEJPrcJmv/CDuKdVnYS7HE4INiysoWzEqfsCprGzV3nkjobjT2+qxxSEnxqH7qNjNOpYBo/KC++OYvtpneBTChOB95EW3TnaC3MyiSkU/VRp7bZO0fYXT3CHeakNux4JRixonCONFJ6HRre1hPishUi9lxRva6SO+VElXR5aaDDY4HHmaIfqo2+twm63ZdI8W6sdxpkmw7kMg8rnxQzDixhB7NZCXXd+w68jdLx2n21F55otaiVUDrFK3tspE1M4GAZOi5Teb+EUb+buw8GVqNFenTeCxOjccxuwGDD4oZJ5bQzU5WSm+kVo7TzBkdIt1cJQlYPH0Uzp7vwAvbaoXHYyW1T9VKz2lellgnWPm5jcfTXa1k5O/GyXNgrEifxsuheaHiccxuwbQLxYwTS+hmlqDVGjRZPc7IlIzSpBC6jL3ryGmh6xb0ycD8KSNQ5HU+9TBr8mDV01iVnlOR9u9A+HOb7P0jRP5uzNzWLCvSp7FuimZGPI7ZTbjyQTHj1BK6kSVoM0287FrqN7P6AnRP1FYccGdGRYmvx/c6vgrg8bUfm9pyrPbcuul012Rfdo82fRqP26fjccxuw+CDYsbJJXTRycroQXV2LfWrLWPrjscDfPVVAF0BCdNKi/DiXeMwb/U+S1uxK94v1NNXj6+tEdperHRNQP25tbN/hGhAwWX36NOn8bh9Oh7H7DYMPiimnCyME5msjNaXWDHOyIlu/KC+pluoSxLwDyv2BCfAm0cX4QVcjYdW7Tc9Pj166SvRx5GXlRaWhrGzOFKLaEBh5TZxp1m5WhNt+jQet0+LjmXXkb+5biXMLSt1DD4o5ty0hC76RjrvhqGYPLQg6nEqTXSijci0hE6AN4/uj5dTPKZSOCKsSl8tv3scUjzKvVacesMUDSjiednd6tWaaNOn8bh9WnQsL2yrxX/v+9I1K2FuWqlj8EGu4JYWzKJvpPOnDI96UlGb6KINPICLKazH136MC50B+HIzsf3HN2DvF2fR4L+AJRs+xdnWjqgPqFs8fRTum1wcdfoqPycNp1ra4cvNxC2j+/dYQXHiDdNIQBGvy+52rNZEmz6Nx+3TemMO5ZaVMLet1HG3C1EIpxo02XU6baQzrZ2Y/4fuU1+/+Ytt8F/owHfGXoZ7ywZFdd8edAcASoEHYHyJPHScoafTOrmjwEhA4bZUgUhzLzub+kWzAymWTdHM0hpzJCdPElbjlpOOQ3HlgyiCE3UoRlcGrNDgb8ODr+2zrD271oQQzRK5HFgsv2sclmzQfsN89I2PkJPeC98YWhD15GQkoHBTqsAt5+JEkz6NRVO0aKmNWUmsV8LcuFLH4INIgd11KGY/EcvL2r0zeuFc+1eGfleesKMNPHIzU/Hc34/RnBBElqXVurHKKY7F62rQ2NqhOZbWji78w+/3IC87Dc/cflVUk5RooHD45DkU9M5AXlYvNF1Qfg2cShW47VycaNKnbqr9EiWPedmmv+CFbUd0bx+rolm3rdQBTLsQqbKzQZPoRJefkx7277zsNORlpxkOPKyUmqL9tiEXh95c6lPN4QPabeAlQDfwCNV0vhMPRpmGGTsgT+h2L2w7grt/t1s18AC6x293qsDoUvrR061C141lYacTTdGslpriweShBUK3jdVz66aVOhlXPohiQLTITi4SPdXShqOnW7Fs82Gnh9rD2fOdqgVqSimAFA/C+oz4vJm4qdSH3+86avnYnvzTJ4Z3mMjB0hsfHLNsHH2z0zBFoeGalYwspU8ozsfqPfqPT+tcHlLn9qJZN46PKx9EMSBaZJfeKwVlQ/rhltH9seaD446OUY/8qVoudlyy/hM8qFAcKq9wfH/yYKyePQk7F5YLT8z5OWm6BX2hGprbsafujO7t5DH/8/pPcM3PN+POV6qwtvqEgXvSdvZ8p9A4omFkKX1P3Rk0NLfr3vaOawbGxWqD27i9aNaN42PwQRQjRnYIiBaoZvTyoHeG/Qua8qfqF7YewbXPbsWdr1ThVZWVDLmG4881DcEcvvxJTO2tTt5N8/SMUsNj05uUN9bUB8f8+11HccZAesfKcUTLyFK66FgGF2RHM6Sk5vYzh9w2PqZdiGJItMhOdPJo/0pC+1fG60HkZdfF00uwZIN4M7Jlm/8idLvIanrR3hDTSovwUooHj6/9WLj/idakbLZtvRl258+NLKWLrsK4qZFXPHJ70aybxsfggyjGRHYI2DkpRE72U0t9WLmrDks2fGr5fYUGUaLbK6eVFqF8ZCEmLd2iu0rhy81QzVs71VtFnvQDAQnrqr+07Q3eSHMvN+b8E5VbGiaqccv4GHwQxQEjHRWNipzsU1M8uG9yMX63s87y+yvonRH2bzmw+K/Ko/jizHkMys/GP5QNRnqv8Ixweq8U/Mttpbqn9D75nStVJ3kneqvIQcCFzi7c/eru4PftamEtGsA5eYgjkQiPJGlteHNec3MzvF4v/H4/cnNzYz0cItcwfdItoPg7syYPRkWJT/PE1jlfT/aRk5XZN43XZ03E5GEXtyUabZ2+saYej739cY9eJSJ9PtZVf4mH11QbGu+3SwqRm9kLb+37Uuhx981Ow1mNPip6z7lZPIWX3MDI/M3ggxS55eRDCtd9RL14/cOsyYPxbk2D6clGbbK645oBprb9Pn/HWMwYe1nw2krBlPxXplYE1xWQUPV5IyprGwFIKLuiAJMEekJU1jbizleqhMYppyH+9e/H4HRrO46ebsXqPcfCdowUeTOxePoo9M3JwKmWNhT0zsCjb1QL7SqJ5YTP/7bJLgw+KCr8dOQso5NBx1cBjH96E1ra9AtLV8+ehAnF+aiqbUTl56cBdOd7J10h3sBJaXwAcO2zWw2nZVbPnoSyIf3QFZBw7bNbVdMg8uS/c2G5ZROjfJ96Y5ZXOCLb0PtyM3HnhIEYXJCNS/tkYvygvsEeLJf26a7xCE21aNELsOIJgxmSMfgg08x+GiVzzAZ67x44gYdW7de8dtHXk/emgw22BJNqaRklkcGE6CqEHKxYRWTMaqmT0P8GAPR4TvOy0tB0Qbx1vR0BltP4QYVCGZm/2eeDgtx48mEii+bE1ptH98cPri9W/bkH3QWEmw422HYqrFrfAKWxAOEFjaJbhxua23RPazVCbcz5OWmYNXkwXv+nicjopfy2KN/zY29/rPicGgk85OvJ24/jkZMnDlPi4W4XCnLjyYeJSi/Q86A70NNqFb7o5hKMubwvfrquJmwLqvzJc0qJD9c+uzWq+9AT2TdAqTZC6WRS0a3DS975JKy+xYpP1Vq9DiprGzVrNiREfzBfpFgdNhYNK/5+Kbkx+KAgN558mKisCvRuHt3dl0NtInUimIzsGzCvfJhuDYDo1uHIwlql01qtGLMsFn/b8djYix9UKFoMPijIjScfJiorA71oJ1KrJ1yRJkZ6fSfUAhK7P1Vb9bctUv8Rz429+EGFosWaDwoSPW8jHt8sY0k+xCy0bsGJQM/twaR6/UW65u/Jn6qrPm9UfG6jofffgKjld4/D6tmT8P3JgwG45zAvq7j9b4vcjysfFMQuiNZT2w2wePoo29tdx0NLbaX6iwb/Bcx/4yPd3539nx8ivVdKWA1GtDUhof8NmCE/p/JW5rIh/TChOF+3A2m8iYe/LXI3brWlHrh9zhp625YfuL4Yv91RB0A50LNiW7NWl1Kr7sNqRpqBRbLqcW2sqcdj//2x4a2zavediL0w4vFvi+zFPh8UtUR8s3SSaBMtpVNkrQ704i2YFG0GpkZ+brf/+IawJmBG/4Z3HT4t3DQMcPdzKsrof/fx9rdF9mLwQRRjRppoyUee2xnodXwV0D28zU3MnmMTKj8nXXELsuikqBdAht9XGqoWVTj+nFr5IcFsIMEPKiRj8EEUY6KHmIWedWKXeP10aib1ocVMOmBjTb3uSboyq7ux6rHydWVnY7ICO5wSxZhVuwGi3c0Rz10op5UWYfld4yy7npkuvdNKizDr6x0repzcVmrl6xrrzsZW71ii+MDdLkQ2sGI3QLSfbBOhC+WkIf2EmpGJMtP8qqLEh1d3HdW9nVPbSq1+XWPZMCxeV+UoeoZXPnbs2IFbb70V/fv3h8fjwR//+Mewn0uShJ/97GcoKipCVlYWKioqcPiw8aO3ieKZvGUTMNfjwYpPtkYmFbfSeh6jYWSVwm39b6x+XWPVMCyeV+UoeoaDj9bWVowZMwbLly9X/Plzzz2HX//613j55Zexe/du5OTkYOrUqWhrY6c7Si5qTbR83kzNHLpVy+CJ0oVS7XnMy05DXnaaqWsaWaWINpA0QiQFYfXrGouGYbFO9VDsGU673HTTTbjpppsUfyZJEn71q1/hpz/9KWbMmAEA+M///E8UFhbij3/8I+64447oRksUZ7QOMVNj1TK42UnFjbsX1J7HqtpGQ9thzTa/kgMgO5uFiaYgrA4WYtEwjGfDkKU1H3V1dWhoaEBFRUXwe16vFxMnTkRlZaVi8NHe3o729ounSDY3N1s5JKKYEznrJJRVn2zNTCpuzsErPY+nW9VPoI0U7SqFUgA0flBf7P3iLNZVfxlVoKa220TpID2rg4VYdDZOlFU5Ms/S3S4NDQ0AgMLCwrDvFxYWBn8WaenSpfB6vcGvAQMGWDkkorhj1Sdbo+mCeMzBG0kF6KW7RMgB0Iyxl8F/oQPf/MU23PlKFR5eU407X6nCtc9uNfw8GU1B2JEGMpsiNItnw1DMt9ouWrQIfr8/+HX8+PFYD4kopqwscBSdVOI1By9yEFxeVhpe/6eJ2Lmw3NKusVYFamYKSO0IFqaVFmHnwnKsnj0Jz98xFqtnT7L0OQvltiJecp6laRefzwcAOHnyJIqKLv7Bnjx5EmPHjlX8nYyMDGRkZFg5DKK4ZvUyuEjdSbzm4EWeq2dmXoXJQwssu0+rt7qaTUGYqSfSYzRFGM398BDL5GbpykdxcTF8Ph+2bNkS/F5zczN2796NsrIyK++KKKFZ/ck2NF1QNqRfjzf1eM7BO50ysHqrazQpCL3X1c2cft3IXQyvfJw7dw5HjhwJ/ruurg7V1dXIz8/HwIED8cgjj+Dpp5/GsGHDUFxcjMWLF6N///747ne/a+W4iRKeHZ9s1bgpB29mt42Tz5XVgVoyH08/rbQI5SML4+rcIbKG4eDjww8/xA033BD894IFCwAA9957L1auXImf/OQnaG1txQMPPICmpiZce+212LhxIzIzWThEZJRTy+BumQCj2W3j1HNldaBmNAXhxq3QZim93r/bWeeK3VVkLx4sR0QALhZRAsoToBVL4VoTZ7wcbiafdqsXqO1cWG4oKBAJvMwEZ24NVuLl9SZxPNWWiEyxs8+H1rWnlPg0j683O6FriWZStitQszo4c2vfFjmAc/L1toJbAzm3YPBBRKbZ8Qb77oETeGjV/h7fl6/6SMUwLNusfwaUVcfWWzEpq13jjmsGYnBBtqWTk5nJ2s0rC5W1jbjzlSrd21n1elvBrYGcmxiZv3mqLRGFsbp24t0D9Zi3umfgAVzcmrpC4NRYwJrdNka6iWqJLHI9eroVq/ccw7LNfwnexqrJyehW6FifaKwXwMbb7iqr/mboIpYUE5FtNtbU46FV+6DVm0wC0HShU+h60e62sbqZmhyoZfRKwa82H0ZDc3i7d6u6wxqdrGN5ovHGmnpc++xWzc6vbtpdpSdeG/C5HYMPIrKF/KYtKi8rzfaOl3ZMyk5MTkYn61itLIh2fo2nDqexDOQSGYMPIrKF3pt2pPsnFwOw99h6OyZlJyYno5N1LFYWjARhdpxPY5d4SxHFCwYfRGQLI2/GRd5MzCsfanvHSzsmZScmJ6OTdSxWFowGYfHS4TSeUkTxhAWnRGQLI2/G8sRpd6dSO5qpWT05qRVrypN15I4Ln0JRayzOTjEThDnZmdYstzTgSzQMPojIFnpv2gCQ4gFeuPPqHhOnXdsr7ZiUrZyc9LZzGpmsjQQrVjAbhFnxetvZf4OH4NmDfT6IyDZqzbhkL941DlNLnf/ka3XPBiuajtnVl8OOiVnpmgCi7vxqZqxO9d9gnw99bDJGRK6h9aYNIGZv6FZPytFMTvHU8VPv9TQbhJl5/pxupMYOp9oYfBCRqyi9aW862ODaDpwi1D79m5mc4qXjp8hkDxgPKM0EEfEUsNnFbcEQO5wSkatE5vVj3YEzWlYvwcfDdk7R12znwnJDRaRm/xaMdn1NNPGeBuJWWyJyXDw3bhJtpGVEPGznNPKaycHmjLGXoWxIP80A0uzfQjwEbHax42/QaQw+iMhx8Tpx2NXNNB46ftr1mpm9bjwEbHZIlHbvDD6IyHHxOnHYtWITDx0/7XrNzF43HgI2O8TzqmEoBh9E5Lh4nTjsXLFxe8dPs69ZV0BCZW0j1lV/icraxh6fyM1eNx4CNjvE66phJBacEpHj4rVxk90rNm7u+GnmNRMpiozmb8HpRmpuEK+rhpG41ZaIYibeKvbl7Z3RNNKKd6KvmdHts9H2SXFjwGYHN/8Nss8HEcWNeJs4rOhmGu/0XjOzPTji7W8hVtz6N8jgg4jIRvG2YuO0eGmaFs/c+DfIJmNERDZyU22GG1cLEqUo0s3c9DdoBoMPIiIT7Dx9V5QbP/0CiVMU6XZu+Bs0i1ttiYjikJu7XMbrVmpyDoMPIqI44/Yul8nag4PEMfggoqSn1wjLbeKhy6Vo07R4e+7JGqz5IKKk5ta6CS3xUtCpVxQZj889WYMrH0SUtNxcN6Elngo61U64jdfnnqzB4IOIkpLb6ya0xHtBZzw/92QNBh9ElJTioW5CTbwXdMbzc0/WYPBBREkpXuom1Lj9FFwt8f7cU/RYcEpESSme6ibUxGuXy0R47ik6DD6IKCnJdRN6p4O6tW5CFo9dLhPluSfzmHYhoqQU73UT8YzPPTH4IKKkFc91E/GOz31y80iS5Kq9TEaO5CUisoIbT4ZNFnzuE4eR+Zs1H0SU9OKxbiJR8LlPTky7EBERkaMYfBAREZGjGHwQERGRoxh8EBERkaMYfBAREZGjGHwQERGRoxh8EBERkaMYfBAREZGjGHwQERGRo1zX4VTu9t7c3BzjkRAREZEoed4WObXFdcFHS0sLAGDAgAExHgkREREZ1dLSAq/Xq3kb1x0sFwgEcOjQIZSUlOD48eNJd7hcc3MzBgwYkHSPnY+bjzsZJOvjBpL3sSfT45YkCS0tLejfvz9SUrSrOly38pGSkoLLLrsMAJCbm5vwL5aaZH3sfNzJhY87+STrY0+Wx6234iFjwSkRERE5isEHEREROcqVwUdGRgaeeOIJZGRkxHoojkvWx87HzcedDJL1cQPJ+9iT9XHrcV3BKRERESU2V658EBERUeJi8EFERESOYvBBREREjmLwQURERI5yZfCxfPlyDB48GJmZmZg4cSL27NkT6yHZ6sknn4TH4wn7GjlyZKyHZYsdO3bg1ltvRf/+/eHxePDHP/4x7OeSJOFnP/sZioqKkJWVhYqKChw+fDg2g7WQ3uO+7777evwNTJs2LTaDtcjSpUtxzTXXoE+fPrj00kvx3e9+F4cOHQq7TVtbG+bOnYt+/fqhd+/emDlzJk6ePBmjEVtH5LF/61vf6vGaP/jggzEasTVeeukljB49OthQq6ysDH/+85+DP0/U11vvcSfiax0t1wUff/jDH7BgwQI88cQT2LdvH8aMGYOpU6fi1KlTsR6ara688krU19cHv3bu3BnrIdmitbUVY8aMwfLlyxV//txzz+HXv/41Xn75ZezevRs5OTmYOnUq2traHB6ptfQeNwBMmzYt7G9g9erVDo7Qetu3b8fcuXNRVVWFTZs2obOzE9/+9rfR2toavM38+fOxfv16vPnmm9i+fTtOnDiB22+/PYajtobIYweA2bNnh73mzz33XIxGbI3LL78czzzzDPbu3YsPP/wQ5eXlmDFjBj755BMAift66z1uIPFe66hJLjNhwgRp7ty5wX93dXVJ/fv3l5YuXRrDUdnriSeekMaMGRPrYTgOgLR27drgvwOBgOTz+aRf/OIXwe81NTVJGRkZ0urVq2MwQntEPm5JkqR7771XmjFjRkzG45RTp05JAKTt27dLktT92qalpUlvvvlm8DaffvqpBECqrKyM1TBtEfnYJUmSvvnNb0oPP/xw7AblkL59+0q/+93vkur1lqSLj1uSkue1NsJVKx8dHR3Yu3cvKioqgt9LSUlBRUUFKisrYzgy+x0+fBj9+/fHFVdcgbvvvhvHjh2L9ZAcV1dXh4aGhrDX3+v1YuLEiQn/+gPAe++9h0svvRQjRozAnDlz0NjYGOshWcrv9wMA8vPzAQB79+5FZ2dn2Os9cuRIDBw4MOFe78jHLnv99ddRUFCA0tJSLFq0COfPn4/F8GzR1dWFNWvWoLW1FWVlZUnzekc+blkiv9ZmuOpgudOnT6OrqwuFhYVh3y8sLMRnn30Wo1HZb+LEiVi5ciVGjBiB+vp6PPXUU7juuutQU1ODPn36xHp4jmloaAAAxddf/lmimjZtGm6//XYUFxejtrYWjz/+OG666SZUVlYiNTU11sOLWiAQwCOPPILJkyejtLQUQPfrnZ6ejry8vLDbJtrrrfTYAeCuu+7CoEGD0L9/fxw4cAALFy7EoUOH8Pbbb8dwtNH7+OOPUVZWhra2NvTu3Rtr165FSUkJqqurE/r1VnvcQOK+1tFwVfCRrG666abg/x89ejQmTpyIQYMG4Y033sCsWbNiODJyyh133BH8/1dddRVGjx6NIUOG4L333sONN94Yw5FZY+7cuaipqUnYWiYtao/9gQceCP7/q666CkVFRbjxxhtRW1uLIUOGOD1My4wYMQLV1dXw+/146623cO+992L79u2xHpbt1B53SUlJwr7W0XBV2qWgoACpqak9qp9PnjwJn88Xo1E5Ly8vD8OHD8eRI0diPRRHya9xsr/+AHDFFVegoKAgIf4G5s2bh3feeQfbtm3D5ZdfHvy+z+dDR0cHmpqawm6fSK+32mNXMnHiRACI+9c8PT0dQ4cOxfjx47F06VKMGTMGzz//fMK/3mqPW0mivNbRcFXwkZ6ejvHjx2PLli3B7wUCAWzZsiUsd5bozp07h9raWhQVFcV6KI4qLi6Gz+cLe/2bm5uxe/fupHr9AeCvf/0rGhsb4/pvQJIkzJs3D2vXrsXWrVtRXFwc9vPx48cjLS0t7PU+dOgQjh07Fvevt95jV1JdXQ0Acf2aKwkEAmhvb0/o11uJ/LiVJOprbUisK14jrVmzRsrIyJBWrlwpHTx4UHrggQekvLw8qaGhIdZDs82jjz4qvffee1JdXZ20a9cuqaKiQiooKJBOnToV66FZrqWlRdq/f7+0f/9+CYD0y1/+Utq/f7/0xRdfSJIkSc8884yUl5cnrVu3Tjpw4IA0Y8YMqbi4WLpw4UKMRx4drcfd0tIi/ehHP5IqKyuluro6afPmzdK4ceOkYcOGSW1tbbEeumlz5syRvF6v9N5770n19fXBr/Pnzwdv8+CDD0oDBw6Utm7dKn344YdSWVmZVFZWFsNRW0PvsR85ckT653/+Z+nDDz+U6urqpHXr1klXXHGFdP3118d45NF57LHHpO3bt0t1dXXSgQMHpMcee0zyeDzS//t//0+SpMR9vbUed6K+1tFyXfAhSZL07//+79LAgQOl9PR0acKECVJVVVWsh2Sr733ve1JRUZGUnp4uXXbZZdL3vvc96ciRI7Eeli22bdsmAejxde+990qS1L3ddvHixVJhYaGUkZEh3XjjjdKhQ4diO2gLaD3u8+fPS9/+9relSy65REpLS5MGDRokzZ49O+4DbqXHC0BasWJF8DYXLlyQHnroIalv375Sdna2dNttt0n19fWxG7RF9B77sWPHpOuvv17Kz8+XMjIypKFDh0o//vGPJb/fH9uBR+n73/++NGjQICk9PV265JJLpBtvvDEYeEhS4r7eWo87UV/raHkkSZKcW2chIiKiZOeqmg8iIiJKfAw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhRDD6IiIjIUQw+iIiIyFEMPoiIiMhR/x96ouMhYG7pmwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "data_url = \"http://lib.stat.cmu.edu/datasets/boston\"\n",
        "raw_df = pd.read_csv(data_url, sep=\"\\s+\", skiprows=22, header=None)\n",
        "data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])\n",
        "target = raw_df.values[1::2, 2]\n",
        "\n",
        "plt.scatter(data[:, -1], target);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9w35NG1B1DLt"
      },
      "source": [
        "Нам надо обучить линейную регрессию. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9v8PLwht1DLt"
      },
      "outputs": [],
      "source": [
        "w = torch.randn(1, requires_grad=True)\n",
        "b = torch.randn(1, requires_grad=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hthWmKNR1DLu",
        "outputId": "882c40a6-062e-4b72-df5b-f5ab4bb86491",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([-0.0249], requires_grad=True), tensor([0.1742], requires_grad=True))"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "w, b"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2Ddb2Nyy1DLv"
      },
      "outputs": [],
      "source": [
        "x = torch.tensor(data[:, -1] / data[:, -1].max(), dtype=torch.float32)\n",
        "y = torch.tensor(target, dtype=torch.float32)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FPhMafkd1DLv"
      },
      "outputs": [],
      "source": [
        "# Наша модель\n",
        "def linear_regression(x):\n",
        "    return w*x + b\n",
        "\n",
        "# Ошибка для модели\n",
        "def mean_square(y_pred, y_true):\n",
        "    return torch.mean((y_pred-y_true)**2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FJuK9Pl81DLv",
        "outputId": "69b207c5-2bfd-4b45-cf77-ede822931b0e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([506])"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ],
      "source": [
        "pred = linear_regression(x)\n",
        "pred.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TMvEhUgr1DLv",
        "outputId": "3e99929c-2947-4bbd-e231-544120ab66cf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(584.6359, grad_fn=<MeanBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ],
      "source": [
        "loss = mean_square(pred, y)\n",
        "loss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dXyGeOrw1DLw"
      },
      "outputs": [],
      "source": [
        "# считаем градиенты\n",
        "loss.backward()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8mw_ltrE1DLw",
        "outputId": "e6e48c73-41e5-4b93-c6f3-0af59142fc05",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "dL/dw = \n",
            " tensor([-12.3620])\n",
            "dL/db = \n",
            " tensor([-44.7339])\n"
          ]
        }
      ],
      "source": [
        "print(\"dL/dw = \\n\", w.grad)\n",
        "print(\"dL/db = \\n\", b.grad)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8ChFoAMc1DLw"
      },
      "source": [
        "Обучим нашу линейную регрессию."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ECdoOjPq1DLw",
        "outputId": "a00597de-2c76-4f00-9eb1-e7c44906913f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 447
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABqK0lEQVR4nO3deXwTZf4H8E/SI22hTS8gLRQolKuUQ1CggrhiWRAEDzwAdUVdVARXYX8uoCKyqHjsLngAKquiq8CKF3JYV0BgwSIuUATLTTmEtkALbSn0IJnfH3FK0+aYmUwyk/Tzfr26K8lk5slQ8nzzPN/n+xgEQRBARERE5CdGrRtAREREjQuDDyIiIvIrBh9ERETkVww+iIiIyK8YfBAREZFfMfggIiIiv2LwQURERH7F4IOIiIj8KlTrBtRns9lw6tQpREdHw2AwaN0cIiIikkAQBJSXlyM5ORlGo/uxDd0FH6dOnUJKSorWzSAiIiIFTpw4gVatWrk9RnfBR3R0NAB742NiYjRuDREREUlRVlaGlJSU2n7cHd0FH+JUS0xMDIMPIiKiACMlZYIJp0RERORXDD6IiIjIrxh8EBERkV8x+CAiIiK/YvBBREREfsXgg4iIiPyKwQcRERH5FYMPIiIi8ivdFRnzFatNwLb8EhSWXkJJRTXim5pgiYlAn9R4hBhdF0Sx2gT8cOgsPt/xKy5WX8bVrePRyRKNbcdKcOrcJSTHRuLa9om4JjUeP+WXIOfIWQAGZLZPQL92CQ3O7akd4vOnyyvRPNpz+9S8N95e09N5tHhvRESkPwZBEASpBz///POYNWuWw2OdOnXCvn37AACVlZX485//jGXLlqGqqgpDhgzBggUL0KJFC8kNKisrg9lsRmlpqWoVTrP3FGDWyjwUlFY2eC7JHIGZI9IxNCPJ6eumfLoLF6utiq4bGxWGl2/vVntuT+0Y2SMJX+8qcHjeXfvU4KxNSq7p6TxqXYeIiPRJTv8tO/j47LPPsHbt2trHQkNDkZiYCACYMGECVq9ejcWLF8NsNmPSpEkwGo3YsmWLTxovRfaeAkz4eAfcvUkDgIX39nLoBLP3FODRj3d4fX0AePveXgDgsR2u2gYn7VODq3sj95qezvPwwFS8uynf6+sQEZF+yem/ZU+7hIaGwmKxNHi8tLQU7733HpYsWYJBgwYBAD744AN06dIFW7duRb9+/eReymtWm4BZK/MkdfizVuZhcLqldupj5opfVGvHrJV5EARBduABAALsnXTd9qnB3b2Rc00p51n034aBh9zrEBFR8JCdcHrw4EEkJyejXbt2uOeee3D8+HEAwPbt21FTU4OsrKzaYzt37ozWrVsjJyfH5fmqqqpQVlbm8KOWbfklTqc46hMAFJRWYlt+Se3risqrVGtHQWklCsuUn69++9Tg6d5IvaaU89jcRF2+eG9ERKRvsoKPvn37YvHixcjOzsbChQuRn5+P6667DuXl5SgsLER4eDhiY2MdXtOiRQsUFha6POecOXNgNptrf1JSUhS9EWdOl3sOPJwdL/d1/qJmu6Sey9NxarVJr/eciIjUJ2va5aabbqr97+7du6Nv375o06YNPv30U0RGRipqwPTp0zFlypTaP5eVlakWgDSPjlB0vNzX+Yua7ZJ6Lk/HqdUmvd5zIiJSn1d1PmJjY9GxY0ccOnQIFosF1dXVOH/+vMMxRUVFTnNERCaTCTExMQ4/aumTGo8ks+dOzQD7yos+qfG1r2sRbVKtHUnmCFhilJ+vfvvUIN4bV1kWUq8p5TzuUjl88d6IiEjfvAo+Lly4gMOHDyMpKQm9e/dGWFgY1q1bV/v8/v37cfz4cWRmZnrdUCVCjAbMHJHusmOsa+aI9NqExxCjAbNu6apaO2aOSMfzI7tKakd94mvqtg+wJ3rmHC7GityTyDlcDKu7xAonxHtT9xqerqn0POOvS4XBy+sQEVHwkLXU9v/+7/8wYsQItGnTBqdOncLMmTORm5uLvLw8NGvWDBMmTMCaNWuwePFixMTE4PHHHwcA/PDDD5IbFEx1PuKiwjDHB3U+1KyZwTofRESkBp/V+Rg9ejQ2bdqE4uJiNGvWDAMGDMCLL76I9u3bA7hSZGzp0qUORcbcTbt403g5/FXh9IfDZ3Hq/CUkx0Xi2naJ6Nc+wWmVT6UVTtWqzeHs3rDCKRERKeWz4MMffBV8+IOvv91bbQIGvLLe5dJWAwCLOQKbpw5ip05ERH4lp//mxnIqEUck6gcGBaWVePTjHVjz8ymvr6FWbQ4iIiItMfhQgZRKqpOW7sSanwu8uo5atTmIiIi0xOBDBVIqqdoE4LElO5C9R3kAolZtDiIiIi0x+FCBnJGGWSvzZC+LFalVm4OIiEhLDD5UIGekwZucDLVqcxAREWmJwYcKpFZSFXmTkzE0IwkL7+0FS73rWcwR3JqeiIgCgqy9Xcg5cUTi0Y93SDre25yMoRlJGJxuYc0MIiIKSAw+VDI0IwkLxl6FSUt3utxCXqzDoUZORojRgMz2CV6fh4iIyN847aKiYd2T8daYXk6fY04GERGRHYMPlQ3rnoS37+3VIAeEORlERER2nHbxAeZkEBERucaRDx/gBmpERESuceRDZdw6noiIyD2OfKjI1eZyhaWVmPCxd6XViYiIggWDD5W421xOfMyb0upERETBgsGHRFabgJzDxViRexI5h4sbBBHc7p6IiEga5nxIICWPg9vdExERScORDw+k5nFwu3siIiJpGHy4ISePg9vdExERScPgww05eRwhRgNmDO/iNFBhaXUiIqIrmPPhhpw8juw9BZi9eq/T5y2s80FERFSLwYcbUvMzjp6twLy1B52OegDAjOEMPIiIiEScdnFDah7H0m3HXQYeBgCzV7O+BxERkYjBhxshRgNmjkgHgAYBiPjn0de0RmFZlctziHkhc7/b77Q+CBERUWPD4MODoRlJWHhvL1jMjlMwFnMEFt7bC20ToySd563vD2PMoq0Y8Mp6llknIqJGjTkfEgzNSMLgdIvTnWpzDhfLOpdYH2Thvb2YB0JERI0Sgw+JQowGZLZPaPC4mBdSWFrpMu+jLgH2KZtZK/MwON3CpbdERNTocNrFS+7yQlzhPi9ERNSYMfhQgau8EE+4zwsRETVGnHaRwWoTnOZ9AI55IVsOncVb3x/yeD7u80JERI0Rgw+J3O1sWz8Z9U83dsDnO351mQdigH21DPd5ISKixojBhwTizrb1A4nC0ko8+vEOxEaF4fzFmtrHk8wRGNkjCe9uyocBcHgd93khIqLGjjkfHkjZ2bZu4AHYg5J3N+Xj4YGpLuuDcJktERE1Vhz58MDTzrbOiMtpv95VgI1P3YDtx845zRMhIiJqjBh8eKB0RYq4nHb7sXNO64MQERE1Vgw+PPB2RYqel9O6W71DRETkKww+POiTGo/4JmEoqajxfLATel1O6271DvNRiIjIl5hw6kGI0YDberaU/ToD7J25HpfTiqt36ueyiPvOcOM7IiLyJQYfEmSlWxS9To/LaaWs3pm1Mg9Wm5SdaoiIiORj8CGBuHmcnDDi4YGpupy+8LR6h/vOEBGRrzH4kKDu5nFSiMts9Th6IDUBVs+JskREFNgYfMhgjgqTdJyeRw+kJsDqNVGWiIgCH4MPCcQEzfqVTD3R4+iBpykkPSfKEhFRcGDw4YG7BE1P9Dh6UHcKqX4Awn1niIjIHxh81GG1Ccg5XIwVuSeRc7i4tgiX3PLqeh89GJqRhIX39uK+M0REpAkWGfuNq6JbN2XIW2YbKKMHQzOSMDjdwgqnRETkdww+cCWno/7USmFpJd7fclTWuSwBVCU0xGjgvjNEROR3jT748FR0ywDAYADcrZqNbxKGGTd3hSVG2ugB91QhIqLGrNEHH1KKbgm/BR4GwCFIEcOFl27rJnmkg3uqEBFRY9foE06lLod9qH9brxM0uacKERERRz4kL4fNSrfg6eHpiqdLpEzvzFqZh8HpFk7BEBFRUGv0wYdYdKuwtNJpYGCAfYRDDDSUJmjK2VOFSaBERBTMGv20i7+KbnFPFSIiIrtGH3wA/im6xT1ViIiI7Br9tIvI10W35EzvEBERBTMGH3X4suiWOL0z4eMdLpfs6r0qKhERkRq8mnZ5+eWXYTAY8OSTT9Y+VllZiYkTJyIhIQFNmzbFqFGjUFRU5G07gwL3VCEiIvJi5OOnn37CO++8g+7duzs8PnnyZKxevRrLly+H2WzGpEmTcPvtt2PLli1eNzYY6HVPFVZdJSIif1EUfFy4cAH33HMPFi1ahBdeeKH28dLSUrz33ntYsmQJBg0aBAD44IMP0KVLF2zduhX9+vVTp9UBTm97qrDqKhER+ZOiaZeJEydi+PDhyMrKcnh8+/btqKmpcXi8c+fOaN26NXJycpyeq6qqCmVlZQ4/5D+sukpERP4mO/hYtmwZduzYgTlz5jR4rrCwEOHh4YiNjXV4vEWLFigsLHR6vjlz5sBsNtf+pKSkyG0SKeSp6ipgr7pqdberHhERkUyygo8TJ07giSeewCeffIKICHXqUUyfPh2lpaW1PydOnFDlvOSZnKqrREREapEVfGzfvh2nT59Gr169EBoaitDQUGzcuBFvvPEGQkND0aJFC1RXV+P8+fMOrysqKoLFYnF6TpPJhJiYGIcf8g9WXSUiIi3ISji98cYbsXv3bofHHnjgAXTu3BlTp05FSkoKwsLCsG7dOowaNQoAsH//fhw/fhyZmZnqtZpUwaqrRESkBVnBR3R0NDIyMhwea9KkCRISEmoff+ihhzBlyhTEx8cjJiYGjz/+ODIzMxv1She9LmNl1VUiItKC6hVO586dC6PRiFGjRqGqqgpDhgzBggUL1L5MwNDzMlZWXSUiIi0YBEHQ1VKGsrIymM1mlJaWBnz+h7iMtf4NFrtyvVQ11XOAREREgUFO/829XXzE0zJWA+zLWAenWzQfWdBr1VUiIgpODD58RM4yVj1UO9Vb1VUiIgpeDD58ROry1MLSS8g5XMwRByIiajQYfPiI1OWps1fvRUlFde2fmWtBRETBTtHeLo2V1SYg53AxVuSeRM7hYrdlx8VlrJ7GMOoGHgD3VCEiouDHkQ+J5K4IcbeM1R29JaMSERGpjSMfEnja+XXNz6ecjogMzUjCwnt7wWJ2nIKJbxLm9nrcU4WIiIIZRz48kLLz66SlO1F3BqbuiIizZayFZZWY/O9cj9cuLOOeKkREFHw48uGBpyWzAFA/9aN+3oa4jPWWni2R2T4BlhhpyajPrdiNNT+fUtRuIiIivWLw4YGSHV3FWGTWyjynSalSk1HLK614bMlOzFmTJ7sNREREesXgwwOlO7q6y9sQk1EBeAxAAOCdTflY8zNXvxARUXBg8OGB1FEKV1yNnIjJqHFNwiWdZ8aKPW6X9hIREQUKBh8SjL6mteSlsvW5GzkZmpGEGcO7SDpPcUW1Zqtf5NQ3ISIi8oSrXdxwVtujLqOhYbKpyADAYraXS3fHYo6U3B4l+Sfe4o63RESkNo58uOCqtodoclYHvDWmFwxomLch/nnmiHSPRcL6pMZ7rPshSmxqknScK3JHMDzVN5FThZWjJ0REJDIIgqCrXqCsrAxmsxmlpaWIiYnRpA1Wm4ABr6x3GXiIoxqbpw7Cd3mFXo8MrPn5FB5bstPjcZYYE54f2VXRiIOcEQyrTcDWw8WYuGQHzl+qcXq+uvfAU4DF0RMiouAnp/9m8OFEzuFijFm01eNxS8f3Q2b7BFhtgkMRMSU7085Zk4d3NuW7PUY848J7e8nqtMURjPp/0c7O52mqqT7xHqhxbSIiClxy+m9OuzghNbdCPK5+ETEl+7FMH5aOBWN7Id7N6hdP9UOckVKhVTyfp6kmZ9zdKznXJiKixoPBhxNSa3sorQHiyrDuSXhz9FVuj5G774unCq3i+bYeKXYZKLjj7h5IvTb3sCEialy42sUJsbZHYWml085Y6koWV+pO0yQ2NQECcLaiCs2jI2SPuqh1XM7hYlkjHlLugdptJCKi4MDgwwmxAumEj3fAADgEIHJWsjjjKafC3bRLXZ5GXcQA52BRucSWSR/zkHoPtBpBIiIifeO0iwtiBVKL2bFjtJgjFCdJSsmpKKmodnsOA+wrRdyNOGTvKcCAV9ZjzKKteOv7w5LOl9ku0e1xdUm9B56qw0p5L0REFHwa5ciH1NUpQzOSMDjd4vVKFvGaSnIqnHE34uBqdYkzdUcw+rVPcDvVBACxkWGYf08v9GsnLanWlyNIREQUuBpd8CG35oS4ksVbnpIvpTAagPHXpboccZAb4FjqvW9PgcLLo7qhf5r0ERLgyghS/Xte/9pERNR4NKo6H/6uOVF3hOVg0QW89f0hr87nqZ1S65NMuiEN/dMSnY7i+KogmBq1UIiISL/k9N+NZuTDU80JA+w1JwanW1TpFOUW65LCUzulrhrp0KKpy9EcNaea6lJrBImIiAJfowk+5NSc8LaTlJN3IZe7dkpdNXL0bIXb5xkoEBGRLzWa1S7+qjmhZmKpO87a2Sc1HpYYzwHI0m3HWVWUiIg002iCD3/VnFAjsVQKZ+0MMRowpk9rj68tLKtiVVEiItJMowk+/FVzwtfVOj21s21ilKTzsKooERFppdEEH2LNCQANAhA1a06oUa0zLioMgLJ26r2qqNUmIOdwMVbknkTO4WJO/xARNUKNJuEU8E/NCU/7wgD2eh11+1xLjAlj+rRG28QmtatLvssrVNROX+9L4w1fLeMlIqLA0qjqfIjk1pyQe7y42gVwXqxr/tirENfE5PF8SmtjrPn5FB5bsrPB476qZyKFv2usEBGRf8npvxtl8CGH0m/r2XsK8PzXv6CwrKr2MUuMCc+P7OrTTtZdfRGtRhmsNgEDXlnvMhFXHI3ZPHUQC48REQUoFhlTiatv64WllZjw8Q4J39YdO9KqyzbYnOQ4eFP902oTsPVIMXIOF+PwmXJ8s6fI5bEzhmszveHPGitERKR/DD5c8KYiqqug5dzFGjy2ZCce+fU8pg9Lrz1WaR5E9p4CTPtiN85frPH4fgwAZq/Ow5AM+RVcvS2N7q8aK0REFBgaV/BRVQLUlAJhZsDkPuFS6bd1KUXG3tmUj2qrgNjIcMxbe0DRyEr2ngI8+lteiRRKRxfUSBLV+wocIiLyr+APPqpKgEPvAEcWA+UHrjzeJBVoew/QebLTQETpt3WpRcY+2HLU5XP1R1bE84ojD73bxOH5r/Mktc9Te93xftrJTs8rcIiIyP+CO/g4lQ3k3AdUnW34XEU+8MsLwC8vAal/AHr93SEIUfptXa2pA3Gk4q31h7Dsp+MOAU1TUwguVFkVnVfq+1JzIz6xxsqEj3fAAOcrgNSosUJERIEheIuMncoGNo5wHng4sAH5i4Evk4Fjy2sflVoRtXebOIeiWYlNTCq9Abu5aw80GElREnjIreAqZ9pJCrHGisXsGPxYzBFcZktE1MgE58hHVYl9xEO4LP01tipgy11A2Wyg27OSvq2P7JGE61/73rEQWEwEmphCUKFwZMIXlIwuFJZeknScnJGeoRlJGJxu8Sp5lYiIAl9wBh+H3pEw4uHC7hlAeBzQdozbiqgjeyTh3U35DaYlispcVzbVitwKrtl7CjB79V5Jx8pNEg0xGricloiokQvO4OPIYu9ev32S/Se6I4a2G4dBfxqPf+0ow7GSi2gTH4Wxfdtg0N83uM2HiAoPQUW1stGP+iMtSj1wbRu0iotCfJNwmCPDYbUJHkcZXCWZOmsjk0SJiEiJ4As+qkocV7V4o/wAsOtpXNoxB+dPj8DeC12xorod5m84jJKKapcvEwBUVFvx5I0d8NHWY26PdcZijsDoa1Iwd+1BRc2OjQrD3Ve3wte7CmQtkZWyTLguJokSEZESwRd81JSqfkpzSDn+nLSk9s+V1lB8UfI7vHL6IZRao12+7rJNwIzhXVBSUY1fz13CBz8cbXCMOMoxOauDw8ZyVpuAD3OkBy639UxGy7hI9G2bgO3Hz2HeuoaBi6clslKXCcc3CcNLt3VjkigRESkSfMFHmNnnl4gIuYyxzdZiTOJa7L+UgvHHn8OJ6oYd8VvfH6r97yRzBB4ZmNpgNCKuSRhu69kSfVITapMvxcJeUgIPcfrjb3f1xHd5hXjq859RWOY8gPC0RFZq8uiMm327Pw0REQW34As+TPFAdEf1pl7cMBiAzlEnsKnTePx0oQvGH3/O5UhIYWkl3t2UX7uj7Xd5hfgq9xRKKqrx3pajeG/LUSS5SWR1ev3f/n/miHR8l1coKVfDXaVTqcmjlhhWIiUiIuWCs85Hu3F+vZzBAPSJ3ovc9DF4vNnHTo8Rg4LZq/fiXEU13t9ytMHIRmFpJd6RGHgAV2pkDE63yMrVAJyPckitbcIkUyIi8kZwBh9pjwCmRL9f1mAA/py0DPndbsaPncYiPeKQw/PiqMNfPv/Z6eulBg+TbmiPpeP7YfPUQRiakSQ5V6MuZ6McYm0ToP5+vPqtRGq1CQ5F3qxOdg0mIiJ9Cb5pF8A+9ZL5L2DDzQD8X+zLYABamMqwusOTqLSF4bFjf8H3FzJrn79QJaP4mRMdWkQ7TJnIKfTlaYmsu9omcmqF+IMam94REZH/BWfwAQDJQ4HfrQK23A3UlGnSBIMBiAypwQftXoRNALZe6IapJ//kNDlVjvqjFnILfXkavQiESqRqbXrnb1aboOv7SkTkDwZBEHQ1Tl1WVgaz2YzS0lLExMR4f8KqEmDzaKDoO+/PpQJBABadHYmXCh6W/Vpx1GLz1EEOHZbVJmDAK+td7horssSY8PzIwF+pIr5fV1NNru6T1jhSQ0TBTE7/HZw5H3WZ4oEb/wP0/xQIidK6NTAYgPGJX2NbpzENckKkcDZq4S5XQzQ5qyO2TLsxKDo5tTe98wdxpKZ+u8WRmuw9BRq1jIjI/4I/+BC1uRO49QTQYw7QtL2mTTEYgOamcqzp+CSOdLsZn6Y+hZRw951PbFSY26kEV7vGJpkj8Pa9vfBEVgfVRwG0SvaUmuMiJxfGl9xVjhUfm7Uyj8myRNRoBG/OhzOmeKDrNPtP9Tng1xXAj+Pl7X6rMuNvy3Q3dRqPXy61xT35c5zWCpk/phf6d3C/gsddroaUXAM5+QhaTiFIzXGRmwvjK3JGarjpHhE1Bo0r+KgrPM5eDyTCAuT8Aag6o2lzDAYgI+ooctPH4KWC+7Ho7J32x2HPX+gnsVNytmuslEBBTjChdbKnWI/EVY6L3ja9C7SRGiIiX5M17bJw4UJ0794dMTExiImJQWZmJr755pva5ysrKzFx4kQkJCSgadOmGDVqFIqKilRvtKqShwI377NPx0R31Lo1MBiAp5M+xIGuIzCoaQ4A72prSMk1kJOPoIcpBE/1SAQAo69JwaqfT+mi9kegjdQQEfmarNUuK1euREhICDp06ABBEPDhhx/itddew86dO9G1a1dMmDABq1evxuLFi2E2mzFp0iQYjUZs2bJFcoNUX+0iV/U54Nv+QPle/1/bCQGAIa43MODfQLS8XBVPq0IA++iGIAgoLKty+nz9lSM5h4sxZtFWj9deOr6fz6cQnI3WxEaFAQDOX6ypfUzrFSWeViPpdXUOEZEccvpvr5faxsfH47XXXsMdd9yBZs2aYcmSJbjjjjsAAPv27UOXLl2Qk5ODfv36qd54n9p4O3DyS+2u70xkayBrveQgRGqgIIUYTKzIPYknluV6PP710T1xS8+Wqlzbnbp5KkfPXsS8tQcadPBid65kOkituhzi6BLgWMnWm7YREemJnP5bcc6H1WrF8uXLUVFRgczMTGzfvh01NTXIysqqPaZz585o3bq12+CjqqoKVVVXvnWXlWlTEKyB678Airfbq6RWFWrdGrtLx4GVaUDbccC1H3g83NXutkqI+Qh6m0IQc1zE0QVX00HudvN1Rc2k2kCqHEtE5Guyg4/du3cjMzMTlZWVaNq0Kb788kukp6cjNzcX4eHhiI2NdTi+RYsWKCx03XnPmTMHs2bNkt1wv0joDYwqsBcq+99k4NhHWrfI7uhi+0+rO4G+b9tX8ThRcsH5VIoSYjCh12RPtVeU+CKpNhAqxxIR+YPsOh+dOnVCbm4ufvzxR0yYMAH3338/8vLyFDdg+vTpKC0trf05ceKE4nP5jCke6P8hMFYAhmyHLSwBuqgL++ty4PME4Jt+9gCpnvgm4ZJO0yTM9a9B/Z1s9br5nJorSnyZVCuO1NzSsyUy2ycw8CCiRkl28BEeHo60tDT07t0bc+bMQY8ePfD666/DYrGguroa58+fdzi+qKgIFovF5flMJlPt6hnxR9cSeuHHXvsx/OA8FNc01UcQcu5HexDyWTP7VNFvLOZISS+vqLE5fVzsFmcM74Jt+SW1xcQGp1ucFjSzmCM0y11QczooECuoEhEFEq/rfNhsNlRVVaF3794ICwvDunXrMGrUKADA/v37cfz4cWRmZno4S2A5XV6JvMo09N67DOaQcixImY3M6Dxo/iW2+izw7dVAaCxw0//QJ7UdkswRbjtSdyzmCIzskYTZq/c6zXvYPHWQbqYQ1JwOYl0OIiLfkhV8TJ8+HTfddBNat26N8vJyLFmyBBs2bMC3334Ls9mMhx56CFOmTEF8fDxiYmLw+OOPIzMzU/JKl0BR99tzqTUa9xx9FeaQcjzabDn+mPgF3Mxi+Mfl8xBWpiEkLAEv//5rjFvufsM5kQH2qZpnh3eBxRyJcxXVmLhE/WJivtjZVZwOmvDxjtpaHyK500F6S6olIgo2soKP06dP4w9/+AMKCgpgNpvRvXt3fPvttxg8eDAAYO7cuTAajRg1ahSqqqowZMgQLFiwwCcN11Kf1HhYYkwOtTFKrdF4pfBBvFL4IGYlzccfEr+BQcOREAMA1BTj+v39cbB7KJ4oeAmrz6S7fY0AoLiiGhZzJPqkxqu+egTwbVl2tVaU6DWplogoWHhd50Ntuqnz4Ub2ngJM+2K3QyErkdgNL74zDtcXPQbh/C6XO836k/Dbz5+OTsGqskFuj319dE80j46QXUzM04iGqxUkate6UGNkhXU5iIjk8Uudj8bKVQcqMkeF4eXbu+H6jCQAufjf/kMoWXcPfm/epvlIiAHAm23+gdeFf+CPR5/B9xec5+KcLa+CTeJKDjHvwdOIhqcVJEpHUpxxtr+NXKzLQUTkOxz5kEFqufK6ZbLF11y6cAZLUqeiS+RxTYOQumwCsKx4MF4perDBTrrxTcJRUlHt8RxLx/dD6aVqtyMa88f2QkHpJcxe7blkvT/Kssvhi/wUIqJgJKf/1jo1MqB4WoIJNFyCKSZCllqjMfzQAgzcvwgFlWZdLNE1GoCxid9hR5cxGBX3H4fnznkIPMT6H73bxLkd0RAATFq6Q1LgAfh2BYnVJiDncHHtkmEpdTpYl4OISH2cdpFB6RLMukP4J0qTkHngE6SEF+DT1KdgCT+v+UhIiBH4e8obeLXlG/jy3O8wu/CRBiMhddVdPbL92DmPAZmcWly+WkHiy0RXIiKShyMfMnizBHNoRhI2Tx2EpeP74cH+bXGiOgmZ+z/GwP2LcOhSki5GQkKMwB0JG5CbPgZL2j4Fc0g5ACC+SZjDcXWLiak1UlG/kqqaxDyd+kGSuGQ4e0+B6tckIiLXOPIhg7dLMEOMBvRJjceUT3NrHztRnYSsg4tgDinHh21noEfUIc1HQgwG4NqYvchNH4PjVc2xr+sXiEns7DTvQa2RCgHA6GtSVDlXXf5MdCXXmDtDRHUx+JBBjUJWrvJGSq3RuPXwPKSEF+CNlFfRM+qgLoKQNhGn0frwABhOtQSyNgLRLR2O8RSQyTF37UEs++mEqlMham84R/JxyouI6uO0i0xi/obSfU08TVOcqE7CbYfnInX3Kjx17DFYbdB8SsYAAJdOAivTgBUdHDaxc7fRnBJqT4WwVLr/OEvo5ZQXETnDkQ8FvNkaXc40xfLSYVheOgx9onZhcerziDTWaD4agopD9k3sEAoMWA60vtVlTQyjQV6yKaD+VAhLpfuHs9ENS4wJlZdtnPIiogYYfCiktJCVkmmKbRd7IP2XL2EOKceq9pPQylSsfRCCy8Dm2wBjFDD8ZwzNaN8gINt6pBivrzso+8xqToWwVLrvuSq8V3f7AWc45UXUeHHaxc/cTVOIVUgfGZiKJHPDb+Kl1mhcd+BD3HxwHiqsJq9zLFRhu2ifjvnudwipOedQE6NdsyZenVqNqRBP9xuQvuEcNeQuoVcqTnkRNT4MPjTgKm/EHBWGJ7M64i9Du2DjUzcgvkm409f/UpmGrr98jrEn3kClwayPIOTMRvt0zKZRtTkh3k5lqDUV4m2eDrkmpfCeJ5zyImp8WF5dQ1abgLfWH8QHW47i/KUrm9QlmSMw+poUzF0rbcrCHFKOz9s/hfYRv+piEzsAQPc5sKZPxYBX1steCSNOhdQtU68GXy73bKxLSVfknsQTy3IVvdZXf89EpA05/TeDDw252+VV7l+KAUBKeAG+7TwZEcIFHeSE2JWa0jFi12Qcr5Y2uhCIu8Y25qWkOYeLJe1+XF8g/j0TkXvc2yUAeCp+JZcA+zLdrruXYfjBeSipaaL5El0AMFflYWPn8fg5/S6kRxzyeHx8k/CA6pAa+1JSMaHXVaxrABAbFQZLDKe8iOgKjnxoROk3RjnMIeV4KXkebjL/CKMOwkxBAM5fjsI9+S8hrzLN6TFz7+qB23q18nPLlPG0y3FjmVYQAzDAeeG9hff2Urw0nYgCB0c+AoCcDH+lH9Gl1mhMPDED7faswrunb9F8JMRgAOLCLmJNxyexr+vNmN7in7X7x4gs5kiNWiefnOqpwUxKQi93ByaiuljnQyNSM/wnZ3XEsp+Oe72i4KXC8fhXyc320u1NDmqemBoRAjzS4is83Pwr7CxPw4MnZiOyabOAqrfB6qlXeFN4j4gaHwYfGpFa/GrSoDRMGpSGbfklKCy9hNmr9+JcRbWivBCxdPvcu3vitrIngZOfefcmVGAwAL1iDmFn+hicib4eITVXAabACEBYPdWR0sJ7RNT4cNpFI3KKX4kf6rf1aoWXbstwej453y8tMRHA9cuBEYeAuKtlt90XDAag+YWNwOeJwLHlWjdHEinJlkmsnkpE1ACDDw0pLX5ljgpr8FhsVBgWjO3ltDJqXQ6dYXR74KafgLECYBmsj2JlEIAtdwHfDXLYwE6PWD2ViEgZrnbRAWcFqgA0eOy7vEKndUFEN2VYAAj4Zk+Ry2stGHsV4pqYnM/LF2+HbdOtMFzSUbEyAGh5B9DvHd1OxzTmOh9ERCIWGQtwzncIjUDlZSvOX6xx80rPYqPCHM5Rv5MUr92q5ie8nvIyksJLdVOwDE3aAYP+Yx+x0ZnGWuGUiEjE4MPP1Ox4XFU99RWxlU9mdUTppWq8v+Wow/Mp4QV4r/UsdIj8VT9BSGgMcON6IKG31i0hIqLfMPjwIzWH3K02Af1fXudxK3ItXN90O95t+1eYjFatm3JF047AkBzdTscQETUmLDLmJ2qX1n5r/SFdBh4AsPFCb/TZ+zHOxo3QuilXXDgA4fME2L7pDZQf9ttlrTYBOYeLsSL3JHIOF8Nqkxa/K30dEVGwYZ0PhTztzWIAMGtlHganWyRNwWTvKcDctQfUbqaqSq3R2JK0ELcMWgzkjANOrdS6STAAMJzbAWFlGgxpE4A+C3yaf6F0pItJqUREV3DkQyE1S2uLgUwgaB4dYZ/m+N3XwKhiIG0S9PBrZACAQwshLDEgb3FX/OXDL/HEslyMWbQVA15Zr8oGb0pHuhr75nNERPVp32sEKDVLa3sKZPTCaAB6t4m78oApHujzJjDWCow8DES00rxWiAFAt4i92NRpPDZ3uh8p4QWqdPJSdiGetTKvwVSK0tcREQUzBh8KqVlae21eobfN8QubAGw/dq72zw45DEVmVI88hgdOzcf5mihdbGLXylSMTZ3G4/VWLwKwd/JbDp5VlHOhdKSLm88RETXEnA+FpO7N4qm0dvaeArxXb3mrnokjOc5yGOKbhKOkog16nv0UKeEF+Kr9FMSHlmu6RNdgAEbG52BE3M04VZ2I8Z88i7zKNADyci6UjnRx8zlqTFjvhqTiyIdC3pbWttoEbDl0FtM+3+27RvpA8+gIlzkMJRXVtf99ojoJvfcuxfCD81B2OVQXIyEtTWexusOTWNz2WQDyci6UjnRx8zlqLLL3FGDAK+sxZtFW1fOtKPgw+PCC0r1ZxH+k9/zzR5y/5F3FUn+KjQpD7zZxeP5r5zkMzuRVpqF73lcYuH8RTldF6yII+V1MLvK73Yyd6XejT9QuSTkXSjeR4+Zz1BgwqZrkYpExFcgZavR3BVM1xUWF4Q+ZbfD6ukOKz5ESXoAv2/8ZCaFluqiYKgjAgcqWKBuwAdd0SnN7rKu/O/Ft1A84xd+LtXmFTqfWXL0uEHB4nURWm4ABr6x3mdskTkFvnjqIvyNBTk7/zZwPFYhb3nvibuVDIDh3scarwAOwT8dcvXcJ0iMO4dP2U9HEWKV5TkinyJPA9g7Azgig40Sg69MNqqZabQL2F5YjMjwEF6sdq7zGRoVhzu3dHAIIZzkxRoM9aVdk0bjOh9IAgjVLqC45SdVSPiepcWDw4UeBsqTWH/Iq05Dxy+dIjziEz9r/HyKNl7UfCbFVAvv+bv/p8z6Q9gAAe2c77YvdLjf1O1fvcVcjJPVndrQcdPSmWJqz9yYOrwfiKA55h0nVpARzPvyI//gayqtMQ/ovX9kTU2siNM8JqbXtQWCJAUezH8C0pRvd7iYsVrO12gRZo1tFZVWazIcrnZ9nzRJyhknVpASDDz/iPz7X8irT0H3vZxi4fxF+vtCuwSiBVtqWLMaO9DH4U+LHLo+pO6wsZ3RLiw7bmwCCNUvIGSZVkxIMPvzoXJ2lqOTcieokjDzyBtrtXoWb9s/DqnP9UK3xRrpGAzAleRkOZ9yM5y0LYA4pd3rc6fJK2aNb/u6wpQYQi7fkNyjGxuF1csbbsgPUODHnw0+sNgGzVwfG/i16sbcqDZNO2GtyzEqejz8kfKNpXkiIERjXfA3ub7YG35RmYvrJP6HUGl37vDcjW1I7bG9XmUi9zuzVe2v/W8wF4fA6uSKWHaifR6R1UjXpF4MPP2nMyaaWGBMqL9vc5k14MvPURPx4oTv+0fofiDBqWxvFYACGxeZgqDkHcwrux6KzdzoMK7urfOuKlA5bjVUmSgIDMRdk/tirVKnqS8FpaEYSBqdbuASbJOG0i5805qHohwa08yrwEK0puw59936E14tGo9Kq/a+u0QA8nfQhDnYdgXm/K0GI0eB2CNoZqfPhahVx8jQ/74wYaMxevRczhnN4nVwTyw7c0rMlMtsn8HeBXNL+E7yRaMxD0SfOXVTtXKXWaMwtuhedf/kadx56ERcvh2m6QsZgAMJCBPTNvwu2T83Y8cv/UHXZhuHdkzxOEUntsNVcZSI3OKp7nYLSSsQ1CVdU1ZeIqC5Ou/iJp43ogtnFqss+Oe9PF3sgPe9Le8GydlPRJETbgmXGy2W4KvcaGCvaY9ev02AT3HfEUubDrTYBi7fkq1LEScwXqbpsw5NZHbF023EUlskbkTtdXolberbUxfA6q6wSBS4GH34ifuOc8PEOGIBGE4CYI0Oxft8Zn14jrzINGXn2gmXvt5mJFuGlmgUhBgPQs+lh/LfzeFTbgCXFwzH39L0OiamxkWGYf08v9GvnfljaWY6HO+6m9pydyxJjwuSsDmib2ARny6sckkxdEUfwpFb19RVWWSUKbJx28SNXG9EFs9JLl1Fy0T9LjPMq09Bv/ydI3b0Ki0/fpHmtkHAjMK7ZauSmj8GKto/XLtE9f6kG+wrKsOrnUw5LWetylePhjqupPVfnKiqrwry1B2EKNWJc/1RYYkwuz62nWg3cxIwo8HFjOQ3UHS4+erYCc9ce1GQ0ZLLCofdAkRJegC/bTUZC2AXtS7fDvond2ctm3J8/C3mVVzaxq/+N3dNGXfW527hL6qZfM4an4+mvnJeQ19MGeNzEjEi/5PTfHPnQQN2M8CeyOuJtDUZDkswRmDQoDVumDcLkrA5+vba37u3bGrGRYR6PO1GdhKv3LcPwg/Nw/nITzUu3GwxAs7BSrO7wJH7oeF/tSEj9b+xylmV7SlqVWlTssSU7XK5IMkeF6SLwAFhllShYMPjQgaEZSdg8dRBmDO/it2uKnVWI0VAbACUFyHTQiXOX8NJt3SQfn1eZhp55/8bA/YtQUBWriyAkOeIcctPH4FDGCAyPWQ/gyooVOcuyPa0yUWOJd2RYCAanW7w+jxpYZZUoODD40IkQowHj+qfKrsEgl9EALBh7VYPOSgyAJt2Q5uKV+rHxwBkcPH1B9utOVCchc//HGLh/EQ5caqWLICTUKODNNv/A/q43I65qD7bll0helj1jeBdsnjrI7YiEGku89TSSwCqrRMGBwYeOeNojwQB7nsbro3ticlaHBgmCSeYIDE5v7vYab9x9FYZ1T3Z5/f5piZLbGx6izZy6AcAHP+Qrfv2J6iT8/uDb6Jm3FP8uvgE2m3ptU8JgAMJDgNUdnkSfH5PQJzJX0kZd4/qnesxrUFJUzBm9jCRwEzOi4MCEUx1ytYxw9DWt0TYxqramAQCndQ7W/HwKz67Yg5KKhnP4npYjWm0CrnnxO6evra+pKQQXqjTe9U0lf2/5Gm6P36iLxFQAqEEk7j30LLZd7OGQiKwk+VNcHQKgwbmk/uNfOr6fpktr63L3fgB9JMYSNUZy+m8GHzpVf0WMfVVKVe3znoKINT8X4LElOxo87ukD2moTMGnJDnyzp1CV9+FLsZFhKL1Uo9oqoZTwAixJfRqtws/oJgixCgZsvZCBaSf/hBPVSYprWbgKaGcM74LZq/e6TeJM0uHqEdb5INIfBh9BRPyWV/8vyV0QoXQ5otyiVlqbnNUB89YeVH2Jckp4AZa1m4rksBLdBCECgFOGbvj1qs9xdcc0RYGAq4qgc9bk4Z1NrqexHhmYiunD0r1ovW+wwimRvnCpbZBQuqeHkuWISopaacloACb8Lg3zx14FtfubE9VJ6L/vIwzcvwh7yltrXqwMsAeMLYXd6LO9I4o/TsSWbdmyz+Fs0y+rTcDXu9wX5fp6V4GkfWP8jZuYEQUuBh86prSmgdzliO6CHL2yCcD2Y+cQ18Tks+DgRHUSbs5fgHa7V2F+0e2ar44B7MmpzUNLcO3Bm1D4dRZyDhdjRe5Jl5VSPZFSU0RPq118zWoTvL6nROSZrOBjzpw5uOaaaxAdHY3mzZvj1ltvxf79+x2OqaysxMSJE5GQkICmTZti1KhRKCoqUrXRjYXSmgZylyPKKWqlJ6fLK/22CuO1ogfRM28p3j59O6pt2m+JZDAAlgvrcE1OImr+ey8eff87DHhlvezS4qybcUX2ngIMeGU9xizaiieW5WLMoq2K7ikReSYr+Ni4cSMmTpyIrVu34rvvvkNNTQ1+//vfo6KiovaYyZMnY+XKlVi+fDk2btyIU6dO4fbbb1e94Y2B0poGcpcjBmrH0jw6AkfPVng+UCWl1mi8XPggOu75CgP2LkLuhfaaj4aEGoE7EjYgN30Mvms5HB988YGszjKQ6mb4clSC+8UQ+ZdXCadnzpxB8+bNsXHjRgwcOBClpaVo1qwZlixZgjvuuAMAsG/fPnTp0gU5OTno16+fx3My4fQKMXG0sLTS6ZSIu30s5CxHzDlcjDGLtqrefl8R3/fGp27AwFfXO6wC8reU8AL8p8NjiDDW6CI5VRCAwssJaH7XfoREel4aq+R3TItET1+ubuF+MUTq8FvCaWlpKQAgPt7+DXr79u2oqalBVlZW7TGdO3dG69atkZOT4/QcVVVVKCsrc/ghO09FxwDXe3q42kHXWTlutQpR+UPd97392DlNAw/AnhfS5Zcv8fixKbhsM2g+EmIwAElhxTB+mQisHwJUuc/VkPs7psXUhK9HJZTmVjE/hEg5xcGHzWbDk08+if79+yMjIwMAUFhYiPDwcMTGxjoc26JFCxQWOq8bMWfOHJjN5tqflJQUpU0KSnKCCGev3Tx1EJaO74fXR/fE0vH9nJbjdtcB6U3d962n6aJVZYOQtmcleuYtxaGKFtoHIQBQ+B/g8wRg81i3QYjU3zEtpiaUrviSQ0neC/NDiLyjOHNu4sSJ2LNnDzZv3uxVA6ZPn44pU6bU/rmsrIwBSD1DM5IwON2iaKhbXI4o5RoL7+2FaV8431Zda91axuDpYekO71sPeQj1lVqjkXX4PaSEF2BR69noFHlc++mY40shHP8UB9Lexb7wIU5/f+r/jiU2MQEG4OyFKuQcLkbvNnFugwAD7EHA4HSLqlMTUkclth4plrU1QF1y815c1d4RgzBWWCXyTFHwMWnSJKxatQqbNm1Cq1atah+3WCyorq7G+fPnHUY/ioqKYLE43xXTZDLBZDI5fY6ukBpEeGNwugXPf50HwHnwYfjtf7T4Vr/7ZBnOVVQ7dGzidJGrfAUtnahOwtBDC2AOKcdnbacgLapA0yDEACs6HnwIra2hmHlqPKYIoxrkS4i/Y9l7CvB/n+1y6PTjm4SjpKLa5fnrTk2o+XsqdVRi4ic78PKoboo6fU+/R2LOR5/UeI8jMb4KwoiCjaxpF0EQMGnSJHz55ZdYv349UlNTHZ7v3bs3wsLCsG7dutrH9u/fj+PHjyMzM1OdFpPPbMsvQWGZ+2+ZYuChxcfqjBV7HIbXA2G6qNQajcGHF2Hg/kXYWdFB0+kYgwGIDL2MV1svxA+ts9B/RzssX/WhQ86Cq6kVd4FHXWpPhUkdlTh/qUbx1I+cvBel+SFE5EhW8DFx4kR8/PHHWLJkCaKjo1FYWIjCwkJcunQJAGA2m/HQQw9hypQp+P7777F9+3Y88MADyMzMlLTShbQlteN4qH/bBvkBSeYIPDIw1adBQHFFNRZvyXcIQFzlK+hJRKgRw/pdiwcK3kLq7lVYWZKpfU6IAYgOrcQdpeNw9c7uePT979D/5XWY9sVur0aR1J4Kk5sMrTT/Q2reC+uiEKlD1lJbg4tx4w8++ADjxo0DYC8y9uc//xlLly5FVVUVhgwZggULFricdqmPS221I3XJ7dLx/dAnNd5pDkr2ngKf5404W2JptQlYvCUfs1fv9dl1lYiLCsOPT2chxGhA/5fX1a7OSQkvwGst56JP0zzVy8MrYbUBU3/9Ez47/3tFr/flclRXORaueLMDr6dlxHL+jehlF2Aif+HGcqSIq51w65Kyw2n1ZRt6zf4OF6ouq93EWgY03FTPU70GLSwYexWGdU9222m1DCvEjTFb8ceEr5BiOqtpbshlG/BpyWC8UvQgSq3Rkl7jj63ss/cUYNrnu3H+kueg9vXRPXFLz5Y+aYc3tXeIgh03liPZrDYBs1fneTzu7qtbeTxm+7FzPg08APvcujjELtZbWPXzKfRuE+fT68o1e/VeZO8pcDsMf7LGgo+Kb8XAA4vx2PGpmm5kF2oExiZ+h53pY/Cvtk8jPeKQx9dIWfbtraEZSZg/tpekY325Csqb2jtEdAVHPgiAvCqnnipLrsg9iSeW5arYOtcmZ3XAsp9OaD7aERsZivOXGgZcYhf0ZFZHzF17QNK5rm+6HQvbvICoEH0sebYKBmwu74lnTz2GE9WOf+cJTcKRM/1GhIf6/nuMnkYdfFlxlShQcdqFZJMTMHgaZg+Ecu3ie3h4YCqWbz8peTVHXZFhRrx4aze0iInAn5fvcrlSSOwUBUGQXJHVHFKOCYmfYlziKkToJAgRBGDVuWvx+K9POzzuz/wGOdsG+JoWZeb9IVjfly/wXjmS039rvz0n6YKcoWpP9Qz0XH9DZKnzLbWzJQaTP90l+xx/v7NHbT6HpyXKBaWVePLGNMxb53kaA/htE7uih/By0UNoFV6IOclvYkD0Lm1rhRiAEfE/YFjszdh6oRumnfoTTlT7t9KsuCql/qiDRYNRB3/U3vE3juhIx3vlHY58EADPQ9quuPrW6+obqjMxEaEoq/RtjkhdM4Z3wbj+qQgxGhSvknlkYCqmD7PP/UsdNTJHhqFUQsKky9eHlOPjtk8jIypf+6qpAGwCsOp8PzQbusrvnTC/carP1aoiLUaV9I73yjlOu5AicgIGkbuVBa6+GcwYno64JuG1HUdh6SVFIw9y1c8JcNY+TxKahGP2LRkY1v3KB4u/p5lSwgvwefv/Q7PQUl0EIQIAQ1gcMOAzIGmQ1s0hBbizr3S8V65x2oUUcTWk7Y676Rqpe9LkHC6W3dabuydh1c/Sq1nWX4kgt3YEYC8xXj/wAPw/zfRrdRJuOfU5FgwV0GX/3TDZSjSt8GoAgJpzwPc3ApGtgGG7AFO8T64VjCMect+TL+6BnMqtwTbVJBfvlToYfJADMWDYergYE5fscFlXoe5+F+5ImReX23nHRYXhH3f1xPZj5yS/pm5OgLv9Odw5V1GNiUt2YKGxV4M9UWaOSMeEj3fAAOmjRkrUDaKuykgCrioGqkpw4IuR6GDbov1IyKVf7TvpthkDa683se0UVOskg3GOXe578tU9YOVW6Xiv1ME6H9RAiNGA/h0S8fKobjDA9/UM6tZOkOLcxRpsP3bOY72FyVkd8Pronlg6vh82Tx1U++Hs6ZuLK+62cPdXmXenNTVM8SjuswI985Yi72JbzUu3AwCOLYXhi2Y4lz0Czy3/r9dbzrvac0bcSTYQt7KX+558eQ/k7uzbmPFeqYPBB7kkdb8LNa8VGxkm6fjT5ZVu2/f2vb3wRFZH3NKzJTLbJzgESd/lFSpup7uNw4ZmJGHz1EGYMbyL4vPXJ7b6wf5tGwRRdfVJjUdU02YYdugtDD84DxetYZoHIUaDgGFxOchNH4PVHSYh7FK+ok7S006ygPI9XbQi9z35+h542kPHAPsIi6eRzsaA90odnHYht6Tmbah1reiIMNzzzx89Hit+q5DbPqtNwFe5p7xuq6sh1RCjAYnRJq/PL5K6hFQcPXr04x3Iq0xD+i9fok/ULnyc+gzCjNB8iW7XyKPY2Gk8tl7IwPOr/4rB6bdK/h0Kxjl2ue/J1/fA3dQhK7c64r1SB0c+yCMxb8PZKIJY2rzutuze6NcuQfa3Cnftq29bfomigmL1uRtSPXr2otfnNxiAfz3Qx+VIhzNDM5IwOatj7Z+3XeyBjr+swvCD81Bc01TzkRCDAciM3oNvWt2Oc9+OBMoPS3pdMM6xy31P/rgHvhrpVPszQg/8OSocrDjyQYr5IvnN198qvO2gPCXaWm0Clm477tU1AHs10dBQo6T3WXf1w9Vt49AiOhxF5VcCrLzKNPTeuwzmkHIsb/cUOkT8qulIiNEAJJ5bBaxchUvGRJxOfhStrnkSIZHOv7EH4xy73Pfkr3ug9khnMCYJi/w5KhyMGHyQIq6WqorJb95E/0qrWEpZgujNh7OU4GdbfonbaqdySAmUnH24x0bZ82bqB2+l1mj8/uDbeHFgFcZcmgBj1WlV2umNSNtZtPn1BZw/9joOpb2NqzPHNjjG02ooqSuv9ETue/LnPVCrcqsvPyP0Ihir3PoLgw+SzVPym7vS61LJ/Vbh6RuWGJgUllUivkkYSirkVxqVkn+h5tC/p0DJ1Yd76UX7ezNHheH8xYbv85lNJvy9yb/wyvCWGFx0P3Buu1pNViw2pBxXHbkH+y7uQecbX3J4Lhjn2OW+p0C7B/74jKDAxgqnJJvUip7+2nDMU6njhwem4utdBbKW1xoNwEMDUjGocwtZQ6pqVTuNjQzD9hmD3SbOSqmyeNfVrfC6m/1kHhmYiunXmYD/9AeqirxutxoEAIYm7YABnwIJvWsfD8YhfL3U+VCb3j4jyD9Y4ZR8Sk8JgFKWIL6zKV/2eW0C8M//5qN3mziX5eOdEYfHldQRqeuB/m09Js5KWf3wr63H3F7nnU356NGqF4aNKgSKtwMbhmsehBgAoOII8O3VQEwGMHgjYIoPyjl2ue9JyeouLe6Xnj4jSJ8YfJBsekoAVFowDPitkzPA7SoQuUPDIUYDZgxPx2NLdihqE2Cv4DppUAe3x0j90JYyvTRjxR4MybAgJKE3MOq3GihFG4Et9wCVJyVdx2fK9tgrprYaBVz1CkKi2wfdN2Vf5Q1oOUqip88I0icGHySbnhIAvfnmJNT+j+vnldROiGsSrrhNADDn9m4eg52jZyu8ukZdxRXVDd9ji+uB23+1L4f98WHg9HrVrqfIr5/bf0wtgLTxQOfJsIbFBdUoiBRSAwqtkz319BlB+sQ6HyRb3XLovi697ok/vjnJDXB8PZScvacAc9ceVPWca11VfY1uD2StA8YKQMs7/LJxnltVRcAvL0D4PBHfLBqKR9//Dk8sy/W6fHsgkFpeXQ8VYfX0GUH6xOCDFNFLkR1PpY7VIDfA8XY5r7uOQexY1Pbp9hP4cqe9CFT1ZZvzolDXL4dhxCGcieoPq031JshigICbo/+D7V3G4uGETwEE9j4vnsgJKORUQ/UlvXxGkD5x2oUUUzMBUGlinC93lHU2NCylnXJ36a3L01SPNzku7pRXWjH537kA7Ct96sY+dYf11+RHYtKP02ETgM6mQ3gm6QP0j94Frb7AhhoFPN3yI0xL/gg/XUjHUycnB+USTjkBhZ6SPYMxSZjUweCDvKJGspy3iXGuipIlmSNwc/ckLPqv59UuUmonSG2nGgGRq47BHx1G/UEXcUTh4YGpDiuH9lWl4b6jL8IcUo6FrV9EZtM9mlVONRqAvtF52NhpPH6q6IwdB1fimk5p2jTGB+QEFHpL9mQhLnKG0y6kKbW2CRd3lF06vh9eH92zdgfYQZ1bSHp9/STR+kPDctvpasg5vom0XXtddQxarA4QYxFXQVypNRpj81/GwP2L8MvFVE33kDEYgD5N96HX9i7AqWztGqIyOQEFd12lQMCRD9KM2lUQnX3DkvqNccbwLrCYIx2GhgF7saTC0kuYvXqv7HY6G3Lu3SYO17/2veJVAOdU2BRPCQHulyQDwInqJAw/9CbMIeXY0vt5NK3c75e2OROCy8CGm4DUcUCvvwOmwO5o5awe0bIaqlZ1RSjwMPggzfhjq3Sp3xgt5kiHazibYlHSTmcBkdKOwWoTMHu1+smmaotq2gyRt+4Fdj0D7J2jbWPyFwO/fgn0WQS0uVPbtnhBbkChdH8kb6hRVyQQg5dAbLMeMPggzfgjMU5JvQFXNRLUaqecjqHuB9vZ8iqfJJuqrbYTvOolIO0hYPPdEM5t9+mKJLdqSoEtdwE7WgIdJwJpjwTkSIjcgMKfyZ5q1BUJlNLxdQVim/WCe7uQZvy1/4P4wQg4/8ZY94PR054parbT0zcmOaMvdU26IQ3tmzXB7NV7UeLnaZoHrm2LmSO7OjxWfdmG37/yBf4vdi5uis1BiEHjjxxTIpD5LyB5qLbtUEhv37Sl7jO0eeogtxtDutufSY9LcwOxzb4mp/9mwilpxl+JcXLqDShZyqq0neKUzC09WyKzfUKDwMNZgqsUcVFhGNmzJV66LUP2a731+64Whz9n7ylAvznrcLQ8EpNOPI1eeUswr2gMLlg1LKtddRbYOAI4thy4kA9UNax3YbUJzmudOCHnWDW4+73Rgrd1RfRQFE2uQGyz3nDahTTjz8Q4qUPQcqd41Ghn/W+yvdvEufxgk2L26r345+Z8zByRjgVje2HS0h0Nls/6Qv0AzNk3w1JrNOYV3YN5RfegT9TPeKv1q2gWdt7/S3SFy/apGFHT9kD7h4C0R5B9sEryUHqgDrurOXri7fSpP3K/1Ca1zYu35GNc/1TNA8S69DJyxuCDNOXPxDgp9QbkLmX1tp3OOq/4JmGSNoRzp+5c+1tjrsJjS3Z6dT53nAVg7r4ZirZd7I4++z6GOaQcr7Sci9+bf4JRqymZC4eBXU/j8q7nsfb4oygo/b3D085yF7TeP0UptQMmb+uK6KkomlRS21L3i4Aefhf0FCwz54N0QS/RuDh/7a46aXyTMMy4uSssMd61U2liqxxiW48XV2DptuMoLKtS/RrOPryk5vPUbeesIUloeXoR0iuWIcJ6RvV2ynHJGo4Pi4dh4Zm7UWqNBuCYuwDA6zwHLfgiT8HTvxlP98JfuV9qkvP7rZccEH/kqDDngwKOXuaxPW2IZQDw0m3dcNtV3rVTysiAGkoqajD537m/bURnwOSsjph7Vw/c0auVKuefMbwLNk8d1OBDS+631JKKGjz+xXHcvnkIbjiyFBs7/QA07aTZRnaRIdV4tPlX2JE+Bv9qOx0p4QUOw/962T9FJCXvxFd5Ct5uIheIRdHk7CmlhxwQPeaoMPggqscfG2L5ao8Wd4rKKjF37QHMWpWHz3b86tW5xA7B1Xy2N5VYC0srMW55Cda0/R5rL1yvacXUEANwXcxu/LfzeBzKGImZSQtw7nyBrqYKsvcUYMAr6zFm0Va3O/z6MmDy5t9MIO6A667Nzvg7GK1Pb8EywJwPIqd8XSNBi/lrsQ8/f9G7fBKRu2JoNkFAbGQYzl9yfS2DwXnVVLFq7IwVe1Bc8RRGxfXAyy3fQphR2610Q402PNBsDS7v/w6fGF8A0NXja3xdDl9O3omvAyZv/s1oURTNW67a7I5WeSt6CpZFDD6IXPDlhlhSO6X4JuEOtTriosJQddmGi9VWn7RLCgOAhwemOu0QpNQmEVc2uRvREAAU//a+Pz/3e6wty8SDiSvwcLPPEGm87FX7vRWKGtxvm4rbukZi8ZmReK/41tq8kLosMSafThXI3Z7g6NkKSef1JmDy5t9MIO6AK7Z58ZZ8zF691+PxWuzNJOe6/mwfp12INCB1nnvr9BtrN8ubnNUB5y7WaBp4APaO7d1N+bXD+mK+weyVv+BRCbVJLOYIPNi/raxrllqjMbfoXnTZ8xVuOjAPO8o7wKrtQAhiQi7hT5Z/Y1uXezA+8dMGz4/p09qnHaecoXSrTcDSbcc9nlPr3Aq95H7JEWI0YFz/VF3nregxr4bBB5EGpM5zh4cakdk+ATd3T8ayn074tY2ezFqZhzU/X8k3eG/LUbfHx0aF4ZOH+mLz1EEYnG5xe6wovklYg/uztzINt+fPRfs9qzDl+JMovRyl7A2oxGS04Znkj5Df7WZs6PQQ0iMOAQDaJjbx6XXlDKVvyy+RtNJp9DW+DZiCld7zVvTYPgYfRBrxdeVVXxK/VT+2RHoV1vMXa2A0GhBiNEj+JvbCLe6rtH5xPgsD97+HVwrux6lqbZdhGgxAW1MRVnd4Et+mPYrkyEs+vZ6coXSpgUrbRG0DuUDmj0R1b+itfcz5INKQryqvyiXWYpgxPB2zV8vfT0Yq8X1IrW47NCMJC40GPP/1Ly6/uZdao7HwzJ1YeOZOjLFsxUutFsBQrc2qAsAehHSK+hXW7Z2Qc+ZDIOkmn+QuyNk0UeoqBq1yEoKF3vNW9NQ+jnwQaUzKPLcvO4W6nf2w7knYPHUQZgzv4pNr1X0f4jexFjEmh2NaxJgcvokNzUjClmk3YnJWR4/nvz7rMRhGHAR6zAGiPR/vSyGwod+x+3A6+3YMe+2rBktfvT6/jKF0Pc75Byu9563opX0MPogCgJyiRnLVH3aVkkCnhNEA9G4T5+QZV13nFSFGA57I6oC37+2F2KiwBs/HRoXhbfE9mOKBrtOAEfuBO0qAkUeAvu9rUrDMYABuiduENa3uQPmG+/Da1z+ouvmc1KF0Pc75U+PG8urklF7KndMVSsux15/WED3Uvy2y0i0u/27F6wENp0WUfmjULZGttNyz1SZg65Fi5BwuBiAgs10i+kn4Bnd809NIOTHH/5vY1VFujcDUE09gp2GwqvUrpP579cXeHvysIJGc/pvBBzWgp82HGgM5H97Zewrc5j/UNzmrA5b9dKLBxnUv3JKBYd2TPb7e1e/CjOFdMHv1Xrd74Djz+uieuKVny9r9QPy5N4rVJuC5N5/DjPhXEBGiTqE1pQqqE/Cv4uHofeNU3Ngz3a/XVjNY4GcF1cXggxTzx+ZDdIWSD2+rTcAb6w7i9XUH3Z476bfO+9s9hXh2xR6HYmVyOghXnZWrkRF3xJEPqRtzffLHvjAaDKp9q87eU4BpSzfi6aT3cEfcOu120f3NOasZMYOWIqTlTZq2Qwl+VlB9DD5IES2+jTZm3n54z1mTh3c25Tt9zvDb6wH4tIOQUtFUvF7d350VuSfxxLJcj+evX6JdjW/VYpsvXjiDCc2W496ENWgaot0yZpshFMZrlwAJVwNhZnvOis7xs4Kc4a62pIgeNx8KVmrsMjl9WDoWjO2F+CbhDo8n/ZZsODjd4vOdLIdm2FfHLB3fDw+5qFrqLKFR6uqd+nvDiHuWeLNyRGzz2w8ORtLv3sCufifwv+4/oSy8nSZJqUbhMrDlLuDrdsDnCcDXacCOp4Dywxq0Rhp+VpC3WOeDaulx86FgJefD291eGcO6J2FIhvN1+zmHi1W5hifi0r3M9gm4JjVe0uZgnmpUuGtz/T1LvGnzFYlAxmF8n/MN2u8bh5Tw09olpl44DOz7m/0nIgno9DiQ9oiuRkT4WUHeYvBBtfS4+VAwcJYzoeaHt6vNvLToIKQWMXJXZMwTMWjaeqQY/dolqLrSIqJ5Hwxc8T4A4N74VXg2aREiQjTcS6eyANj1NLDnBaDfYqDNndq1pQ5+VpC3GHxQLTkVE0kaVwmlo69JkfR6bz68teogpO5s6mpL8tioMJy/6Hk1yviP/ofwUKPDsd7mhNT9N/Bxyc1YWXo9ZiW/jZGxm7RNTrVetE/NlM0Guj2rXTt+w88K8hYTTsmBu9oOADPY5XCXUCrA3smWXqxx++HtTcKemBToqYPQOimw/siQzSbgnvd+VHQuNX5Pnf29mUPK8UDi13gs8VOEazkSAgD9P9XFCAg/K6g+JpySYnrbfChQeUoordvV+6riZKBUtaxf7rlf+wTF1VXrJtJWX7Yh53AxVuSelFVVVPw3EBt5pZJqqTUa84ruQcdfVmBp8Y3Q9Cvbjw8ABeuBo/8GireremqrTZB8z/hZQd7gyAc5xaqF3pFax2JyVkcs++m4T4s02QuT5aGwLHAKQSmt5lpXfJNwxbVNAGDLwbMuR2BSwgvwcss3kdn0Z2j9z0IAUBHaCkfbv4EuV93q94Jh/KwgkZz+mzkf5JTUeXtyTs4W5punDnL54a3eB7tjN66z7xwNiN+qp32+u8FyW6nqBh7AlWW6Ur+ViyMwzlYMnahOwj35LwEARsWuxV9bLkCTkOoGx/mDAUDTy78iY//tuJhnwqmkCUi7boas1TGugj0p98zbzwoGL40TRz6IfEDqyEfdvU7qU6N0daBXoXQ3+qCE3DyX7D0FePS3vAZ3zCHl+PraL9Gm7HNA0DgnBIAAAwytRwPXvOUxCNGyYBjLswcXn+Z8bNq0CSNGjEBycjIMBgO++uorh+cFQcBzzz2HpKQkREZGIisrCwcPui8DTRRsvN3CXAwa6ncIcopsqVHITGve5H84I7f41dCMJEzO6uDxuFJrNE51XgDcfhro+hxgauFlS71jgAAcXwp8lQKcynZ7rFYFw9T4HafAJTv4qKioQI8ePTB//nynz7/66qt444038Pbbb+PHH39EkyZNMGTIEFRWstgMNR7eJHuqFTQEQxVKd/fRG3Jqm0wa1AGWGNfLkR0CSVM80GMWMKoQGPApEK7x1KX1IrBhGHBkMVDl/O9Zi3owwRAYk3dkBx833XQTXnjhBdx2220NnhMEAfPmzcOzzz6LW265Bd27d8dHH32EU6dONRghIQp2SlcDqBU0KO1U5Kx48AdX9zE2KgxNTcrS1uTUNgkxGvD8yHQYIDOQbH0nMOIA0GMOEN1RUTvVIQBbH7hSuv2XOQ6BiBb1YIIhMCbvqJpwmp+fj8LCQmRlZdU+Zjab0bdvX+Tk5GD06NENXlNVVYWqqivbg5eVlanZJCJNSa34WZda30SVdCp6nYN3dR+/zj2JyZ/uknwepcWvXBVEs5gjMPqa1qj6bVlvg79bUzzQdZr9p/ocUH0eKP4f8MNYQLgsqw2quHC4QcVULQqGsTw7qRp8FBYWAgBatHCc72zRokXtc/XNmTMHs2bNUrMZRLoidzWAWt9E5XYq3qx48Adn99FijpT8em9rm9QPgI6ercDSbccxd+2B2mPcBmrhcfafpqlAWDSQ8weg6ozsdqhCrJha9BhCesx2WereV/VgWJ6dNC8yNn36dJSWltb+nDhxQusmEWnK22RVkZy8k0Cdg/d0r+pSo/iVGACZQo2Yt/YgCsuqHJ6XnCyZPBS4eR/QYw6qQ+IUt8drhxYAnydg6P7u+C5rLTrGOS4X9lXBMLV+xylwqRp8WCwWAEBRUZHD40VFRbXP1WcymRATE+PwQ9SYqVmZVGreSaDOwUtJSH2of1ssHd8Pm6cOUqUTVS1Q+21K5ptOu/HnE0/iks3kddsUqzqLtNPzkN1uHL65swKvj+6p6j2rL1Cq75LvqDrtkpqaCovFgnXr1qFnz54A7DkcP/74IyZMmKDmpYiCmrscA7n5F1LyTvQ0By+36JSre+WrXBU5gZqU6bbm0RH4/FwW1pb1xdj4bNwd/x+0NWmzzNRQU4Yu+8egS+83gLhrgRqDrGJlcoh/b/Wr7yr5HafAIzv4uHDhAg4dOlT75/z8fOTm5iI+Ph6tW7fGk08+iRdeeAEdOnRAamoqZsyYgeTkZNx6661qtpso6ClJVnXFU96JXubglSa8qnmvPFE7ULuSmwMsPHMnFp65EzEhFxBjvICksCJ82v4ZGPw6AGADtk+68sfwRKDDo0DnyT4KRAKr+i6pQ3aF0w0bNuCGG25o8Pj999+PxYsXQxAEzJw5E++++y7Onz+PAQMGYMGCBejYUdpSM1Y4JfI/f+2A625UI1CqsapRvbY+dzvEvtbqHxgVv17VOieKhMUA/f9tz1dRQaD8fZN0cvpvllcnIgC+3yLd3ajG4HSL30t8K91TxFeBmttRn7JpEI5+pH0AAiPwu9VeByBalnT3BvehcY/BBxEp4qs6H2t+PoXHluxs8Lj4sf1kVgfMXet5GwY5ownuePs+3QVqAoAH+7fF4HSL7M7JVedmtQl46B8L8Vz8i0g1nfLzNEw94Qn24mleTMH4YvTI1/RaA0dPGHwQkWJqf7tb83MBJi3dAVeLPwwAzJFhknavfX10T9zSs6XitgDqDfc764yMBji8T7U6p/qd9R8TP8dTlg9hMtq8Oq9iPeYAaQ8DNaVAmLlBIOLpd2hF7kk8sSzX42XU+PtWA6eIpJHTf6u62oWIAp+3W6TXlb2nAI8tcb8rrABICjwA7xNePS2TNcC+THZwusVjwFU3yXVtXiHe23K0QYClVoG2+smr/zw7CsvP/R4PJq7AfQmrER9arvjcivz8HLBr+pU/N20PpNwGpD2K7GNRHkcI9JLgLIWavzN0heZFxogoOIkf2lLFRob5vOiU2vVMQowG9EmNx5o9zis4q1WgzVknXGqNxtyie9Erbym6/7IMNx14A0dbzlR8DVmEesHihcPA3r8BK9PQ43/dcFv4RzCHXAmI6hdfC6QiY4FaA0fvGHwQkU94+tCu74H+qQB8W3TKF/VM/NE5eeqsy61Ncd6UjpTrZgLdZjv9lu4vSeHF+EvSR9jQaTyub7odQMMgLJCKjOmpBk4wYfBBRD4h58M4yRyBSYPSFO0CLIcvhvvV7pyc7Sosq7Pu9ixs1/4bF23aTlnEhV7A4tSZWJM2CSnhBQ2CMKW7PvtbIE0RBRLmfBCRT8j5MBY7Tl8XC/PFDq5qdk6eVlRIrXob0vYu5JzPwP/WvYpxiSvRIkybKQGDAUiPOopNncbj54tpmH5yEk6X96x9Xq2/b18ugdVi19/GgKtdiMgnPNXDAOyrQ964uycSoiP8VjtB7XomatX9kLqiQk5HKwYzPYW1eCVlHmJCtJ8asCEERsuNwDULgOj2Ds8pCSL8sQTW1zVwggWX2hKRLrj60BaNvy4Vq34u8HvtBDU6rLod5dGzFZi79qDLLek9dU6+LLoltvPc+QJ0K/8Qrc4uhqH6rKxz+IYB6PZXoNuzAJT9nfhzCawe63zoregZgw8i0g1XH9ojeyTh3U35mtVO8OaD29l7io0KAwCcv3hlJYjUzsnvRbeqzwEXjgFntwDbnwAEq/fnVKrbbGQbHpIdRGhRJVVPnb0egyHW+SAi3XA2r9+7TRyuf+17TWsnKK1n4urbdulvQcfkrI5omxglq3Py+4qK8DggPg6I72mv0bFltL1gmAaE3c9hXaEVw2IqkF/dEnmVaVeeg+vfBbV3F5ZCzRo43nD1O6hWXRl/YPBBRD5X/0M753Cx3zsONUgpOLXsp+Oyv21ruqIieSgw8giwfx5wcCFQVX9KJgSA70ZGDBDwmuX52j8LAlBQk4Bnfn0U31/IdPm70FiXwAZL0TMutSUivwvUjsNXNT00L7plige6/xUYdQa4owS4aScwdKf9v3vM9s01XTAYgOTwYnzQ7kUc7nYzPmw7AynhBQ1+FxrrEthgKXrG4IOI/C5QOw5fBU26KroVHgfE9bRPyYTHAWmPAKZE31/XiRADcH3MTmzs9DCuKpvv8JzmAZtGAjVwr4/BBxH5XaB2HL4MmnRbdMsUD2T+C4JBu1l6o0FAyq9zgN0v1D6mq4DNjwI1cK+POR9E5HdixzHh4x0ul6fqsePwdcEpXxdZUyx5KAzXr0TVf++FyVqsSRMMALD7OaDtmNr6IHIKrwWLYCl6xqW2RKQZPS4X9KRRF5yqKsH+LX9D3K+L0DxUo1ohliHAoGyHh/S0BNYf9Po7yDofRBQwArHjCMSgSU1Wm4Bde3cgZf8EJFb+5HL6zCcMYcCYan9eUZf0+DvI4IOIyMf0EjRp3o7dLwC7Z/jveoB9aXDTVMfHqkrstUrCzPY8lUZA87/7elhkjIjIx/RQcEoX3367PWvPw9h8F3Buh3+uWXHCHnxUlQCH3gGOLAbKD1x5Proj0G7cbyt1gjcQ0cPvoFIc+SAiCkD+3NdEsqoSIO9lCIfegaGmzHfXGXkEKNsP5NznpChaHaZEIPNf9kJq5HNy+m8utSUiCjCeqlwC9iqXVpufv1ua4oGrXoXhzlJsSfsGW8u7otoWou41DGH2wGPjCPeBB2B/fuMI4FS2++PI7zjtQkSNnt7mzj3RYl8Tufr3GYrsqB64fmUejBePIjnsDJoYK/BB6gsweHNrmw2wj3gIl6UdL1wGcv4A3LwvqKdgAg2DDyJq1HSRNyFToFS5dFa3xFZ0DCFH/6XwjAYgvhdw+nt5L6s6Axx6F+g6TeF1SW2cdiGiRkvMm6g/iiDuDpq9p0CjlrkXSFUuxaTIW3q2RGb7BIRc+xHQ9g9Op4w86vZX4ORKZQ058oGy15FPMPggokZJt3kTEgRqeXqRtd9iPHBqPo5UJkPakgcD0G020PExx1UtcpQfAKrPKXstqY7BBxE1SoG8O2ig72uyLb8EG862waAD7yJ19yqMOzwDv1xsi2qbY3tthjB7RdMRB+1LemtKvbtw9XnvXk+qYc4HETVKgZI34Uog72tS/55uqOiLDYf6AgBahhUiOewMTtU0w19GDcUtPVteOTDM7N2Fw2O9ez2phsEHETVKgZQ34YpuN6LzwN09PVljwckai/PjTPH2AmJKpl6iOwLhcfJfRz7B4IOIGqVg2R00EKtcenXv240Ddj0t/6LtHpD/GvIZ5nwQUaMU6HkTgcyre5/2iL1yqRymZkDaw7LbSb7D4IOIGi0xb8Jidhzet5gjNNuWvLFQfO9N8faS6QaJA/eGUCDzI/UKjFWVABfy7f9PinFvFyJq9AKtwmkwUXzvT2XbK5dWnXF9jKmZPfDwdm+XRr6BnVRy+m8GH0REFJiqSuyVS4984CQoeMA+1eJtUHAq2/MGdmGxQPfZQNuxjToIYfBBRESNS/U5ex2P8Fj1VrWcyrZvTCd1HxkACE8EOjwKdJ7c6AIR7mpLRESNS3gc0DRVvcCjqkTeBnai6rPALy8AK9oAx5ar05YgxOCDiIiovkPvuJ9q8eTyBWDLXcCXrYBf5jBBtR4GH0RERPUdWazOeS6dtNclWdXJPo1DABh8EBEROaoqUb6BnctznrXnjzAAAcDgg4iIyJG3G9i5Ilz+bXkwp2AYfBAREdXl7QZ27lSdsS8PbuQYfBAREdUlbmDnK0c+8N25AwSDDyIiovrajfPducsP2OuSNGIMPoiIiOpTsoGdHNXnfXfuAMDgg4iIqD65G9jJFR7rm/MGCAYfREREziQPBa5fad+gTk3RHdWrxBqgGHwQERG5kjwUuHkf0GMOENlSnXO2e0Cd8wQwBh9ERETumOKBrtOA234F+n8KhEZ7ca5m9t12GzkGH0RERFK1uRO45SiQMUN+QqohFMj8qNHtdusMgw8iIiI5TPFA978Co84Ad5QAV78FhHnI4TA1s+ePJA9Vty1VJcCF/ICrmuqjNF4iIqJGIDwO6DgRaDPGXrn0yAeO+8JEd7TneKQ9rN6IR1WJfdfdI4udXGvcb8uE9T26YhAEQdC6EXWVlZXBbDajtLQUMTExWjeHiIhInupz9joe4bHqr2o5lQ3k3GffqM4VU6J9mbDaoyweyOm/Oe1CRESkpvA4oGmqbwKPjSPcBx5AQOygy+CDiIhI76pK7CMewmVpx+t8B10GH0RERHp36B3PIx71udpBVwdJqkw4JSIi0rsjixW+7gN7jRKdJan6bORj/vz5aNu2LSIiItC3b19s27bNV5ciIiIKXlUljgGDHOUHgGPLgVWdgF1PNzxP+QH746s6+TVHxCfBx7///W9MmTIFM2fOxI4dO9CjRw8MGTIEp0+f9sXliIiIgldNqXev3zJGd0mqPgk+/vGPf2D8+PF44IEHkJ6ejrfffhtRUVF4//33fXE5IiKi4BVm9vIEVmmH+TFJVfXgo7q6Gtu3b0dWVtaVixiNyMrKQk5OToPjq6qqUFZW5vBDREREvzHF23Mz/MFVkqrKVA8+zp49C6vVihYtWjg83qJFCxQWFjY4fs6cOTCbzbU/KSkpajeJiIgosLUb579rHfnA55fQfKnt9OnTUVpaWvtz4sQJrZtERESkL2mPyN/ITqnyA/YqrT6kevCRmJiIkJAQFBUVOTxeVFQEi8XS4HiTyYSYmBiHHyIiIqrDFG8vmW6QWCFD6nGuVJ/37vUeqB58hIeHo3fv3li3bl3tYzabDevWrUNmZqbalyMiImockofad8Y1NXN/nKkZcO0S764VHuvd6z3wSZGxKVOm4P7778fVV1+NPn36YN68eaioqMADDzzgi8sRERE1DslDgZv3SdtB9+eOyuqDRHdUf1+aenwSfNx99904c+YMnnvuORQWFqJnz57Izs5ukIRKREREMpni7VVLu05zv4Nuu3H2AmJytfP9QIFBEATB51eRQc6WvERERORCVYm9cqmcPWFMzewjKwpKrcvpvzVf7UJEREQ+oCRJNfMjv+zxwuCDiIgoWMlJUr1+pf14P2DwQUREFMzEJNUecxpWSo3uaH/85n1+CzwAHyWcEhERkY5ITVL1EwYfREREjUl4nGZBh4jTLkRERORXDD6IiIjIrxh8EBERkV8x+CAiIiK/YvBBREREfsXgg4iIiPxKd0ttxa1mysrKNG4JERERSSX221K2jNNd8FFeXg4ASElJ0bglREREJFd5eTnMZrPbY3S3q63NZsP+/fuRnp6OEydOcGdbPysrK0NKSgrvvZ/xvmuH914bvO/a8dW9FwQB5eXlSE5OhtHoPqtDdyMfRqMRLVu2BADExMTwl1IjvPfa4H3XDu+9NnjfteOLe+9pxEPEhFMiIiLyKwYfRERE5Fe6DD5MJhNmzpwJk8mkdVMaHd57bfC+a4f3Xhu879rRw73XXcIpERERBTddjnwQERFR8GLwQURERH7F4IOIiIj8isEHERER+ZVmwcf8+fPRtm1bREREoG/fvti2bZvb45cvX47OnTsjIiIC3bp1w5o1a/zU0uAj594vWrQI1113HeLi4hAXF4esrCyPf1fknNzfedGyZctgMBhw6623+raBQUzuvT9//jwmTpyIpKQkmEwmdOzYkZ85Csi97/PmzUOnTp0QGRmJlJQUTJ48GZWVlX5qbXDYtGkTRowYgeTkZBgMBnz11VceX7Nhwwb06tULJpMJaWlpWLx4sc/bCUEDy5YtE8LDw4X3339f+OWXX4Tx48cLsbGxQlFRkdPjt2zZIoSEhAivvvqqkJeXJzz77LNCWFiYsHv3bj+3PPDJvfdjx44V5s+fL+zcuVPYu3evMG7cOMFsNgu//vqrn1se2OTed1F+fr7QsmVL4brrrhNuueUW/zQ2yMi991VVVcLVV18tDBs2TNi8ebOQn58vbNiwQcjNzfVzywOb3Pv+ySefCCaTSfjkk0+E/Px84dtvvxWSkpKEyZMn+7nlgW3NmjXCM888I3zxxRcCAOHLL790e/yRI0eEqKgoYcqUKUJeXp7w5ptvCiEhIUJ2drZP26lJ8NGnTx9h4sSJtX+2Wq1CcnKyMGfOHKfH33XXXcLw4cMdHuvbt6/wyCOP+LSdwUjuva/v8uXLQnR0tPDhhx/6qolBScl9v3z5snDttdcK//znP4X777+fwYdCcu/9woULhXbt2gnV1dX+amJQknvfJ06cKAwaNMjhsSlTpgj9+/f3aTuDmZTg4y9/+YvQtWtXh8fuvvtuYciQIT5smSD4fdqluroa27dvR1ZWVu1jRqMRWVlZyMnJcfqanJwch+MBYMiQIS6PJ+eU3Pv6Ll68iJqaGsTHx/uqmUFH6X3/61//iubNm+Ohhx7yRzODkpJ7//XXXyMzMxMTJ05EixYtkJGRgZdeeglWq9VfzQ54Su77tddei+3bt9dOzRw5cgRr1qzBsGHD/NLmxkqr/tXvG8udPXsWVqsVLVq0cHi8RYsW2Ldvn9PXFBYWOj2+sLDQZ+0MRkrufX1Tp05FcnJyg19Wck3Jfd+8eTPee+895Obm+qGFwUvJvT9y5AjWr1+Pe+65B2vWrMGhQ4fw2GOPoaamBjNnzvRHswOekvs+duxYnD17FgMGDIAgCLh8+TIeffRRPP300/5ocqPlqn8tKyvDpUuXEBkZ6ZPrcrULSfbyyy9j2bJl+PLLLxEREaF1c4JWeXk57rvvPixatAiJiYlaN6fRsdlsaN68Od5991307t0bd999N5555hm8/fbbWjctqG3YsAEvvfQSFixYgB07duCLL77A6tWrMXv2bK2bRj7g95GPxMREhISEoKioyOHxoqIiWCwWp6+xWCyyjifnlNx70d/+9je8/PLLWLt2Lbp37+7LZgYduff98OHDOHr0KEaMGFH7mM1mAwCEhoZi//79aN++vW8bHSSU/M4nJSUhLCwMISEhtY916dIFhYWFqK6uRnh4uE/bHAyU3PcZM2bgvvvuwx//+EcAQLdu3VBRUYGHH34YzzzzDIxGflf2BVf9a0xMjM9GPQANRj7Cw8PRu3dvrFu3rvYxm82GdevWITMz0+lrMjMzHY4HgO+++87l8eScknsPAK+++ipmz56N7OxsXH311f5oalCRe987d+6M3bt3Izc3t/Zn5MiRuOGGG5Cbm4uUlBR/Nj+gKfmd79+/Pw4dOlQb8AHAgQMHkJSUxMBDIiX3/eLFiw0CDDEAFLgFmc9o1r/6NJ3VhWXLlgkmk0lYvHixkJeXJzz88MNCbGysUFhYKAiCINx3333CtGnTao/fsmWLEBoaKvztb38T9u7dK8ycOZNLbRWSe+9ffvllITw8XPjss8+EgoKC2p/y8nKt3kJAknvf6+NqF+Xk3vvjx48L0dHRwqRJk4T9+/cLq1atEpo3by688MILWr2FgCT3vs+cOVOIjo4Wli5dKhw5ckT4z3/+I7Rv31646667tHoLAam8vFzYuXOnsHPnTgGA8I9//EPYuXOncOzYMUEQBGHatGnCfffdV3u8uNT2qaeeEvbu3SvMnz8/eJfaCoIgvPnmm0Lr1q2F8PBwoU+fPsLWrVtrn7v++uuF+++/3+H4Tz/9VOjYsaMQHh4udO3aVVi9erWfWxw85Nz7Nm3aCAAa/MycOdP/DQ9wcn/n62Lw4R259/6HH34Q+vbtK5hMJqFdu3bCiy++KFy+fNnPrQ58cu57TU2N8Pzzzwvt27cXIiIihJSUFOGxxx4Tzp075/+GB7Dvv//e6We2eK/vv/9+4frrr2/wmp49ewrh4eFCu3bthA8++MDn7TQIAseziIiIyH+YwUNERER+xeCDiIiI/IrBBxEREfkVgw8iIiLyKwYfRERE5FcMPoiIiMivGHwQERGRXzH4ICIiIr9i8EFERER+xeCDiIiI/IrBBxEREfkVgw8iIiLyq/8HKZpdqMxuTr4AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "loss =  38.59655\n"
          ]
        }
      ],
      "source": [
        "from IPython.display import clear_output\n",
        "\n",
        "epochs = 1000 # число эпох \n",
        "\n",
        "w = torch.randn(1, requires_grad=True)\n",
        "b = torch.randn(1, requires_grad=True)\n",
        "\n",
        "for i in range(epochs):\n",
        "\n",
        "    y_pred = w * x + b\n",
        "    loss = torch.mean((y_pred - y)**2)\n",
        "    loss.backward()\n",
        "\n",
        "    # делаем шаг градиентного спуска с lr = .05\n",
        "    w.data -= 0.05 * w.grad.data \n",
        "    b.data -= 0.05 * b.grad.data\n",
        "\n",
        "    # обнуляем градиенты, чтобы на следующем шаге опять посчитать и не аккумулировать их\n",
        "    w.grad.data.zero_()\n",
        "    b.grad.data.zero_()\n",
        "\n",
        "    # рисуем картинки\n",
        "    if (i+1) % 5 == 0:\n",
        "        clear_output(True)\n",
        "        plt.scatter(x.data.numpy(), y.data.numpy())\n",
        "        plt.scatter(x.data.numpy(), y_pred.data.numpy(),\n",
        "                    color='orange', linewidth=5)\n",
        "        plt.show()\n",
        "\n",
        "        print(\"loss = \", loss.data.numpy())\n",
        "        if loss.data.numpy() < 0.5:\n",
        "            print(\"Done!\")\n",
        "            break\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QqBx6EMo1DLx"
      },
      "source": [
        "### Задание 6:\n",
        "\n",
        "Реализуйте для функции \n",
        "\n",
        "$$\n",
        "f(w) = \\prod_{i,j} \\ln(\\ln(w_{ij} + 7) \n",
        "$$\n",
        "\n",
        "процедуру градиентного спуска. Каким получилось минимальное значение? "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KcWoMCQA1DLy",
        "outputId": "4695c97b-bdb5-447b-8ede-bd401a79e558",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[4.9900, 9.9948],\n",
            "        [0.9775, 1.9825]], requires_grad=True)\n"
          ]
        }
      ],
      "source": [
        "w = torch.tensor([[5., 10.], [1., 2.]], requires_grad=True)\n",
        "alpha = 0.001\n",
        "\n",
        "for _ in range(500):\n",
        "    # it's critical to calculate function inside the loop:\n",
        "    function = (w + 7).log().log().prod()\n",
        "    function.backward()\n",
        "    w.data -= alpha * w.grad\n",
        "    w.grad.zero_()\n",
        "\n",
        "print(w)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qht1Yf8w1DLy"
      },
      "source": [
        "# 4. Моя первая нейросеть \n",
        "\n",
        "Для того, чтобы разобраться как обучать нейросетки, нужно освоить три вещи: \n",
        "\n",
        "1. Как обрабатывать поток данных и пихать его в сетку\n",
        "2. Как сделать сетку\n",
        "3. Как написать цикл обучения"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6MZOGNcq1DLz"
      },
      "source": [
        "## 4.1 Как формировать батчи и пихать их в сетку\n",
        "\n",
        "С данными помогают работать две абстракции: `Dataset` и `DataLoader`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ww3mraJd1DLz"
      },
      "source": [
        "`Dataset` умеет выдавать по индексу некоторый элемент и помогает итерироваться по данным. Чтобы работать с данными и применять к ним преобразования, например, аугментации, о которых вы узнаете позже — нужно создать свой класс унаследованный от `torch.utils.data.Dataset`.\n",
        "\n",
        "Вот пример из документации:\n",
        "\n",
        "```\n",
        "class FaceLandmarksDataset(torch.utils.data.Dataset):\n",
        "    \"\"\"Face Landmarks dataset.\"\"\"\n",
        "\n",
        "    def __init__(self, csv_file, root_dir, transform=None):\n",
        "        \"\"\"\n",
        "        Args:\n",
        "            csv_file (string): Path to the csv file with annotations.\n",
        "            root_dir (string): Directory with all the images.\n",
        "            transform (callable, optional): Optional transform to be applied\n",
        "                on a sample.\n",
        "        \"\"\"\n",
        "        self.landmarks_frame = pd.read_csv(csv_file)\n",
        "        self.root_dir = root_dir\n",
        "        self.transform = transform\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.landmarks_frame)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        if torch.is_tensor(idx):\n",
        "            idx = idx.tolist()\n",
        "\n",
        "        img_name = os.path.join(self.root_dir,\n",
        "                                self.landmarks_frame.iloc[idx, 0])\n",
        "        image = io.imread(img_name)\n",
        "        landmarks = self.landmarks_frame.iloc[idx, 1:]\n",
        "        landmarks = np.array([landmarks])\n",
        "        landmarks = landmarks.astype('float').reshape(-1, 2)\n",
        "        sample = {'image': image, 'landmarks': landmarks}\n",
        "\n",
        "        if self.transform:\n",
        "            sample = self.transform(sample)\n",
        "\n",
        "        return sample\n",
        "```\n",
        "\n",
        "Как вы видите, у такого класса должно быть два метода: \n",
        "\n",
        "* `__len__` — возвращает информацию о том, сколько объектов у нас в датасете\n",
        "* `__getitem__` — возвращает семпл и таргет к нему\n",
        "\n",
        "Теперь давайте напишем такой сами, в качестве датасета сгенерируем рандомные данные."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eV2Sulnw1DL0"
      },
      "source": [
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J6Kyu4PX1DL0"
      },
      "outputs": [],
      "source": [
        "class RandomDataset(torch.utils.data.Dataset):\n",
        "    \"\"\"Our random dataset\"\"\"\n",
        "    \n",
        "    def __init__(self, x, y):\n",
        "        self.x=x\n",
        "        self.y=y\n",
        "        \n",
        "    def __len__(self):\n",
        "        return len(self.x)\n",
        "    \n",
        "    def __getitem__(self, idx):\n",
        "        return {'sample': torch.tensor(x[idx, :], dtype=torch.float), 'target': y[idx]}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Te4AObI51DL0"
      },
      "outputs": [],
      "source": [
        "x = np.random.rand(1000, 5)\n",
        "y = np.random.rand(1000)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dT-c3aC71DL1"
      },
      "outputs": [],
      "source": [
        "our_dataset = RandomDataset(x, y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VlM9SbQy1DL1",
        "outputId": "c73ab9e7-a5b1-465b-a178-a9e20d6e205d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'sample': tensor([0.2818, 0.9121, 0.7825, 0.3733, 0.5294]),\n",
              " 'target': 0.5043001422790946}"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ],
      "source": [
        "our_dataset.__getitem__(1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "22bHEw5O1DL1"
      },
      "source": [
        "`Dataset` умеет говорить сколько в выборке объектов и брать из неё очередной. Этого мало. Данные хочется как-то видоизменять. Для этого есть такая сущность как `DataLoader`.\n",
        "\n",
        "Он принимает на вход класс унаследованный от `torch.utils.data.Dataset` и преобразовывает его."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dwB131Yq1DL1"
      },
      "outputs": [],
      "source": [
        "dataloader = torch.utils.data.DataLoader(our_dataset, batch_size=4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whFWq8YN1DL2"
      },
      "source": [
        "Работают с ним следующим образом:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZRbg8jZ71DL2",
        "outputId": "eccc3e9f-2698-4726-81e1-2ffd66b8b91a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sample: tensor([[0.3722, 0.0114, 0.2744, 0.7695, 0.3046],\n",
            "        [0.2818, 0.9121, 0.7825, 0.3733, 0.5294],\n",
            "        [0.2418, 0.7356, 0.4305, 0.0160, 0.5177],\n",
            "        [0.2764, 0.3557, 0.4687, 0.4140, 0.2503]])\n",
            "Target: tensor([0.4432, 0.5043, 0.6372, 0.5250], dtype=torch.float64)\n"
          ]
        }
      ],
      "source": [
        "for batch in dataloader:\n",
        "    batch_x = batch['sample']\n",
        "    batch_y = batch['target']\n",
        "    break\n",
        "    \n",
        "print('Sample:', batch_x)\n",
        "print('Target:', batch_y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LolXR6HE1DL2"
      },
      "source": [
        "## 4.2 Нейронная сеть\n",
        "\n",
        "Есть несколько способов собрать нейросетку."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pQBEeziq1DL2"
      },
      "source": [
        "__Способ 1:__ Для того, чтобы в high-level pytorch создавать нейросети используется модуль `nn`. Нейросеть должна быть унаследована от класса `nn.Module`. Пример как это может выглядеть:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7PDNkqFU1DL2"
      },
      "outputs": [],
      "source": [
        "from torch import nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "class Model(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(Model, self).__init__()\n",
        "        self.layer1 = nn.Linear(5, 3)\n",
        "        self.layer2 = nn.Linear(3, 1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = F.relu(self.layer1(x))\n",
        "        return F.relu(self.layer2(x))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6iYY52Eg1DL3"
      },
      "source": [
        "Как мы видим на данном примере, у данного класса должно быть метод `forward`, который определяет прямой проход нейросети. Также из класса выше видно, что модуль `nn` содержит в себе реализацию большинства слоев, а модуль `nn.functional`—  функций активаций."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FDnoTVst1DL3",
        "outputId": "af1cfd24-c374-4325-d5f2-4a9e673e1a05",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.6637],\n",
              "        [0.5861],\n",
              "        [0.5621],\n",
              "        [0.6586]], grad_fn=<ReluBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ],
      "source": [
        "model = Model()\n",
        "model(batch_x) # получили предсказания модели"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dkp9H57L1DL3"
      },
      "source": [
        "Если вы не очень поняли, что произошло, не пугайтесь. На следующих семинарах мы более подробно будем говорить про объектно-ориентированное программирование и научимся писать подобные модели. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5f6VojfO1DL3"
      },
      "source": [
        "__Способ 2:__ Можно собрать модель с помощью класса `Sequential`. В его рамках сбор модели будет выглядить как строительство башни из конструктора LEGO. Модель описывается последовательно. Мы как бы создаём коробочку `model` и постепенно добавляем туда детальки нашей сетки. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "au525An61DL4"
      },
      "outputs": [],
      "source": [
        "model = nn.Sequential()                 # создаем пустую модель, в которую будем добавлять слои\n",
        "model.add_module('l1', nn.Linear(5, 3)) # добавили слой с 5-ю нейронами на вход и 3-мя на выход\n",
        "model.add_module('l2', nn.ReLU())       # добавили функцию активации\n",
        "model.add_module('l3', nn.Linear(3, 1)) # добавили слой с 3-мя нейронами на вход и 5-ю на выход"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8PjkpSqg1DL4",
        "outputId": "dc31e552-6be6-4855-c597-78de5a0cda6d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.2556],\n",
              "        [0.4704],\n",
              "        [0.4711],\n",
              "        [0.3831]], grad_fn=<AddmmBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 69
        }
      ],
      "source": [
        "model(batch_x) # получили предсказания модели"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I1O3b8N01DL4"
      },
      "source": [
        "__Способ 3:__ Описать модель функционально, в явном виде прописав какие аргументы идут на вход какому слою. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XIhJ5gQW1DL4",
        "outputId": "565ca990-25ef-4221-8ee6-d536e499fa91",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.2670],\n",
              "        [0.4930],\n",
              "        [0.5484],\n",
              "        [0.3923]], grad_fn=<AddmmBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ],
      "source": [
        "h = nn.Linear(5, 3)(batch_x)\n",
        "h = nn.ReLU()(h)\n",
        "out = nn.Linear(3, 1)(h)\n",
        "out"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "12kkhcCr1DL4"
      },
      "source": [
        "Обычно модель объявляют в виде класса, а второй и третий способ используют, чтобы вычисления внутри него выглядели более красиво."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p_MKAJsX1DL5"
      },
      "source": [
        "## 4.3 Цикл для обучения модели\n",
        "\n",
        "Осталось научиться писать циклы для обучения. Давайте подгрузим настоящие данные, соберём модель и обучим её."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "32AdluVs1DL5"
      },
      "outputs": [],
      "source": [
        "from torchvision.datasets import MNIST\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "\n",
        "# внутри модуля transforms есть много полезных преобразований\n",
        "# пока что нам из него понадобится только T.ToTensor() \n",
        "# для конвертации картинки в тензор \n",
        "import torchvision.transforms as T"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S0IWa9Zb1DL5"
      },
      "source": [
        "Не будем выпендриваться и как все обучим [наборе рукопистных цифр MNIST.](http://yann.lecun.com/exdb/mnist/)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mx8ZuGYL1DL6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c17ca365-9266-4dad-8ca5-8abf5d42e8c0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to .MNIST/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 9912422/9912422 [00:00<00:00, 123546033.65it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting .MNIST/MNIST/raw/train-images-idx3-ubyte.gz to .MNIST/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to .MNIST/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 28881/28881 [00:00<00:00, 30721707.79it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting .MNIST/MNIST/raw/train-labels-idx1-ubyte.gz to .MNIST/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to .MNIST/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 1648877/1648877 [00:00<00:00, 38186770.23it/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting .MNIST/MNIST/raw/t10k-images-idx3-ubyte.gz to .MNIST/MNIST/raw\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to .MNIST/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 4542/4542 [00:00<00:00, 4130643.71it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting .MNIST/MNIST/raw/t10k-labels-idx1-ubyte.gz to .MNIST/MNIST/raw\n",
            "\n"
          ]
        }
      ],
      "source": [
        "train_set = MNIST('.MNIST', transform=T.ToTensor(), train=True, download=True)\n",
        "test_set = MNIST('.MNIST', transform=T.ToTensor(), train=False, download=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ESYRom0f1DL6"
      },
      "source": [
        "`train_set` в данном случае это датасет. Давайте нарисуем несколько изображений. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r7pYe8Xv1DL6",
        "outputId": "17b460b0-92d8-43d2-e52e-b56d834a7b2f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 398
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1600x500 with 16 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABOwAAAF9CAYAAAC3R/fEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABI9UlEQVR4nO3debyd09kw/nVEQsqhpiQiYooKIuYhUSXGxlDeV4I81BCUShFBG20Rs2hVtIpoPVpjpBQtrYiUeOJNDCEq8sRcMohUSyaRyDnn98fze72991p6dnb2Ofd9zvl+Px9/rOuz9r2vyMp97n1l57pqGhoaGgIAAAAAUAir5Z0AAAAAAPD/KNgBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIGsXs6m+vr6MHfu3FBbWxtqamqaOicKrqGhISxatCh07do1rLZa09V8nTv+VXOduxCcPbLc88iDc0cenDvy4BmPvLjnkYeVOXdlFezmzp0bNt1006okR+sxa9as0K1btya7vnNHSlOfuxCcPdLc88iDc0cenDvy4BmPvLjnkYdyzl1ZZeTa2tqqJETr0tTnwrkjpTnOhbNHinseeXDuyINzRx4845EX9zzyUM65KKtg52ubpDT1uXDuSGmOc+HskeKeRx6cO/Lg3JEHz3jkxT2PPJRzLgydAAAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACUbADAAAAgAJZPe8EAAAAAGgZHn/88Si22267ZdZ77LFHtOedd95pspxaI9+wAwAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACMXQCAAAAgMgJJ5wQxfbbb78otmzZssx6zTXXbKqU2gzfsAMAAACAAlGwAwAAAIACUbADAAAAgAJRsAMAAACAAjF0Agpm0003jWJjxoyJYn379s2s6+vroz177713FJsyZcoqZAcA0HodccQRmXWXLl2iPTfeeGMUSzVXr6mpyawbGhrKyuGxxx7LrP/whz9EeyZMmBDF3nnnnbKuDyGkhwY89dRTmfXTTz8d7Zk4cWIUS+1LxWiZTj311Ci2xhprRLHp06dn1jNmzGiynNoK37ADAAAAgAJRsAMAAACAAlGwAwAAAIACUbADAAAAgAIxdAIKJjVgYo899ohipUMmUkMn7r///ig2efLkKHbcccetTIoArV5qAND111/f6OvOP//8KDZr1qyq5ASUr0ePHpl1qml6v379oljPnj0z69ra2rLeLzVQotwhE6UOPfTQf7sOIYTvf//7Uey+++6LYnPnzq0oB4opNSgiFbv00ksz60oHQFT6fqn3TP15o3h69eoVxbbZZpsolrq/XXXVVU2SU1vmG3YAAAAAUCAKdgAAAABQIAp2AAAAAFAgbbaHXceOHaPY6qtn/3ek+nqV9sMIIYR33303s+7QoUO0509/+lOjrwshhLq6ujhZ2pQ+ffpEsVSPgJqamsx6tdXi+nuqB1O3bt2i2KhRozLrKVOmNJYmbUS5fUpKTZw4MYqNGDGiChlRVGPHjo1iqftN3759myOdlZK6V6Z6gJb+eo455phoj351rVu7du2i2DnnnBPF1l9//Sg2dOjQzHrGjBnRntQ9d+nSpeUn2EbtvffeUeyBBx7IrDt16tRc6TSL6667LooNGjQoipX2v5s/f36T5UR1pe4HTz31VNWuVY5U77tyr1W6r5zPMzS/9u3bZ9ZPPPFEtGfjjTeOYnfffXcUe+ihh6qXGCEE37ADAAAAgEJRsAMAAACAAlGwAwAAAIACUbADAAAAgAJpdUMnUs2ATzzxxCg2fPjwKLb11ls3SU4hxE39Q0g3avznP/+ZWacays6dO7dqeVE8qYas9fX1Uax0yEQ5e75sX2lz9WOPPTbaYxBF61PaDDg1TKLSJsXlvs4gipZpr732Kis2e/bs5khnlZUOAwghPQBo2LBhmbX7YsvVq1evKPbtb3+70df17Nkzih1xxBEV5bD77rtHsVRD+dKBCgaUxbbbbrsoVumQiYULF2bWjzzySEXXKVdqEN6AAQMqutbOO+8cxb761a9m1oZOtByVPoOF0PzDHFLPc/vuu29mXe4QjX79+lUrLUqUDtkMIYRbbrkls04NmJg+fXoU+8EPflC9xPhSvmEHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAXS4odOlA6ZOOecc6I9119/fXOls1JOOOGERvccdNBBUezggw+OYnPmzKlKTuQv1SQ2NTyidF85e75s36abbppZP/vss9Ge1EAXWo5UU99Kmxlfdtllje4pbTQcQnqohaETLVP37t2jWOl9JITiDp0oHZBROkwihBAmT54cxW644YYmy4nqad++fRQ766yzMusLLrgg2rPJJps0WU4hxD+TU0OmevfuHcVKf24bOhH74x//GMX+1//6X5n15ptvHu0ZMmRIFFu0aFFm/eKLL65aco3o0KFDFFuwYEFmfeqppzZpDrCqUs9zpbHUc+eqDNZg5fXv3z+KlXN/SdVYDMJsHr5hBwAAAAAFomAHAAAAAAWiYAcAAAAABdKietitvfbaUey2227LrI877riKr//+++9n1v/5n/8Z7TnllFOi2J133plZp3pRpPLq2rVrFCvtu7LttttGe5544okoVtrrzr8pbznOO++8zDrV06a+vj6Klfa0Se0ZNWpUFBs6dGhF16KYUj1DUr3iyvH0009HsX79+lV0rZTU2S7NX0+7lmHAgAFl7Uvdg5pbqrfez372s8x61qxZ0Z5UXzuqI9UTtVOnTlHsqKOOyqx79epV1vW7dOkSxUr7mRXV1VdfHcU+//zzHDJpWebNmxfFDj300BwyWXnLly+PYqWfSWibUs9llT7j5SHVz7hU6tdI0+nRo0eje959990o9tJLLzVFOpTBN+wAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEBqGlJdwEssXLgwrLvuus2Rzxc22WSTKFbaJDqEEAYOHFi193z55Zcz61133bVq10559dVXo9j2229f0bVmzpyZWR944IHRnmoPoliwYEFYZ511qnrNf5XHuWtq119/fRQrHQJRU1MT7Un9MX3ggQcy6xtuuCHaM2XKlCh2//33R7HSP0ep90s1Cc9DU5+7EIp99kqHMqxK8+HSgRJN3fi3jB83SanBF3k0KW7L97zSwQ2phuipwQ3du3dvspzKdcwxx0Sx0vvg7373u7Jel4eWdu5KhxiFEMJpp52WWacakQ8aNKhqORRV6c/31D0x9Xu9ePHiJsvpy7S0c9dSpAbOpYZj/PznP8+s11hjjYrfs3SA3RtvvFHxtZpaW3/GK8dTTz0Vxfbbb78oVvqcVM0hYtUceJb63JOH1njPS9UVSusdIcSf8VI/oydNmlS9xPhCOefON+wAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEBWzzuB/6u0SfHll18e7al0wMTy5cuj2LPPPhvFRo4cWdH1K/Wb3/wmil111VWZdYcOHcq6Vs+ePTPrJ598MtrTu3fvKLZixYqyrs/KO++886JY6YCJEEKor6/PrFMNu1PDKr7//e9XlFeqyXVprDSnENK/ntSgC6on1US4nKa+qYEM1Ww2XKlUXqlfI8UzYMCARvekBjc0t9LhGCGkB+2UDshIDbWiMmuttVYUu/XWW3PI5N+bPXt2FFu6dGkU+/Wvf51ZpwapDBkypKz3LP1ZO27cuGjPsmXLyroWLcN6662XWffv3z/aM3r06Kq9X+o+nDrrtFyXXXZZFEs9S5XGUsMqUs+GpftW5Tmt9LkvlTtN58wzz4xi7du3j2Kff/55Zm3ARLH4hh0AAAAAFIiCHQAAAAAUiIIdAAAAABSIgh0AAAAAFEhhhk78x3/8R2Z9yimnVO3aP/nJT6LYxRdfXLXrVyo1SGD99dfPrC+66KKKrl06hCKEEGpqaiq6Fo1L/V6mBkykfg9Kh0xccMEF0Z5qDndI5VAaSw2+SA19KW1urLFxdbXkARPlNEBOSf16UjGaTmpwQ+lQhtKhDSGEcP755zdZTuUqZzhGCCFMmTLl366pXGpo0Wuvvdbo61KNsL/2ta81+rr3338/ii1atCiK3XPPPZn17bffHu35+9//HsXWWWedzPquu+5qNKdyPfPMM1GstPk3LUfpgIkQQnjkkUcy67333rtq7/fZZ59FsV/96ldR7NNPP63ae5K/cp/7yhkekRpEV02lQyY8zxXTzJkz806hLDvttFMU22GHHaLYoEGDMusNN9ywrOtPnz49s/7pT38a7ZkxY0ZZ16om37ADAAAAgAJRsAMAAACAAlGwAwAAAIACKUwPuxEjRlTlOj//+c+j2K233lqVazeH119/vcmuXfrvuUMI4c4772yy92tLUv3qUn18Ur3hSvdVs19dSqpfRWkslXtdXV2T5UTcaySEdL+R0v4fefSrK71f77vvvtGecvrVpUycOLGi11E9qftZqdL+lXkop9deCOl+e8ccc0yT5EQIS5YsiWKpHjOlvvKVr0Sxgw8+uNHXPf/881Fs7ty5jb6uXMcff3xmfcQRR1R8rY8++iizvvnmmyu+Fvl68803o9i6664bxTbYYIOqvF+qv+LJJ58cxSZMmFCV96NlqfSZqxzl9sxLfZbXs65lGD9+fLO+X+rzcK9evTLr1LyB1M/fNdZYo2p57b777pn1YYcdFu35xje+EcWasn4Tgm/YAQAAAEChKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBFGboxFZbbZVZpxrjl+PVV1+NYnPmzKnoWnn48MMPM+tly5ZFeyptrnjppZdGMUMnKrPXXntl1jU1NdGeVEPN1L7UMJCmlMqhNJbKPdXEe/bs2dVLrI0rt2FwpUMZUtcvjVVzeESlqjWAiPKU3stCCGHYsGGNvu65556LYk09yGHs2LGZ9fXXX1/W65p6kA/V8emnn0axhx9+uFlzWG+99aJYOUNYylV6FhcsWFC1a9N0Bg4cGMVSQ2/at2/fHOl84bvf/W5ZsdLm7amBGUuXLq1eYjSpcoeUNTfPb6RstNFGUezGG2+MYuV8Hv7888+j2KRJk6LYmDFjMuvSgU8hhNCtW7codsUVV2TWnTp1ivZccsklUey0006LYtW8p/qGHQAAAAAUiIIdAAAAABSIgh0AAAAAFIiCHQAAAAAUSGGGTvA/xo0bl1mXDqEIIYTu3btXdO0tttiiotcRK21CnRqSUl9fH8WmTJlSVqwppRqwDxgwILNO5V7pIBiqKzU8ppI91dSvX78o9vTTT0cxZ6h4yhkwkXL//fdX/J6zZs1qdE+qofuqvGep0mEbzX0fJn+pARNnnHFGFNt6660ruv68efOi2OjRoyu6Fvn63e9+F8Vuu+22KNaUQydSjdsPP/zwsl5bui91L/2P//iPyhKjSa3KgInS57DUs1pK6bNa6v1Sz3OpoXa0PT179sysH3/88WjPZptt1uh13n777Sh27LHHRrGpU6euRHb/3mOPPZZZT58+PdqTuldedtllUeyNN96oWl6+YQcAAAAABaJgBwAAAAAFomAHAAAAAAVSmB52b731Vma91VZb5ZRJvrp165ZZd+zYsWrXLv1/THmuv/76KDZw4MDMOtW3YbXV4nr4PvvsU73EKnTeeedFsdL8U7nrTdG0Ur1FUr1LSq1Kv7pUz4VSqV50qVipESNGrHxCNLnS3m2l97JypXo6/exnP4ti1ewNN3bs2Mw6lXuqP16fPn0qej997Vq3Y445JopdffXVTXr9f/7zn1W7Pvk65ZRToliqr90GG2yQWc+YMSPas91221UvsTKkzuZnn30Wxc4999zMetGiRU2WE/+jtF9cuf3qUs9zlT6HlT7vl9t/OPV+ngVbhrXXXrui162//vpR7N57782sU/3qPvrooyh20EEHZdb//d//He1ZtmzZyqa4UmbOnJlZF6X3tm/YAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBFGboxJgxYzLrH/3oRzllkq/vfve7mfVGG21UtWsfcMABVbtWWzJ06NAoVl9fn1mnhjSU7slDaYP5EELYc889o1hpU81U7jfccEP1EiOSGuTQFgZ9lDP4gqYzefLksvYNGzYss27qgQype1fpkInUgInu3bs3WU60bKXN/7/3ve9V7do///nPo9gLL7xQtetTPKmhUKlBFKXD41588cVoz2677VZRDqnBdDfeeGMUq62tzaxTz6wnnXRSFBs9enRm/dxzz61siqykcoaNVXPARDnKHYqWGoJW+mxbztAymt/gwYMz68svvzzaM2fOnCiWOos777xzZv3SSy9Fe04//fQoNm3atMbSrKr11lsvin3729/OrNu1axftefTRR6PY3Llzq5dYgm/YAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBFGboxGuvvVaV62y33XZVuU61devWLYrdddddUaxPnz5NlsO8efOa7NqtWarxf2nD3tSepm7KXo7/83/+TxQrHTARQpx/qiFxEX49tByp5sPkr/TPcd++fXPK5N8rHXKRcsEFFzRDJvw7q68eP0buscceUSz1s6i5jRo1KrPefvvtK7rOE088EcV+8IMfRLFly5ZVdH2KZ7/99otijz32WBT78MMPo9iWW27Z6PX/9re/VZJWUuqzxcyZMzPrrbfeuqxrlf6ZacrPKG1RpYMimntwQ+r9yh1EUfpnx9CJ5vXLX/4yih111FFRrLROkRqadPvtt0ex0iENIcT3m3322Sfa8+mnn0axppQaZHbsscdGsdJBk4sWLYr2XHTRRVFs8eLFlSdXBt+wAwAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACKczQifvvvz+z3n///aM9p512WqPXOfvss6PYJ598EsWuuOKK8pNbSeecc04UGz58eBTr0qVL1d6ztEH3o48+Gu1ZsWJF1d6vtTrvvPOiWGpIQ319fWadGtJQ2qy3qVWaewhx/tdff331EqPVSzXkLpcGxGy66aZRbODAgVFs8uTJmfXYsWObLCfKk3quKMKAiQ033DCK7bjjjlW59rPPPhvFDJhoXQ455JDM+p577on2rLnmms2VzkpJ3U87duyYQyY0lSI8N5WbQ+kAskoHbVCZ0gEQIYQwePDgKDZmzJjMeuONN472/PjHPy7rPZcvX55ZX3nlldGe1FCL0me8lLXWWiuKnXrqqVFs0KBBmXWnTp2iPanP7rNnz86sUwN6Pvvss0bzrDbfsAMAAACAAlGwAwAAAIACUbADAAAAgAIpTA+70l5bTz31VLSnnB52q68e/5IuvvjiKPatb30rio0ePTqz7tatW7Tn5JNPbjSHVG+6Dh06NPq6cu2+++5RbNq0aZl1XV1d1d6vLdlrr72iWE1NTRQr/XfvqT2pa6X+vXyp0n6OIaR70ZW+Zzl7viyHCy64ILO+4YYbGs0T/i897FgVAwYMKGtfaa9WCCGEDTbYIIql+hv26tWrouuX9nS97rrrKroOLccPf/jDzHq99dYr63WpZ65q9bq79tpro9j6668fxbbddtsolvo8U47PP/+8otfRtFLPXM39LLUqz33ka/z48VFszz33zKx/+9vfRnv69u1b1vU32mijzDrVY725pfrMpnqTDh06NLPOo19dim/YAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBFGboRKlU4/3a2tooduuttzZ6rdQgil133TWK3XbbbWVm13Ref/31zPqoo46K9rz11ltRzJCJ6kgNbkjF6uvrM+vUIIfSxpVftq/0WuW8X+pa5ewJIW6gHYIhE0B+Bg4cGMUmT54cxaZMmdIc6dDCbLbZZlGs0obos2bNimK/+MUvMutU82oIIYTu3btHsSVLluSQycr761//GsWOO+64HDKhManBjKmhExMnTsysR4wYUdH7pe6nqRxSLrvssorek+ZVWlvYf//9oz2pn7Wpeko5wylSAzRLB1+kLF26NIrdfvvtUey+++7LrOfNmxfteeeddxp9v6LwDTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKJDCDp1INdD/9a9/HcUef/zxzDrV6P+b3/xmFOvZs2flyVXg7bffjmKHH354FJszZ05mvXjx4ibLiVhNTU1ZsdJhDuXsqfa1Hnjggcw6NTiiqZu0n3feeY3mAP+X5sOEEMJee+2VWffp0yfa87vf/a650qGFSzWvLsfs2bOj2GGHHRbF3n///YquT8s1Y8aMzLpHjx7Rni5dujRXOqusdKDEzTffHO157LHHotjcuXObLCfSQyBKh0eUO9whNRiiNHbppZeWmVllUs94lQ66IF+p4UpvvPFGWbHSgQ+sOt+wAwAAAIACUbADAAAAgAJRsAMAAACAAilsD7uUVF+70t4iw4YNi/b89Kc/jWKDBg2KYqNHj240h+7duze6J9XvZMWKFVHss88+a/RaNK9UD7YBAwZEsdKzmOoxlzqv5exLnc2GhoYo9uCDD0ax5qZnHdAUSnt0wpc55JBDKnrdzJkzo9j06dNXNR1age9+97uZ9W9+85toz0knnRTFzjjjjKZKKenWW2+NYi+99FIUK72fLliwoMlyYtWU9rDr169ftCfVr64p+9OV5hRCul9dah+w6nzDDgAAAAAKRMEOAAAAAApEwQ4AAAAACkTBDgAAAAAKpEUNnajU3Llzo9j1119f0bVmzJixqulQYFOmTIli7dq1yyETaH323XffvFOgAObMmdPontTwJkiZPHlyFDvqqKMafV2qmfu0adOi2De/+c3Met68eWXnRuvw3HPPlRU766yzmiMd2pDUIIdUbMSIEU2eC5AP37ADAAAAgAJRsAMAAACAAlGwAwAAAIACUbADAAAAgAJpE0MnAGg6qWbHl156afMnQoswa9aszLqmpianTGgN7rnnnih2+umnR7EePXpk1qmBUvfdd18UM2QCAMiLb9gBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEYOgFA1RkkADSHuXPnRrGvfe1rOWQCAFBdvmEHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAVSVsGuoaGhqfOgBWrqc+HckdIc58LZI8U9jzw4d+TBuSMPnvHIi3seeSjnXJRVsFu0aNEqJ0Pr09TnwrkjpTnOhbNHinseeXDuyINzRx4845EX9zzyUM65qGkoo6xXX18f5s6dG2pra0NNTU1VkqPlamhoCIsWLQpdu3YNq63WdP+q2rnjXzXXuQvB2SPLPY88OHfkwbkjD57xyIt7HnlYmXNXVsEOAAAAAGgehk4AAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2JXpl7/8Zdh8883DmmuuGfbcc8/w/PPP550SbYSzR3NbtGhRGDp0aNhss81Cx44dQ9++fcMLL7yQd1q0Iddee22oqakJQ4cOzTsVWrnNN9881NTURP8NGTIk79RoxW655ZbQu3fvsM4664R11lkn9OnTJ/z5z3/OOy1aubq6unDxxReHLbbYInTs2DFstdVW4YorrghmUNIc5syZE0444YSwwQYbhI4dO4YddtghvPjii3mnVXgKdmW4//77w7Bhw8Kll14aXnrppbDjjjuGQw45JMyfPz/v1GjlnD3ycNppp4Xx48eHu+66K7z66qvh4IMPDgceeGCYM2dO3qnRBrzwwgth9OjRoXfv3nmnQhvwwgsvhA8++OCL/8aPHx9CCGHgwIE5Z0Zr1q1bt3DttdeGqVOnhhdffDHsv//+4cgjjwyvvfZa3qnRio0cOTLccsst4aabbgr//d//HUaOHBmuu+668Itf/CLv1GjlPv7447D33nuH9u3bhz//+c9hxowZ4frrrw/rrbde3qkVXk2Dknqj9txzz7D77ruHm266KYQQQn19fdh0003D2WefHYYPH55zdrRmzh7NbenSpaG2tjY88sgj4bDDDvsivuuuu4b+/fuHK6+8MsfsaO0WL14cdtlll3DzzTeHK6+8Muy0005h1KhReadFGzJ06NDw6KOPhjfffDPU1NTknQ5tyPrrrx9+8pOfhFNPPTXvVGilDj/88NC5c+dw++23fxE7+uijQ8eOHcPdd9+dY2a0dsOHDw/PPvts+K//+q+8U2lxfMOuEcuXLw9Tp04NBx544Bex1VZbLRx44IFh8uTJOWZGa+fskYcVK1aEurq6sOaaa2biHTt2DJMmTcopK9qKIUOGhMMOOyxz34Pmsnz58nD33XeHwYMHK9bRbOrq6sKYMWPCkiVLQp8+ffJOh1asb9++YcKECeGNN94IIYTwyiuvhEmTJoX+/fvnnBmt3R/+8Iew2267hYEDB4ZOnTqFnXfeOfzqV7/KO60WYfW8Eyi6jz76KNTV1YXOnTtn4p07dw4zZ87MKSvaAmePPNTW1oY+ffqEK664Imy77bahc+fO4b777guTJ08OPXr0yDs9WrExY8aEl156Sb9EcvPwww+HTz75JJx88sl5p0Ib8Oqrr4Y+ffqEzz77LKy99trhoYceCtttt13eadGKDR8+PCxcuDD07NkztGvXLtTV1YWrrroqHH/88XmnRiv3zjvvhFtuuSUMGzYs/PCHPwwvvPBCOOecc0KHDh3CSSedlHd6haZgB0DGXXfdFQYPHhw22WST0K5du7DLLruEQYMGhalTp+adGq3UrFmzwrnnnhvGjx8ffbsTmsvtt98e+vfvH7p27Zp3KrQB22yzTZg2bVpYsGBBeOCBB8JJJ50UJk6cqGhHkxk7dmy45557wr333hu23377MG3atDB06NDQtWtXRROaVH19fdhtt93C1VdfHUIIYeeddw7Tp08Pt956q7PXCAW7Rmy44YahXbt24cMPP8zEP/zww9ClS5ecsqItcPbIy1ZbbRUmTpwYlixZEhYuXBg23njjcOyxx4Ytt9wy79RopaZOnRrmz58fdtllly9idXV14Zlnngk33XRTWLZsWWjXrl2OGdLavffee+HJJ58Mv//97/NOhTaiQ4cOX3xzfddddw0vvPBCuPHGG8Po0aNzzozW6sILLwzDhw8Pxx13XAghhB122CG899574ZprrlE0oUltvPHG0V9GbLvttuHBBx/MKaOWQw+7RnTo0CHsuuuuYcKECV/E6uvrw4QJE/SZoEk5e+RtrbXWChtvvHH4+OOPw7hx48KRRx6Zd0q0UgcccEB49dVXw7Rp0774b7fddgvHH398mDZtmmIdTe6OO+4InTp1ygzbgeZUX18fli1blncatGKffvppWG217Mf/du3ahfr6+pwyoq3Ye++9w+uvv56JvfHGG2GzzTbLKaOWwzfsyjBs2LBw0kknhd122y3sscceYdSoUWHJkiXhlFNOyTs1WjlnjzyMGzcuNDQ0hG222Sa89dZb4cILLww9e/Z07mgytbW1oVevXpnYWmutFTbYYIMoDtVWX18f7rjjjnDSSSeF1Vf3aEzTu+iii0L//v1D9+7dw6JFi8K9994bnn766TBu3Li8U6MVO+KII8JVV10VunfvHrbffvvw8ssvh5/97Gdh8ODBeadGK3feeeeFvn37hquvvjocc8wx4fnnnw+33XZbuO222/JOrfA8lZTh2GOPDX//+9/DJZdcEubNmxd22mmn8Pjjj0fDAKDanD3ysGDBgnDRRReF2bNnh/XXXz8cffTR4aqrrgrt27fPOzWAqnvyySfD+++/70MrzWb+/PnhxBNPDB988EFYd911Q+/evcO4cePCQQcdlHdqtGK/+MUvwsUXXxzOOuusMH/+/NC1a9dwxhlnhEsuuSTv1Gjldt999/DQQw+Fiy66KFx++eVhiy22CKNGjTLwpAw1DQ0NDXknAQAAAAD8Dz3sAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQFYvZ1N9fX2YO3duqK2tDTU1NU2dEwXX0NAQFi1aFLp27RpWW63par7OHf+quc5dCM4eWe555MG5Iw/OHXnwjEde3PPIw8qcu7IKdnPnzg2bbrppVZKj9Zg1a1bo1q1bk13fuSOlqc9dCM4eae555MG5Iw/OHXnwjEde3PPIQznnrqwycm1tbVUSonVp6nPh3JHSHOfC2SPFPY88OHfkwbkjD57xyIt7Hnko51yUVbDztU1SmvpcOHekNMe5cPZIcc8jD84deXDuyINnPPLinkceyjkXhk4AAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBrJ53Am3ZD37wgyh2zTXXZNZdunSJ9syfP7/JcgIAaClqa2uj2I9//OMotvnmm2fWxxxzTLSnoaEhio0bNy6zbt++fbSnY8eOUWzq1KlRrNSCBQui2MiRI6PY4sWLG70WwL9z5plnRrGbb745itXU1GTWM2bMiPY88MADUezKK6+MYp9//vnKpAgk+IYdAAAAABSIgh0AAAAAFIiCHQAAAAAUiIIdAAAAABSIoRPNpEOHDlHs4IMPjmKphscAAMRuvfXWKHbcccc1+rr6+vqyrp96VivHXnvtVdHrpkyZEsUee+yxiq4FtF19+/bNrEeNGhXtefHFF6PYn/70p8w69Rl22LBhUeyTTz6JYjfccEMjWVIEa665Zma92267RXt69OgRxY488shGr92zZ88otuGGG0axSZMmZdapgSWlZzOEEGbOnBnFXn755cx62bJljeZZZL5hBwAAAAAFomAHAAAAAAWiYAcAAAAABaKHXTM5++yzo9h+++3X/Imw0tZdd90o9tBDD2XWqd/LmpqaKJbqTdOnT5/Kk4M2oLS3RgghtGvXrtHXpXpULV26tCo5Ubl+/fpl1ieccEK0Z9asWVFs9OjRmfUHH3xQ3cQovE6dOkWxSnvFQR622267KDZhwoRGX3f55ZdHsVtuuaUqOdE6HX/88Zl16vnn0EMPjWIfffRRo9deffW4hHDeeedFsTvvvDOz/sc//tHotame9dZbL4qdeeaZUez000/PrDfffPOq5ZDqbZiKlX4eTv28HzBgQFnv+dZbb2XWAwcOjPa88sorZV2rCHzDDgAAAAAKRMEOAAAAAApEwQ4AAAAACkTBDgAAAAAKxNCJZnL44YeXte+vf/1rZr1kyZKmSIeVsMcee0SxcgaGNDQ0RLFevXpFsUsuuaSivP7yl79k1u+99160Z/HixVHs448/ruj9aNl22223KPbiiy82+rpdd901iq2zzjpVySmEELp165ZZ9+/fP9qz7777RrGuXbs2eu1XX301ivXu3XslsmNlpH7vbrzxxii28cYbZ9Zf+cpXyrr+17/+9cw61YQ95Z133olis2fPLuu15Gu11bJ/r/y9730v2lNuc+w5c+Zk1qmm13fffXcU22GHHTLrHXfcsaz3K8ekSZOi2PTp06t2ffL361//OrM+9thjoz1rr712o9c5+OCDo5ihE6yMzz//PIqVM2AiZfjw4VEs9Vm39DPUn//854rej8r853/+ZxQ78sgjG33da6+9FsWee+65KHbTTTc1eq158+aVFSsd8rjFFltEe1Kxq6++Oopts802mXXqc4ShEwAAAABARRTsAAAAAKBAFOwAAAAAoEAU7AAAAACgQAydaAL9+vWLYnvvvXdZr50wYUJmbehE/gYMGFC1a6211lpRbMSIERVdq5zXpRqr33bbbVHsb3/7W0U5PPTQQ5m189r0OnTokFmnhqKk7jcXXnhhFDvwwAMz66OPPrqs162xxhqN5knrNmTIkCh21VVXRbHa2too9s9//jOzLnfoxP77759Zp37Wprz77rtR7OGHH86sUw2Wx44dG8Xc45pX6aCmH/3oR2W9LtVc/YQTTsisn3nmmcoTg4SePXtGsZNPPjmzbteuXbQn1fj/1FNPzayffvrpVcqNtmfp0qWZdceOHaM9qUFcpQMQU0oHBISQ/sxR+lnltNNOi/akhv2Ufr6gMqXDur7MxRdfnFmPHDky2rNixYqq5PRlFixYkFlPmzYt2pOK1dXVRbHSZ7zUsMif//znK5NernzDDgAAAAAKRMEOAAAAAApEwQ4AAAAACqRN9LBbffX4l/nd7343ipX+2/6bb7452rN48eJG3++iiy6KYqmeFameOZX2M6N1mz59ehQr7e2T0q1btyh2+eWXVyWnEEJ48803M+ttttmmatcm7bLLLsushw8fXvG1Xn755VVNh1aoa9euUezGG2/MrI844ohoT0NDQxS76667otgPfvCDzDrVK66cviupfjmpfnhbbLFFFDvvvPMavX5pniHEffpSvz6q51vf+lZFr7v11lujmJ51VNOWW24ZxVL3jNLn/3/84x/RntL+iiGEMG7cuMy6R48e0Z5Uz7znn38+TpY2acyYMZn1sGHDoj2pvtbnnHNOZn322WdHe0p7IIcQwpprrhnFSp8dRo8eHe354IMPohjV8Ytf/CKKpWoNc+fOzaybul9dNe2yyy6N7vnTn/7UDJk0Hd+wAwAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACaXVDJ1IDJs4///wodvXVVzd6rdKG+iGE8NBDDzX6ui5dujS6J4T0AItyhlrQvFJNzCu1ZMmSKHbUUUdl1rNmzYr2pJqrlw6U2GqrraI92223XRQbPHhwFNt2222jWDmWL19e0esozzrrrBPFUr+n5airq4tic+bMyaw33XTTaE9NTU1F7/f+++9X9LqUTp06RbFUc+NSqSbdd955ZxQ78cQTK0usFUjd3+67774oVjoEIvVnf+TIkVGsnEFK8+fPj2LXX399FCv9vSs9vyHEw6NCSJ+Do48+OrNODTfYeuuto9jtt9+eWe+8887RnlRjbxq36667RrFUE/9ynHHGGVHs008/bfR1f/jDH6LY66+/nll//PHHFeVEy1H686V3797Rnvvvvz+Kbb755lHs3Xffzay/853vRHuefPLJRq/11FNPRXtSPxvXWGONKEbb9NJLL2XW//Vf/xXt2WeffaLYpEmTMuuZM2dGe37zm99EsWuvvTaKLViwoLE0aUKpQYPVHD7Y3NZbb70odsoppzT6urfffrsp0mk2vmEHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAXS6oZODBw4MIqVM2AihBAWLVqUWT///PNVyenLpJrMUjzHHnts1a41Y8aMKDZhwoSKrvXGG2/823UIcbPjENJDWMrx4YcfRrHjjz++omsR22CDDaLYo48+GsX22muviq6fapS+5557ZtapoQGpgSSlAyWmT58e7Sln2EC5HnvssSg2b968zDrVAHzhwoVRbMqUKVXLqzVI/Z6XDpgIIW4cnbqP3HHHHRXlcM0110Sxd955J4p98sknjV4rdc7nzp0bxZ555pnMOjXc4Nvf/nYUK22qffbZZ0d76uvro9gFF1wQJ0vGgQceGMUqHfrUoUOHKFbOAIvUntKfraVnJ4T0PSo1wIKWYaeddsqsJ0+eXNbrSj9HhBBCz549M+tyh3WdeuqpmXXpoLGVuRZtU+fOnTPrcgdAlH526Nu3b7THkESa2mqrxd8rSz2zpgbmPfjgg5l1auBKS+IbdgAAAABQIAp2AAAAAFAgCnYAAAAAUCAKdgAAAABQIC1+6MSuu+6aWQ8ePLis16Uaww4aNCiznjNnTlnX2mijjTLr2traaE+qMewTTzxR1vVpPv37949iqaaX5ViyZEkUa+4hDcccc0wU69KlS0XX+tWvfhXF/vrXv1Z0LWLjx4+PYjvvvHPVrr/hhhtGsbfffjuzfvbZZ6M9P/rRj6LYiy++mFmnGv1XU2qYUF1dXWa9bNmyJs2htdhmm20y69NPPz3a8/nnn0ex0gEMqSb7lXrppZeqdq1KffbZZ1Esdc+bOHFiZp26B5533nlRzNCJrHbt2kWxQw89NIdMGve1r33t365DSD973n333VHs4osvzqxnz569itmRl9TniHXWWafR1+24445RbMCAAVHshBNOqCwx+P+dfPLJmfXhhx8e7Xnvvfei2NZbb51ZH3DAAdGeRx55ZNWSg0aceOKJUey0006LYkuXLo1iN998c2a9YsWK6iWWA9+wAwAAAIACUbADAAAAgAJRsAMAAACAAmnxPezOP//8zHr//feP9qT+3XKqt1elPeV69+6dWXfv3j3aM3/+/Cg2adKkit6PpjNs2LAoVmkPu1Rfprfeequia5Vj9dXjP85HHXVURddavHhxFPvZz35W0bUoT6r3ZVP7yle+klkfdNBB0Z5U7J133smszz333GjPo48+uorZ/T+ffvpp1a7V1l166aWZdfv27aM9V1xxRRSrZs+6luyNN97IrC+77LJoz5VXXtlc6bRY55xzThT7+te/3qTvWdpX9rXXXov2lPZFDiHdb69U6jkh1X+nT58+mXXqmXXu3LmNvh/5K7dfdanUWSnnjMG/s8suu0SxH//4x5n1m2++Ge3Zaaedoti8efMy64suuijaU9rPNYQQPvnkk0ayhC/Xq1evzPqXv/xlWa8bMWJEFHvqqaeqkVJh+IYdAAAAABSIgh0AAAAAFIiCHQAAAAAUiIIdAAAAABRIixo6kRoUcdxxxzX6uo8++iiKVTpgItWc87bbbsusa2pqoj2dO3eOYqnGwqXNvkuvHUIIdXV1jaVJmUqb7nfq1Klq17722murdq1yDBkyJIqlzms5Ro4cGcU0k21aPXv2jGKp5tT9+vXLrLfaaquq5fDVr341ip1xxhlRbMstt8ysH3744WhPauBJNQdRUJl99tmn0T3PPfdcM2TSOvTo0SPvFNqUjz/+OIrdeOONUWzcuHFRrHSY0owZM6I9qcEXp59+emad+j3fa6+94mQTtt5668z6kksuifaceeaZZV2LplM6NOyHP/xhtGfw4MFRrNL7QWmT/xBC+Mtf/pJZ77333tGeTTbZJIrtuOOOUeyVV16pKC9ajm9961tR7O23386sv/nNb0Z7li5dGsVKP79cddVV0Z5DDjkkit1///2N5gkhpD/zjB07NrPu2LFjtOeWW26JYm1hKKJv2AEAAABAgSjYAQAAAECBKNgBAAAAQIEo2AEAAABAgdQ0NDQ0NLZp4cKFYd11122OfP6tVNPL4cOHN/q6t956K4rNnDmzohyOOOKIKFbG/8Kyffjhh5n1dtttF+0pSvP/BQsWhHXWWafJrt8c5650KENpo+FypV6XakK9YsWKiq5fjmeeeSaKpRpop8yfPz+z3nnnnaM9H3zwQWWJVVlTn7sQinPPK4INNtggij399NOZda9evaI9pc2OQwhh9913j2KpJvJF1RrueaU/r0p/5oSQbpxe2rC/rVp77bUz6zfeeCPa06VLlyiWGiBTrtZw7kqlBjx95zvfiWKlPzN/+ctfRnsWLVpUvcTK0KFDhyh25JFHRrE777yz0dfW19dHe6688soodtlll61MilXRGs9dNW266aZRLPXz8pprrsms//rXv0Z7UgPmSn+G3nfffdGeo48+OoptttlmUawoz2/l8IzXuN69e0ex1KCd3/zmN5n1RRddVNb1SweQTZ06Ndqz5pprRrGDDjooiqWG+xSVe151rLHGGpl1apDSDTfc0Oh1fvvb30axCy+8MIqlhou2JOWcO9+wAwAAAIACUbADAAAAgAJRsAMAAACAAlk97wRWxvTp0yt6XaofTypWBE899VRmXZR+da3Vr371q6pc580334xiTdmvLoS4V0SqZ165nn322cy6JfU7oWn94x//iGKTJk3KrFM97Lbaaqsodswxx0Sx0aNHr0J2rKzSHnYvv/xytEe/ui93/vnnZ9adO3eO9syZM6e50mmxSvumhpDu3VZEy5cvj2K/+93voth6660XxW655ZbMOtXbcFV+ltN8Zs2aVVasf//+FV2/9HNKv379oj2pHtqe31q/ww47LIqlfhaNHDmyouuXfvacMGFCtOe0006LYqmz3pJ62LHySvv6hhD3dU/VXD777LModvvtt2fWqd6tLb1fXaV8ww4AAAAACkTBDgAAAAAKRMEOAAAAAApEwQ4AAAAACqRFDZ34wx/+EMVKm5+nmvWuvnrz/jKfeeaZKPbEE0+U9dp777232unwbyxZsqQq10k1Gm5qZ555ZmZd7jlPNfu+/PLLq5IT1bXGGmtEsY4dO1Z0rfr6+ii2cOHCiq41duzYzLr0LH6ZH/7wh1HM0Inm9ec//zmzPuCAA6I9qQbBb731VpPlVFSnnHJKFEud4VKjRo1qgmxoaVLPgqXN3L/61a82TzK0OGuuuWZmvc466+SUCUVz6KGHRrHx48dHsUqf8Uq99tprVbkOrU/qLJYz2LP0c0QIIZx99tlVyak18g07AAAAACgQBTsAAAAAKBAFOwAAAAAoEAU7AAAAACiQFjV0IjUgYN99982sDzrooGjP8OHDo1g5jdt32mmnKFbaBDYl1TRx+vTpjb6O5lfaLPPII4+M9gwePDiKXXfddZn1xIkTq5tYidS569WrV0XXmjp1ahR75ZVXKroW1ZNqPv773/8+ivXr16+i6y9dujSKPfbYY42+7o9//GMU23333SvKYfHixRW9juq57bbbMutvfvOb0Z4LLrggipU7WKSl+MpXvpJZX3rppdGec889N4qVDveZMWNGtOfhhx9eteRoFd59990o9ve//z2zNnSCL1N6b0n9LD766KOj2CGHHBLFxo0bV73EaFbrrbdeFNt8882jWOkQxhDSw8agUhtuuGEUGzNmTKOvu//++6PYaaedVpWc2grfsAMAAACAAlGwAwAAAIACUbADAAAAgAJRsAMAAACAAmlRQyfKMX78+LJiKd/4xjcy63KbtL744ouZ9TvvvFPW68jfp59+mlnfd9990Z5UrLmlGgtvvfXWFV2rrq5uVdOhCWy55ZZRbJ999qna9VODdgYMGNDo68rZU64HH3ywateiMk8++WRm/cILL0R7dtlllyjWvn37zPrzzz+vbmJVstZaa0Wx//2//3cUKx1G1bNnz7Ku/+tf/zqzHjlyZLTHM0Dbs8Yaa0SxW2+9NYpV+nObtqdz586Z9XbbbRftST3PGTDRumywwQZRbO21127WHNZdd91mfT+K6fHHHy9r33PPPZdZn3LKKdGeFStWVCWnPGy88cZRLPXcXM5gv3L5hh0AAAAAFIiCHQAAAAAUiIIdAAAAABRIq+thtyr69euXWXfo0KGs1y1cuDCzLu2LBqsq1cOuHKn+Jj/96U9XNR2awEsvvRTFZs6cGcV69erVHOmsshkzZkSx6667LodM+FdLlizJrK+//vpoz5gxY6LYX/7yl8z6Jz/5SbTn2WefjWL/+Mc/Gs1ptdXivztM9XQsleqveOKJJ0axbbbZptFrpc5rqj/dQw89lFmX/v8kbaeddsqsb7zxxmjPlClTothBBx2UWZ9zzjnRnkGDBkWxVE+5zTffPLN+4IEHUqlWZMiQIVEs1XMMyvXBBx9k1tOnT4/2fO1rX2uudMjJhx9+GMUWLFgQxVI/R2tqajLrhoaGst5zo402yqxPPfXUaE99fX0Uq2bPLvKVepZK9WlLPQOdfvrpmfVnn31WvcSq6Ktf/WoUK32eGDhwYLRnr732imIXX3xxFNPDDgAAAABaKQU7AAAAACgQBTsAAAAAKBAFOwAAAAAoEEMn/sX666/f6J7SBp7QHEqbZZcrNXTi/fffX8VsaC6lg3BCCOGee+7JrPfff/9oz+qrN+2tffny5Zn1E088Ee257LLLotjixYubLCcqk/q9e+aZZ6LYN77xjcy6dPhCCCHMnTu3rFipVLPsVHPjShtopxp0jx49OrO+/PLLoz1Lly4t6/pkpYZA3HnnnZl16vf861//eqPXfvrpp6NYpc9lqftrc/vnP/8ZxVJ/Jml7tt9++8z68MMPzykT8rRo0aIolrpHnHbaaVHs8ccfz6zvuOOOaE+nTp2i2Pjx4zPrTTfdNNrz/e9/P4qlBqXRMp199tll7bv99tujWGpATrWkPt9069Ytih188MGZ9d577x3tOfTQQ6NY+/btM+sHH3ww2nPmmWdGsTfeeCNOtop8ww4AAAAACkTBDgAAAAAKRMEOAAAAAApEwQ4AAAAACsTQiX9R2uA1pdwm11CpVAPtSptqf/7551Hsb3/7W0XXovl99NFHUeyQQw7JrFPNT4877rgotu+++zb6fgsXLoxijz76aBS78cYbM+vnn3++0WtTTKmBDEcddVQUO++88zLr888/P9rTtWvXsmKVKm3QP3bs2GjPhx9+GMV+/etfR7E5c+ZULS+yUsMcUkMmKtHaBn+l7t+pJte0PcuWLcusU0Nw1lhjjSiWGlJW2pT9448/jva89tprK5khefntb38bxVKDmm655ZbMOjUMrGPHjlFsgw02yKzffffdaE/q2ZDWY8MNNyxr37PPPlvR9ddee+0o1qNHj8y6e/fu0Z7Us+c+++zT6PulBi6mftaW/pl55ZVXGr12c/ANOwAAAAAoEAU7AAAAACgQBTsAAAAAKBA97Krg5ZdfzjsFWpEdd9yxrFg5Pvjgg1VNh4K79dZbo9ivfvWrKJbqdVOqvr4+in322WeVJUaLleprN2LEiMz67rvvjvZsueWWUSzVH6e0D+PDDz9cVl5PPPFEWfvI17Bhw6LYk08+mVnvvffe0Z5XX301iu2www6Z9f777x/t2W677VY2xapbsmRJFLvjjjui2L333ptZ6//Jl3nrrbcy6wkTJkR7Uv1qx48fH8W22GKLzPq2226L9px11lkrmyI5SfUNS/3+/ehHP8qsjzjiiLKuf+6552bWqXvZ4sWLy7oWLdNTTz0VxU4++eQoNmbMmCh23XXXNXr91GeSLl26ZNapnrWzZs2KYo888kgUe+CBBxrd05LOsG/YAQAAAECBKNgBAAAAQIEo2AEAAABAgSjYAQAAAECBGDrxLw488MC8U4CwfPnyKLZs2bIoVs4QgXHjxlUlJ1qWurq6KPbpp5/mkAmtVWlD9C+LpaSa/9J6pBo5jx079t+ugX8vNdgkpUePHlGsdHjUyJEjq5ITxZEaYHPkkUfmkAmtwXe+850olhp8M2DAgChWOoCsdHhUCCF89NFHUax0ANmDDz4Y7fn9738fxZYuXRrFWhvfsAMAAACAAlGwAwAAAIACUbADAAAAgAJRsAMAAACAAjF0AgpmxowZUeyPf/xjFCtt9Pnxxx9He+65557qJQYAQLMbMWJEFDv11FOjWKoxfOmQiffee69qeQGtz4oVK6JY6jOlz5nNwzfsAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBADJ2AFuCYY47JOwUAAHIwe/bsKFZTU5NDJgA0J9+wAwAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACUbADAAAAgAJRsAMAAACAAlGwAwAAAIACKatg19DQ0NR50AI19blw7khpjnPh7JHinkcenDvy4NyRB8945MU9jzyUcy7KKtgtWrRolZOh9Wnqc+HckdIc58LZI8U9jzw4d+TBuSMPnvHIi3seeSjnXNQ0lFHWq6+vD3Pnzg21tbWhpqamKsnRcjU0NIRFixaFrl27htVWa7p/Ve3c8a+a69yF4OyR5Z5HHpw78uDckQfPeOTFPY88rMy5K6tgBwAAAAA0D0MnAAAAAKBAFOwAAAAAoEAU7AAAAACgQBTsAAAAAKBAFOxW0rXXXhtqamrC0KFD806FVm7zzTcPNTU10X9DhgzJOzVasREjRkRnrmfPnnmnRRvxy1/+Mmy++eZhzTXXDHvuuWd4/vnn806JVs49j7wsWrQoDB06NGy22WahY8eOoW/fvuGFF17IOy1asbq6unDxxReHLbbYInTs2DFstdVW4YorrghmUNLUrrnmmrD77ruH2tra0KlTp3DUUUeF119/Pe+0WoTV806gJXnhhRfC6NGjQ+/evfNOhTbghRdeCHV1dV+sp0+fHg466KAwcODAHLOiLdh+++3Dk08++cV69dX9qKDp3X///WHYsGHh1ltvDXvuuWcYNWpUOOSQQ8Lrr78eOnXqlHd6tGLueeThtNNOC9OnTw933XVX6Nq1a7j77rvDgQceGGbMmBE22WSTvNOjFRo5cmS45ZZbwm9/+9uw/fbbhxdffDGccsopYd111w3nnHNO3unRik2cODEMGTIk7L777mHFihXhhz/8YTj44IPDjBkzwlprrZV3eoVW06CkXpbFixeHXXbZJdx8883hyiuvDDvttFMYNWpU3mnRhgwdOjQ8+uij4c033ww1NTV5p0MrNWLEiPDwww+HadOm5Z0Kbcyee+4Zdt9993DTTTeFEEKor68Pm266aTj77LPD8OHDc86O1so9jzwsXbo01NbWhkceeSQcdthhX8R33XXX0L9//3DllVfmmB2t1eGHHx46d+4cbr/99i9iRx99dOjYsWO4++67c8yMtubvf/976NSpU5g4cWL4xje+kXc6heafxJZpyJAh4bDDDgsHHnhg3qnQBi1fvjzcfffdYfDgwYp1NLk333wzdO3aNWy55Zbh+OOPD++//37eKdHKLV++PEydOjXzM3a11VYLBx54YJg8eXKOmdEWuOfR3FasWBHq6urCmmuumYl37NgxTJo0KaesaO369u0bJkyYEN54440QQgivvPJKmDRpUujfv3/OmdHWLFiwIIQQwvrrr59zJsXnO/9lGDNmTHjppZf0lSA3Dz/8cPjkk0/CySefnHcqtHJ77rln+M1vfhO22Wab8MEHH4TLLrss7LPPPmH69OmhtrY27/RopT766KNQV1cXOnfunIl37tw5zJw5M6esaAvc88hDbW1t6NOnT7jiiivCtttuGzp37hzuu+++MHny5NCjR4+806OVGj58eFi4cGHo2bNnaNeuXairqwtXXXVVOP744/NOjTakvr4+DB06NOy9996hV69eeadTeAp2jZg1a1Y499xzw/jx46O/BYPmcvvtt4f+/fuHrl275p0Krdy//i1r7969w5577hk222yzMHbs2HDqqafmmBlA9bnnkZe77rorDB48OGyyySahXbt2YZdddgmDBg0KU6dOzTs1WqmxY8eGe+65J9x7771h++23D9OmTQtDhw4NXbt2DSeddFLe6dFGDBkyJEyfPt23icukYNeIqVOnhvnz54dddtnli1hdXV145plnwk033RSWLVsW2rVrl2OGtHbvvfdeePLJJ8Pvf//7vFOhDfrqV78avva1r4W33nor71RoxTbccMPQrl278OGHH2biH374YejSpUtOWdEWuefRXLbaaqswceLEsGTJkrBw4cKw8cYbh2OPPTZsueWWeadGK3XhhReG4cOHh+OOOy6EEMIOO+wQ3nvvvXDNNdco2NEsvve974VHH300PPPMM6Fbt255p9Mi6GHXiAMOOCC8+uqrYdq0aV/8t9tuu4Xjjz8+TJs2TbGOJnfHHXeETp06ZZoSQ3NZvHhxePvtt8PGG2+cdyq0Yh06dAi77rprmDBhwhex+vr6MGHChNCnT58cM6Otcc+jua211lph4403Dh9//HEYN25cOPLII/NOiVbq008/Dautlv34365du1BfX59TRrQVDQ0N4Xvf+1546KGHwl/+8pewxRZb5J1Si+Ebdo2ora2N/m31WmutFTbYYAP/5pomV19fH+64445w0kknhdVX98eVpnfBBReEI444Imy22WZh7ty54dJLLw3t2rULgwYNyjs1Wrlhw4aFk046Key2225hjz32CKNGjQpLliwJp5xySt6p0Yq555GXcePGhYaGhrDNNtuEt956K1x44YWhZ8+e7nk0mSOOOCJcddVVoXv37mH77bcPL7/8cvjZz34WBg8enHdqtHJDhgwJ9957b3jkkUdCbW1tmDdvXgghhHXXXTd07Ngx5+yKTQUACuzJJ58M77//vh+kNJvZs2eHQYMGhX/84x9ho402Cl//+tfDlClTwkYbbZR3arRyxx57bPj73/8eLrnkkjBv3ryw0047hccffzwaRAHV5J5HXhYsWBAuuuiiMHv27LD++uuHo48+Olx11VWhffv2eadGK/WLX/wiXHzxxeGss84K8+fPD127dg1nnHFGuOSSS/JOjVbulltuCSGEsN9++2Xid9xxh6GKjahpaGhoyDsJAAAAAOB/6GEHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAWiYAcAAAAABaJgBwAAAAAFomAHAAAAAAWiYAcAAAAABaJgBwAAAAAF8v8BmD+I/Cggcb8AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "cols = 8\n",
        "rows = 2\n",
        "fig = plt.figure(figsize=(2 * cols, 2.5 * rows))\n",
        "for i in range(cols):\n",
        "    for j in range(rows):\n",
        "        random_index = np.random.randint(0, len(train_set))\n",
        "        ax = fig.add_subplot(rows, cols, i * rows + j + 1)\n",
        "        ax.grid(False)\n",
        "        ax.set_xticks([])\n",
        "        ax.set_yticks([])\n",
        "        ax.imshow(train_set[random_index][0].squeeze(0).numpy().reshape([28, 28]), cmap = 'gray')\n",
        "        ax.set_xlabel(train_set[random_index][1])\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hw-_O-3T1DL6"
      },
      "source": [
        "Каждая картинка это матрица из чисел. Если число большое - пиксель яркий. Если маленькое - тёмный."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cFKVh5Pp1DL7",
        "outputId": "10bf4b14-5e53-4fe9-dda4-9cd9b761682e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 28, 28])"
            ]
          },
          "metadata": {},
          "execution_count": 74
        }
      ],
      "source": [
        "train_set[5][0].shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rpv0rLFM1DL7"
      },
      "source": [
        "Будем прогнозировать тип картинки по её пикселям. Подготовим данные с помощью `DataLoader`. Пока что попробуем настроить его максимально просто. В будущем мы увидим с вами другую магию, на которую способна эта абстракция. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lJhxGFxG1DL7"
      },
      "source": [
        "### Задание 7:\n",
        "\n",
        "Напишите `DataLoader` для тренировочной и тестовой выборок. Попробуйте проитерироваться по нескольким его первым объектам с помощью цикла.\n",
        "\n",
        "В обучающей выборке данные должны перемешиваться каждую эпоху. Размер батча поставьте равным 64. В тестовой выборке данных перемешивать не надо."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nns6s22O1DL7"
      },
      "outputs": [],
      "source": [
        "# берем датасет, перемешиваем его и формируем батчи\n",
        "train_loader = torch.utils.data.DataLoader(\n",
        "    train_set, batch_size=64, shuffle=True\n",
        ")\n",
        "\n",
        "test_loader = torch.utils.data.DataLoader(\n",
        "    test_set, batch_size=64, shuffle=False\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f96Y5z_61DL7",
        "outputId": "6d675733-0c7d-4379-d4e6-34c4b464bd6b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([64, 1, 28, 28]) torch.Size([64])\n"
          ]
        }
      ],
      "source": [
        "for images, labels in train_loader:\n",
        "    print(images.shape, labels.shape)\n",
        "    break"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qlc3_zDk1DL8"
      },
      "source": [
        "### Задание 8:\n",
        "\n",
        "Напишите  двухслойную полносвязную нейросеть. Выходной слой должен состоять из $10$ нейронов с `Softmax` в качестве функции активации, так как мы решаем задачу классификации $10$ классов. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ea0GZt8H1DL8"
      },
      "outputs": [],
      "source": [
        "class MLP(nn.Module):\n",
        "    def __init__(self, in_features, num_classes, hidden_size):\n",
        "        super().__init__()\n",
        "        self.model = nn.Sequential(\n",
        "            nn.Linear(in_features=in_features, out_features=hidden_size),\n",
        "            nn.ReLU(),\n",
        "            nn.Linear(in_features=hidden_size, out_features=hidden_size, bias=False),\n",
        "            nn.LeakyReLU(0.1),\n",
        "            nn.Linear(in_features=hidden_size, out_features=num_classes),\n",
        "            # nn.LogSoftmax(), # в функцию потерь пойдут логиты\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        return self.model(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JMj78iB51DL8"
      },
      "outputs": [],
      "source": [
        "model = MLP(28**2, 10, 20)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tYYPh0bu1DL8"
      },
      "source": [
        "Веса моделей хранятся в виде матриц и выглядят так:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wR9NpUmy1DL9"
      },
      "outputs": [],
      "source": [
        "# [x for x in model.named_parameters()] "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xmolZ1KA1DL9"
      },
      "source": [
        "Объявляем модель и оптимизатор."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FQvzKS2V1DMI"
      },
      "outputs": [],
      "source": [
        "IMG_SIZE = 28\n",
        "NUM_CLASSES = 10\n",
        "HIDDEN_SIZE = 64\n",
        "NUM_EPOCHS = 10\n",
        "\n",
        "device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n",
        "model = MLP(in_features=IMG_SIZE ** 2, num_classes=NUM_CLASSES, hidden_size=HIDDEN_SIZE).to(device)\n",
        "\n",
        "# оптимайзер\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9)\n",
        "\n",
        "# функция потерь \n",
        "criterion = nn.CrossEntropyLoss()  # nn.NLLLoss()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mnsLmySp1DMI"
      },
      "source": [
        "Осталось всё это дело обучить. Для этого нам придётся написать цикл"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HaDNGy9D1DMI",
        "outputId": "9d59a76e-cabe-4533-b470-d8e292f62fe6",
        "colab": {
          "referenced_widgets": [
            "655bd6cd9d874a46ba95eb68bc5b7af5",
            "1ed90c0b0dc04e499c1989910c57ab49",
            "e29e398af70846d8b9b57bb4bee419dd",
            "36c9d9ed89924e609207f0ccdc0a7483",
            "9cbf00567de94e4a81eaa12b732f7706",
            "1cfb140194214295910db72f9647ec5e",
            "30b25b6f911c4fc8a1c6aac2152e5a10",
            "d8426e6cc2b34bee996134de95dd8936",
            "42408d5c7eab4c549ab074ae86fc549e",
            "9b91677f282b4f08aa820e011560e45b",
            "ad35a9e93cb848e39ae73cd7b898c6ce",
            "ab9d3ef6324c43dd893a59dd0d130ae6",
            "e9537757914a45ba8f21edc334f4f6f4",
            "22228801a5a648f3bc4f01af5e1ac568",
            "2fae853c2458451b89a1bc8cb5151c6b",
            "cd361deaca3744329d83b6a075b68b85",
            "e27644cc3358474a8d36e97c3ea1f814",
            "2eaf7038387f44a0b5f0dcf9c3bf249d",
            "0bbc27b535f1470a8508232bd9493e73",
            "53f86e1f54d7455ebec92ef05a117f5e",
            "bc3bb79d5cb349a7b13db03779bfa2ea",
            "6bc80afbf0a84d1491e4ecb23f896075",
            "9c94a87dfdb14bdaaaa8dd661fcbcef6",
            "0c5a7a4166d349e3ab695a31181cac26",
            "47e1df05e2d5488dad17af36cd9ef45b",
            "41096f87ad3842cc80db11111dad7400",
            "bb919864e6a747d180d9d3f212a9ad37",
            "35d2e63a2cdc47efab63c76fc20c82f2",
            "4fc0f865b6c6497d916de816268003ba",
            "4ae4d353b2fb4d4395980ac9c37c0092",
            "6ac76a3538a343738850a1cb3b453697",
            "aa376b2e7ddc455aafc4280186f49ed6",
            "e5ab876de85647e79869c30ba598ad4e",
            "49e78df0ee9845669d2168e10d21165f",
            "753c4132efa6411db4145333ae2a53ba",
            "fdbe3e0d95ac42a19313cb0e0a5e1480",
            "7ab1e5495c104b69a18daabab53f1af6",
            "2d95b1e4d34b4861ba4589a503a03932",
            "fbc4dd8e2fd04469bd9959b90c0cf00e",
            "cdf8678a603a494db2e2850694cb9db4",
            "6ace1dc26db84a028081ec80582fe158",
            "33b3677ccf344fe59c00a836f6db1ee0",
            "b62932a96a1e4ef295c06b7198862ba1",
            "19250d2a609440bda3539f5ce9c152fb",
            "56d35e96c88843f890c31c8093479c52",
            "4dacb8b42d4f42a9bb8531acd65c0ac1",
            "220dc32e4709490d9b38d4f7c4b6b1cd",
            "c2c6fff4a8514f358a25d373cec4fd56",
            "a19dd26df92a4f8b848fb5300b5c0ac0",
            "614c23b1c6214b0ab32c8b72866fd6b6",
            "9232c55d071c4cf19155e3822242edb7",
            "9b572e1e6da042f7a4c1bdd3b14cecee",
            "a386f581a5b74d42b182d8415962677c",
            "9a0225a6e4324736a0c48d07e4d199f9",
            "5ab8371c65a74cc49013b1aebf69f069",
            "d9695e1d1a364298b029e3c499eb1e79",
            "0c1e7ccc772f4e1e83dfb8d9f07f1941",
            "9a8a3c5bc1904723ab7eed99994b30d3",
            "bb7025f964f4434aa2286d472734b7da",
            "4419601c1ae84346b17a48973c3c3898",
            "6d43f3dad0e244339141716a81662cb6",
            "a0df5ac2671d4365aca74763228cd6be",
            "f109311dfc334bfb8d4ba3776239908a",
            "3b3db09f2b4b4c01ad3ff682cadce8fb",
            "0b1b1a8779994b8f936a0632c8e7ed90",
            "b3a48e91fef944f29cf17bec23d90628",
            "90f589afbb564e8199116fa3ab2f4c4b",
            "91b87c66669f4759b15a27cf0c151804",
            "7d8ae387a7a1413c85ae4262ecc6b2da",
            "968693b7673e4e588861bc5a60151a98",
            "f9a744d01b8144d29f63d5226be6357f",
            "985a558a468e44cd96a7136213d07645",
            "6b06e7e3552f499c9110a3c38229673d",
            "3adea2f2d6b740fbb77bed8ea4deb69f",
            "188f185dc2d24e30ab7e618710be116f",
            "6b0b24e71f26414e871820c42483aeba",
            "a3d9c4a018f249d4808eaec1a7e66879",
            "6ef77b5a220640af9dffba8f754a85b1",
            "23b5ec6d5a9a4c12b817f11837433039",
            "6e73d366c1774a16aef73ad151f25231",
            "413e20adaab14611aca3ea13cd5ea142",
            "85fd06a8e3064c3781d19dadd5046b42",
            "e6c3e6cf8548465e96679c2a27c88d64",
            "67db24504c9a444fb809fd2925f0024b",
            "86421b4adf8443009ec875a9fbeb6fcd",
            "3728b0f225a1474cb67c18ddd799614b",
            "d772f687f8b645ada4265795b1cf475c",
            "b0c6768a822c46a4a385613ade669334",
            "915f23839f8b497d9c8984d574b3ea5b",
            "d39be31621b547d0bdf9fc9ad2eadfa5",
            "1624b03bfe9c4e27b9ba943e1bd308dd",
            "874271be60d84bae915cb99916139b63",
            "970538766c164c05be2abf067965b317",
            "9e0ec7b0d6914c7ebe7e9c71fccb2fe3",
            "97afb5c58b8c499a88c63548eed070be",
            "093970309ea74ba6b3e67e3f3860dffa",
            "c6fe7df2bcb0458fb108c2485a3d982f",
            "9c67eafd73e642eea1067b1515676365",
            "8fa1e1ed605b4f8eaa992f440785fa1b",
            "c3930b87c3114ec393b3edcdb1fab83e",
            "97e45169c241476fb8041cd620307909",
            "bd74e010036840f0b46fc20802b07c64",
            "3eb54ce864724adabc1e571fee0feb79",
            "dd51bfadcc1e4495bbafb0cf9ad279b9",
            "852ac15273724ff4b4a7778b98d934c9",
            "473391b3ded14b358ce6905f36231b05",
            "e9ea8066a2de471a85129036a6967706",
            "2de9fd8ab3284ea1ad7d0090d6dceb8f",
            "167f8a23fc3648aca3b3b0ed6a1ca0a8",
            "58c62817024f4c6d825dde4c7962fe19",
            "de423df179cc4a3599eeb8b4eab54a1b",
            "0575ceb6d27b46508465f97ddd628c30",
            "dc8bcb406a8e45d68b3bca226d3aba53",
            "32d17218d3ec424a8f06ddd4fd730691",
            "886e2b318c7a482d985039360f36433a",
            "1205188b13384a98b19cf66841b21a2b",
            "88e8537104914278bad91ef6f7eb1c90",
            "6358d881e51b494cb58dcdcacf74e425",
            "615a655ea4644a7b9ed4656183087af3",
            "37bb3d494d3c405789576040dbb56259",
            "3245100768cc44f982cd0c6ed4465af6",
            "4878939ae6734768a3d5290a5001d0cb",
            "37323af8eb9a45a981d5d39dfae6c8ff",
            "c3a3a8390e8544a288ab0eeed900237a",
            "ce0736fd5d384e5f8094e6d5ac3f3aa8",
            "839d8ec957124aff8cbe36951ca261d2",
            "db66a1bf27ae4fc4bd550e741f755944",
            "cd03b9f8bb4c434a90386c5d1d350319",
            "c464076e27404b5ea59cb62859c49c3c",
            "45b160b0705642b1884d215cee80a790",
            "9f152de2b37a442cab8e77b24b47d523",
            "b3f886fe14c64ec489c0ef383f30e656",
            "3acbe984b43f4d0092265ca295ca34eb",
            "23d5953e691a4f0a8136e95ed13d2b66",
            "3263179a0ff84f2a8d4ce4106d0e3c3b",
            "7869bfc19a464f298a32409eedcfe5f2",
            "a19b363b48d54faba3ae4390e78ab2fa",
            "b485bd8a06f9479083b54134e1eb8dd4",
            "4676152d5ce041b38019416de1f27b07",
            "4740330d27d3493aab720a06bae3d1cf",
            "4edc432da3624f82a502a64e97cc5481",
            "c6766600aa0d4f53805359e093badb86",
            "46ddbff341cc4019a7aa2a1a9fa7823a",
            "c9478d4c32ec4669be2c05427439a9bc",
            "0c19b5023cae406e9dcf9ad318ca0d0e",
            "57f06e09791e4f4882af7def7c7f30df",
            "c87be05509ed47758ab4aac2ae9fe40e",
            "80227e7f33cd481a8a47886d5a0116b4",
            "025856940d964c13a16b2767580ac333",
            "2726defb9d1843988a8c7fb9b7383204",
            "c07f39e5150c4de7a14aa6f0de3f8152",
            "d9615eda7ce74a72b141d555983f1106",
            "5ba6bab8a246441f9d453ab2ff0123f7",
            "76b8c81ec8e2434281f900535d6e3c0d",
            "24f4c0ec566c47e0b79f6a82cdbbd429",
            "4416cadd96d34b3ea3836399860359d5",
            "1dacb028065c4c87910826068d96b6e3",
            "8106c7778de4416e9cea305382b566cf",
            "fe669078abe14009b16e60767a6d75d2",
            "261dfc8ed39e43dd920593dee011f31d",
            "56927cad1d8a4b4284ee7c60976fae8a",
            "705c62ac4aa948ff9dcd89772abff7b4",
            "e8d63926f8c24e25986377bf2327e477",
            "09faa896dabc4c419b74d1b1cdf82dc1",
            "272329eb2916471c88e2a5c35915d5e0"
          ],
          "base_uri": "https://localhost:8080/",
          "height": 584
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "655bd6cd9d874a46ba95eb68bc5b7af5"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ab9d3ef6324c43dd893a59dd0d130ae6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch: 0, loss: 0.254931777715683, accuracy: 0.9243\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9c94a87dfdb14bdaaaa8dd661fcbcef6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "49e78df0ee9845669d2168e10d21165f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "56d35e96c88843f890c31c8093479c52"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch: 2, loss: 0.12739315629005432, accuracy: 0.9614\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d9695e1d1a364298b029e3c499eb1e79"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "90f589afbb564e8199116fa3ab2f4c4b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6ef77b5a220640af9dffba8f754a85b1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch: 4, loss: 0.10643548518419266, accuracy: 0.9685\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "915f23839f8b497d9c8984d574b3ea5b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c3930b87c3114ec393b3edcdb1fab83e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "de423df179cc4a3599eeb8b4eab54a1b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch: 6, loss: 0.08490326255559921, accuracy: 0.9739\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4878939ae6734768a3d5290a5001d0cb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3acbe984b43f4d0092265ca295ca34eb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Validating:   0%|          | 0/157 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c9478d4c32ec4669be2c05427439a9bc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch: 8, loss: 0.085875004529953, accuracy: 0.9733\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/938 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "24f4c0ec566c47e0b79f6a82cdbbd429"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from tqdm.notebook import tqdm\n",
        "\n",
        "# NUM_EPOCHS раз подряд пройдемся по всем батчам из трейна\n",
        "for epoch in range(NUM_EPOCHS):\n",
        "    \n",
        "    # берем батч из трейн лоадера\n",
        "    for images, labels in tqdm(train_loader):\n",
        "        \n",
        "        # отправляем тензоры на девайс к сетке\n",
        "        images = images.to(device)\n",
        "        labels = labels.to(device)\n",
        "        \n",
        "        ### Пишем самые важные 5 строк\n",
        "        ### Мы их дальше будем встречать везде\n",
        "        \n",
        "        # зануляем старые градиенты иначе они сложатся\n",
        "        optimizer.zero_grad()\n",
        "        \n",
        "        # вызываем модель\n",
        "        logits = model(torch.flatten(images, start_dim=1))\n",
        "        # (batch_size, x, num_classes) - размерность\n",
        "        \n",
        "        # считаем функцию потерь \n",
        "        loss = criterion(logits, labels)\n",
        "        \n",
        "        # считаем градиенты обратным проходом\n",
        "        loss.backward() \n",
        "        \n",
        "        # обновляем параметры сети (шаг спуска)\n",
        "        optimizer.step()\n",
        "        \n",
        "    if epoch % 2 == 0:\n",
        "        mean_val_loss = [] # сюда будем складывать средний лосс по батчам\n",
        "        val_accuracy = []\n",
        "        \n",
        "        # берем батч из вал лоадера для валидации\n",
        "        for images, labels in tqdm(test_loader, desc='Validating'):\n",
        "            images = images.to(device)\n",
        "            labels = labels.to(device)\n",
        "            \n",
        "            # мы считаем качество, поэтому мы запрещаем фреймворку считать градиенты по параметрам\n",
        "            with torch.no_grad(): \n",
        "                # делаем предсказания\n",
        "                logits = model(torch.flatten(images, start_dim=1))\n",
        "                \n",
        "                # считаем ошибку на валидации\n",
        "                loss = criterion(logits, labels)\n",
        "                \n",
        "                mean_val_loss.append(loss.numpy()) # добавляем в массив \n",
        "                val_accuracy.extend((torch.argmax(logits, dim=-1) == labels).numpy().tolist())\n",
        "        \n",
        "        # выводим статистику\n",
        "        print('Epoch: {epoch}, loss: {loss}, accuracy: {accuracy}'.format(\n",
        "                epoch=epoch, loss=np.mean(mean_val_loss), accuracy=np.mean(val_accuracy)\n",
        "        )) \n",
        "        "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EjwiUM2i1DMJ"
      },
      "source": [
        "Кажется, что всё работает. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8ZTiEmCM1DMJ"
      },
      "source": [
        "### Ещё одно задание:\n",
        "\n",
        "На следующей паре мы продолжим обучать эту сетку и попробуем улучшить получившийся у нас пайплайн. Попробуйте для удобства вынести один шаг обучения модели и один шаг валидации модели в отдельные небольшие функции, чтобы код стал более читаемым. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "csIrj4Tb1DMJ"
      },
      "source": [
        "## Почиташки\n",
        "\n",
        "__Откуда взялась эта тетрадка?__ \n",
        "\n",
        "Когда я писал эту тетрадку, я ориентировался на четыре источника:\n",
        "\n",
        "- [Введение в pytorch от ШАД](https://github.com/yandexdataschool/Practical_DL/blob/fall22/week02_autodiff/seminar_pytorch.ipynb)\n",
        "- [Тетрадка с введением в pytorch с ИАД](https://github.com/hse-ds/iad-deep-learning/blob/master/2022/seminars/sem01/sem01.ipynb) (её взяли из ШАД)\n",
        "- [Введение в pytorch от Samsung](https://github.com/SlinkoIgor/Neural_Networks_and_CV)\n",
        "- [Экспромт Ильдуса с семинаров по DL на ФКН](https://github.com/isadrtdinov/intro-to-dl-hse/tree/2022-2023/seminars/202)\n",
        "\n",
        "Я добавил к ним свою больную фантазию и получилось то, что получилось.\n",
        "\n",
        "\n",
        "__Что можно почитать?__\n",
        "\n",
        "* [Документация торча](https://pytorch.org/docs/master/)\n",
        "* [Использование pytorch на GPU](https://pytorch.org/docs/master/notes/cuda.html)\n",
        "* [Хорошая книга про pytorch](http://download.webclark.org/Deep-Learning-with-PyTorch.pdf)\n",
        "* [Pytorch за 60 минут](http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)\n",
        "* [Как устроено автоматическое дифференцирование в pytorch](http://videolectures.net/site/normal_dl/tag=1129745/deeplearning2017_johnson_automatic_differentiation_01.pdf)\n",
        "* [PyTorch examples](https://github.com/pytorch/examples)\n",
        "* [Practical pytorch tutorials](https://github.com/spro/practical-pytorch)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    },
    "varInspector": {
      "cols": {
        "lenName": 16,
        "lenType": 16,
        "lenVar": 40
      },
      "kernels_config": {
        "python": {
          "delete_cmd_postfix": "",
          "delete_cmd_prefix": "del ",
          "library": "var_list.py",
          "varRefreshCmd": "print(var_dic_list())"
        },
        "r": {
          "delete_cmd_postfix": ") ",
          "delete_cmd_prefix": "rm(",
          "library": "var_list.r",
          "varRefreshCmd": "cat(var_dic_list()) "
        }
      },
      "types_to_exclude": [
        "module",
        "function",
        "builtin_function_or_method",
        "instance",
        "_Feature"
      ],
      "window_display": false
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "655bd6cd9d874a46ba95eb68bc5b7af5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1ed90c0b0dc04e499c1989910c57ab49",
              "IPY_MODEL_e29e398af70846d8b9b57bb4bee419dd",
              "IPY_MODEL_36c9d9ed89924e609207f0ccdc0a7483"
            ],
            "layout": "IPY_MODEL_9cbf00567de94e4a81eaa12b732f7706"
          }
        },
        "1ed90c0b0dc04e499c1989910c57ab49": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1cfb140194214295910db72f9647ec5e",
            "placeholder": "​",
            "style": "IPY_MODEL_30b25b6f911c4fc8a1c6aac2152e5a10",
            "value": "100%"
          }
        },
        "e29e398af70846d8b9b57bb4bee419dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d8426e6cc2b34bee996134de95dd8936",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_42408d5c7eab4c549ab074ae86fc549e",
            "value": 938
          }
        },
        "36c9d9ed89924e609207f0ccdc0a7483": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9b91677f282b4f08aa820e011560e45b",
            "placeholder": "​",
            "style": "IPY_MODEL_ad35a9e93cb848e39ae73cd7b898c6ce",
            "value": " 938/938 [00:14&lt;00:00, 47.99it/s]"
          }
        },
        "9cbf00567de94e4a81eaa12b732f7706": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1cfb140194214295910db72f9647ec5e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "30b25b6f911c4fc8a1c6aac2152e5a10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d8426e6cc2b34bee996134de95dd8936": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "42408d5c7eab4c549ab074ae86fc549e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9b91677f282b4f08aa820e011560e45b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad35a9e93cb848e39ae73cd7b898c6ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ab9d3ef6324c43dd893a59dd0d130ae6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e9537757914a45ba8f21edc334f4f6f4",
              "IPY_MODEL_22228801a5a648f3bc4f01af5e1ac568",
              "IPY_MODEL_2fae853c2458451b89a1bc8cb5151c6b"
            ],
            "layout": "IPY_MODEL_cd361deaca3744329d83b6a075b68b85"
          }
        },
        "e9537757914a45ba8f21edc334f4f6f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e27644cc3358474a8d36e97c3ea1f814",
            "placeholder": "​",
            "style": "IPY_MODEL_2eaf7038387f44a0b5f0dcf9c3bf249d",
            "value": "Validating: 100%"
          }
        },
        "22228801a5a648f3bc4f01af5e1ac568": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0bbc27b535f1470a8508232bd9493e73",
            "max": 157,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_53f86e1f54d7455ebec92ef05a117f5e",
            "value": 157
          }
        },
        "2fae853c2458451b89a1bc8cb5151c6b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bc3bb79d5cb349a7b13db03779bfa2ea",
            "placeholder": "​",
            "style": "IPY_MODEL_6bc80afbf0a84d1491e4ecb23f896075",
            "value": " 157/157 [00:02&lt;00:00, 79.64it/s]"
          }
        },
        "cd361deaca3744329d83b6a075b68b85": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e27644cc3358474a8d36e97c3ea1f814": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2eaf7038387f44a0b5f0dcf9c3bf249d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0bbc27b535f1470a8508232bd9493e73": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53f86e1f54d7455ebec92ef05a117f5e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bc3bb79d5cb349a7b13db03779bfa2ea": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6bc80afbf0a84d1491e4ecb23f896075": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9c94a87dfdb14bdaaaa8dd661fcbcef6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0c5a7a4166d349e3ab695a31181cac26",
              "IPY_MODEL_47e1df05e2d5488dad17af36cd9ef45b",
              "IPY_MODEL_41096f87ad3842cc80db11111dad7400"
            ],
            "layout": "IPY_MODEL_bb919864e6a747d180d9d3f212a9ad37"
          }
        },
        "0c5a7a4166d349e3ab695a31181cac26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_35d2e63a2cdc47efab63c76fc20c82f2",
            "placeholder": "​",
            "style": "IPY_MODEL_4fc0f865b6c6497d916de816268003ba",
            "value": "100%"
          }
        },
        "47e1df05e2d5488dad17af36cd9ef45b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4ae4d353b2fb4d4395980ac9c37c0092",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6ac76a3538a343738850a1cb3b453697",
            "value": 938
          }
        },
        "41096f87ad3842cc80db11111dad7400": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aa376b2e7ddc455aafc4280186f49ed6",
            "placeholder": "​",
            "style": "IPY_MODEL_e5ab876de85647e79869c30ba598ad4e",
            "value": " 938/938 [00:13&lt;00:00, 75.89it/s]"
          }
        },
        "bb919864e6a747d180d9d3f212a9ad37": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "35d2e63a2cdc47efab63c76fc20c82f2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4fc0f865b6c6497d916de816268003ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ae4d353b2fb4d4395980ac9c37c0092": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6ac76a3538a343738850a1cb3b453697": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "aa376b2e7ddc455aafc4280186f49ed6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e5ab876de85647e79869c30ba598ad4e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "49e78df0ee9845669d2168e10d21165f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_753c4132efa6411db4145333ae2a53ba",
              "IPY_MODEL_fdbe3e0d95ac42a19313cb0e0a5e1480",
              "IPY_MODEL_7ab1e5495c104b69a18daabab53f1af6"
            ],
            "layout": "IPY_MODEL_2d95b1e4d34b4861ba4589a503a03932"
          }
        },
        "753c4132efa6411db4145333ae2a53ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fbc4dd8e2fd04469bd9959b90c0cf00e",
            "placeholder": "​",
            "style": "IPY_MODEL_cdf8678a603a494db2e2850694cb9db4",
            "value": "100%"
          }
        },
        "fdbe3e0d95ac42a19313cb0e0a5e1480": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6ace1dc26db84a028081ec80582fe158",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_33b3677ccf344fe59c00a836f6db1ee0",
            "value": 938
          }
        },
        "7ab1e5495c104b69a18daabab53f1af6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b62932a96a1e4ef295c06b7198862ba1",
            "placeholder": "​",
            "style": "IPY_MODEL_19250d2a609440bda3539f5ce9c152fb",
            "value": " 938/938 [00:13&lt;00:00, 79.77it/s]"
          }
        },
        "2d95b1e4d34b4861ba4589a503a03932": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fbc4dd8e2fd04469bd9959b90c0cf00e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cdf8678a603a494db2e2850694cb9db4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6ace1dc26db84a028081ec80582fe158": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "33b3677ccf344fe59c00a836f6db1ee0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b62932a96a1e4ef295c06b7198862ba1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "19250d2a609440bda3539f5ce9c152fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "56d35e96c88843f890c31c8093479c52": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4dacb8b42d4f42a9bb8531acd65c0ac1",
              "IPY_MODEL_220dc32e4709490d9b38d4f7c4b6b1cd",
              "IPY_MODEL_c2c6fff4a8514f358a25d373cec4fd56"
            ],
            "layout": "IPY_MODEL_a19dd26df92a4f8b848fb5300b5c0ac0"
          }
        },
        "4dacb8b42d4f42a9bb8531acd65c0ac1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_614c23b1c6214b0ab32c8b72866fd6b6",
            "placeholder": "​",
            "style": "IPY_MODEL_9232c55d071c4cf19155e3822242edb7",
            "value": "Validating: 100%"
          }
        },
        "220dc32e4709490d9b38d4f7c4b6b1cd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9b572e1e6da042f7a4c1bdd3b14cecee",
            "max": 157,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a386f581a5b74d42b182d8415962677c",
            "value": 157
          }
        },
        "c2c6fff4a8514f358a25d373cec4fd56": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9a0225a6e4324736a0c48d07e4d199f9",
            "placeholder": "​",
            "style": "IPY_MODEL_5ab8371c65a74cc49013b1aebf69f069",
            "value": " 157/157 [00:01&lt;00:00, 80.50it/s]"
          }
        },
        "a19dd26df92a4f8b848fb5300b5c0ac0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "614c23b1c6214b0ab32c8b72866fd6b6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9232c55d071c4cf19155e3822242edb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9b572e1e6da042f7a4c1bdd3b14cecee": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a386f581a5b74d42b182d8415962677c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9a0225a6e4324736a0c48d07e4d199f9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5ab8371c65a74cc49013b1aebf69f069": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d9695e1d1a364298b029e3c499eb1e79": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0c1e7ccc772f4e1e83dfb8d9f07f1941",
              "IPY_MODEL_9a8a3c5bc1904723ab7eed99994b30d3",
              "IPY_MODEL_bb7025f964f4434aa2286d472734b7da"
            ],
            "layout": "IPY_MODEL_4419601c1ae84346b17a48973c3c3898"
          }
        },
        "0c1e7ccc772f4e1e83dfb8d9f07f1941": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6d43f3dad0e244339141716a81662cb6",
            "placeholder": "​",
            "style": "IPY_MODEL_a0df5ac2671d4365aca74763228cd6be",
            "value": "100%"
          }
        },
        "9a8a3c5bc1904723ab7eed99994b30d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f109311dfc334bfb8d4ba3776239908a",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3b3db09f2b4b4c01ad3ff682cadce8fb",
            "value": 938
          }
        },
        "bb7025f964f4434aa2286d472734b7da": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0b1b1a8779994b8f936a0632c8e7ed90",
            "placeholder": "​",
            "style": "IPY_MODEL_b3a48e91fef944f29cf17bec23d90628",
            "value": " 938/938 [00:14&lt;00:00, 78.07it/s]"
          }
        },
        "4419601c1ae84346b17a48973c3c3898": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d43f3dad0e244339141716a81662cb6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a0df5ac2671d4365aca74763228cd6be": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f109311dfc334bfb8d4ba3776239908a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3b3db09f2b4b4c01ad3ff682cadce8fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0b1b1a8779994b8f936a0632c8e7ed90": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b3a48e91fef944f29cf17bec23d90628": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "90f589afbb564e8199116fa3ab2f4c4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_91b87c66669f4759b15a27cf0c151804",
              "IPY_MODEL_7d8ae387a7a1413c85ae4262ecc6b2da",
              "IPY_MODEL_968693b7673e4e588861bc5a60151a98"
            ],
            "layout": "IPY_MODEL_f9a744d01b8144d29f63d5226be6357f"
          }
        },
        "91b87c66669f4759b15a27cf0c151804": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_985a558a468e44cd96a7136213d07645",
            "placeholder": "​",
            "style": "IPY_MODEL_6b06e7e3552f499c9110a3c38229673d",
            "value": "100%"
          }
        },
        "7d8ae387a7a1413c85ae4262ecc6b2da": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3adea2f2d6b740fbb77bed8ea4deb69f",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_188f185dc2d24e30ab7e618710be116f",
            "value": 938
          }
        },
        "968693b7673e4e588861bc5a60151a98": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b0b24e71f26414e871820c42483aeba",
            "placeholder": "​",
            "style": "IPY_MODEL_a3d9c4a018f249d4808eaec1a7e66879",
            "value": " 938/938 [00:13&lt;00:00, 73.63it/s]"
          }
        },
        "f9a744d01b8144d29f63d5226be6357f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "985a558a468e44cd96a7136213d07645": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6b06e7e3552f499c9110a3c38229673d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3adea2f2d6b740fbb77bed8ea4deb69f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "188f185dc2d24e30ab7e618710be116f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6b0b24e71f26414e871820c42483aeba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a3d9c4a018f249d4808eaec1a7e66879": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6ef77b5a220640af9dffba8f754a85b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_23b5ec6d5a9a4c12b817f11837433039",
              "IPY_MODEL_6e73d366c1774a16aef73ad151f25231",
              "IPY_MODEL_413e20adaab14611aca3ea13cd5ea142"
            ],
            "layout": "IPY_MODEL_85fd06a8e3064c3781d19dadd5046b42"
          }
        },
        "23b5ec6d5a9a4c12b817f11837433039": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6c3e6cf8548465e96679c2a27c88d64",
            "placeholder": "​",
            "style": "IPY_MODEL_67db24504c9a444fb809fd2925f0024b",
            "value": "Validating: 100%"
          }
        },
        "6e73d366c1774a16aef73ad151f25231": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_86421b4adf8443009ec875a9fbeb6fcd",
            "max": 157,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3728b0f225a1474cb67c18ddd799614b",
            "value": 157
          }
        },
        "413e20adaab14611aca3ea13cd5ea142": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d772f687f8b645ada4265795b1cf475c",
            "placeholder": "​",
            "style": "IPY_MODEL_b0c6768a822c46a4a385613ade669334",
            "value": " 157/157 [00:01&lt;00:00, 79.74it/s]"
          }
        },
        "85fd06a8e3064c3781d19dadd5046b42": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6c3e6cf8548465e96679c2a27c88d64": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67db24504c9a444fb809fd2925f0024b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "86421b4adf8443009ec875a9fbeb6fcd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3728b0f225a1474cb67c18ddd799614b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d772f687f8b645ada4265795b1cf475c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b0c6768a822c46a4a385613ade669334": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "915f23839f8b497d9c8984d574b3ea5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d39be31621b547d0bdf9fc9ad2eadfa5",
              "IPY_MODEL_1624b03bfe9c4e27b9ba943e1bd308dd",
              "IPY_MODEL_874271be60d84bae915cb99916139b63"
            ],
            "layout": "IPY_MODEL_970538766c164c05be2abf067965b317"
          }
        },
        "d39be31621b547d0bdf9fc9ad2eadfa5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9e0ec7b0d6914c7ebe7e9c71fccb2fe3",
            "placeholder": "​",
            "style": "IPY_MODEL_97afb5c58b8c499a88c63548eed070be",
            "value": "100%"
          }
        },
        "1624b03bfe9c4e27b9ba943e1bd308dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_093970309ea74ba6b3e67e3f3860dffa",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c6fe7df2bcb0458fb108c2485a3d982f",
            "value": 938
          }
        },
        "874271be60d84bae915cb99916139b63": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c67eafd73e642eea1067b1515676365",
            "placeholder": "​",
            "style": "IPY_MODEL_8fa1e1ed605b4f8eaa992f440785fa1b",
            "value": " 938/938 [00:14&lt;00:00, 31.72it/s]"
          }
        },
        "970538766c164c05be2abf067965b317": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e0ec7b0d6914c7ebe7e9c71fccb2fe3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "97afb5c58b8c499a88c63548eed070be": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "093970309ea74ba6b3e67e3f3860dffa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c6fe7df2bcb0458fb108c2485a3d982f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9c67eafd73e642eea1067b1515676365": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8fa1e1ed605b4f8eaa992f440785fa1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c3930b87c3114ec393b3edcdb1fab83e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_97e45169c241476fb8041cd620307909",
              "IPY_MODEL_bd74e010036840f0b46fc20802b07c64",
              "IPY_MODEL_3eb54ce864724adabc1e571fee0feb79"
            ],
            "layout": "IPY_MODEL_dd51bfadcc1e4495bbafb0cf9ad279b9"
          }
        },
        "97e45169c241476fb8041cd620307909": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_852ac15273724ff4b4a7778b98d934c9",
            "placeholder": "​",
            "style": "IPY_MODEL_473391b3ded14b358ce6905f36231b05",
            "value": "100%"
          }
        },
        "bd74e010036840f0b46fc20802b07c64": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e9ea8066a2de471a85129036a6967706",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2de9fd8ab3284ea1ad7d0090d6dceb8f",
            "value": 938
          }
        },
        "3eb54ce864724adabc1e571fee0feb79": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_167f8a23fc3648aca3b3b0ed6a1ca0a8",
            "placeholder": "​",
            "style": "IPY_MODEL_58c62817024f4c6d825dde4c7962fe19",
            "value": " 938/938 [00:25&lt;00:00, 71.60it/s]"
          }
        },
        "dd51bfadcc1e4495bbafb0cf9ad279b9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "852ac15273724ff4b4a7778b98d934c9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "473391b3ded14b358ce6905f36231b05": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e9ea8066a2de471a85129036a6967706": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2de9fd8ab3284ea1ad7d0090d6dceb8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "167f8a23fc3648aca3b3b0ed6a1ca0a8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "58c62817024f4c6d825dde4c7962fe19": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "de423df179cc4a3599eeb8b4eab54a1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0575ceb6d27b46508465f97ddd628c30",
              "IPY_MODEL_dc8bcb406a8e45d68b3bca226d3aba53",
              "IPY_MODEL_32d17218d3ec424a8f06ddd4fd730691"
            ],
            "layout": "IPY_MODEL_886e2b318c7a482d985039360f36433a"
          }
        },
        "0575ceb6d27b46508465f97ddd628c30": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1205188b13384a98b19cf66841b21a2b",
            "placeholder": "​",
            "style": "IPY_MODEL_88e8537104914278bad91ef6f7eb1c90",
            "value": "Validating: 100%"
          }
        },
        "dc8bcb406a8e45d68b3bca226d3aba53": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6358d881e51b494cb58dcdcacf74e425",
            "max": 157,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_615a655ea4644a7b9ed4656183087af3",
            "value": 157
          }
        },
        "32d17218d3ec424a8f06ddd4fd730691": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_37bb3d494d3c405789576040dbb56259",
            "placeholder": "​",
            "style": "IPY_MODEL_3245100768cc44f982cd0c6ed4465af6",
            "value": " 157/157 [00:02&lt;00:00, 61.69it/s]"
          }
        },
        "886e2b318c7a482d985039360f36433a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1205188b13384a98b19cf66841b21a2b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "88e8537104914278bad91ef6f7eb1c90": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6358d881e51b494cb58dcdcacf74e425": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "615a655ea4644a7b9ed4656183087af3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "37bb3d494d3c405789576040dbb56259": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3245100768cc44f982cd0c6ed4465af6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4878939ae6734768a3d5290a5001d0cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_37323af8eb9a45a981d5d39dfae6c8ff",
              "IPY_MODEL_c3a3a8390e8544a288ab0eeed900237a",
              "IPY_MODEL_ce0736fd5d384e5f8094e6d5ac3f3aa8"
            ],
            "layout": "IPY_MODEL_839d8ec957124aff8cbe36951ca261d2"
          }
        },
        "37323af8eb9a45a981d5d39dfae6c8ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db66a1bf27ae4fc4bd550e741f755944",
            "placeholder": "​",
            "style": "IPY_MODEL_cd03b9f8bb4c434a90386c5d1d350319",
            "value": "100%"
          }
        },
        "c3a3a8390e8544a288ab0eeed900237a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c464076e27404b5ea59cb62859c49c3c",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_45b160b0705642b1884d215cee80a790",
            "value": 938
          }
        },
        "ce0736fd5d384e5f8094e6d5ac3f3aa8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9f152de2b37a442cab8e77b24b47d523",
            "placeholder": "​",
            "style": "IPY_MODEL_b3f886fe14c64ec489c0ef383f30e656",
            "value": " 938/938 [00:13&lt;00:00, 54.83it/s]"
          }
        },
        "839d8ec957124aff8cbe36951ca261d2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db66a1bf27ae4fc4bd550e741f755944": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cd03b9f8bb4c434a90386c5d1d350319": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c464076e27404b5ea59cb62859c49c3c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "45b160b0705642b1884d215cee80a790": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9f152de2b37a442cab8e77b24b47d523": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b3f886fe14c64ec489c0ef383f30e656": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3acbe984b43f4d0092265ca295ca34eb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_23d5953e691a4f0a8136e95ed13d2b66",
              "IPY_MODEL_3263179a0ff84f2a8d4ce4106d0e3c3b",
              "IPY_MODEL_7869bfc19a464f298a32409eedcfe5f2"
            ],
            "layout": "IPY_MODEL_a19b363b48d54faba3ae4390e78ab2fa"
          }
        },
        "23d5953e691a4f0a8136e95ed13d2b66": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b485bd8a06f9479083b54134e1eb8dd4",
            "placeholder": "​",
            "style": "IPY_MODEL_4676152d5ce041b38019416de1f27b07",
            "value": "100%"
          }
        },
        "3263179a0ff84f2a8d4ce4106d0e3c3b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4740330d27d3493aab720a06bae3d1cf",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4edc432da3624f82a502a64e97cc5481",
            "value": 938
          }
        },
        "7869bfc19a464f298a32409eedcfe5f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c6766600aa0d4f53805359e093badb86",
            "placeholder": "​",
            "style": "IPY_MODEL_46ddbff341cc4019a7aa2a1a9fa7823a",
            "value": " 938/938 [00:14&lt;00:00, 48.64it/s]"
          }
        },
        "a19b363b48d54faba3ae4390e78ab2fa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b485bd8a06f9479083b54134e1eb8dd4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4676152d5ce041b38019416de1f27b07": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4740330d27d3493aab720a06bae3d1cf": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4edc432da3624f82a502a64e97cc5481": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c6766600aa0d4f53805359e093badb86": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46ddbff341cc4019a7aa2a1a9fa7823a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c9478d4c32ec4669be2c05427439a9bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0c19b5023cae406e9dcf9ad318ca0d0e",
              "IPY_MODEL_57f06e09791e4f4882af7def7c7f30df",
              "IPY_MODEL_c87be05509ed47758ab4aac2ae9fe40e"
            ],
            "layout": "IPY_MODEL_80227e7f33cd481a8a47886d5a0116b4"
          }
        },
        "0c19b5023cae406e9dcf9ad318ca0d0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_025856940d964c13a16b2767580ac333",
            "placeholder": "​",
            "style": "IPY_MODEL_2726defb9d1843988a8c7fb9b7383204",
            "value": "Validating: 100%"
          }
        },
        "57f06e09791e4f4882af7def7c7f30df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c07f39e5150c4de7a14aa6f0de3f8152",
            "max": 157,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d9615eda7ce74a72b141d555983f1106",
            "value": 157
          }
        },
        "c87be05509ed47758ab4aac2ae9fe40e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5ba6bab8a246441f9d453ab2ff0123f7",
            "placeholder": "​",
            "style": "IPY_MODEL_76b8c81ec8e2434281f900535d6e3c0d",
            "value": " 157/157 [00:02&lt;00:00, 80.99it/s]"
          }
        },
        "80227e7f33cd481a8a47886d5a0116b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "025856940d964c13a16b2767580ac333": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2726defb9d1843988a8c7fb9b7383204": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c07f39e5150c4de7a14aa6f0de3f8152": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d9615eda7ce74a72b141d555983f1106": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5ba6bab8a246441f9d453ab2ff0123f7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "76b8c81ec8e2434281f900535d6e3c0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "24f4c0ec566c47e0b79f6a82cdbbd429": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4416cadd96d34b3ea3836399860359d5",
              "IPY_MODEL_1dacb028065c4c87910826068d96b6e3",
              "IPY_MODEL_8106c7778de4416e9cea305382b566cf"
            ],
            "layout": "IPY_MODEL_fe669078abe14009b16e60767a6d75d2"
          }
        },
        "4416cadd96d34b3ea3836399860359d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_261dfc8ed39e43dd920593dee011f31d",
            "placeholder": "​",
            "style": "IPY_MODEL_56927cad1d8a4b4284ee7c60976fae8a",
            "value": "100%"
          }
        },
        "1dacb028065c4c87910826068d96b6e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_705c62ac4aa948ff9dcd89772abff7b4",
            "max": 938,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e8d63926f8c24e25986377bf2327e477",
            "value": 938
          }
        },
        "8106c7778de4416e9cea305382b566cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_09faa896dabc4c419b74d1b1cdf82dc1",
            "placeholder": "​",
            "style": "IPY_MODEL_272329eb2916471c88e2a5c35915d5e0",
            "value": " 938/938 [00:13&lt;00:00, 74.89it/s]"
          }
        },
        "fe669078abe14009b16e60767a6d75d2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "261dfc8ed39e43dd920593dee011f31d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "56927cad1d8a4b4284ee7c60976fae8a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "705c62ac4aa948ff9dcd89772abff7b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e8d63926f8c24e25986377bf2327e477": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "09faa896dabc4c419b74d1b1cdf82dc1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "272329eb2916471c88e2a5c35915d5e0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}