HW7_ac0.696.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2f8c4c12",
   "metadata": {
    "id": "view-in-github",
    "papermill": {
     "duration": 0.017353,
     "end_time": "2021-11-20T11:12:43.975144",
     "exception": false,
     "start_time": "2021-11-20T11:12:43.957791",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/ga642381/ML2021-Spring/blob/main/HW07/HW07.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97eab95a",
   "metadata": {
    "id": "xvSGDbExff_I",
    "papermill": {
     "duration": 0.013608,
     "end_time": "2021-11-20T11:12:44.004627",
     "exception": false,
     "start_time": "2021-11-20T11:12:43.991019",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# **Homework 7 - Bert (Question Answering)**\n",
    "\n",
    "If you have any questions, feel free to email us at ntu-ml-2021spring-ta@googlegroups.com\n",
    "\n",
    "\n",
    "\n",
    "Slide:    [Link](https://docs.google.com/presentation/d/1aQoWogAQo_xVJvMQMrGaYiWzuyfO0QyLLAhiMwFyS2w)　Kaggle: [Link](https://www.kaggle.com/c/ml2021-spring-hw7)　Data: [Link](https://drive.google.com/uc?id=1znKmX08v9Fygp-dgwo7BKiLIf2qL1FH1)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28467904",
   "metadata": {
    "id": "WGOr_eS3wJJf",
    "papermill": {
     "duration": 0.013559,
     "end_time": "2021-11-20T11:12:44.032091",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.018532",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Task description\n",
    "- Chinese Extractive Question Answering\n",
    "  - Input: Paragraph + Question\n",
    "  - Output: Answer\n",
    "\n",
    "- Objective: Learn how to fine tune a pretrained model on downstream task using transformers\n",
    "\n",
    "- Todo\n",
    "    - Fine tune a pretrained chinese BERT model\n",
    "    - Change hyperparameters (e.g. doc_stride)\n",
    "    - Apply linear learning rate decay\n",
    "    - Try other pretrained models\n",
    "    - Improve preprocessing\n",
    "    - Improve postprocessing\n",
    "- Training tips\n",
    "    - Automatic mixed precision\n",
    "    - Gradient accumulation\n",
    "    - Ensemble\n",
    "\n",
    "- Estimated training time (tesla t4 with automatic mixed precision enabled)\n",
    "    - Simple: 8mins\n",
    "    - Medium: 8mins\n",
    "    - Strong: 25mins\n",
    "    - Boss: 2hrs\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b0a7db95",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:12:44.070077Z",
     "iopub.status.busy": "2021-11-20T11:12:44.069349Z",
     "iopub.status.idle": "2021-11-20T11:12:44.091381Z",
     "shell.execute_reply": "2021-11-20T11:12:44.091983Z",
     "shell.execute_reply.started": "2021-11-20T08:54:25.450090Z"
    },
    "papermill": {
     "duration": 0.046419,
     "end_time": "2021-11-20T11:12:44.092261",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.045842",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/bert-question-answering/hw7_train.json\n",
      "/kaggle/input/bert-question-answering/hw7_dev.json\n",
      "/kaggle/input/bert-question-answering/hw7_test.json\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83a58ed0",
   "metadata": {
    "id": "TJ1fSAJE2oaC",
    "papermill": {
     "duration": 0.014878,
     "end_time": "2021-11-20T11:12:44.123028",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.108150",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Download Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d0cf9f3",
   "metadata": {
    "id": "TevOvhC03m0h",
    "papermill": {
     "duration": 0.014334,
     "end_time": "2021-11-20T11:12:44.151864",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.137530",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Install transformers\n",
    "\n",
    "Documentation for the toolkit:　https://huggingface.co/transformers/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b7ef9723",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:12:44.184947Z",
     "iopub.status.busy": "2021-11-20T11:12:44.184164Z",
     "iopub.status.idle": "2021-11-20T11:12:44.186140Z",
     "shell.execute_reply": "2021-11-20T11:12:44.186552Z",
     "shell.execute_reply.started": "2021-11-20T08:54:25.476394Z"
    },
    "id": "tbxWFX_jpDom",
    "papermill": {
     "duration": 0.02051,
     "end_time": "2021-11-20T11:12:44.186704",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.166194",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# You are allowed to change version of transformers or use other toolkits\n",
    "#!pip install transformers==4.5.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5f9146e",
   "metadata": {
    "id": "8dKM4yCh4LI_",
    "papermill": {
     "duration": 0.014577,
     "end_time": "2021-11-20T11:12:44.215845",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.201268",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "81c39b34",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:12:44.251546Z",
     "iopub.status.busy": "2021-11-20T11:12:44.246966Z",
     "iopub.status.idle": "2021-11-20T11:12:49.812569Z",
     "shell.execute_reply": "2021-11-20T11:12:49.811677Z",
     "shell.execute_reply.started": "2021-11-20T08:54:25.699070Z"
    },
    "id": "WOTHHtWJoahe",
    "papermill": {
     "duration": 5.582639,
     "end_time": "2021-11-20T11:12:49.812752",
     "exception": false,
     "start_time": "2021-11-20T11:12:44.230113",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np\n",
    "import random\n",
    "import torch\n",
    "from torch.utils.data import DataLoader, Dataset \n",
    "from transformers import AdamW, BertForQuestionAnswering, BertTokenizerFast\n",
    "\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "\n",
    "# Fix random seed for reproducibility\n",
    "def same_seeds(seed):\n",
    "\t  torch.manual_seed(seed)\n",
    "\t  if torch.cuda.is_available():\n",
    "\t\t    torch.cuda.manual_seed(seed)\n",
    "\t\t    torch.cuda.manual_seed_all(seed)\n",
    "\t  np.random.seed(seed)\n",
    "\t  random.seed(seed)\n",
    "\t  torch.backends.cudnn.benchmark = False\n",
    "\t  torch.backends.cudnn.deterministic = True\n",
    "same_seeds(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "47ce9e37",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:12:49.847375Z",
     "iopub.status.busy": "2021-11-20T11:12:49.846632Z",
     "iopub.status.idle": "2021-11-20T11:12:49.849135Z",
     "shell.execute_reply": "2021-11-20T11:12:49.848709Z",
     "shell.execute_reply.started": "2021-11-20T08:54:31.212670Z"
    },
    "id": "7pBtSZP1SKQO",
    "papermill": {
     "duration": 0.021981,
     "end_time": "2021-11-20T11:12:49.849252",
     "exception": false,
     "start_time": "2021-11-20T11:12:49.827271",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Change \"fp16_training\" to True to support automatic mixed precision training (fp16)\t\n",
    "fp16_training = False\n",
    "\n",
    "if fp16_training:\n",
    "    !pip install accelerate==0.2.0\n",
    "    from accelerate import Accelerator\n",
    "    accelerator = Accelerator(fp16=True)\n",
    "    device = accelerator.device\n",
    "\n",
    "# Documentation for the toolkit:  https://huggingface.co/docs/accelerate/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "679ffe20",
   "metadata": {
    "id": "2YgXHuVLp_6j",
    "papermill": {
     "duration": 0.013916,
     "end_time": "2021-11-20T11:12:49.877084",
     "exception": false,
     "start_time": "2021-11-20T11:12:49.863168",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Load Model and Tokenizer\n",
    "\n",
    "\n",
    "\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e72f8da3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:12:49.910526Z",
     "iopub.status.busy": "2021-11-20T11:12:49.909736Z",
     "iopub.status.idle": "2021-11-20T11:13:27.163653Z",
     "shell.execute_reply": "2021-11-20T11:13:27.164072Z",
     "shell.execute_reply.started": "2021-11-20T08:54:31.221440Z"
    },
    "id": "xyBCYGjAp3ym",
    "papermill": {
     "duration": 37.272822,
     "end_time": "2021-11-20T11:13:27.164222",
     "exception": false,
     "start_time": "2021-11-20T11:12:49.891400",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4feb27d51b194b69a800676244a746d7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/624 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2094897a57344e1f8f299b3de7abbdc4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/412M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at bert-base-chinese were not used when initializing BertForQuestionAnswering: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']\n",
      "- This IS expected if you are initializing BertForQuestionAnswering from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing BertForQuestionAnswering from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of BertForQuestionAnswering were not initialized from the model checkpoint at bert-base-chinese and are newly initialized: ['qa_outputs.weight', 'qa_outputs.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "76939c8565ca4380add4d7508d722281",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/110k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0cec9d3d60c34e35a75be11f664feaf4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/269k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a28075a734544392921753ed84662c0b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/29.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = BertForQuestionAnswering.from_pretrained(\"bert-base-chinese\").to(device)\n",
    "tokenizer = BertTokenizerFast.from_pretrained(\"bert-base-chinese\")\n",
    "\n",
    "# You can safely ignore the warning message (it pops up because new prediction heads for QA are initialized randomly)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67f48858",
   "metadata": {
    "id": "3Td-GTmk5OW4",
    "papermill": {
     "duration": 0.164558,
     "end_time": "2021-11-20T11:13:27.347479",
     "exception": false,
     "start_time": "2021-11-20T11:13:27.182921",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Read Data\n",
    "\n",
    "- Training set: 26935 QA pairs\n",
    "- Dev set: 3523  QA pairs\n",
    "- Test set: 3492  QA pairs\n",
    "\n",
    "- {train/dev/test}_questions:\t\n",
    "  - List of dicts with the following keys:\n",
    "   - id (int)\n",
    "   - paragraph_id (int)\n",
    "   - question_text (string)\n",
    "   - answer_text (string)\n",
    "   - answer_start (int)\n",
    "   - answer_end (int)\n",
    "- {train/dev/test}_paragraphs: \n",
    "  - List of strings\n",
    "  - paragraph_ids in questions correspond to indexs in paragraphs\n",
    "  - A paragraph may be used by several questions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "411ae89b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:13:27.658136Z",
     "iopub.status.busy": "2021-11-20T11:13:27.657610Z",
     "iopub.status.idle": "2021-11-20T11:13:28.228734Z",
     "shell.execute_reply": "2021-11-20T11:13:28.228154Z",
     "shell.execute_reply.started": "2021-11-20T08:55:22.100773Z"
    },
    "id": "NvX7hlepogvu",
    "papermill": {
     "duration": 0.595131,
     "end_time": "2021-11-20T11:13:28.228867",
     "exception": false,
     "start_time": "2021-11-20T11:13:27.633736",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def read_data(file):\n",
    "    with open(file, 'r', encoding=\"utf-8\") as reader:\n",
    "        data = json.load(reader)\n",
    "    return data[\"questions\"], data[\"paragraphs\"]\n",
    "\n",
    "train_questions, train_paragraphs = read_data(\"../input/bert-question-answering/hw7_train.json\")\n",
    "dev_questions, dev_paragraphs = read_data(\"../input/bert-question-answering/hw7_dev.json\")\n",
    "test_questions, test_paragraphs = read_data(\"../input/bert-question-answering/hw7_test.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01a0508d",
   "metadata": {
    "id": "Fm0rpTHq0e4N",
    "papermill": {
     "duration": 0.016742,
     "end_time": "2021-11-20T11:13:28.262806",
     "exception": false,
     "start_time": "2021-11-20T11:13:28.246064",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Tokenize Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "280ef2e5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:13:28.316447Z",
     "iopub.status.busy": "2021-11-20T11:13:28.304418Z",
     "iopub.status.idle": "2021-11-20T11:13:38.487124Z",
     "shell.execute_reply": "2021-11-20T11:13:38.486465Z",
     "shell.execute_reply.started": "2021-11-20T08:55:22.621977Z"
    },
    "id": "rTZ6B70Hoxie",
    "papermill": {
     "duration": 10.207979,
     "end_time": "2021-11-20T11:13:38.487267",
     "exception": false,
     "start_time": "2021-11-20T11:13:28.279288",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Token indices sequence length is longer than the specified maximum sequence length for this model (570 > 512). Running this sequence through the model will result in indexing errors\n"
     ]
    }
   ],
   "source": [
    "# Tokenize questions and paragraphs separately\n",
    "# 「add_special_tokens」 is set to False since special tokens will be added when tokenized questions and paragraphs are combined in datset __getitem__ \n",
    "\n",
    "train_questions_tokenized = tokenizer([train_question[\"question_text\"] for train_question in train_questions], add_special_tokens=False)\n",
    "dev_questions_tokenized = tokenizer([dev_question[\"question_text\"] for dev_question in dev_questions], add_special_tokens=False)\n",
    "test_questions_tokenized = tokenizer([test_question[\"question_text\"] for test_question in test_questions], add_special_tokens=False) \n",
    "\n",
    "train_paragraphs_tokenized = tokenizer(train_paragraphs, add_special_tokens=False)\n",
    "dev_paragraphs_tokenized = tokenizer(dev_paragraphs, add_special_tokens=False)\n",
    "test_paragraphs_tokenized = tokenizer(test_paragraphs, add_special_tokens=False)\n",
    "\n",
    "# You can safely ignore the warning message as tokenized sequences will be futher processed in datset __getitem__ before passing to model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "516a65b3",
   "metadata": {
    "id": "Ws8c8_4d5UCI",
    "papermill": {
     "duration": 0.017128,
     "end_time": "2021-11-20T11:13:38.522209",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.505081",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Dataset and Dataloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "41fedc9b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:13:38.578015Z",
     "iopub.status.busy": "2021-11-20T11:13:38.576377Z",
     "iopub.status.idle": "2021-11-20T11:13:38.578693Z",
     "shell.execute_reply": "2021-11-20T11:13:38.579113Z",
     "shell.execute_reply.started": "2021-11-20T08:55:31.073669Z"
    },
    "id": "Xjooag-Swnuh",
    "papermill": {
     "duration": 0.03975,
     "end_time": "2021-11-20T11:13:38.579239",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.539489",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class QA_Dataset(Dataset):\n",
    "    def __init__(self, split, questions, tokenized_questions, tokenized_paragraphs):\n",
    "        self.split = split\n",
    "        self.questions = questions\n",
    "        self.tokenized_questions = tokenized_questions\n",
    "        self.tokenized_paragraphs = tokenized_paragraphs\n",
    "        self.max_question_len = 40\n",
    "        self.max_paragraph_len = 150\n",
    "        \n",
    "        ##### TODO: Change value of doc_stride #####\n",
    "        self.doc_stride = 75\n",
    "\n",
    "        # Input sequence length = [CLS] + question + [SEP] + paragraph + [SEP]\n",
    "        self.max_seq_len = 1 + self.max_question_len + 1 + self.max_paragraph_len + 1\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.questions)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        question = self.questions[idx]\n",
    "        tokenized_question = self.tokenized_questions[idx]\n",
    "        tokenized_paragraph = self.tokenized_paragraphs[question[\"paragraph_id\"]]\n",
    "\n",
    "        ##### TODO: Preprocessing #####\n",
    "        # Hint: How to prevent model from learning something it should not learn\n",
    "\n",
    "        if self.split == \"train\":\n",
    "            # Convert answer's start/end positions in paragraph_text to start/end positions in tokenized_paragraph  \n",
    "            answer_start_token = tokenized_paragraph.char_to_token(question[\"answer_start\"])\n",
    "            answer_end_token = tokenized_paragraph.char_to_token(question[\"answer_end\"])\n",
    "\n",
    "            # A single window is obtained by slicing the portion of paragraph containing the answer\n",
    "            mid = (answer_start_token + answer_end_token) // 2\n",
    "            paragraph_start = max(0, min(mid - self.max_paragraph_len // 2, len(tokenized_paragraph) - self.max_paragraph_len))\n",
    "            paragraph_end = paragraph_start + self.max_paragraph_len\n",
    "            \n",
    "            # Slice question/paragraph and add special tokens (101: CLS, 102: SEP)\n",
    "            input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102] \n",
    "            input_ids_paragraph = tokenized_paragraph.ids[paragraph_start : paragraph_end] + [102]\t\t\n",
    "            \n",
    "            # Convert answer's start/end positions in tokenized_paragraph to start/end positions in the window  \n",
    "            answer_start_token += len(input_ids_question) - paragraph_start\n",
    "            answer_end_token += len(input_ids_question) - paragraph_start\n",
    "            \n",
    "            # Pad sequence and obtain inputs to model \n",
    "            input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "            return torch.tensor(input_ids), torch.tensor(token_type_ids), torch.tensor(attention_mask), answer_start_token, answer_end_token\n",
    "\n",
    "        # Validation/Testing\n",
    "        else:\n",
    "            input_ids_list, token_type_ids_list, attention_mask_list = [], [], []\n",
    "            \n",
    "            # Paragraph is split into several windows, each with start positions separated by step \"doc_stride\"\n",
    "            for i in range(0, len(tokenized_paragraph), self.doc_stride):\n",
    "                \n",
    "                # Slice question/paragraph and add special tokens (101: CLS, 102: SEP)\n",
    "                input_ids_question = [101] + tokenized_question.ids[:self.max_question_len] + [102]\n",
    "                input_ids_paragraph = tokenized_paragraph.ids[i : i + self.max_paragraph_len] + [102]\n",
    "                \n",
    "                # Pad sequence and obtain inputs to model\n",
    "                input_ids, token_type_ids, attention_mask = self.padding(input_ids_question, input_ids_paragraph)\n",
    "                \n",
    "                input_ids_list.append(input_ids)\n",
    "                token_type_ids_list.append(token_type_ids)\n",
    "                attention_mask_list.append(attention_mask)\n",
    "            \n",
    "            return torch.tensor(input_ids_list), torch.tensor(token_type_ids_list), torch.tensor(attention_mask_list)\n",
    "\n",
    "    def padding(self, input_ids_question, input_ids_paragraph):\n",
    "        # Pad zeros if sequence length is shorter than max_seq_len\n",
    "        padding_len = self.max_seq_len - len(input_ids_question) - len(input_ids_paragraph)\n",
    "        # Indices of input sequence tokens in the vocabulary\n",
    "        input_ids = input_ids_question + input_ids_paragraph + [0] * padding_len\n",
    "        # Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]\n",
    "        token_type_ids = [0] * len(input_ids_question) + [1] * len(input_ids_paragraph) + [0] * padding_len\n",
    "        # Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]\n",
    "        attention_mask = [1] * (len(input_ids_question) + len(input_ids_paragraph)) + [0] * padding_len\n",
    "        \n",
    "        return input_ids, token_type_ids, attention_mask\n",
    "\n",
    "train_set = QA_Dataset(\"train\", train_questions, train_questions_tokenized, train_paragraphs_tokenized)\n",
    "dev_set = QA_Dataset(\"dev\", dev_questions, dev_questions_tokenized, dev_paragraphs_tokenized)\n",
    "test_set = QA_Dataset(\"test\", test_questions, test_questions_tokenized, test_paragraphs_tokenized)\n",
    "\n",
    "train_batch_size = 16\n",
    "\n",
    "# Note: Do NOT change batch size of dev_loader / test_loader !\n",
    "# Although batch size=1, it is actually a batch consisting of several windows from the same QA pair\n",
    "train_loader = DataLoader(train_set, batch_size=train_batch_size, shuffle=True, pin_memory=True)\n",
    "dev_loader = DataLoader(dev_set, batch_size=1, shuffle=False, pin_memory=True)\n",
    "test_loader = DataLoader(test_set, batch_size=1, shuffle=False, pin_memory=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbac9ccd",
   "metadata": {
    "id": "5_H1kqhR8CdM",
    "papermill": {
     "duration": 0.017987,
     "end_time": "2021-11-20T11:13:38.614843",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.596856",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Function for Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2c7479ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:13:38.660318Z",
     "iopub.status.busy": "2021-11-20T11:13:38.658642Z",
     "iopub.status.idle": "2021-11-20T11:13:38.660951Z",
     "shell.execute_reply": "2021-11-20T11:13:38.661385Z",
     "shell.execute_reply.started": "2021-11-20T08:55:31.095505Z"
    },
    "id": "SqeA3PLPxOHu",
    "papermill": {
     "duration": 0.028167,
     "end_time": "2021-11-20T11:13:38.661511",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.633344",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def evaluate(data, output):\n",
    "    ##### TODO: Postprocessing #####\n",
    "    # There is a bug and room for improvement in postprocessing \n",
    "    # Hint: Open your prediction file to see what is wrong \n",
    "    \n",
    "    answer = ''\n",
    "    max_prob = float('-inf')\n",
    "    num_of_windows = data[0].shape[1]\n",
    "    \n",
    "    for k in range(num_of_windows):\n",
    "        # Obtain answer by choosing the most probable start position / end position\n",
    "        start_prob, start_index = torch.max(output.start_logits[k], dim=0)\n",
    "        end_prob, end_index = torch.max(output.end_logits[k], dim=0)\n",
    "        \n",
    "        # Probability of answer is calculated as sum of start_prob and end_prob\n",
    "        prob = start_prob + end_prob\n",
    "        \n",
    "        # Replace answer if calculated probability is larger than previous windows\n",
    "        if prob > max_prob:\n",
    "            max_prob = prob\n",
    "            # Convert tokens to chars (e.g. [1920, 7032] --> \"大 金\")\n",
    "            answer = tokenizer.decode(data[0][0][k][start_index : end_index + 1])\n",
    "    \n",
    "    # Remove spaces in answer (e.g. \"大 金\" --> \"大金\")\n",
    "    return answer.replace(' ','')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40a5275a",
   "metadata": {
    "id": "rzHQit6eMnKG",
    "papermill": {
     "duration": 0.018388,
     "end_time": "2021-11-20T11:13:38.698197",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.679809",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a0a4878a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:13:38.751329Z",
     "iopub.status.busy": "2021-11-20T11:13:38.750778Z",
     "iopub.status.idle": "2021-11-20T11:25:24.469530Z",
     "shell.execute_reply": "2021-11-20T11:25:24.469033Z",
     "shell.execute_reply.started": "2021-11-20T08:55:31.109448Z"
    },
    "id": "3Q-B6ka7xoCM",
    "papermill": {
     "duration": 705.753334,
     "end_time": "2021-11-20T11:25:24.469696",
     "exception": false,
     "start_time": "2021-11-20T11:13:38.716362",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start Training ...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a67e72d918fc4aeeb90b15a4e5a27673",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1684 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 | Step 100 | loss = 1.793, acc = 0.394 \n",
      "Epoch 1 | Step 200 | loss = 0.995, acc = 0.611 \n",
      "Epoch 1 | Step 300 | loss = 0.876, acc = 0.663 \n",
      "Epoch 1 | Step 400 | loss = 0.765, acc = 0.686 \n",
      "Epoch 1 | Step 500 | loss = 0.708, acc = 0.704 \n",
      "Epoch 1 | Step 600 | loss = 0.650, acc = 0.711 \n",
      "Epoch 1 | Step 700 | loss = 0.568, acc = 0.738 \n",
      "Epoch 1 | Step 800 | loss = 0.557, acc = 0.762 \n",
      "Epoch 1 | Step 900 | loss = 0.535, acc = 0.751 \n",
      "Epoch 1 | Step 1000 | loss = 0.547, acc = 0.746 \n",
      "Epoch 1 | Step 1100 | loss = 0.513, acc = 0.767 \n",
      "Epoch 1 | Step 1200 | loss = 0.515, acc = 0.774 \n",
      "Epoch 1 | Step 1300 | loss = 0.503, acc = 0.768 \n",
      "Epoch 1 | Step 1400 | loss = 0.479, acc = 0.771 \n",
      "Epoch 1 | Step 1500 | loss = 0.423, acc = 0.799 \n",
      "Epoch 1 | Step 1600 | loss = 0.449, acc = 0.786 \n",
      "Evaluating Dev Set ...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aa8a992eda6541439bc6fbeaf2d2c180",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3524 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation | Epoch 1 | acc = 0.697\n",
      "Saving Model ...\n"
     ]
    }
   ],
   "source": [
    "num_epoch = 1 \n",
    "validation = True\n",
    "logging_step = 100\n",
    "learning_rate = 1e-4\n",
    "optimizer = AdamW(model.parameters(), lr=learning_rate)\n",
    "\n",
    "if fp16_training:\n",
    "    model, optimizer, train_loader = accelerator.prepare(model, optimizer, train_loader) \n",
    "\n",
    "model.train()\n",
    "\n",
    "print(\"Start Training ...\")\n",
    "\n",
    "for epoch in range(num_epoch):\n",
    "    step = 1\n",
    "    train_loss = train_acc = 0\n",
    "    \n",
    "    for data in tqdm(train_loader):\t\n",
    "        # Load all data into GPU\n",
    "        data = [i.to(device) for i in data]\n",
    "        \n",
    "        # Model inputs: input_ids, token_type_ids, attention_mask, start_positions, end_positions (Note: only \"input_ids\" is mandatory)\n",
    "        # Model outputs: start_logits, end_logits, loss (return when start_positions/end_positions are provided)  \n",
    "        output = model(input_ids=data[0], token_type_ids=data[1], attention_mask=data[2], start_positions=data[3], end_positions=data[4])\n",
    "\n",
    "        # Choose the most probable start position / end position\n",
    "        start_index = torch.argmax(output.start_logits, dim=1)\n",
    "        end_index = torch.argmax(output.end_logits, dim=1)\n",
    "        \n",
    "        # Prediction is correct only if both start_index and end_index are correct\n",
    "        train_acc += ((start_index == data[3]) & (end_index == data[4])).float().mean()\n",
    "        train_loss += output.loss\n",
    "        \n",
    "        if fp16_training:\n",
    "            accelerator.backward(output.loss)\n",
    "        else:\n",
    "            output.loss.backward()\n",
    "        \n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "        step += 1\n",
    "\n",
    "        ##### TODO: Apply linear learning rate decay #####\n",
    "        optimizer.param_groups[0][\"lr\"] -= learning_rate/1684\n",
    "        \n",
    "        # Print training loss and accuracy over past logging step\n",
    "        if step % logging_step == 0:\n",
    "            print(f\"Epoch {epoch + 1} | Step {step} | loss = {train_loss.item() / logging_step:.3f}, acc = {train_acc / logging_step:.3f} \")\n",
    "            train_loss = train_acc = 0\n",
    "\n",
    "    if validation:\n",
    "        print(\"Evaluating Dev Set ...\")\n",
    "        model.eval()\n",
    "        with torch.no_grad():\n",
    "            dev_acc = 0\n",
    "            for i, data in enumerate(tqdm(dev_loader)):\n",
    "                output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "                # prediction is correct only if answer text exactly matches\n",
    "                dev_acc += evaluate(data, output) == dev_questions[i][\"answer_text\"]\n",
    "            print(f\"Validation | Epoch {epoch + 1} | acc = {dev_acc / len(dev_loader):.3f}\")\n",
    "        model.train()\n",
    "\n",
    "# Save a model and its configuration file to the directory 「saved_model」 \n",
    "# i.e. there are two files under the direcory 「saved_model」: 「pytorch_model.bin」 and 「config.json」\n",
    "# Saved model can be re-loaded using 「model = BertForQuestionAnswering.from_pretrained(\"saved_model\")」\n",
    "print(\"Saving Model ...\")\n",
    "model_save_dir = \"saved_model\" \n",
    "model.save_pretrained(model_save_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7baec4a",
   "metadata": {
    "id": "kMmdLOKBMsdE",
    "papermill": {
     "duration": 0.02228,
     "end_time": "2021-11-20T11:25:24.514941",
     "exception": false,
     "start_time": "2021-11-20T11:25:24.492661",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "36730552",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-11-20T11:25:24.568135Z",
     "iopub.status.busy": "2021-11-20T11:25:24.567379Z",
     "iopub.status.idle": "2021-11-20T11:28:04.946870Z",
     "shell.execute_reply": "2021-11-20T11:28:04.947414Z",
     "shell.execute_reply.started": "2021-11-20T09:06:09.825799Z"
    },
    "id": "U5scNKC9xz0C",
    "papermill": {
     "duration": 160.410547,
     "end_time": "2021-11-20T11:28:04.947642",
     "exception": false,
     "start_time": "2021-11-20T11:25:24.537095",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluating Test Set ...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ee1b7c6b2931478ebf0741078f5c3d94",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3493 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completed! Result is in result.csv\n"
     ]
    }
   ],
   "source": [
    "print(\"Evaluating Test Set ...\")\n",
    "\n",
    "result = []\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    for data in tqdm(test_loader):\n",
    "        output = model(input_ids=data[0].squeeze(dim=0).to(device), token_type_ids=data[1].squeeze(dim=0).to(device),\n",
    "                       attention_mask=data[2].squeeze(dim=0).to(device))\n",
    "        result.append(evaluate(data, output))\n",
    "\n",
    "result_file = \"result.csv\"\n",
    "with open(result_file, 'w') as f:\t\n",
    "\t  f.write(\"ID,Answer\\n\")\n",
    "\t  for i, test_question in enumerate(test_questions):\n",
    "        # Replace commas in answers with empty strings (since csv is separated by comma)\n",
    "        # Answers in kaggle are processed in the same way\n",
    "\t\t    f.write(f\"{test_question['id']},{result[i].replace(',','')}\\n\")\n",
    "\n",
    "print(f\"Completed! Result is in {result_file}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 931.598548,
   "end_time": "2021-11-20T11:28:08.147785",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2021-11-20T11:12:36.549237",
   "version": "2.3.3"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "0218def1301f4c179b4c5bcee87d3651": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "05c90be74d364272bd1c378fdc5870a5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "064f35f27b674206ab041f42a280489f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "06f376e9786848e9ac33d1a7156cb6cb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0864329a07214e5ebe0a0f3d1dd34104": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c3c2e0ba50a8420db89b11567e332348",
       "placeholder": "​",
       "style": "IPY_MODEL_06f376e9786848e9ac33d1a7156cb6cb",
       "value": " 269k/269k [00:00&lt;00:00, 864kB/s]"
      }
     },
     "0a202fbbbc1448ccbd7dd3e931e1c0e8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "0aec29a0c39c488fae37ee35a65d5aa6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0b692fc7d2ec4fff82c5167ed8cba5dc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0cec9d3d60c34e35a75be11f664feaf4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_5e2da6a653dc4401bb679c3478c63657",
        "IPY_MODEL_c2ab7830d2634456bf30ac5f7a80885d",
        "IPY_MODEL_0864329a07214e5ebe0a0f3d1dd34104"
       ],
       "layout": "IPY_MODEL_b37025b2cd1547c180e1ded404350f94"
      }
     },
     "0d027db93daa416b8ce0abb7065a5cf6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "116d3fe4cc4f4877a78c63d26344514e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_97ad960f0cb641859034fa4e44e2b0f7",
       "placeholder": "​",
       "style": "IPY_MODEL_2c39ebebd3844fd980b17eaa99e1585e",
       "value": " 412M/412M [00:22&lt;00:00, 12.3MB/s]"
      }
     },
     "1257613a7ee04adc94eaa678a741464f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "1d3b0ef4688d48d89a3ed110d4b50ef2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1dd7df6df87c4c3aa63750a45d328cd2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "1f98762116d1424ea1f83eab5874e38d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c27952ff52974b178a45d46180f2a471",
       "max": 109540.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_9ce8b4334a3245d49442a85e43894a57",
       "value": 109540.0
      }
     },
     "2094897a57344e1f8f299b3de7abbdc4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_3167ae81074d4a3b96b2d27cc14bf53a",
        "IPY_MODEL_59fd43661d4c4449a80e47809ff77e70",
        "IPY_MODEL_116d3fe4cc4f4877a78c63d26344514e"
       ],
       "layout": "IPY_MODEL_a3eeb3c82e194f15a546119e110b4503"
      }
     },
     "23715952043b4d0d934ac66d2bcf6acb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "2acca5830df74bdc92bbfbc9e87956e8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_0218def1301f4c179b4c5bcee87d3651",
       "max": 3524.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_46ab8aae49ff4d998fe099ccc26c3c1d",
       "value": 3524.0
      }
     },
     "2b3a318dff5148c3b9c02aa62c49e5bc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2c39ebebd3844fd980b17eaa99e1585e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "2e6ef16ce3b74f42a7970f6a72934314": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_4229d314f9cf490280e08e134f872efc",
       "placeholder": "​",
       "style": "IPY_MODEL_0b692fc7d2ec4fff82c5167ed8cba5dc",
       "value": "100%"
      }
     },
     "2f893806706d48f594abf924efa3eb97": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3120a32da70e4c80ad3a661615757b61": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3167ae81074d4a3b96b2d27cc14bf53a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6622c6fda7454a2b9d1121015ab66f6a",
       "placeholder": "​",
       "style": "IPY_MODEL_59c9792a16c547cfae3214ef59251417",
       "value": "Downloading: 100%"
      }
     },
     "356a608b0a9641249f510028ecf734bd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_845796aa320946e4ba8e6223d9c4070c",
       "max": 3493.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_23715952043b4d0d934ac66d2bcf6acb",
       "value": 3493.0
      }
     },
     "3861df3f53c040349d065f432a86dbb8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3cd5628685654a9f8e36c5a8781c7bad": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4013c5ce817a46e3ad01fced58344058": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4229d314f9cf490280e08e134f872efc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "44166611d83d43579c09a254bfef0a54": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "45592eaf37d0454381c2ff48c4f0712b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_638071e1e44f49f7bbbf1c6bb72b9a7e",
       "max": 1684.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_98790649e37a42ef8c2001cebcb89da4",
       "value": 1684.0
      }
     },
     "46ab8aae49ff4d998fe099ccc26c3c1d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "4985824116b341e5ae82e2e96f206146": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4feb27d51b194b69a800676244a746d7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_9ad75a4d85b740129f893308ce0bb8aa",
        "IPY_MODEL_588a09812f1d4cd88f36493b67246bc8",
        "IPY_MODEL_f5216ec582a44ffdbe42879ba6276a8c"
       ],
       "layout": "IPY_MODEL_b1076dea4226432193fb9e0228f9478e"
      }
     },
     "588a09812f1d4cd88f36493b67246bc8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6464c309b0cb486bad3ab56e13cfeedc",
       "max": 624.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_1257613a7ee04adc94eaa678a741464f",
       "value": 624.0
      }
     },
     "59c9792a16c547cfae3214ef59251417": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "59fd43661d4c4449a80e47809ff77e70": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_cbcd29bfa14e45bcbbbd0d5812bf805c",
       "max": 411577189.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_dcbd22effe0d4fa58bac1e5915ab03ec",
       "value": 411577189.0
      }
     },
     "5e2da6a653dc4401bb679c3478c63657": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_762d806259614cddbd58a542ae757dc9",
       "placeholder": "​",
       "style": "IPY_MODEL_44166611d83d43579c09a254bfef0a54",
       "value": "Downloading: 100%"
      }
     },
     "638071e1e44f49f7bbbf1c6bb72b9a7e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6464c309b0cb486bad3ab56e13cfeedc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6622c6fda7454a2b9d1121015ab66f6a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "699094ff6e2e49d1ac875a3cfe7ed6a2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6b186e631cc94570a21364fc15ece208": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "711c77f5b5104ce3a6b024cfc90c161b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "762d806259614cddbd58a542ae757dc9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "76939c8565ca4380add4d7508d722281": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_f719b921029e48b591be17c0551bc28b",
        "IPY_MODEL_1f98762116d1424ea1f83eab5874e38d",
        "IPY_MODEL_949a097468f1473fa3106bf6b110cabd"
       ],
       "layout": "IPY_MODEL_2f893806706d48f594abf924efa3eb97"
      }
     },
     "77e1ee0502a0411dad129bd3371f8139": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "833d41ce4e274756b812fd2fbc127103": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "842ee690456d4c3e99e0b1b68882c41b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "845796aa320946e4ba8e6223d9c4070c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "9067c16f1c7c4965845cfb68bbc6408a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_77e1ee0502a0411dad129bd3371f8139",
       "placeholder": "​",
       "style": "IPY_MODEL_d259d791a91746b1a38cfb35998b1e6e",
       "value": " 3524/3524 [02:46&lt;00:00, 22.56it/s]"
      }
     },
     "9234744775fd432980f3d4f9715021c5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "949a097468f1473fa3106bf6b110cabd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3120a32da70e4c80ad3a661615757b61",
       "placeholder": "​",
       "style": "IPY_MODEL_833d41ce4e274756b812fd2fbc127103",
       "value": " 110k/110k [00:00&lt;00:00, 803kB/s]"
      }
     },
     "96ed1a9beef64e56a301e1f325670513": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "97ad960f0cb641859034fa4e44e2b0f7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "98790649e37a42ef8c2001cebcb89da4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "9ad75a4d85b740129f893308ce0bb8aa": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d29df587ed1149e3999572ba5b6b981f",
       "placeholder": "​",
       "style": "IPY_MODEL_05c90be74d364272bd1c378fdc5870a5",
       "value": "Downloading: 100%"
      }
     },
     "9ba61440136f4ff09b26cca0082ecbfa": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "9ce8b4334a3245d49442a85e43894a57": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "a28075a734544392921753ed84662c0b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_db1a69e1b3ba4782bc3e89583eb022d3",
        "IPY_MODEL_b05f5a673e234dd4a777f05a5d670189",
        "IPY_MODEL_c512cf61f5164037b88dc9c7bfa5f8b5"
       ],
       "layout": "IPY_MODEL_4985824116b341e5ae82e2e96f206146"
      }
     },
     "a3eeb3c82e194f15a546119e110b4503": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "a67e72d918fc4aeeb90b15a4e5a27673": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_a72d7121a104463aa854b68cacd8bec5",
        "IPY_MODEL_45592eaf37d0454381c2ff48c4f0712b",
        "IPY_MODEL_e10a70de62bc4dc18a5dd0a36749d96c"
       ],
       "layout": "IPY_MODEL_b4c8a65e2dd145c6b5d621b6f0efbaf7"
      }
     },
     "a72d7121a104463aa854b68cacd8bec5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d3d424b2c8eb4a6e9e6f052f77db02ca",
       "placeholder": "​",
       "style": "IPY_MODEL_9234744775fd432980f3d4f9715021c5",
       "value": "100%"
      }
     },
     "aa8a992eda6541439bc6fbeaf2d2c180": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_2e6ef16ce3b74f42a7970f6a72934314",
        "IPY_MODEL_2acca5830df74bdc92bbfbc9e87956e8",
        "IPY_MODEL_9067c16f1c7c4965845cfb68bbc6408a"
       ],
       "layout": "IPY_MODEL_acb0d75d126244f2b927c667cbab1a6b"
      }
     },
     "ab75582bbfa140efb91f67bc29914a16": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "acb0d75d126244f2b927c667cbab1a6b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b05f5a673e234dd4a777f05a5d670189": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1d3b0ef4688d48d89a3ed110d4b50ef2",
       "max": 29.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_9ba61440136f4ff09b26cca0082ecbfa",
       "value": 29.0
      }
     },
     "b1076dea4226432193fb9e0228f9478e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b1bf6a01e86343f5a6b6e9a33bb45c31": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_699094ff6e2e49d1ac875a3cfe7ed6a2",
       "placeholder": "​",
       "style": "IPY_MODEL_711c77f5b5104ce3a6b024cfc90c161b",
       "value": "100%"
      }
     },
     "b37025b2cd1547c180e1ded404350f94": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b4c8a65e2dd145c6b5d621b6f0efbaf7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b9e7c8cfb6ad43a2a51cf94da62b7e1e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c27952ff52974b178a45d46180f2a471": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c2ab7830d2634456bf30ac5f7a80885d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6b186e631cc94570a21364fc15ece208",
       "max": 268943.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_064f35f27b674206ab041f42a280489f",
       "value": 268943.0
      }
     },
     "c3c2e0ba50a8420db89b11567e332348": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c512cf61f5164037b88dc9c7bfa5f8b5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3861df3f53c040349d065f432a86dbb8",
       "placeholder": "​",
       "style": "IPY_MODEL_ab75582bbfa140efb91f67bc29914a16",
       "value": " 29.0/29.0 [00:00&lt;00:00, 790B/s]"
      }
     },
     "c7b1f87c32124bc59bc1c8e45257c912": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_4013c5ce817a46e3ad01fced58344058",
       "placeholder": "​",
       "style": "IPY_MODEL_96ed1a9beef64e56a301e1f325670513",
       "value": " 3493/3493 [02:40&lt;00:00, 24.28it/s]"
      }
     },
     "cbcd29bfa14e45bcbbbd0d5812bf805c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d259d791a91746b1a38cfb35998b1e6e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "d29df587ed1149e3999572ba5b6b981f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d3d424b2c8eb4a6e9e6f052f77db02ca": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "db1a69e1b3ba4782bc3e89583eb022d3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_0a202fbbbc1448ccbd7dd3e931e1c0e8",
       "placeholder": "​",
       "style": "IPY_MODEL_0d027db93daa416b8ce0abb7065a5cf6",
       "value": "Downloading: 100%"
      }
     },
     "dcbd22effe0d4fa58bac1e5915ab03ec": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "e10a70de62bc4dc18a5dd0a36749d96c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2b3a318dff5148c3b9c02aa62c49e5bc",
       "placeholder": "​",
       "style": "IPY_MODEL_0aec29a0c39c488fae37ee35a65d5aa6",
       "value": " 1684/1684 [08:58&lt;00:00,  3.68it/s]"
      }
     },
     "e2c6ed5043b547caa83315cb0b9143bc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ee1b7c6b2931478ebf0741078f5c3d94": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_b1bf6a01e86343f5a6b6e9a33bb45c31",
        "IPY_MODEL_356a608b0a9641249f510028ecf734bd",
        "IPY_MODEL_c7b1f87c32124bc59bc1c8e45257c912"
       ],
       "layout": "IPY_MODEL_b9e7c8cfb6ad43a2a51cf94da62b7e1e"
      }
     },
     "f5216ec582a44ffdbe42879ba6276a8c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e2c6ed5043b547caa83315cb0b9143bc",
       "placeholder": "​",
       "style": "IPY_MODEL_1dd7df6df87c4c3aa63750a45d328cd2",
       "value": " 624/624 [00:00&lt;00:00, 24.1kB/s]"
      }
     },
     "f719b921029e48b591be17c0551bc28b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3cd5628685654a9f8e36c5a8781c7bad",
       "placeholder": "​",
       "style": "IPY_MODEL_842ee690456d4c3e99e0b1b68882c41b",
       "value": "Downloading: 100%"
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}