2_Amazon_Customer_Reviews_Sentiment_Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis on Amazon Customer Reviews\n",
    "\n",
    "For this project, we’ll be using the Amazon customer reviews dataset which can be found on [Kaggle](https://www.kaggle.com/bittlingmayer/amazonreviews). The dataset contains a total of 4 million reviews with each review labelled to be of either positive or negative sentiment. I will only be using 100k reviews in this implementation to speed things up, however, feel free to run it yourself with the entire dataset if you have the time and computing capacity. You can run the codes on GPU to speed up the training process significantly. You can also use [My Notebook on Google Colab](https://colab.research.google.com/drive/1pFduf8iVOuFGm9-nkPxQEm3GniPhEBzw?usp=sharing) if your hardware is not powerful enough.\n",
    "\n",
    "Our goal at the of this implementation will be to create an LSTM model that can accurately classify and distinguish the sentiment of a review. To do so, we'll have to start with some data-preprocessing, defining and training the model, followed by assessing the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our data pre-processing steps, we'll be using *regex*, *numpy* and the *NLTK (Natural Language Toolkit)* library for some simple NLP helper functions. As the data is compressed in the *bz2* format, we'll use the Python *bz2* module to read the data. The process flow of our implementation looks like this."
   ]
  },
  {
   "attachments": {
    "Slide23.jpg": {
     "image/jpeg": "/9j/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCAFyA4EDASIAAhEBAxEB/8QAHAABAAIDAQEBAAAAAAAAAAAAAAYHAQUIBAID/8QAWBAAAQMDAQMFCQ0EBggEBgMAAAECAwQFEQYSITEHQVFhcRMUFyJVgZTR0hYyNjdUdJGTobGys/AVg5LBI0JSU+HxMzQ1YnJzgsJEZHWiJCYnQ2WjY4Sk/8QAGgEBAAIDAQAAAAAAAAAAAAAAAAEDAgQFBv/EADQRAAIBAgQDCAAGAwEAAwAAAAABAgMRBBIUITEzUQUTFTJBUmFxIjRCgaHBI5Hh0WKx8P/aAAwDAQACEQMRAD8A6pGQfLlwmVAM5BD77yhWCz1CwSVTqidq4cynbtbK8FyvDKdGcmm8MFhTOKO6fVs9svjhasldRZU60E7NlkgrbwwWH5Hc/q4/bHhgsPyO6fVx+2To6/sZGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskFbeGCw/I7p9XH7Y8MFh+R3T6uP2xpK3tY1FPqWSCtvDBYfkd0+rj9seGCw/I7p9XH7Y0lb2sain1LJBW3hgsPyO6fVx+2PDBYfkd0+rj9saSt7WNRT6lkgrbwwWH5HdPq4/bHhgsPyO6fVx+2NJW9rGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskFbeGCw/I7p9XH7Y8MFh+R3T6uP2xpK3tY1FPqWSCtvDBYfkd0+rj9seGCw/I7p9XH7Y0lb2sain1LJBW3hgsPyO6fVx+2PDDYfkd0+rj9saOt7GNRT6lkmSvKTlY09PKjZEradF/rSxIqJ/CqqTa23CkuVI2poaiOogdwfG7KdnUvUV1KNSn51YyhVjPgz2gICssAAAAAAAAAAAAABjO/qAANBqLVtn0+mzcqxrJlTLYWJtPX/pT713EWdyv2BrsJS3JydLY2Y+1+S6GHqzV4xbRVKtCOzZZAK28MFh+R3T6uP2x4YLD8jun1cftmWkre1kain1LJBW3hgsPyO6fVx+2PDBYfkd0+rj9saSt7WNRT6lkgrbwwWH5HdPq4/bHhgsPyO6fVx+2NJW9rGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskFbeGCw/I7p9XH7Y8MFh+R3T6uP2xpK3tY1FPqWSCtvDBYfkd0+rj9seGCw/I7p9XH7Y0lb2sain1LJBW3hgsPyO6fVx+2PDBYfkd0+rj9saSt7WNRT6lkgrbwwWH5HdPq4/bHhgsPyO6fVx+2NJW9rGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskFbeGCw/I7p9XH7Y8MFh+R3T6uP2xpK3tY1FPqWSCuGcrtge7C09xYnS6Nv8nKS6wajtV/idJaqxkyt98z3r29rV3p28DCdCrTV5RsZRqwlsmbrIMIZKiwAAAAAAAAAAAAADgAYBr7zeKCzUvfFzqo6eLgivXeq9CJxVepCFT8rWnonq1kVfM1F98yJuF6/Gci85ZCjUqK8I3K5VYR2bLFBW3hhsPyO6fVx+2PDBYfkd0+rj9ss0df2Mx1FPqWSCtvDBYfkd0+rj9seGCw/I7p9XH7Y0lb2sain1LJBW3hgsPyO6fVx+2PDBYfkd0+rj9saSt7WNRT6lkgrbwwWH5HdPq4/bHhgsPyO6fVx+2NJW9rGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskFbeGCw/I7p9XH7Y8MFh+R3T6uP2xpK3tY1FPqWSCtvDBYfkd0+rj9seGCw/I7p9XH7Y0lb2sain1LJBW3hgsPyO6fVx+2PDBYfkd0+rj9saSt7WNRT6lkgrbwwWH5HdPq4/bHhgsPyO6fVx+2NJW9rGop9SyQVt4YLD8jun1cftjwwWH5HdPq4/bGkre1jUU+pZIK28MFh+R3T6uP2x4YLD8jun1cftjSVvaxqKfUskbitvDBYV/8Hc/q2e2beycoenrtO2nZVOp53LhrahuznsX3vRuzvIlhqsVdxZKr027XJmD5auejHMfWSgtAAAMdBWvLPqWa1WyG20Mix1NYiq97V8Zsac3TvVcdiKWUpQ/Lu5V1bSN5komqnne//A3MBTVSulI1sVJxpuxXC86/r9dZgA9Zw4HGAAF2NgAANgAANgAANgAANgAANgAANgAANgAANgAANgAANgABuB95K+TjUc2n9Q06LIqUNQ5I52Zw1EVcI/HSi7/pIoEVUVFRVRU4L0FVenGpBxkZ05OMk0dgNXKZMoYbwTsMoeMO8uAAAJAAAAAAAAAMEf1ze00/purr2o10zURkTV53uXCduOPmUkBW/Luqt0lSIi7nVrEX+B/qQuw8FOrGL4XK60ssGyjKupmrKmSoqpHSzSrtPe9d6qfkOgHsYxUVZI4Lbe7AA6eom5AAAJ2AAA2AAA2AAA2AAA2AAG4sAABsAABsAAAAALsbAAAbA9dpuNVarhFWUEqxVESorVTgu/ei9S9HQeQGMoqSaaJi8u6OrtPXKO8WWjuEO5k8bX7Oc7Krxb2ouUNkQ7kiXPJ7alVePdfzXkxPGVYqM5RXozvU3eKZkAGBmAAAAAAAAAD8amVkMMksrkbGxqucq8EROJ+3MabWLlZpO9OavjNoplTt2HExV2kYydlc5x1dqCp1Hepqyoc7uWVSGP8Au2cydvX09WDS+b9eoA9nShGEFGJwZycndgAFn0YWAABOwAAGwAAGwAAGwAAGwAAGwAAGwAAGwAAGwAAGwAAGwH6/X6yABw4F5ci2pJrlb57XXSLJPSbKxOcuVdGu7HSuF+9Cz0KF5Cc+6+rT/wAk9f8A3sL5TnPJ9oQUK7UTtYWTlT3MgA0zYMKULy7/AAvpfmLPxyF9KULy7/C+l+Ys/HIb/Zn5hGrjOWVyAD1JxwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADGflZMeKOwG+9TsMoYb71Owyh4k9AuAAAJAAAAGQAAAAYK25evgnR/PmfgkLJK25evgnR/PmfgkNjCc6H2U4jlsogAcynsDhonPJ9brVNY9RXK60CVy0EbHxxrIrP7SqmU7E45NhYqXS2s3yW+mtclmufclkhkjnWViqnNhd3buyqc45L6ZlZpTVtPLUR00ckUbVmkXDWIqP3r1GLG7T+iZprkt3ju1zSN0cENMxdhFXiquyu7z7kzuVcHFrNupOzeZPbj8G7BfhjfgQF1BUtnqYWwSPdTbSS7DNrY2VXOVTdjdx3Ce31tPSxVNRR1EVPLhI5XxKjH5TO5efdknvJRVI6t1JVVsSVDVo3yyxu3o/eqqnYvXki971NeNRuSmrqjbgfMj2Qo1rWMXeiYxvxvVN6ru3m7GvVc3BLZcSl045b9TVUVura/a7xo6mp2ePcYlfjtwfhNFJDK6OZjo5G7nNe3Cp286Fy6ptF7ou9LZpmupbfb6aJucVCQySyc7nc/R28+ebQco8DpNKWepuk9JNfI5FgmfA9rlkYu0qKuP8Ahb2KqlVPHOcltszKVBJMgDLdXPqGQMo6l08jdtkaRLtOb0onHHXwPzqqWoo5liq4JYJU37EjFav0LvLQ5QNRVlmorNTWrZppqihidLVMx3RzU3IzK70Teq9O/m358VyrJdT8lctxuapJX22rSNs+ERz2qrU346npnHQhMcXUaU5L8LdiHRjuk9yARW6tlWBIqOpetRlYUbG5e6Y47OOPXxwflUU09NUup6iCSGdqoixyNVrkVcY3cd+ftLQud+qLFybaafb2sjrZ2SRtqVRHOjblFcjc8M4b9HTg8OiKqSaHUeq7miVdxooGJC6RqYR+zhFXCJvTDU7FUlYuplc3Hbh+9x3MbpXINPaLlT0/d6i31kUGM90khcjfpweEnOldcXt+paRtfWvq6WqlbDNDIibGHKjconMu/ONycym9sunqGg5UL1mBHUlthWsjiwiojla1yJw4JtOx2Jv6ZeLlSbVWO9roKip+UraW03KKl75lt9Wym491dC5GfSqYPJFG+aRscTHPe5cNa1FVV5+HElScoGou/wCWofW90jkyjqd7EdFjfu2f8ckj5PLbURaLuV1tfe0d3mm73hnnc1EiYmMqirneqqvbhCZ4mpSjmqJb8CI0ozlaJXVZbq6hRi1tHU06P96ssTmI76eJ+dJSVFbN3Kjp5qiVUzsRMVy46cJvLf05Q3Z9PcaLVtzo6u11EDt8lU2RzJN2FRV4JjPUipk0XJ7ItVpO52uz10NBf5Z0ex7nbDpY0RPFRyb/AO1w4ZK9bLK9t1/oy7hXRX9bRVVDL3Otpp6eTGdmZisXHTv34P2pbRcqun7vSW+sngTP9JHC5zd3WiG01c7UMc1NR6nWoc+DaWJZcKqtdjOHp77gnZw3E/1H+3bmylumhbg6S1xQtalJSvRrolTmVnBebdx3YwplPFTio8N/X0IjSTbK50jLDT3lFq7S+6s2HJ3s1u0uenHPwXsya1sMtbWyMo6V7nuc5yQxtV6tTPQm9cJu4E15KppqjlG7tV574kSZ8mUx4y5Vcpzb+bHUY5J/jEbjhszfcpMq+SU5W4JMKGayIbT2yvqad09PRVUsDeMkcTlamOO9Nx5Ov9dBPqHXF3XVtHHBK2ntzahsDaONqJG2PaRMcOOOfp4dCe6a20EHLalNNGxKV8ySpGqble6LaRMf8fqGrnG6qL0uiO5T8rIIyxXeSFssdrr3RO969Kd+Fz14wu48dVSz0k3cquGWCXCLsSNVq7+fC78FtafqtXryivZW9+pQd1kSRsjXJAkaZwqc3RheK/SVxq64PuWoq+Z1S+piSV7IXucjv6NHLsoi9GFJoYidSpllbh6CpSUI3NOADeKDo/kh+L21/vfzXkxIdyQ/F7a/3v5ryYnjMRzZfbO9S8iMgAqLAAAAAAAAADBpta/BC9/MZ/y1NyabWnwQvfzKf8tTKHmRjPys5YAB7WPBHnwACQAAAAAAAB1gbgDnwAAAAAAAAAAAAN/QAAAAAAAAAAAAAWPyEfDCq+Yv/HGXynAobkI+GFV8xf8AjjL5TgeW7T/MM7GD5YABoG0FKF5d/hfS/MWfjkL6UoXl3+F9L8xZ+OQ3+zPzCNXGcsrkAHqTjgAdXnAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMZ+Vkx4o7Ab71OwyhhvvU7DKHiT0C4AZC8DHnBJk/CaojhkiZI5GukXDc86n7EY1u5W01KrVVFR6qiouFRcGUI5nYhuyuSdDJodN3hK+JIZ1Tvlib/APfTpN8hEouLswncAAgkwVty9fBOj+fM/BIWSVty9fBOj+fM/BIbGE50fspxHLZRBhTIPYHDRNtF3GjpNH6sp6mpjinqIWNiY5yI5+525E5+KEJ7QCqnSUJSl1MpTckl0JlydXGkoItQJWVEUKzW97I9t2Npy53J0r1cSGoqoqKm5c7u39YAIjSUZSn7g6l0l0LIvn7H1y2muSXiktl0bC2Kpgq/FY5UyuUd2qvTuwRLUFqoLXFTtpLxBcalyr3RsDF2I0REwqO595pOfAKoYZ09oy26Fkqubii4NaWuz3aCxwVl0ZbLi2gjdHJMirE9mOCrlMKip1cefmjeprharRo+PTVlrG18ks3d6upYi7GUxuTfv3o1NyrhE4ml1jf47++2rFC+LvWkbTu21Rcq1eO7mI6UYfCSyrO+HoZ1Kqu8qJnqq40lVobS9LBURyVMDZe7RtciqzKpxTmPz0Be6ChS52u8q5luucXcnytTKxuTOF7N678LvxuIgDZ0sXTdN9blaqvNmLCtNk09YbnDc6/UlJWwUzkmigp02pHubvaipndv3/evE8+ntZRRa7rbrc41Sir0dDK1E2lYxcI3twiJnz9hBQYaNSv3ju2T3zXlJ3LpbTFNO+pn1TBLb87TYoG7U7k5k4rheG/H0H46PulsqdOXDTd8qFpIaiRJ4KnZyjJExuXo4J9u9OJCgHhc0cspNhVbO6RLqrS1qooJpqnVNulRrFWNlKiyPeuNyYRfF39p5bFYbbdbcki36joa5rlR8NX4jcJwVHfTu/SxsGSoTUWs/wD9Ed4r3sT7W91oJbNY7I24rcpaJ6rPXNTOEVfetVd7typ0phE5z0WC0WuyXmmu0Gr6FKKFyPVrFVJntRcqxWZVd/Dp6iuQYPB/48ilsSq34s1iw9LX+3ycqNRdpZI6SimWVWulVGomUxleZFXGe1TX8m1xpKDXLaqtqI4KfZl/pHuw3ei448CGAyeEjZq/FWI753RsbfNHHqKmnkciRNqmvV+d2Eei5+hCyKWsir+Wt1ZbWxVtOjEVZWP8Vje5I1X56lXH63VMbrSN/n03d21tPGyZHMWKSN6qiPYqpuzzcE/xTcRiMPmi3HjaxNOpZ7k2q6qy3BZbe3W94ZE56xtbM1zmKi9LkxlvauMfSQXVNkn09ep7fUvbI5iorXtTCOau9F+3hzL1YU3kd80pBOlVBpmdahF2mxSVblia5N6budOrGOoj2oLvU3y7z3Ct2e6yr71vBqJuRE/x3lOEpzpz2Vlb1sZ1pqS+TXAA6XWxrHR/JD8Xtr/e/mvJiQ7kh+L21/vfzXkxPGYjmy+2d6l5EZAUZ6yosCn5Tytghklk3MY1XL1Im8/RFTpPHeP9lVmP7l/4VCVyD1RPbIxHxuRzXJlFTnPshGl7z3rIlLUu/oXL4jlX3qrzfrgTZMLwMpwcHZiLujIAMSTBptafBC9/Mp/y1NyabWnwQvfzKf8ALUyh5kYz8rOWAAe1jwRwGB5usAXtdkWvxJzFoi3Q2e11131HFQLcIu6xMdSudzIvHa5kVOg8110NUQ2yS5WavprvQx5WR9PufHjiqtz9PFTYco3wN0P80f8AhiPDyS181JrSkhY5e41SPhlZxRybKqi9uUTf0dpzITrd260ZcG9jaahmyWIb5gTy16WpKzWN/jq9ptotbpZZGRe+VrVXZjTn4Iv0Y58nrss1i1BdY7TUaYS3wVG0yKphe7uka4yiquN+/HSiZ35LpY6K3S9DBUGRHR9l90GoqS2ukdFHKq7b2plUaiKq/djzm1vUGlpbnDQWzv2kcyrSGWqncj2LHwV6Jndv4bkTHHoJJyWdxtOs6qzVFDFJWxyStSs2lRWo1MYROhcZ85oKyttt71haqWGy09FH3+2OdGPVUmR0jU37k5kX6SideU6r4pJXM1BKG/Ejd9paWhu1RTW+rbWUsapsTtTCOyiL2dKdG7J4CY3a2UcXKky2xQNbQrXwxLD/AFdlytyn2m51FUaW0xqCpo47J+0pVk2pllkVrIUXC7DG4Xgi8evj0X6vKlFJt2uYd1e7ZWzURzkRVwirxxnBtdT2+gttxbDarg24U7o0csrURMO37t3VhfPg3HKRY6Kz3WlltLXMoa6nbURscudnPN04xg++Va10do1T3tbqdsEPe7HbLOGVVUz9iExxCnKElezTIdNpNEOBamtotL6UvioyztrqqVjXpTuerIoW47N6quc5z/JdBr6021lrtF8stOtNTV7HI+DayjHJ0Z8/Qm7OBTxqm1dNJkyoON9yFAsW8Ns2jI7fRS2SC6Vc9M2onmnkXG9VTDU344Lw5scT8uU5lppqS1NtVpp6VtZAyrSVqqj8Ki+Irejguc8whjVOaio8SHRsrtlfmwWzVqWNt37k3vBZe4o/Ke+TPN0Gv6y30vtsZyZRVbrBTOpUre596rKuztY9/nGfsMsXWnSy5Ve7IpQUr3ZCeTyyUd+uddT16SLHDRPnbsuVvjNc3HDj75SLdP0lk8lE1PWayvE0cDaOmloJV7mxcpGivZwU+NP1mkq+7QWNlhV1LM7uTK6SVe6udzOVMbsrzcOrmTX1UqdWd02tv2LO6UoKzK5BLrXpHv3X01hdK9KeCV/dJd213Nq7l7Vy36eBtae46emukduTSuLa+RIkqVe/u29cbarjd04yXzxsYuyV9rmCoNorwG61hZv2BqKstzXK+OJyLG5eLmqmUVebO/GURDSmzTmpxUo+pVKOV2YABmQWPyEfDCq+Yv8Axxl8pwKG5CPhhVfMX/jjL5TgeW7T/MM7GD5YABoG0FKF5d/hfS/MWfjkL6UoXl3+F9L8xZ+OQ3+zPzCNXGcsrkAHqTjmxstluN7nfDa6V9TIxu05GqiYTPSqonrPfWaN1DRxufUWmp2G71VjUkx5mqu7pXmJDyTvdHTanfG5zXtoHKjm7lRUR30KRu2atvtuq2Tw3Ore5q+8lkWRjupUVeHHrTmNF1a0qkowt+EvUYKKcvU0XWgJ9yo0VLPBaNQ0ESQtucW1NGiYRHoiLnrVc4XsyeRNNadp5GU9fquFKx2EXvendLGxV3b3ouFT6Mc+DOOMg4KT4mMqLTsiGb+gEivWmUsepWWy7VzIaZ7UelW2JXpsKi4XZTfxTHr4k31/bLJNb9Pd93/vXudC1sC95Pf3duEw7cvi5wm5SJ4yMZRSV0yVRbTb9Cpgb+wWa21lDLWXe9Q2+Fknc2xtYskr1wiqqNTfs7+O/fu3HtvWlKaHT7r1ZLq2429j0jlzEsb41yib0XjvVOjjnrLHiqallfExVKTVyJgLz47fOSqw6Ujq7O68Xm5Mtlr2+5xyOjV7pXcF2UTjz8M8F3c5nVrRpK8jGMHJ2RFedMG41RYKjTtfFS1csUr5IWzI6NVxhVVOdOlFP31JZaCgpaestN4iuVLK5W4RixyRuTpaq5+7+a7/AJZ/hPRcMd4R9X9Z32FGozThl4O5Z3dosgPT1AmbtI0FspqZ2p7223VVRGj2UzIHSva3CYV+OC5ymOpeOFNTqvTktglpnpUR1dFVM7pT1Me5JERE4pzcU3ZXd9BZDFU5yypmLpSSuaIEzTTOnad7Kev1VC2sd4q9707pY2KvMr0XCp9HXg0urLBPpu8OoaiRkuWpJHKzcj2rnfjO7ei53/TzoYqnOWVCVKSVzTdgTmVPuLAumgaSzStlvN9jpLe9jVjf3JXySOVN6NYiqu7dv6/OaPV+l1sLaSpp6tldbqxuYahjdnOMLhU383Df07tymMMZSm8qZLoyW7NXcrNX2ykoqmtp1igrGd0gcrkXbbhF5lym5yLvNf8A5Em1VaqmgsenamouM9XHWU6vjieq4gREZ4rcruTxkTm4GzTQkUdnt12rrvDSW6op0lllkj2la52FRjWouXfZhE5yFioxgnN8WHSbdkQbmyvaOpSVal0pHbbPT3i03Flytkz9juiMVjmOxzpvXm4rhc83Tm1aWpP2HDdr/dEttJO5W07GxLJJLjiuE4J5usy1dPLmRDpSTsRQEz1FpG2Wqww3Wnv/AH1HUpmmYlG5O6YXe3aRcNVE6cLuVCGfy3fr6SyjWjWTcfQxnBwdpAAGc/KzFcUdgN96nYZTgYb71Owyh4k9CuBFNcSyR95dze5mdvOyqpn3pF++p/7+X+JfWSbXnGh/6/8AtImb9BJw3KZt3P276n/v5f4l9Z8vmlkwkkj3onM5VXf5z8wXWXQwuz7hlfBKyWJytkYuUVP114LBsd0juVNtbmzN3SN6F6U6iuz0UFXLQ1LZ4HYcnFOZU6F/W7iVVaSmroyjJplog8VsrorhStmiXGdzmrxavQp7TntW2ZetzBW3L18E6P58z8EhZJW3L18E6P58z8EhsYTnR+yrEctlEAA9gcIAAxzR6mVmP8/8SQaf0heb/SSVVspmyQMf3NXuka3xsZ5+1CPlgUGfAtcV/wDyLcf/AKzXxVSdOKyerLKUFJ7kevmkL3ZIO7XCheyn3IsrHo9qZ6ccOKcTQdfVksvktlnfYtRMuKvfY20q57ovio7C7m56uOOoi9kodOd4Nqr/AHOqbI9yo2lo4/Haibsq5yKnYnR15K6eKks0aiu10MpUk7OJHOfAJfqPTVuh09BfNP1k9Rb3S9xkjqGokkbt/FUwn2c6b1Qab0xRT2KW+airZKO2I/uUaRNzJM7q47ubgvBVXGMlurp5M/7GPcyvY1mndPuvVDd6llQ2JLdTLUKit2trCOXHFMe94mj8xbmmLXa6bTGqq2x3F1XSS2+Rjo5k2ZYnIx6+NjCYwvHGN24hej9MxXemrbldKtaK00SIssyJlzndDdy9XSucIiLkop4tOU5S4KxnKhsrHj0dp9+pbz3hHUNgXubpNtW7W5MbuKdP2H42GwVt7ZWLQ9yTvSLusm27Z3b+G7juXju6SzeTK1WL3QrX6euc0zY4Xxy01S1GyplUw5FRERUyn2p0kC0XZI7zFeVlqJoe9aN86dzXG2qZ3O6uox1blKTTslbiie6sokZ+7gP8iUaG0xHqeW5QOmdFLBTLJFvRGq7gm0uF8U3Nt0ppi51brTQ32pku+y5GSdzRIHuRMqjd2V4LjfzZ3mxPGU4NxfoVqjJq5X2/oBv9OaYnu+oprVJI2n7221qZFTKRoxcO3dqp96ko09prR1/rJqS33C8LURMdJ47WJ3RE52+LnnTcuPvFTF04fIjRlIrgHrujKOO4TNtkk0tGi/0b5Uw9Uxz+fJi108NXcIYampZSQPXD53plGJx+ndhDYzrLmK7b2PKnQnEE9ttj0Zda2O20d1uqVkqqyOaSJvcnP343YRePZ0cTQUGn3O1nFYq16tXvnuEj48c2d6Z4Jwxnm+goji4Sv6WLHSkjQ/5AsCt0tpiy3N9DfL1UJUukVGNp2ovcWZ8VZHKi78YVU4p2byPar01UWLUX7LY7vhZNl1O5qJmRHLhExzLnKdH2YU8XTqOyEqMlxNB0Y/zN5fdPutNms1e6obK25RukRiM2dhE2d2d+ffISSu0ppmyPioL7eqmO6OYjn9wj2ooc78O3ZVPPnG9cbj0cqtGtv07pCk7tHOkUMrEljXLX4SPxkKdWp1IRp8GZ9zaLbK3ABvmudH8kPxe2v97+a8mJDuSH4vbX+9/NeTE8ZiObL7Z3qXkR5Luqpa6tWrhUhfvTim5SuO+qj++l/iUse8/7Jrf+S/8ACpWXT2l2GSd7kVD9u+p/7+X+JfWYWoncio6aRUxvTb4p0H5A2bIquOnBMNK3nurW0VU7+kTdG9V98nR2/wAiHmWuc1yK1VRUXKKi4xjnMalNTViYysy10Mmi05eG3CHuUzkSpYm//eTpN4hzpRcXZmwnfcGm1p8EL38yn/LU3JptafBC9/Mp/wAtSYeZET8rOWAAe1jwRwGAB1kvoQWlq2x3K86N0b+y6OWp7jSO29jHi5bHjjz7lPx0dp6TSVS7UOqFZSNp2PSnp3PRZJXqipuTsVURM5yuVwQWnv13pYGQU11uEMLEw2OOoe1qJ1Ii4Q8dVVVFXJ3Srnlnk5nSOVy/Spzo4WrldNytF/7Nl1YOWa25YnJlepKq8ajibO2muV0jfLTvdjCS5cqInThX58ynwtfykpVrT7Fx7oiqm13sxWfxbOzjryVy1ytcitVUVOCoqpwNq/Ul7kg7i+73B0S8WrUOwvbvyvZwE8G1PNC1n1IVVNWZLOTGSqk5S3PuDldWYmSVXYXxsb+G7j0Edsfw/t//AKnH+chpqSrqaOoSekqJoJ0TCSRPVjt6f2kXJ+cc0sc7Jo5Htma7ba9q4cjundvz9pbLDvNJ9VYx71WROr4v/wBZo1//ACVP+JhqOUvfrq74/vU/C00D6yqkrEq5KmZ1Wjkf3Zz1V+0nBc8c7u3cfFTPNUzvmqZXzTP3ufIu05y4xvXiKWGlCSk3wVhKqmnYnnKqqLRaWXO79nN+5p8ctSp7tM/+Wj+9SE1VZVVaRNqqiedsLUZGkj1dspwwmeCfQK2sqq6bu1bUzVMuMbcr1cuE7d/OvYY0sLKGW74J/wAkyqqVybctfw0//rR/9x+mqsLyWaUwmV7pJ97iC1tZU10yy1tTNUy4xtyvVy4TfjK78cd3WfrFXzKlLFVyzT0UEiOSne9VYiZ3oic2d400lCC9v/g71NyfUui722qqoLetFYbffaCGljSmqJ50bKu7+ttblRd27h0lZa/S/PvDJ9RUa0r3NRkLGonc2tT+q1UynPv3539hIdW01q1bdP2pSaloaeJ0bWd7VeY3Q4Tg3qzle1V4mu1neaFdN2uwUNa65OpJFklrHIqIq7/FbninjceG5OxNLCRlCcdr/wC9i6q04shHQWNbKGpu/JFJBboXz1EFesjo2Jl6pjmTn4lc9H2J+vOey3XOutcjnW+sqKVz/fdyerdroyiblOliKTqpZHuma1KSi9yb8lFNNQ6nvNPVxOhmjt0rXMdxTxmL/MjOhfhlZfnUf3mtjuVdFUzVMVbUsqJmqkkrZXI96Lxyqb14Jznnhllp5mSwSPiljcjmvYuFaueKL057DDTybm2/MjLvEsq6Fq2i4wW7lpua1EiRsqFfA16rhEcqNVOPTs47Tw3Gr5RqOvkpkdXzbLtlJYqZr2OTgiouzuTtXKc/PiuaieWpnfNUSPlmeu0571VXL2rzqbGLUd7ip0gju1e2FEwjEqHYTHNxKXg5Jpxs9rbmffJ8TGp5LpJeqj9vOc64Nwkm0rcomEwni7uH2mrPqR75HufI5z3OXLnOXKqvTlec+TfpxyxUehryd3cAAzILH5CPhhVfMX/jjL5TgUNyEfDCq+Yv/HGXynA8t2n+YZ2MHywADQNoKULy7/C+l+Ys/HIX0pQvLv8AC+l+Ys/HIb/Zn5hGrjOWVyAD1JxywOSr/U9U/wDpzvucQKCKWeZkUEbpJXqiNYxuVVVxuRE3qbvSepajTc9U+np6eoZUR9zkjnaqtVuf8VTsNx4Q6ynYqWq02e3PVMd1p6bxm9ac3mVFOe41YVZOEbpmwnCUUm+Bs+UKB1Na9KaZY5q1sUad1YioqNe/CJv7Vd5jx3qg0vpauS211HXXSujRqzO7t3KPLkRcNxv/AFxIVVVtVVVzqypnllqnO2lle5VdlOvq3Y6E3biXzcoVRUJDUVVotU90iRGtrZIcu3cFx/a5+jPNzFcsPVhGMVvxv9syVSDbbPVyzbK3q0qxisb+zmYau9UTadu7UPjlP32vSTv6q2xm/wD6Wka1NqGr1HU0s9e1iTQwNg2mf18Kqq5eZFyqm2pNbzMs1NQV9rt1w71asdNNUR5dGmOvzJuxuRBDD1IRpu18tyJVFJyv6n3YrDbKfS66h1AtVLA+buEFPTKiK9yZ3qvRud0cPMSCnqLZU8leo3Wi3S0UTZItpJJlk2nbTN6Z6sES09q6otNsltlRR0twtsjtvvepZ712OZeb6N2Mn7XDW9bWWWstLaOip7fOjGshgj2GxbLtrKdq8c/R0xUoVpzu169fQyjUgl+xFCwtcNV3J3o99Oi96ox7X4/vFRPU8r1crnn/AF9xJtOawq7PbpbdNS0lwtr3bfe9SzaRFznd59+FzvQ28TTm3GcFdoppSW6ZGla7YV2yuxnG1zcPozuLH5UnRs11ZnT/AOhSmp1k/wCFJHZ+zJF9SannvUEFKyko6GhgcrmQUseymV51/wAMH4aov9RqKvhq6qOOKSKFsCJHngiqueni5ecwdOdScZNW4kqSjFonPKZX2ek1dUNuenpKudY2ObP38+LbbspwajVxzpx4opGr5f4dQW602O2WvvKOnl2YtqoWVV21xjKt3b1Tp5kPuj11P3hT0l4tdvuzKduzE+qjzI1E5s9nVlec1N+1BJdX0vcqOjoIqVVWKOkj2MKuFyvXuQoo4ecGlKO69b/0WVKqd3ckd7t+l9LV/wCza6jrrrXxo1Zn917jHlURcNxv5/8AHjj9OWbZ/bVpWNisatvjw1y5VE2n7l6TyTcoNRUJFPVWi1T3SJEa2tkhy7dwXH9pOPRnm5jRan1DV6jqaWor2xpNBA2BXMTG3hV8ZebO8inQrZ4ymuF/UmVSGVpEm5Z3L7paFNpVRtBHuz/vP8x86g+KLTCrvXviZM9SOk3Eb1Rf6jUdfDVVUUcT44WwokecKiKq8/PvMVl/qKrTVvsr4o0p6OR0jHpnaVXK5VReb+sWRw81CnG3B7/yYOpHNIk3KLj3JaK3b+83fdGZ5RVX3IaJRM4WkcuP+mMjF7v9Rd7baaKeKJkduiWKNzM5cmGpv6/F+0ze9QVF2ttqopoomR26JYo3MzlyKjU35/4UJp4eayXXBsl1VvYklv38jN0zwS5IiZ5kxHw8+TZXGltlToPTEGoq99vnSOR8EzIHSsViqnirjnxs/QQmnv8AUQ6XqLE2KNaeadJ1k37aOwnDfj+qn63ljVtTf7PpPT8OlIJq2jkpkklmZH3wu2u9W4XOyiZXdjqzuNSvTlB24Xk2iynJSV/gh2r71bpbLa7FY3SzUVCrnunlbsrK92VyiLvRMq77utYj0KnmLR1KtRW8nk1ZqqggpLuk7WUbkj7lLImW5ynHht8d25N2Srv8/wCZvYFpwaRRXVpXYABty8rKVxOwG+9TsMoYb71Owyh4k9CuBEtecaH/AK/+0iZLNecaH/r/AO0iZv4fyFE+IABeYgb+bj5voHR0Eq0tZlerK2rbuTfExfvMKk1BXJjG7PfpW1PooHTTK5ssqJ4m/wAVOvrN+gTcZObKWZ3ZsJWRgrbl6+CdH8+Z+CQskrbl6+CdH8+Z+CQvwnOj9lWI5bKIH3AHsPSxwydM5Tb01qIlLbMJu/1dfaMryn3pU/1W1+jr7RBAa2ko8cpZ3s7WufpUSOnnklcjdqRyuXZTCZVVXcidaqWZpS6zWbkmrqunigmkSvwjZ2bTd6MTKpuKvNrHfayPTstlakXecs3d3LsrtbW7dnhjxU5lIxNB1VGK6k0qmRtnsv8ArG8Xum72q6hsdJlFWGFiMYuOnG9U6s4NzR2q0WTRtDfLtQyXOorZXMjh7s6OOPG1xVOK+L9uMbsrBd+7H6/XmJJYtY3Kz21bfGylqqLaVzYaqLbazOeG9Offjh5zCvh2oqNFW339Lk06m95kruc8NTyRVE1Na2W6F9c1WRtc5Ucm7xkVd/Vu3ZQ8N8jdUckNglpkV0VPUSNm2d+Fc5+FXqyv2oaG7axu91tk1BWyRPp5JGybKMxsbOMI3HBu5F6cn5ab1Xc9PxzQ0bo300y5fTzt241XhnGUwvNzecojhakY3XFO5Y6sW7eliTcm9LO3SusKt0T0pn297GyLuRyox64TpU+bSx1TyOXaOlRXSw1zZZWt4q3xN+E5sb/Nk09Rry91LKuOWSHuFTTuplgbHhjGu4q1M++61yazTmorjp2qfNbZUb3REbJG9Npj0ThlOno4L0cQ8NVm3NpXumFVgkkS3kRpZ5NUy1TInLTx072vkxuRXKmEz0r0H4clf+h1N/6ZJ/M8acot9bWwzxLTQxxIuzTRxbMWXJjKoi5X6eJorHfKuytrW0fc8VcK08m23OGrzphU/mJYetUzuSSvb+AqsY5UvQlPJWv9Dqbp/Zsn8zUcmi411Z1//lXh/wALvs3qa2yXyrsza1tH3PFXCsEm21Vw1edMY3/SfhZ7jPaLnT19Js93gcrm7e9N6Kn3KvOXPDyvU+UYd4vw/BZGlWPTlK1XMkazQRtqu6wMbtOmbt+9anTn1c54aWtsuk++a+zUV6fcZInRRd+xIxkOeddyKvD+XZjk3qXXC83+4NkhTUMsTn0bXLstV79pXYRd3HZ49K9putNP13HdoVvcix2pjs1T6vuWxsb87+P8t/E51ROEmpeiW1+JsxaaTRT+cr2m80XZW6g1JSW+WRYopFVXubjOy1M7s8+7r6V6Dzamko5dQXCS2IiUbpnLFhMJhV4p0J/I8ttrqm210NZQyuiqIl2mPb2L5uHm5jsSzTpXjs2jSVlPcsGxVlli1pSW216a/pIqtGd8SzyOkTZdhX7PMqYz0Jg/Cb47ExzV7fuQ1s/KLe5HpJEyhpplc10ksEGy+XC5w5c8Poym40jr/WO1J+21SLv3uvdvers5Tmx0ec0YYWpduS9LcbmxKrFJW6n766+GN66O+pPvJzrWSOPlD0lLUY2EhpnOcvN/SrvX7FKzudbLcrjUVtRs92nkWR+zuRFXo4nr1Bfay/VFPNX9y24IkgbsNVE2UVVyuVXK5UtlhZSyLotzBVUrm35TqaeLXdybIxyule2SPcq7bVamMdWUVO1DbcpVNNR6U0ZT1Mbopo6aVrmO4tXEe410HKLfo6OOFz6aaWJuyypliR0rebjwz5lzx3mlvmobhe6ShguMjZe9NvZkVF23baoq7S9O5MdRFOjWvDOlaP8A4TKcLSt6moAB0TWOj+SH4vbX+9/NeTEh3JD8Xtr/AHv5ryYnjMRzZfbO9S8iPJef9k1v/Jf+FSsuntLNvP8Asmt/5L/wqVl09pdhvUxqAAG3ZlYHm4bwbWwWp9yqfGy2nZ79yfci/rCeYxlLKrsJXZ7dKWuSWdtbIrmxRr4mFwr19RNk858QxsijbHG1GsYmyjU4Ih+hzpzc3c2YqyCml1p8EL38yn/LU3Kmm1p8EL38yn/LUiHmRjPys5YAB7WPBHBYABJAAAAAAAAL2oo6St0np6wVLGNbc7bIsb1ThKxrHIqdfjOd5jUxOJ09tr3LaVLvLlEg/SphkpqmWCZqslicrHNXmVqqmO3JenJpSU9mo6C1yxtWur6WSvqM8Wt2moxF8y8OlFIxWLVCKkt7k0qPeNroUOD0W+iqbjVx0tDC+eokXDY2Jlf8ExnK53Epk5ONQthe5sNPJIxNp0DJ2rIiY6OHMu4uliKcLKb3MFTk+BDjLWq5zWtRVc5URETnzzH6spah9YlK2CV1Sr+59yRvj7WeGOOf5kt8HuoqRsVS+ljfsORz4o5UdI1Mpvx5ubK7lFTEU4L8TJjTlJ8CLXK31dsq3U1fA+CdqIqsem/f/I8pN+WNM65qE3/6KPh/wnlpeT6/TwRvWCCGSRNpkM0zWSu83N2L5yuGJj3cZzaTZMqbzNRItTQyVNRFBAxZJpXIxjG8XOXgh91tJPQVUlNVxOhnjXD2O4ouEX7sG30/RVFv1zaaWuhfDURV8DXsfxTx2/SmF7FTeenlL+HV3/5qfhaZKtesoLha5Dp2jmZGAAbBWAAAAAAAAAWPyEfDCq+Yv/HGXynAobkI+GFV8xf+OMvlOB5btP8AMM7GD5YABoG0FKF5d/hfS/MWfjkL6UoXl3+F9L8xZ+OQ3+zPzCNXGcsrkAHqTjgAAAAAAAAAAAAAAgAAEgAAIAAAAAAAAAdp7aG7XGgYrKGvq6ZirlWwzOYir2Ip4gYyipKzV0Sm1wP3ra2qrpu61tTPUScNuV6vVE6N69Z+ABMYqKslYhtviAARPysmPFHYDfep2GU4GG+9TsMoeJPQLga662qC59y74V6dzzjYXHHHqPB7laD+1P8AxJ6iQAyU5R2TDimaD3K0H9qf+NPUaXUtoprbBC6BZFV7tldpc/QTlTU3u2LcnUrFXETHq56534xwQzhVkpXbMZRViNaasy1siVFQ3/4Zi4ROZ6p/InSIiJhNx8QRMhibHE1GsamEROY/QxnNzd2TGNkEABgZGCtuXr4J0fz5n4JCyStuXr4J0fz5n4JDYwnOj9lOI5bKIAB7A4YAAAAAAG/PAdhKrvYKSk0JZrxEsvfVZI9kiK5NnCK7eicU96hXOqoNJ+pKi5EVA47+oGd11IswB2Ez0FQUlbZtVS1VNFNJT29z4XPai7Dtl65TzohVWrRpRzMyjBydkQwD9c3bzAtzXMWmAB+uCdgzIBF2VRUVcouejf8ASftLVVEzEZNPLI1u5Ec9VRPMqmy1G+yufSfsCOpjakKd37svF/Sn6wacri4zWZqxk7xdkP1+sAAtzLjcx3bsAOoBNPdC1uIAAAAAAABAOj+SH4vbX+9/NeTEh3JD8Xtr/e/mvJieNxHNl9s71LyI/KphbPBJC/OxI1WrjoVDS+5Wg/tT/wAX+BIAVqTXAzaTNB7laDpn/jT1HmuGm6KChqJWLKr443PTLk4omeglB5rjE6agqYo8K+SNzU343qmDJVJX4kZUV1a6GW41SQxJhP6zv7Kfr7SxaGljo6ZkMLcMb08/Weez22O20qRs3vXe93SpsUMqtRzZEY2AAKjMwabWnwQvfzKf8tTcmm1p8EL38yn/AC1MoeZGM/KzlgAHtY8EcBgAEkAAAAAAAszV9dLa7Dye11P/AKWngWREzjOEi3L1cU7Cs+jtJPqvUFNeLBp2hp45mS26B0UqyImFVUZwwvDxV6DTxNLvJw22/wCF1KWVMl1+0vFeeUK11dM3atV1jbWPdjdhqIr06sps7+lx7NC3j9ucq91rGLmBKR8UKf7jXsRFTqXj51I5adfJRaDns6xTOuCNfDBPhFRsb1TOXKuUVN+MdCGq5OtRUumb7LW1sc8kTqd0KNhRFVFVzVzvVOhevJovDVZU5KS4bIv7yKkmvU2/JbmmtmqrlAiJW0lCqwuxvblrlz9LUIlp+vqaTUFFWU8r++Enau1tb35VMovUuVTr6j2aM1HJpy5yTLClRSzsWKeB39dq+Zd6b+PE3tLeNF2utbc7dbbrNWMXbhgne1I43pw35VfpyXzhKE5Xje6ME04qzsTCioaVvLbWva1u0yl74REThIrWtXz4VV85VNDeK9uo47m2pkZWvmR7pE4rld6Y6OremMIey3asrqXV66gkRstQ96rJHlURzVTGynQiJjCrnGE4m7qL9pGnr5LrbbTWyXFzlkZBUq3veN6qvjYRcrzrjh0Y4lcaU6TtKOa6sTmUuDtub+/U8NVy40MVQiOj/o34cmd7Y1VPtRDU6opdO1Wp7hUV2pqqOrSoe1WpROXuStdhGovOiYwaTVWqUuGs2320pLCsfc1YkqIi5anBUReH3703GxuV40dfqh1fdKO70dc9MzNo3MWN7uGU2uf6DHuZxUJNO1rbGTnGV7P1PVe7vbb1yh6ZqbZM6dWy00c0qxqzbc2Xjhd+SP8AKX8Orv8A81PwtPwW42mm1Vbq610lRBQUs0T3Ne7bkfsOyruhFXHDJ59W3OK86jrrhTNkZDO9HNbJ75NyJvxz7jYoUXCrGydrFVSacX9moAB0TXAAAAAAAAALH5CPhhVfMX/jjL5TgUNyEfDCq+Yv/HGXynA8t2n+YZ2MHywADQNoKULy7/C+l+Ys/HIX0pQvLv8AC+l+Ys/HIb/Zn5hGrjOWVyAD1JxwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADGflZMeKOwG+9TsMoYb71Owyh4k9AuAAAJAAAAAAAAAMFbcvXwTo/nzPwSFklbcvXwTo/nzPwSGxhOdH7KcRy2UQAD2BwwAAAAAB+vtLB1J8Uel93/iJPxPK+6f1zk+1FKx3JPpqNr2q9s8qq1FTKb3+tDTxSeam11/oupcJG2rLpTWHQOma1lBT1FzdHIyCWZu02JMornY53bkx59/MvluNX7rOTeuutwggS526oa1J42I1XsVWpv8zl3dSefX6wlY/k+0gxr2q5rZcoiplN6H6adlY3ko1NGr2pI6eNUblEX3zPV9hpqlaHeLjm/svzXeV8LHotE0WkdAUt5p6eCW73GZzI5JWZSFjVcm7+HPaqZ4YNtprULb/pnVT6qigiuMdukR9RC3Z7szYdjKdKLnf182DV2qGPV/J/SWemqqeG7W2ZzmRzv2e6sVXcPMvQvvU6cnv05Z6fT2ntT01VcaaW6TW6VXU8L9pImI1yJlenK8Obr4lVbK4yzee5lC6atwsa3k+sz2aZuF9pbc243NsvcKSF7ctZjZy/HDPjfZg39ooL9qSCut+sLWxInQOdT1axNY+KRMIiIre37N+4jmiKyK46SuOnHV7bfXOmSopZXv2GuXdluebh9ucLjB56vSmpKWmnmud1hpqdjFcrpa5V293BETj5+wmcXKcs7s/T/hCaUVZHt07HHrXSL7I7ubb3bk7pSSLu7rGn9VfpRP4V5lPDr6eltNFSaWtuw5tH49ZMib5Jl4pnjhEX7k5j1WmduidIpco3xrfrqiJAmEXuMOcqqp17vOqdCn5coEFLe7XR6qtysa6o/oq2FF95Im7PT/AIYXnUzhfvt75b7fZEmsnyfHKsxjJrAkbGtzbY13JjpJBrS+x6cisz7bQU37VmoIv/iZWbWwxEXc1OnKrv7OPNH+VWRkk1gVj2uRLbGi7K56egcqsrJZrB3N7XYtsaLsrnpFOHeKnGXDciUsuZo9OtHR33Qtp1FJTxQ3FahaaZ0TURHp429elfFTsyqH73O51GjtM6dXT0cMS1tOk89UsSPc9y4VW5XmTK9aJ2KeKulYvI3b40e3uiXBy7Ocqnv/AFp5iYPqZaKyWFtHe6KyQvo2p3nXQNVcoi5fld6Lv7Ocqm3FKLW13sZpZnf4NDym3aqm0vYGyxwwvr4e6zxJGiK1zdlUVOdEVVKwJjru0XjZjvVfcqa7U8zkibU0z0c1FTOG4REREzncnP2kO4YwdLBKKpbM1q/m3AANwpAAAAAIB0fyQ/F7a/3v5ryYkO5Ifi9tf73815MTxuI5svtnepeRGQAVFgUc4AAAAAAABg02tPghe/mU/wCWpuTTa0+CF7+ZT/lqZQ8yMZ+VnLAAPax4I4DAAJIAAAAAAAAAAAAAAAAAHEgAAEgAAAAAAAAAAAAAAFj8hHwwqvmL/wAcZfKcChuQj4YVXzF/44y+U4Hlu0/zDOxg+WAAaBtAoXl3+F9L8xZ+OQvpSleXq3SNuFuuKIqxPjWnc7HvXIquTPbtO+hTe7NaWIjc1sWr0yqQPN1/r6AeqOMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD22S3y3W70lDAiq+eRGbkzhFXeu7mxlc8ybzCbSi2yYq7VjrNPeoZU+W+9Q+lPFHoFwAABIAAAAAAAABgrbl6+CdF8+Z+XIWSpBeWS3SV+i5nwtVy0sjahWonMiKir5kcq9iF+FaVaLfUqrK8Gkc8gA9jscIAAAAAAyn+RjdwXP6/SgEWA+/iOz+f+XFEAJsibsDo3bvp+wAhpN3sQOhc7+PAeff0gBpPih8GOrzcPUZ7f0vSALLoBuHRn/IAWQuOnHm/X88E/vV70zqt9PWXh1xoK+OJIpEgY18bsZXdz8VXH6UgAKqtCNRp8LFkajhsiWX+/25NPRWDT8NS2hSXu001SqbcjuxNyJw6OCdpEwCaNGNJWiYyk5u7AALTEAAAABd3m6Rw3Y+Do/kh+L21/vfzXkxI3ydW+W16MtdLOitlSNXuau5UVzldhezawSU8XXadSTXVnepK0EAAVlgAAAAAAAAANJrT4IXv5jP+W43Z4L1Sd/2ito0VGrPC+LKplE2mqn8yYbSRjJXTOTQfc8MlPPJDM1WSxuVjmO4oqKu7tyh8HtotNJnAa3aAAJIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA5wCx+Qj4X1XzF/wCOMvnmKT5BbfK+63C4qipCyFIEVU4q5Udu60RqZ7S7DyvaTTxDsdjCK1NXMgA0TaMLwNZf7RS322T0FezbhlTG7i1eKORelDaGAm07ohq6sygr7yU3yknd+zEiuECr4uHNjenUqO3fb9G5DV+DfVnklfSIvaOkQdGPaleKsarwdNu5zd4N9WeSV9Ii9oeDfVnklfSIvaOkgZeK1/gjRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/A0UDm3wb6s8kr6RF7Q8G+rPJK+kRe0dJAeLV/gaKBzb4N9WeSV9Ii9oeDfVnklfSIvaOkgPFq/wNFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8DRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/AANFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8DRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHOFPyaaqlmax9vZCir/pHzsw3rXZcq/YpaWgNAU+mV78q3tqbk5uzto3xYkXijefz/dvJ4ZKa+PrVo5ZPYsp4aEHdGG9ZkA0jYAAAAAAAAAAAAB+csbZY3MkajmOTCou9FQ/QKCPspbVvJPUJUSVOnXxvicu13rIuFb1NdwVOpcefmi3g31Z5KX0iL2jpEydCn2nXgst7mrLCQk7nNvg31Z5JX0iL2h4N9WeSV9Ii9o6SBn4tX+CNFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8DRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/AANFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8DRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/A0UDm3wb6s8kr6RF7Q8G+rPJK+kRe0dJAeLV/gaKBzb4N9WeSV9Ii9oeDfVnklfSIvaOkgPFq/wNFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2pycar57V5++It3/uJtovksdSVkNbqCSKTuao5lLHvaqpw2lXGU58dXPwLbMldXtKtUjlvYyhhKcXcw1ERMJ9xkA0DaAAAAAAAAAAAACny7fzKfSn5RTRyPkaxyK6NdlydCgFd6+5N2X6qfX2uVlNXORO6MkRe5yr05TguOpebcVvJybaqY9yNtiORP6yVEePtdwOj1wib13EO1Hflm2qWhdiPg+RF992HQw2NrxWSL2NWrhqcndlOx8neqZGI6O2I5q8FbUxKi/8AvPrwb6s8k/8A+iL2i0tP3h9um2JNp1M9fGTirV6U/mhPYZGSxNkjcjmOTKKnOXVO0sRDZ2Ko4SmznHwb6s8kr6RF7Q8G+rPJK+kRe0dJAw8Wr/BnooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/AANFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8DRQObfBvqzySvpEXtDwb6s8kr6RF7R0kB4tX+BooHNvg31Z5JX0iL2h4N9WeSV9Ii9o6SA8Wr/A0UDm3wb6s8kr6RF7Q8G+rPJK+kRe0dJAeLV/gaKBzb4N9WeSV9Ii9oeDfVnklfSIvaOkgPFq/wNFA5t8G+rPJK+kRe0PBvqzySvpEXtHSQHi1f4Gigc2+DfVnklfSIvaHg31Z5JX0iL2jpIDxav8AA0UDm3wb6s8kr6RF7Q8G+rPJK+kRe0dJAeLV/gaKBzb4N9WeSl9Ii9o2Vk5Kb7VzJ+0Uit8KL4yq9Hv8yN3fb6joAwYy7VrtWCwVNcTVaes1JYLZHQ2+NGRM3q5ffPcvFzl51/yNsYMnObcndm2lZWQyBgAkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwvSQu9Vk1r1HJPCu57Wq9nM9MImF693EmikL1xHi4QyJwdHj6Fz/ADLaKTlZmE9kfnfNQLWRpBSI5kSp46rxdnm7CP8A66wDfjBQVkUt3ButPXp1vl7lMqupXL/B1p1GlHaRKKkrMJ23LWie2RiPjcjmuTKKi7lPsgenb0tC9IKhVWmcuc/2P8CdMcjmo5q5RU4nPqQcHZl8ZXPoAGBkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAE4EU13HmOkk6Fcn04X+RKyPa1j2rSx2PeSIv0oqfzM6TtNGMuBBwOC45wdM1wOtP0oJZpWzRua2tqFa9f/tsyi4616/uK6k1BXZKjdmNOWBfFq65m/jHG7719RLU4YCdhk585ubuzYSUVZAAGJIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABjJhXInFU+kA+tw3GNpOkxtJ0jci6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcNx87SdI2k6RZi6PrcD52k6RtJ0izF0fWBgxtJ0mNtucZT6RuLo+gMgEgBAAAAAAAAAAAAAAAAAAAAAFMZROKoAZwMHztN6U+kztJ0jci6M7huPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9bhuPnaTpG0nSLMXR9H5VEMVRGsc8bXsXi1yZRT72k6RtJ0jcXRqZtPW2TK977C/7jlT/A8U2k6Vc9ynmZ24cSPaTpQK5qcVQzU5rgY2iyHTaSqG74KmN/8AxorfWfraaK7WifdCk1M737WOT6URcbyW57DOSXWk1ZjIuKPli5TO/f0n0AVmYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGQYXgAQvlI1g3S9BGynRstwqMpEx3BqJxcvVv85Qt1vt0u0yy3CvqJ3LncrlRqZ44bwRPMhJOWKpfPrurjeqq2njjjanQitR2P/cQk9N2fhYQpKbV2zj4mtKU2kDGDIOhkj0NbM+pjAwZAyR6DMzGBgyBkj0GZmMDBkDJHoMzMYGDIGSPQZmYwMGQMkegzMxgYMgZI9BmZjAwZAyR6DMzGBgyBkj0GZmMDBkDJHoMzMYGDIGSPQZmYwMGQMkegzMJ+tx6KStqqORH0lTPA9OCxSK3H0bzzgh04tWaCk16lx8mPKDUVtbFab5IkkkiYhqFwiqqf1XY5+hcdW8txDkWkqH0tXDUQqqSRSJI1ehzVyn2nXTE3Iec7Tw0aNROOyZ1cHVc00z6QBAc03AAAAAAAAAAAAAAAAAADxXStgttBPWVb9iCBiyPXqQ571br27X2pkbFUSUdBlWsgicrcp/vKm9ezh0dJZ3LhUvg0a2NjlRtRUsjd1phzvvahQPR0fcdzsvDQnF1ZK5zcZWknlR9Pc97tqRznO61yfGDIO1kj0NDMzGBgyBkj0GZmMDBkDJHoMzMYGDIGSPQZmYwMGQMkegzMxgYMgZI9BmZjAwZAyR6DMzGBgyBkj0GZmMDBkDJHoMzMYGDIGSPQZmYwMGQMkegzMxgYMgZI9BmZjB9xSvhdtRPcx3S1VT7sHyCHTi/QZpdSeaK5RLlZ6uKG6VElbbnLh3dFV0keedFXevYud3QX/TysmhZJE5HRvRHNc1coqL0HInq3/rtwdKclk8lVoO0ySuVzkY+NF6mvc1E+hEOF2rhoU7VIbXOjg6rl+FktAQHHOgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFHSAgBzbyt/GFdu2L8phECX8rfxg3bti/KYRA9fhORD6OFW5jAANkqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB2AnvU7Dj87ATgnYcLtnjD9zo4D9RlAEBxDogAAAAAAAAAAAAAAAAAFa8vXwSo/nzPwSFEF78vXwSo/nzPwSFEHpOyeS/s5GM5gAB1DUAAAAAAAAAAAAAA7QAAAAAAAAAAAAAAAAAAAAAAdH8kPxe2r97+a85wOj+SL4vLV+9/OecjtjlR+zdwPnZMkAB546oAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAIAc28rfxg3bti/KYRAl/K38YN27YvymEQPX4Pkw+jhVuYwbxmk7+9jXMs9crVTKL3FTR8+4tblTq77Be6FtpqLnHTrQxqqUsj0artp+/xefCIYYitOE4whbfqKcFJNsra52qvtT2NuNHPTOflWpKzZzjGfo3HiJRZrdctWX1lsutzqmVCRudF32r5FzuVURHLu3ZXPUaBaKf9pd49zXvru3cdjCZR+cbPbndwLadX9MnujGUOnA8w7OBJ7npJaPVKWRLjSq9saPlnlXubI/FyqLvVeGF8/MbKm0Rb7k91PZtT0dZcURVSBYVYjlTPB2VRencnXuMXi6UbMlUZPYi1htFTfLrFb6LuaVEudlZFwibKZ3rx5jyVUD6Wqlp5cbcT1jdhedFUl/JPG6LlDoYpGqx7O7Nc1eKKkbs5/XSeGksrtQa5qrayZIFmqJ17ordrGyrl4ZTo6TF18tSSflSuTk/CiMg9UlIrbm+jR6ZSbuSP2cc+M45jcX7TEtq1Q2ypVRSSP2MTP8A6NibXTxx9pe60E0m+O5hkfEjvPhB1+cnNNou01U7aOn1ZQyXB67LYWxKrHO6Efnf28cmrsViYzVy228VcNDJTSplZG7TZHI5MNThuVOcq1dNp24oz7mV0RtOKp5uOD2T2qugtsFwmpnsoqh2zHKuMOX7+ZeJYHKtY7a2819Y2700NUkbFSgSPxlVERE354rx4Eau9urodDWetqLjJNQzyvSKkVFxEqK7fnO/gv085hDFKpGMltcmVFxbRF/X+v5AmVu0JUVthobw64U1PRz7bpnzeK2FGuVqb/62VThu3n5X3RqUVi/bFqudPdLex6MkfG3ZWNV3b0yvSmeC85msXScstyO5la5EgSTT+mEuNrmutxuENstkb+5pPKxXue/oa1FTPPw+3ebS46IoKXT77vBqKnnpfGSHECp3R6Iq7HvtyrgSxdKMstyFRk1cg4H3cedPOW/Jp20O5NoKdb5Rth78V/f3ccI5y58XGejHPzDEYlUbXV7k06TncrXTthrL/UVENCsSPghWd3dHKnipjduRek1X056/uLL5KaVkOqr3S0lQ2qY2ikZHLGmEf4zd6Gvo+T+OpkfRJf7el5aiqtEnjYVP6u3nj07soUrGRjUkpvbYz7huKaIIDYU1orqm9fsmKFzq7uqwqxcblTjleGNy9WN5KI9D0L6v9nN1NQLd87He2w7Y2/7HdM4+zPNjmL6mJp0+LK1SlLgQcHouNFPbq6ejrI1jqIHKx7V6elOpebmx9J5y+LTSa3TK2rOzB2AnBOw4/OwE4J2HD7Z4w/f+jo4D9RlAEBxDogAAAZQGvu1xit1Mssq5cvvWc7l/XOEr7Ijge9FM5NBpatmr21c07suWRMInBqY4G/JlHK7MJ3GRkAgkADIAAABWnL18EqP58z8EhRJe/L18EqP58z8EhRB6Tsnkv7ORjeYAAdQ1D6jY6WRscbVc9y4RqJvVVXgnXk3fuQ1D5GrvqVNIx7o3tfG5zXNXLXN3Ki8dy8clh8lV7u1brSkgrLnW1ELmSKscs7ntXxVXOFXHE1sTOpTi5wtZcS2lGMnlZE6jS99poJJ6i01scUbVc57olRGp0+s03qJFdb1eKu71dFJd7gsEs74VY6oerNlXYxjOMYXGPMb+s5OWW6omZeL/AEdBCrkbTPkbvm8VFXxVduRM4zld+eHPgsT3aXfW34WJdLM/wehXw/XQSOx6do6ygbXXW+Udtge5WxtVFlldhce8RU3Z6z9NSaTW1WqnulBcIbla5n9zSaJuwqO37lTK7t3HPEt1NPPkZj3UrXNN+ya79kpc1pn94K/uaTbsZzjHTx3ZPDnO/wA6Endb67wfx163F629azufee/COx77OfswftY9E1N4sEd1hq4IolnWKVZvFbExqLl6uz1ImMc+9cGOpjFNzfrYnum7JES35Ttwe6ltVdVW+prqemkfSU3+lkRU8Xh/Lo4eckdz0WxlkqLnZrvTXWGm/wBYbGxWuYnTxXPTzZTeh5LJb66fR97q6e4yQUkCxpNStziXKpz53fb0EPExlG8H6pBUmnZkZ845skm0xpCq1Ha62qopo2yU0jGdzemEXaXequzuRE3/AHbzYu0JHU0NXJZr5R3KrpG7ctPE1UXGP6q5Xa6uCEyxVKMsrYVGTV0iEA3WmNPz3+pnRk0VNS0zO6z1Eu5sbfXjPVuXeSWh0Ja7hRVNXQ6qp5YKZMzOWlcmwm/eqK7hu44wvmJqYqnTeWTIjRlJXRAB/mfpUMbFPJHHIkrGOVGyJnD0RVTOOPmLI5GbXQz3CWqmrYH1HcZI1o3My5G5b4+edObhzk166o0+84inDPKxW0MbppWRMVNqRyNTznv1BZ6mw3SWgrljWeNEVVjVVb4yZ58b/MbG42mgtdytqUF5gufdJU2+5M2djxkwvFeOSb8oemIqzVlTXXW70trpZWxshWTx3yKjURcN3bslEsYlON+DRmqLcWVKDe6t03UabrYop5Y6iCaNJIZ4/evb603cN3A2NJo+KG3UtZf7xT2plUm1BG6NZZHNXg5WoqYTfx4fYXvE01FSb4lapSvaxEQb3VGm57CtNKs8VXQ1TdqCqiXxXomN3VxT9ZNEWU6kakc0WYyi47MAAzIB0fyRfF5av3v5zznA6P5Ivi8tX73855yO2OVH7N3A+dkyAB546oAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAIAc28rfxg3bti/KYRAl/K38YN27YvymEQPX4Pkw+jhVuYxu5+HOW1yoaovNlvVDTWyufTwuoY3uajGrlyucirlU6kKmT9fr1k8rOUGGvWJ9x05bKqWONI0kl8ZcJncmU3Jv4FWKpSlOMlG6RlRmlFq9iOw6kuC6kpL1WTvnqoHtVXKiN2mou9MImN6Kv0llPsNO3lO/bni/snvX9qd0x4ucY+/Dys9R3imuz4HUtpo7b3JHZSmTG3np3cycO0279d1i6MTT/cG4Rnce+dvxu5oqbsY4YTHHgVV6FSSUqat6P6Mqc4xupb+p+Nkt0+utX1SyzdwbMr6mZ6pnYYipuT6Wp2b+okehnaWj1jb4rSy7zVKPejJ53sRi+I7K7KJnGM9CkJ0vfqrTl3jr6FGOeiKx0b/AHr2qqKqdKb8Y9WUJHTa9p7fXtq7Pp2hopHO2pnNdtOenO1q4RG57CK9Go24RV422Jp1I7N8T2aN3cs06f8AmqtPsefjoNUTlc389RUp/wC1/qI9adSyW7V0l+bTtkkdLLKsO1hPHzz46+g8MF3qaa//ALWpFbFUpO6dE4oiqudnswqpzfaZPDTlmX/xSIVVK32eieGT3YSQoxzpe/lbsomVztru7ckw5QbTLfOVRlugejJJ2Roj137KI1VVevci+fHDieCs5QduaSsobHQUl2kTDq1E23plMKrUxuXz8+/nNXdtX1dZq6PUFPFHTVUezsszttXCYVF3IuFRccy4IUK0pKWW1k/9k3gla5vKKm0na9UUlDCl3q66CsZF3baYyNJEdhcJjKptY6+fPOeTXmG8q0/M1KinVf4Yz6n15ClatwotP0FNdHO2nVWdpVXiqomNyrzr1qRzUt5kvl/qbo6NIJJla7YaudlWtRNy9O5OYijQq58016EznHLZG/5YUVNeVm7cscSp/Cn+J7NS/FHpf/nyfe819/1rHfLe5tdZaJ1ydGka1qKu0nDgnH7fo4GruGopK3S1tsrqdrGUT3PSba3uztbsY3e+Uyp0qmWnFq2V/wBGLnG8mvUkuo3KnJDplqOdsuqJMonP4z/pPz0a5V5N9XtX3qJE7HQvSn0IR646ifW6VttkdTtYyie56So9VV2VXm5vfdKmLRqB9usF2tbadsjbgjEWRXKmxsr0Y37uwnTy7u1t81/5HeLN+39EvoaWjreS+2Ut7uDbaySskfRzdzc9HYyi7WOC5c7iqcENPqG5Wqg0jBp6z1bq9zqjvqoqe5qxmcYRqIu/mTp4cSSU1VdLHyc2ZbHTOuTKp75p3PiWZsO/3uynDn39KLzqfEyvvOirxW6kstJbpqdqLR1DIFhdI9UXxd+VVOCdC56jThK07y3WYuavGy6FV8M/rnLAnTPItT434uK56vfFf86Y5uslGmtWrabTUWqst1PcbfM/uncpVxsu3b84VObPbznSxVOU0nBXs7mrRkk3c3PIw5zb1dnNVUVLe9UVP+Jv2kd0I96azs7kcqOWpYmd/Ou9D0WPVf7IvVyr6S3wMZWQuibAx2y2NFVF3Ljfw6DS2WvW13ajrmMSRaeVsiMVcZwvTzdu8q7mTdSTXFbGfeJKKLR0kkacsl820TbRkys5vG2m5x5s/aRaK56RpaxsrbNdWVET0cmalMo5q5385pKjUNWuqZb7R4p6l0yytRFyiZ3YXpRU48OJvJ9Z2yrmWqrdKW+aucu06Tuio1zulWY3r2qpRppxabTey4Mz72LVkzSa0vMWoNRVVyhgdA2ZGJsOVFXxWo3P2J5jSHqudYtwr5qpYYIO6LlIoG7LG7kTCJ5jynTpQyQUUa0nd3B2AnBOw4/OwE4J2HG7Z4w/f+jfwH6jKAIDiHRAB4LtcYbdTLJKuXLlGsTi5Qld2RDdjF2uMNtplkl3uX3jEXe5fUV/X1k1dUumndly8E5kToQV9ZNX1Lpp3ZVeCJwROhP1vPMb9GkoK74lMpXNtZr1Ja45GRxMftrnxl4bsGx910/yWL+JSMAydKEndkKTJP7rp/ksX8Sj3XT/ACaL+JSMAjuIdBnZJvddP8mj/iUldDN3xRQTOTCyMa9U6Mpkq7o7SzLP/smj/wCSz8KGvXpqFrGcJN8T2gA1y0rXl6+CVH8+Z+CQogvfl6+CVH8+Z+CQog9J2TyX9nIxvMAAOoagJpyQfDuj/wCXJ+BSFm40pe36evkNxZC2dY2uTYc7ZztJjjhe3gU4mDnSlGPFmdJ2kmz5n+FUipn/AF1V/wD2LvJDyxyvk1xUte9ysjija1qr71Fairj6SIvq3OuTqzYTaWbu2z17WT36svjtRXua4yQpAsiNbsNdte9THHCFSovvYytsl/4ZuaytG5oNPWmj01TXvUc9YrKt6tp6aka3bdjO9VdzLhfp4m7uLrbJyS1TrPDVRUyXBERKl6OdtYbld27GP5kfs+sG01iZaLtaqa6UUTldC2RysdGqqq8cL1/TjsXjWk1y0/LaEoKampVlbJE2HxUianNjG/K5XOedTUlQrSmsy9f2sWKcFHbobJ+PAzHjyl/2qZc5zORhqNcqbVxw5E4OTCrhfoz2kbdqB66QbYe927DanvhJtrfwxjGPtyHageukG2HuDdhKnvju23v4cMfzyZ6efT9V/wBh3i/gkXJtn3P6zTm/ZrlXtRr/AF/aZ0on/wBMtW7t+1D+JCO6ev77LRXemZA2Vtwplp3KrtnYyipnguffC16gfQabutoSna9lerVWTawrML0Y3kzw83OTS4tfwRGoklfoSPSDnM5M9WqxzmuzEm5cblVN2egxyKr/APOapnctK9F697f5oR216gkt+nLraGwNe2vVirJt42VRejHV1DR2oH6ZvHf8VO2oXubo9hz9ncuOrqTmMp0JuNRW48P9EKorx+CScm7IV03qlLlJ3vbJYomSVDWq5zHKrkaqIm9U8bfw+8+YKqyaWsF4goLt+1a+4w97okcLmMjaqKiquc79/DOebHFT2cnDqug0ff7namOrK/bbClIibTdnd46sTeq4cv0L1mz0nVV2oaqSl1Lp2iZbGwuWWqdSrCseE/tLu5ubenHdg0pu05N+W6ui+PlSRUXT+vpJ/wAiu/V0/T3nJ+JpBJ0Y2eVsLldEjlRiqmMpnCKvXw85sdMXyp07eIrhRtY+RqK1WP4OavN1cy56erKHVxEHVpOMUatN5Zps8ls/2lSpz91Z+JNykr5YXudrmrRzlVGxRoiKq7kVqbk8+VNVf9QUlyq6WoorNS25YZFkekS/6Rcpx3Jjh9p59WXt2ob3NcZIUgWRGt2Gu2sI1MZyuOfqKo05TqRm1ayf9EuSUWkyU67Vq6S0Ksyrsd7vRy8+yiR/yN3yn1OnodRNbebbXzzdwZsPin2WbGVxhO3JX991A+7WezUD4GxJbY3Rtej87edlM8N3DrNrR61bJbKeiv8AaKW7R0zUZDI9yxyNaiImNpE6ETfuzjeazw01ldr2b/ll3erdfRjU+pbbX6YorNaaGop4qWZZWumk2sZ2t3TvV3mwQ83eob3BdI4IqO0UVughVVRsLcucv+87dk0nN2bjew1NQha1jWqSuwADYMAdH8kXxeWr97+c85wOj+SL4vLV+9/OecjtjlR+zdwPnZMgAeeOqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgCAHNvK38YN27YvymEQJfyt/GDdu2L8phED1+D5MPo4VbmMAA2fUqAAAAAA3ABM+TnTltvqXaa8SzR01DCkqrE9G4RUcqqu5eZpVWrKjBzkZQhneVEMBvNaWJ2nNRVNvy50LcPhe5Ey5juC7ty78pnnwftoawxX67vjrJHw2+midUVMjf6rET6P8EUjv4Kl3voSqcnLIR0Eq5SLBR6cv0VHb3SugfTtl/pXI5UVXOToTmRCK/wCRNKrGpBTXBkSg4uzAA9ZY5K3wY2A6l5ya/s6k8E/f/e0ffi1+z3bHj46NrjghXNncVUqyqJtejM5U3GyNja75dLUjkt1fU0zHLlWseqIq9OOGT5ul4uV1Vv7TrqipRu9EkflE68cPsPBw4828mt+t1JByZafrYqeNlXNNI2SVrURzkRX8V5+BhN04TTy7syjmknuQoDp5wbJUAAAAAAAAADsBOCdhx+dgJwTsOF2zxh+/9HRwH6jKAIDiHRPzkzsLsoiuxuReCqVvd5qqeukWtRWyouzscyJ1dX38d5Za7zUX6zx3KHabhlQ1MNeqcepeotozUZbmE02ivusH6TwyQTOjmarZGrhUX9b+0/M6Cd1co4bMAAkAAADo7SzbP/smj/5DPwoVln7yzbN/smi/5LPwoamK9CymewAGoXFa8vXwSo/nzPwSFEF78vXwSo/nzPwSFEHpOyeS/s5GN5gAB1DUAAAAAAABsNP0UdyvlvopnObFUVDInK3iiOXG7m8+/BE5qKcn6EpZnZGvBP8AlF0bRWOiirrNLNLA2Z1PUNlcjlY9E3LnHV9qEEgifUTxwwtV8kjkYxqJvVVXh2qU0cRCtDPHgZTpuDys/MFk6y0Ra7HpV9XTVE8tfTSxwzKrk2NpURVTGOtOftK2Tm3bxQxEa0XKIqUnTdmAB09mS6+xhbex6rdcKy3Td1oKqamkVMK6J6tynWey46jvFyhWKvuVVNCu5WOeqNXtRMIu43vJLb6S56s7hX08dRD3B7tiRqOTO7mXn3kMThwx+vuNdOnOo1l3XqWNSjFO4384BM+Tm30ldFqBayninWGgfJH3RqLsuROKZ5zOrWjThm4mMIOTsQwD9frrBajEAAkAAAAAAA6P5Ivi8tX73855zgdH8kXxeWr97+c85HbHKj9m7gfOyZAA88dUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAEAObeVv4wbt2xflMIgS/lb+MG7dsX5TCIHr8HyYfRwq3MYABslQAAAAAALA5NPg3rX/0534JCvye8m8rGad1kjnta51vVGoq8fFk4fT9pqY5XotfRbQdp3P11Snuh5PrRfGptVdAveVUvOqIuGuVf4V/6z8U/wDl3ky2ve19+l3dLYGev7nmeSqrp6ia5aeuL0bR3OFURVXg9ucKnMi4+1qGt5SrpFcNSPgo8JQ0DG0kCN4IjeOPPu7EQ0ownm09tlv/APv3L21bvPU33K1Svr9e2qkY7ZfUU0ETV442nuTP28D71VqV2kLo6y6apaWmgpWtSWV0SPfM5URVyq9S/fzYPz5VLg2m17aa2BWy9708EqbLuKtkc7B+2q9Mrq66rfNN1lJPBVNaskckuw+JyJhUVOjCJu48TCmlaCq+Wz/2TK7csnE8OsoqW96Mt+qIaWKlrlmWnq2wtw167/G+xP4sLnCGy1FdXaBpbZaLNTUyVb4Gz1VRJHtOkVVxjsyjundjBqNX1NHZ9JW/TNHVw1dTHMtTVywrljXLnDUXn446d29MqbjU9q93lNbrxZ6yk74ZTJDVQSy7Kxqi53bulzt+77VC/Tn8l3/wdcvEzqK4wXbkmSrp6KOjfLXJ3aOLcxXoi5c1OZF/WeK/VrtNzsWkLXV6atjau63BndpatY0esTFRFa1u1w3O6+C7l3Y898p6G38lbrfSV8NbLDXok74l8TuiplUavOiIqb+nPDgInSat0faKa0XJtLdraxYX07p1j7ozCIiovOvipjmRVVFMEnk28uYybV9+Nj9dSW6vumhay46htzaW80MrVZMkbWLPGqoiouOPFfoTz63UvxR6Xyv/AIiTz73mq1Fp+82u1yS3i5wJlyNbSLVK+R2/iib0xz9XUbLUUrHclGmY0e1Xtnky3KKvvn+suhC2Szus39GDa3+iAgcwOwaYAAAAAAAAAOwE4J2HH52AnBOw4XbPGH7/ANHRwH6jKAIDiHRAVAADT36zx3KHabhlQ1PFdjj1L1ECnhkgmdHM1WSNXei9P806y1FNRfrPHcoVVuGVDU8V3T1L1F9Gs4Oz4Fc4XIZb7XVV8bnUrGva1cL42N/nPX7m7n/ct/jQ3mjYZKeKsjmarJGyIiovYSQynXknsRGCa3K/9zdz/uW/xoPc3c/7lv8AGhYAMdTMy7tFf+5u5Lu7i3+NCbW6J0NvpopEw9kbWu7URD1DJXOpKfEmMVEAAwMiteXr4JUfz5n4JCiC9+Xr4JUfz5n4JCiD0nZPJf2cjG8wAA6hqAAAAAAA3WivhfZfnkX4kNKbnRjms1bZnPVGtSsiyqr/ALyFOI5Uvozp+ZFmbbbvqjWOmKhyIlZien2l3JK1jfU1f+lekiHJnbmxXqsutxjVKazROmlRyf8A3EyjUwvPuVe1EManurrVyo1Vyp3I7uFS167K++RGojm+dMoSXlRrbfbbI+ks0jXLep+/J1a7OWoibulEV2F/iOUozglTitppf9NtyUvxv0NbV1stx5J7lWVC5mqLosj138VVv0JzdiH5WzuGktC0l7jpoZ7vcplbC6dm0kLGqqbkXsz/ANSdh+NPLGnI1Ux7bUk/aCLjKZXgem2Ng1doOjs0VZBT3e2yuWNk7tlJmOVdyLjoXmzw6FykqOSLTX4c25F7u/rY9mmbquvobhZ75T0761tO6amq2Ro17HJhMLhelycMIqZReY1uioqayaPuWp5qaKprI5kpqVsrctY5cZd9vbuwmMnu09QN5P4LhdLvV0q3F8CwUtJFJtuVVwuV6N6J5s8/HxaKfTXvRdz0xLVRU1a6VKmldMuGuVMZbnpyi8N+HZ37xK34u78l1/0L0v5tyRcmWqnag1D3O60dM6vZC58VXEzYds7std08Ux0Y85FuS+yx1qXW6yUXf8lBG1aemVN0kq5xnqTZ6+OSScnOnotNaj27vcaRK+aN8cVLE/aVERMq5y83BMJ/kRnk2udMyC8Wauq+80uUSJDUK7CMkblE3+fpThjiYtcx0eG3/SU/Kp/JLtPt1Ldrj+z9WWWN9onRyLmFrUg3KqbKpv4pjp3pvNBoGl7wr9Z0iOV/e9FPFtLz7KqmfsPwfo7U8b3OqLxTw0qb++H169zVOn6OnB8cmsjIk1Qj5mOVbfKiP2vf8d6Bw/xycXfhshf8SuiBKADtrgjSYABJAAAAAAAOj+SL4vLV+9/Oec4HR/JF8Xlq/e/nPOR2xyo/Zu4HzsmQAPPHVAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQBADm3lb+MG7dsX5TCIEv5W/jBu3bF+UwiB6/B8mH0cKtzGAAbJUAAAAAAB+u3qAD3F+g86/d5x24zw5v1gAetwOsfrgARZWtYXY+7gPP+uj/EANbWYuxv8AV+ukdX6x0fSAGlawuwvSvb5/8xv+3Pn6ubpAC24E3Y6voABJAAAAAAAAAAOwE4J2HH52AnBOw4XbPGH7/wBHRwH6jKAIDiHRAAAAUAA+WtRFVcJleKn0AAAAAAAAAAAVry9fBKj+fM/BIUQXvy9fBKj+fM/BIUQek7J5L+zkY3mAAHUNQAAAAAAD7/N+uOAAPW43cw6t365gBsLsb+jn39v6TjgdvPuAIt1A3+sdvAAJbWBjtT+ef1vM/rm+4AWS9BxA48f11ABRS4IXYABIAAAAAAAAAB0fyRfF5av3v5zznA6P5Ivi9tX73815yO2OVH7N3A+dkyAB546oAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMGTC44L0AHN3K38YN27YvymEQJnyvROj19cHPRUbIkb29eGNT70Uhh6/Bv/BD6OFWX+RgAGyVAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7AacgxRvllZHGiq97tlqccqq49R163e1OxDhds8YHRwH6j6ABxDogAAAAAAAAAAAAAAAAAFa8vXwSo/nzPy5CiC+uXWJ0mkIHNRcR1bHr2bLm/8AcUKvFe09J2Tyf3ORjOYAAdQ1AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAdH8kXxfWv8Ae/mvOcOKY6TpHkmY5mgbWj0VFVJHb05lkcqfYpyO2OVH7N3A+cmACA88dUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKYMgArflb0fLfaaK422Pbr6duw6NOMkfHCdaZXdz5XqKJmikglfFOx0crFVHseitVF+/OfOdeuwqcMmvr7Pbbg9r6+30lS9uUas0LX4+lDo4TtGWHjkaujTr4RVHmRyeDqdNLaf8h2v0WP1D3Laf8h2z0WP1G74yvaUaCXU5YB1P7ltP+Q7Z6JH6h7ltP+Q7Z6JH6iPGY+0aB9TlgHU/uW0/5DtnokfqHuW0/wCQ7Z6JH6h4zH2jQPqcsA6n9y2n/Ids9Ej9Q9y2n/Ids9Ej9Q8Zj7RoH1OWAdT+5bT/AJDtnokfqHuW0/5DtnokfqHjMfaNA+pywDqf3Laf8h2z0SP1D3Laf8h2z0SP1DxmPtGgfU5YB1P7ltP+Q7Z6JH6h7ltP+Q7Z6JH6h4zH2jQPqcsA6n9y2n/Ids9Ej9Q9y2n/ACHbPRI/UPGY+0aB9TlgHU/uW0/5DtnokfqHuW0/5DtnokfqHjMfaNA+pywDqf3Laf8AIds9Ej9Q9y2n/Ids9Ej9Q8Zj7RoH1OWAdT+5bT/kO2eiR+oe5bT/AJDtnokfqHjMfaNA+pywDqf3Laf8h2z0SP1D3Laf8h2z0SP1DxmPtGgfU5YHQv67Dqf3LWDyHa/RI/UfcGnLJTytkgtFvilbvR7KZiKnnRB4zH2jQPqU7yV6Lq625092uMDoaGnckkSPTCyuTeip/uou/PYnSXuiYQNRETCcD6OTicRLETzSN2jSVKNkEABQXAAAAAAAAAAAAAAAAAAGr1Faob3Zqq31WUinZsqqf1Vyio7tRURfMc0aj0/cNP1zqa4wOaiKqMkRFVj0TnavR1cd+MIdVKfhU00NVC6Kphjmid75kjUci9qKbmExssM7cUzXr4dVdzkYHUyaW0/5DtfosfqM+5bT/kO2eix+o6XjK9pqaB9TlgHU/uW0/wCQ7Z6JH6h7ltP+Q7Z6JH6iPGY+0aB9TlgHU/uW0/5DtnokfqHuW0/5DtnokfqHjMfaNA+pywDqf3Laf8h2z0SP1D3Laf8AIds9Ej9Q8Zj7RoH1OWAdT+5bT/kO2eiR+oe5bT/kO2eiR+oeMx9o0D6nLAOp/ctp/wAh2z0SP1D3Laf8h2z0SP1DxmPtGgfU5YB1P7ltP+Q7Z6JH6h7ltP8AkO2eiR+oeMx9o0D6nLAOp/ctp/yHbPRI/UPctp/yHbPRI/UPGY+0aB9TlgHU/uW0/wCQ7Z6JH6h7ltP+Q7Z6JH6h4zH2jQPqcsA6n9y2n/Ids9Ej9Q9y2n/Ids9Ej9Q8Zj7RoH1OWAdT+5bT/kO2eiR+oe5bT/kO2eiR+oeMx9o0D6nLAOp/ctp/yHbPRI/UPctp/wAh2z0SP1DxmPtGgfU5YB1P7lrB5EtnojPUPctYMov7EtmfmrPUPGY+0aB9TnnR+la/U1eyKmicylR39LUuauwxE5uhVxzJvz1ZU6Wt9JFQUUFJTt2YYWJGxvQiJhD9YYmQxoyFjWMTciNTCIfoc3F4uWJd3sjcoUFSRkAGoXgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABRgAAwZwAAMDAAAwMAADAwAAMDAAAwMAADAwAAMDAAAwMAADAwAAMDAAAwMAAGDOAAAgAAAAAAAAAAAAAAAAAAAAAAAAUAAwZwAAMDAAAwMAADAwAAMDAAAwMAADAwAAMDAAAwMAADAwAAMDAAAwMAADBgyAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP//Z"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Slide23.jpg](attachment:Slide23.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /home/masoud/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import bz2\n",
    "from collections import Counter\n",
    "import re\n",
    "import nltk\n",
    "import numpy as np\n",
    "import random\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "random.seed(8)\n",
    "\n",
    "nltk.download(\"punkt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_file_all = bz2.BZ2File(\"./data/train.ft.txt.bz2\").readlines()\n",
    "test_file_all = bz2.BZ2File(\"./data/test.ft.txt.bz2\").readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training reivews: 3600000\n",
      "Number of test reviews: 400000\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of training reivews: \" + str(len(train_file_all)))\n",
    "print(\"Number of test reviews: \" + str(len(test_file_all)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset contains a total of 4 million reviews - 3.6 million training and 0.4 million testing. We won't be using the entire dataset to save time. However, if you have the computing power and capacity, go ahead and train the model on a larger portion of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_train = (\n",
    "    80000  # we're training on the 80000 randomly selected reviews in the dataset\n",
    ")\n",
    "num_test = 20000  # Using 20000 reviews from test set\n",
    "\n",
    "train_file = [x.decode(\"utf-8\") for x in random.sample(train_file_all, num_train)]\n",
    "test_file = [x.decode(\"utf-8\") for x in random.sample(test_file_all, num_test)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The format of the sentences are as such:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__label__2 Strong Like Bull!: I purchased this as a gift for my son. The bench is very sturdy and well made. If your looking for the right tool for the job, this is it!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(train_file[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "del test_file_all, train_file_all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some pre-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll have to extract out the labels from the sentences. The data is the format ```__label__1/2 <sentence>```, therefore we can easily split it accordingly. Positive sentiment labels are stored as 1 and negative are stored as 0.\n",
    "\n",
    "We will also change all *url*s to a standard \"<url\\>\" as the exact url is irrelevant to the sentiment in most cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extracting labels from sentences\n",
    "\n",
    "train_labels = [0 if x.split(\" \")[0] == \"__label__1\" else 1 for x in train_file]\n",
    "train_sentences = [x.split(\" \", 1)[1][:-1].lower() for x in train_file]\n",
    "\n",
    "\n",
    "test_labels = [0 if x.split(\" \")[0] == \"__label__1\" else 1 for x in test_file]\n",
    "test_sentences = [x.split(\" \", 1)[1][:-1].lower() for x in test_file]\n",
    "\n",
    "# Some simple cleaning of data\n",
    "\n",
    "for i in range(len(train_sentences)):\n",
    "    train_sentences[i] = re.sub(\"\\d\", \"0\", train_sentences[i])\n",
    "\n",
    "for i in range(len(test_sentences)):\n",
    "    test_sentences[i] = re.sub(\"\\d\", \"0\", test_sentences[i])\n",
    "\n",
    "# Modify URLs to <url>\n",
    "\n",
    "for i in range(len(train_sentences)):\n",
    "    if (\n",
    "        \"www.\" in train_sentences[i]\n",
    "        or \"http:\" in train_sentences[i]\n",
    "        or \"https:\" in train_sentences[i]\n",
    "        or \".com\" in train_sentences[i]\n",
    "    ):\n",
    "        train_sentences[i] = re.sub(\n",
    "            r\"([^ ]+(?<=\\.[a-z]{3}))\", \"<url>\", train_sentences[i]\n",
    "        )\n",
    "\n",
    "for i in range(len(test_sentences)):\n",
    "    if (\n",
    "        \"www.\" in test_sentences[i]\n",
    "        or \"http:\" in test_sentences[i]\n",
    "        or \"https:\" in test_sentences[i]\n",
    "        or \".com\" in test_sentences[i]\n",
    "    ):\n",
    "        test_sentences[i] = re.sub(\n",
    "            r\"([^ ]+(?<=\\.[a-z]{3}))\", \"<url>\", test_sentences[i]\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After quickly cleaning the data, we will do tokenization of the sentences, which is a standard NLP task. \n",
    "Tokenization is the task of splitting a sentence into the individual tokens, which can be words or punctuation, etc.\n",
    "There are many NLP libraries that are able to do this, such as *spaCy* or *Scikit-learn*, but we will be using *NLTK* here as it has one of the faster tokenizers.\n",
    "\n",
    "The words will then be stored in a dictionary mapping the word to its number of appearances. These words will become our **vocabulary**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1,\n",
       " 'strong like bull!: i purchased this as a gift for my son. the bench is very sturdy and well made. if your looking for the right tool for the job, this is it!')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_labels[0], train_sentences[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "650ac3ebc0464c55a482ca0bc24a3a3b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/80000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Dictionary that will map a word to the number of times it appeared in all the training sentences\n",
    "words = Counter()\n",
    "\n",
    "for i, sentence in enumerate(tqdm(train_sentences)):\n",
    "    # The sentences will be stored as a list of words/tokens\n",
    "    train_sentences[i] = []\n",
    "    for word in nltk.word_tokenize(sentence):  # Tokenizing the words\n",
    "        words.update([word.lower()])  # Converting all the words to lower case\n",
    "        train_sentences[i].append(word)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['strong', 'like', 'bull', '!', ':', 'i', 'purchased', 'this', 'as', 'a', 'gift', 'for', 'my', 'son', '.', 'the', 'bench', 'is', 'very', 'sturdy', 'and', 'well', 'made', '.', 'if', 'your', 'looking', 'for', 'the', 'right', 'tool', 'for', 'the', 'job', ',', 'this', 'is', 'it', '!']\n"
     ]
    }
   ],
   "source": [
    "print(train_sentences[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "print(words[\"stuning\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To remove typos and words that likely don't exist, we'll remove all words from the vocab that only appear less than a `min_occurrence_threshold` throughout.\n",
    "To account for **unknown** words and **padding**, we'll have to add them to our vocabulary as well. Each word in the vocabulary will then be assigned an integer index and thereafter mapped to this integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "min_occurrence_threshold = 1\n",
    "# Removing the words that only appear once\n",
    "words = {k: v for k, v in words.items() if v > min_occurrence_threshold}\n",
    "# Sorting the words according to the number of appearances, with the most common word being first\n",
    "words = sorted(words, key=words.get, reverse=True)\n",
    "# Adding padding and unknown to our vocabulary so that they will be assigned an index\n",
    "words = [\"_PAD\", \"_UNK\"] + words\n",
    "# Dictionaries to store the word to index mappings and vice versa\n",
    "word2idx = {o: i for i, o in enumerate(words)}\n",
    "idx2word = {i: o for i, o in enumerate(words)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('_PAD', '_UNK')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx2word[0], idx2word[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the mappings, we'll convert the words in the sentences to their corresponding indexes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "14bd06cfee7c4f0cb302ee55a5809234",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/80000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i, sentence in enumerate(tqdm(train_sentences)):\n",
    "    # Looking up the mapping dictionary and assigning the index to the respective words\n",
    "    train_sentences[i] = [\n",
    "        word2idx[word] if word in word2idx else 1 for word in sentence\n",
    "    ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d47ee74b3c414b91a2d60548c7070db3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/20000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i, sentence in enumerate(tqdm(test_sentences)):\n",
    "    # For test sentences, we have to tokenize the sentences as well\n",
    "    test_sentences[i] = [\n",
    "        word2idx[word.lower()] if word.lower() in word2idx else 1\n",
    "        for word in nltk.word_tokenize(sentence)\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the last pre-processing step, we'll be padding the sentences with 0s and shortening the lengthy sentences so that the data can be trained in batches to speed things up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2983,\n",
       " 13,\n",
       " 173,\n",
       " 11,\n",
       " 25,\n",
       " 256,\n",
       " 455,\n",
       " 200,\n",
       " 47,\n",
       " 3,\n",
       " 360,\n",
       " 10,\n",
       " 743,\n",
       " 10,\n",
       " 3,\n",
       " 1546,\n",
       " 4,\n",
       " 9,\n",
       " 12,\n",
       " 13694,\n",
       " 9270,\n",
       " 21,\n",
       " 3,\n",
       " 750,\n",
       " 4323,\n",
       " 10,\n",
       " 3,\n",
       " 201,\n",
       " 4,\n",
       " 580,\n",
       " 9485,\n",
       " 4,\n",
       " 580,\n",
       " 1083,\n",
       " 1,\n",
       " 8,\n",
       " 2274,\n",
       " 136,\n",
       " 5183,\n",
       " 504,\n",
       " 10,\n",
       " 11,\n",
       " 25,\n",
       " 4,\n",
       " 6,\n",
       " 214,\n",
       " 16,\n",
       " 7,\n",
       " 2066,\n",
       " 4,\n",
       " 4356,\n",
       " 1945]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_sentences[10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Defining a function that either shortens sentences or pads sentences with 0 to a fixed length/ Pad from right\n",
    "def pad_input(sentences, seq_len):\n",
    "    features = np.zeros(\n",
    "        (len(sentences), seq_len), dtype=int\n",
    "    )  # instances * features (seq_len)\n",
    "    for idx, review in enumerate(tqdm(sentences)):\n",
    "        if len(review) != 0:\n",
    "            features[idx, -len(review) :] = np.array(review)[:seq_len]\n",
    "    return features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bf6939dce1db49558ac91ef0aa7b11e8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/80000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4cf71f0977f54200bbda9000438e415b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/20000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "seq_len = 200  # The length that the sentences will be padded/shortened to\n",
    "\n",
    "train_sentences = pad_input(train_sentences, seq_len)\n",
    "test_sentences = pad_input(test_sentences, seq_len)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Converting our labels into numpy arrays\n",
    "train_labels = np.array(train_labels)\n",
    "test_labels = np.array(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A padded sentence will look something like this, where 0 represents the padding: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([    0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
       "           0,     0,     0,     0,  2983,    13,   173,    11,    25,\n",
       "         256,   455,   200,    47,     3,   360,    10,   743,    10,\n",
       "           3,  1546,     4,     9,    12, 13694,  9270,    21,     3,\n",
       "         750,  4323,    10,     3,   201,     4,   580,  9485,     4,\n",
       "         580,  1083,     1,     8,  2274,   136,  5183,   504,    10,\n",
       "          11,    25,     4,     6,   214,    16,     7,  2066,     4,\n",
       "        4356,  1945])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_sentences[10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can save the final result of our preprocessed data as Numpy arrays and load them later to save the time of all above preprocessing steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.save(\"./data/train_sentences.npy\", train_sentences)\n",
    "np.save(\"./data/test_sentences.npy\", test_sentences)\n",
    "np.save(\"./data/train_labels.npy\", train_labels)\n",
    "np.save(\"./data/test_labels.npy\", test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our dataset is already split into *training* and *testing* data. However, we still need a set of data for validation during training. Therefore, we will split our test data by half into a validation set and a testing set. A detailed explanation on dataset splits can be found [here](https://machinelearningmastery.com/difference-test-validation-datasets/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "split_frac = 0.5\n",
    "split_id = int(split_frac * len(test_sentences))\n",
    "val_sentences, test_sentences = test_sentences[:split_id], test_sentences[split_id:]\n",
    "val_labels, test_labels = test_labels[:split_id], test_labels[split_id:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Val len: (10000, 200), Test len: (10000, 200)\n"
     ]
    }
   ],
   "source": [
    "print(f\"Val len: {val_sentences.shape}, Test len: {test_sentences.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, this is the point where we’ll start working with the PyTorch library. We’ll first define the datasets from the sentences and labels, followed by loading them into a data loader. We set the batch size to 100. Please note that number of instances must be divisible by the `batch_size` in our implementations. This can be tweaked later according to our needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "import torch.nn as nn\n",
    "\n",
    "train_data = TensorDataset(\n",
    "    torch.from_numpy(train_sentences), torch.from_numpy(train_labels)\n",
    ")\n",
    "\n",
    "val_data = TensorDataset(torch.from_numpy(val_sentences), torch.from_numpy(val_labels))\n",
    "\n",
    "test_data = TensorDataset(\n",
    "    torch.from_numpy(test_sentences), torch.from_numpy(test_labels)\n",
    ")\n",
    "\n",
    "batch_size = 100\n",
    "\n",
    "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size)\n",
    "val_loader = DataLoader(val_data, shuffle=True, batch_size=batch_size)\n",
    "test_loader = DataLoader(test_data, shuffle=True, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# free memory\n",
    "del train_sentences, test_sentences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also check if we have any GPUs to speed up our training time by many folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU is available\n"
     ]
    }
   ],
   "source": [
    "# torch.cuda.is_available() checks and returns a Boolean True if a GPU is available, else it'll return False\n",
    "is_cuda = torch.cuda.is_available()\n",
    "\n",
    "# If we have a GPU available, we'll set our device to GPU. We'll use this device variable later in our code.\n",
    "if is_cuda:\n",
    "    device = torch.device(\"cuda\")\n",
    "    print(\"GPU is available\")\n",
    "else:\n",
    "    device = torch.device(\"cpu\")\n",
    "    print(\"GPU not available, CPU used\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([100, 200]) torch.Size([100])\n"
     ]
    }
   ],
   "source": [
    "dataiter = iter(train_loader)\n",
    "sample_x, sample_y = dataiter.next()\n",
    "\n",
    "print(sample_x.shape, sample_y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we will be defining the architecture of the model. At this stage, we can create Neural Networks that have deep layers or and large number of LSTM layers stacked on top of each other. However, a simple model such as the one below works quite well and requires much less training time. **We will be training our own word embeddings in the first layer before the sentences are fed into the LSTM layer.**\n",
    "\n",
    "The final layer is a fully connected layer with a sigmoid function to classify whether the review is of positive/negative sentiment."
   ]
  },
  {
   "attachments": {
    "Slide24.jpg": {
     "image/jpeg": "/9j/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCAFhAvMDASIAAhEBAxEB/8QAHQABAAICAwEBAAAAAAAAAAAAAAUGAQcDBAgCCf/EAGQQAAEDAwIDAwUGDBIIAwYHAQEAAgMEBREGIRIxQQcTURQiYXGBFTKRlLPTFzZCU1VWdZOhsdLwCBYjMzU3RlJic3SEkrK0wcPRJCU0cpXU4fFEgqImJ0NUZYMYOEVjZHajwv/EABsBAQACAwEBAAAAAAAAAAAAAAABAgMEBQYH/8QAMhEAAgEDAwEFBwQDAQEAAAAAAAECAwQRBRIhMRMUMkFRBhUiNFJhkRYzcaFCgcEksf/aAAwDAQACEQMRAD8A9UoiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAISiwd0BjiWQVX7hTtr9RMpKl8xpm0ve92yV7AXceN+EjIx0Oy7tFY7ZRTtnpaKCOYZAka3zt+e/41LSRXJKIiKCwREQBYKxnp1woCu1EDM+mtEHl9Sxxa9/FwwRHqHP8AHns0OI5EBVlJRWZBc8IsGVGXG/Wu2v4K64UsMp5RukHGfU3mfYFXpaOsryTd7jNI12/k9I408Q9Gx43e12D4Bdiht9HQMLKGlgp2nn3UYbk+JxzPpWhU1KnDw8meNvJ9TsP1W15/0C03Wrb+/wC5EDf/APVzTj2LhdfL5Kf1CzUkTfGorjxexrWEfhXY3RactUqf4oyq2Xmzqm46kdsG2dmeuJH49m2U8t1L9fs/xaT5xdpPgWP3lWLd3gdUXLUY5x2iQDnh0jc/jx+Ffbb9eoj/AKRZKeUeNLWhxPse1uPVkrse1YwpWp1V1IdvE+WasgZ+yFuulCP30kAlb7TEXgD0nClLbebdcwfc+upakt982KUOLfWOY9qjV0662UNeQaykgmePevezzm+kO5g+kELYhqq/ziY3bNdGW7KyqZDDcrdg2u4vliB/2avJlb6hJ78H0kuHgFLWzUEVTUtpK2F9DXO97HKQWy+PdvGzvVs7bJaF0KNzTreFmGVOUepOosZWVsFAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAsLKFAQv7sP5j/iKZUN+7D+Y/4imVaXkVRlERVLBCnRQmsKyWi07WSUzzHUPDYInj6h8jwxrvYXAqG8cgjrhWPvU0tPTSPjtkTjHJIx3C6ocNi0Ebhg5EjcnIGADxc0MUcELIoGNjiYA1rGDAaB0A5BfNLTxUlNFTUzQyGJgYxo6NA2XKvNXVxKtLrwdCnTUUPFEVK7Q7tW2urtTjXVtrsT+9NbcaOmbM6FzQO7DuJjwyMkuy7hOCGjLRutelBzltRkbwXVMqrS6rjpRS0lHT1eoaw0bayV9sbEGmM7Nky6QNw4g4a0uOx6brjq9dULXwtttuul272hbcWmjiYB3DuIZzI9uDlvvffHIAB3xbsJ+hG9FuWNlSaPWk9frGgt9vtc9TZ6y2sro6xndtOHuaA88UgIYA7BHDxZ5AjdcNk1mIrRY4ZY7pfLlcm1LonQ00ULpO5k4XcQ4wxmARg8W4b4kAz2E/Qb0XxFULFr233ea2BlBc6WnuQe2mqaqJrGPkY0ufERxcQcOF+5AaSw4JXPatZ0lxqaENobhBRXBz2UVdOxghqS0F2G4eXty1rnN42tyBnwydCa8huRaE9ap1q17SXKSg7q0XiKK4Nk8immjiY2oexhcYx+qZDsNdguAacZBI3XZ7OtRV2ptOxV9xtslFI57wHEs7uTD3N8wCRztuHBzjfltyiVKUVlhSRaFw1dNDVwOhqo2yxuxkOHhuPby3HI7rnWFjTcXlMlrPU+LTcJqGrht9wldLFLltLUv98T9beepxyd9UBv53vrL61ULvSGttlRAxxZKW8Ucg5skHnNePSHAEelWCxV3unZLfX8PD5VTxz8PhxNDsfhXo7K4dan8XVGhWhslwSCIEW6YgiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIUQoCF/dh/Mf8RTKhv3YfzH/ABFMq0iqMoiKpYHkofVVDLcbBV09MAanhEkIOwMjHB7B/SaN1MLHjhQwVahqo62jhqYM8EgDgCMEHqCOhHIg7hdhcVzt81uqJq63Qvmp5ncdRSxjzg7rJGOp6ubzdzHnZDvikqYKyATUsrZIzkcTehB3B8D4joV5u7tZUZ5xwb9KopI51E3yjvFS9jrNdKajHA5kkdVReUsdnk4YexwcMHqW4PJS2UwFqwk4vJlayjXE3ZXQRm3GhFqndS0TaJwvFsbXMcGuLg9o4mFjsudnB4SCNtgVZLfphtJVmdklNE11sZb+4paYQxMLXPcXNaHHhbl/veniVZMLHoWSVxOSSbI2optm0dU2er09PRXSJ3ubbGWudstKXeURgtPE3DwY3Zb/AAxg8tl92HRnuTPYZPL++9y46yPHc8Pe9+9r88zw4xjrn0clb0Tt5t5GxFPt2ihR0em6d1d3gs9VPUZ7nh77vWyt4cA+bjvee+celdHSnZ3S6crqR1NDp6SmpSe6mNma2uDTnhzUCTBcMjzuDfHpyr9hMKVcTSxkbEVSg0gKSl0rD5bxe4UjpM9zjvuKKSPHPzf1zP1XJdzRtjqdO219vlroqukZI91NiAxyMa57nlrzxkOOXYBAbyVgWFR1ZNbWFFArKx6l1q2tgoo2uqH4c93DHG0cT5HfvWtG7j6Bv1VYwlJ4iS3jlnFeqp9LbZn044qp47qnZ++ldswerJGT0GTyVjs9Ey22qioYt2U0DIGnxDWgD8Si7NbZZatlxuUfdysBEFMSD3AOxJO4LyNjgkAHAzkk2Fejs7fsKeH1ZoVZ72ZREW4YgiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiKq9pmqzojRNy1CKLy7yMRnyfve64+J7We+wce+zy6IC1IqHprUmtrhdqWC9aEitVukBMlWL1HUGLzSR+ptYCcnA59c9Fe87oDKKn9qepqzSWiqq8W6OnkqYpoIw2oa5zCJJmMJwCDnDjjfml+1LWW/tF0rYIY6d1HdYauSZ72u7xpia0t4SCABuc5B9iAuCLGUz16IDKLGUyfBAZRYycplAZRYymUBlEysA5QGUWM8lASaoo4tbU+l3RVBuE1C6va8NHdCNrwwgnOQ7J8MelAWBFjKZQGUVL7W9UVujtC116tkVNLVQSQMa2oaXM8+VjDkNIPJxxvzVya7LQfFAfSLGeSZQGUWOJQWubvUWHRt8u9G2KSpoaKapjbKCWFzGlwBwQcZHiEBPIofSVymvGlrPc6prGz1lHDUSNjBDQ57A44yScZPUlTCAIUQoCF/dh/Mf8AEUyob92H8x/xFMq0iqMoiKpYJhEQHzhQ100/T1c7qumkkoa4854MAv2+radnD1jIHIhTaweWyq0msMJtdCnzC72/PltCK6If/HoOePF0Tjkf+UvXxTXu3VEwgZWRsqfrEuY5R/8Abdh34F0O1LUl7sEEItUNOIZQczk8cjCOfmYxjlvuMnG22ZbRlIbjougN9a24SVLO/f5S0SB3GcjY7DYgYWvW0mnKCqdMloXclLYdrpkJ7Fl+kLUCTStqqLwbSVUkTB/9sO4PwLhOmauPelv9wH8GeKGQD28Ad8JXOlpcvKRsq5Xmjl+FMrrGxXxg8y90Zxy47eSPbiUfjC6tDR3iuhdLSX6zzRh7oi5tskIDmktI/wBo6EFY/ddbqT3qHQk/Yi64sV7d7++Uozz7ugx8GZD/AHrkbpieQjyq/XJ46siEUTfhDOL/ANSmOl1H1YdzE5DsNzgcyVGSX22tlMEVSKqobsYaUGd4PpazJHrUqzSFnzmpglrj1FbPJO0/+V5LfgC7V3ouHT9bTW0eSv7h4h7gcHA7hOCMct1s09Kjn42Y5XLxwiHhhvVxx3NMy2U55y1REkp9UbTge12R1HMKYtVipbdIZgZaiscOF1TO7ieRnOB0a3O/C0AehULsr1bfr5Umjr4Y6mmibl1W53A9vPGw99uMbY8T4LaYAXTdnG1exLk1lVdVZGFlEViQiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgC1f+iYH/ALkdS4/ewf2iNbQUBrnTFFrPS1dYbpLUxUdXwB76dwbIOF7XjBII5tHQ7ICI0rZNX0Nwgnver6S6UAYQ6ljs7acuONjxiQ4wd+SrvYXgaa1g0nBbqK4gg9PPC2s1oa0AdBha9ruyey1V4uVWy5X2ko7pKZq+2UtcY6Wre73zntA4vO64cAeXLZAatp//AMolsz9fi/t4WzNY/t4dnn8luXycakWdmlmZ2cwaKbU1/uVC9r2ScbO+y2bvR53Dj323veXwqcuWmqO4aps1/nlqG1lqZPHAxhaI3CUAO4hjJI4RjBHtQGp4r+OzC7dpVrlc0U7YP0wWmMj3xmJY6MDw7/haOfvlYtKU9/0JoGwUNs0xU364VEbqm5PZWQwFlRIQ9/EZHDiJc5w25Bq63alpL9N3ajoSJ1uqn0lvdNWVtaGEQ920scyEuxglz2A8OcgZPq25gIDS+pY5tb9pOm9N6ro6mhtDrRNc6m1eVZbNOJQwRvdGcPDR52M436r67SLJS6E0LT2jR76u0Ud7vFLRTviqHu8ljldwvdGXk8GQMbbZOwV61roa26smoaqoqLhbrpQlxpbjbZ+5qIg7HE0OwQWnG4IPsXXh7OrRJpi42K81N0vlNcHiSoludW6WUkY4eEjAZwkAgNAAQGu+0PR1k7NaKzaj0XTTW27RXKmppBFUPd5dHJJh8Uoc7D8jfPPIzlSOoNN0Wqu3upoLw+pktrdPRSzUkczo2VBFRIA2ThILmjiJ4c4JxnwVns/ZdbqO62+vuV71FfXW5wko4brWiWKneAQHhoa3LgDsXcRHrVij01Rs1pNqZslR5fLQtt7oy4d13bZC8HGM8WTzzjHRAULsuoo7BqrtD0xbnStsltkppaOmfIXin76Ave1pcSQ3I5dN/EqgaY0BaK/9D3+matdWzX6mts9VR1hqpGuozFxlgiAcAxvmAnbJOSfRvu26XorfqG/3mGWpdVXoQioa9wLGd0zgbwDGRsd8k78scl1LXom3W3s+do6CesdbHUktGZHvaZuCTi4jkNAz5xx5vhsUBrDtGvd4uWk+zW2tp6q4i/tjkr6emqRSyVgbAJDF3pcA1riSTuMgYHgefs705fLD2hUVRZtE1WltOTQSx3OB92iqYXvDQYpGxtkJD8jhJA5H1lbFu2g7PddKWyw1flfc21sQo6uKYx1MD428LZGPbjDwOuMehfOltER2G4+XT6g1HeagR93H7qVxlZGOpDGta3J23IJ25oDXHZrouy9o2n6jVGsYamuvlZXVAErqqWN9E2OVzWRRcLgGcIbnbnnclcN+7P8AS9y/RB0lJcLPDVQVVikq52yuce9mbM1veOOcl3Dt/wBVdqzsmtT7jW1FrvepbNS10pnqaC2V/c08j3HzncPCXNLts8JbsFJat7P7fqK4W24Mud5tNyt8TqeKrtlSI5HRO5scXNdkbZ5Zz1QGsdcMq9RdrFzsVRpSp1PZbNQ0/k9siuDKSFjpASZXtc9oeduEcw3HIHdfMT9W6F7K+0CQWussVBA1j7JBPXsq5KVrwGSBr2udhrT5zQTtnrutoam7PbdfbhS3KK43m03inhFN7oWyq7maWIHPdyEgteM77g7ruWnRdvorBcLTXVNyvMFwDhVy3SqdPJMHDhI6BoxthoaAgNJ9q/Zlp7TvZCbpaRWRXNppDU1Tap+a8ulZkzAktflzuIcsEDG2y3hry4W+16Lu1Zeauro7fHTu72ekJbMwHbzCNw7JAB8fhVOn7ErFVWwW65XvU1woogBRwVVeJGUeDsY2luMgeaC7iw3ljmtg6islDqKw1lnu0JmoauMxSsyQSPEEciDgg+IQHnmKhZYdYaHuen9F3bS0NddIqeSuqbiyR9bFI1xMc0Qkc7iOOLLtwR44Vz1Dpui1V2+VNBeHVL7a3T0Ms1JHM6NlSRUSANk4SC5g4s8OcE4Jyp2j7I7bHcbRXV+odUXWotNTHUUhr68StjLMjh4QwAg5GTji2G4HO2x6ao49aTambJUeXy0Lbe6MuHdd215eDjGeLJ55xjogNaaJ4tHXHtSs9lMnuZZI4au30j3OkbA6SmMjmtyc8PEBty+EqqfQ+stf2CVmrauWul1RV2iW4VF1bVyCaV7mFxjdvwmPGGFuMY9O63ra9MUNu1HfrzE+eSovXc+UxyFpjb3TOBoaMZGQd8k+zkqZ9BXT/klRbnXTUTrDIXOjs5ryaSBzjkFjMZ2d5wDi5vFvgoC29mv7XemM8/cym+SarIuhY7bDZ7NQWyldI6Cjgjp43PI4i1jQ0ZwACcDwXfQBCiwUBDfuw/mP+IplQ37sP5j/AIimVaRVGURFUsERCgB5LpXKuht9I+on4uFuwawZc9x2DQOpJwAueonjghfLM9rI2NLnOccAAcyfQoS2xSXSsZdaxjmQMz5FA/ILQdu9cP3x6fvWnxJUxXmyrZz2q3SPdNWXVjH1lS3gdHnibFGeUQ6H0nqfQABKwxMhjZHE0MjYAGtAwAOQC5APQs4RvJKWBhYCyigkitR+XvslW2zBpr3s4Ii53CGk7cWfRufYqP2PWq82hldFWtjNvkkeGkPyWyxvMbtvA8P/AKQtmEbKF0p+xk/8urP7TKssarUHDHUxShmaZNY6LOFhZWFGUYWCPHksopBCXG0hkFPLaI4oKqjB8na0BjC04zG7H1JwPUQD0XdtVfHcaNk8Qc3JLXxv2dG8HBYR0IOQu7gKAubTZ6191pw51NJgVsbRnYDAlA8WjAPi0fwQFZPdwyuMdCwIuON4kY17C1zXDIcDnI8VyKpYIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAh2CIUBgnYkpn0LUEzrx2ka81Ba6a/XKx6YsEjKV5tcghqKupI4nZlwS1rdm4HPmT4drU+mKnTPZhriOTUd5vNHLapjFHdZWzSQOEcmcSYDiCC3Y5xw89ygNqg59SyqHQampNM9nWl56mmr6yaehpooKWgpnTzTP7lpwGjYbAnJIG3Nd3Ruu6DU9wrbaKG52q7UbGyy0NzpxDN3bsgPGCQ5pO2QfXjKAt+EKoFs7TqG73hlLaLFqKvoHVXknutT0QNHxB3CTxlwcWAgguDSBjwS5dplDT3uttltsWor26hkENXPbKISwwSHBLC4uBc4ZBIaDhAX7Po2TO+ypAraJnavWQtqr7JcY7K2c0Qlb5EY+9IBazP69kYzsOHbqtW9n1XQ6t7U9SXW+aX1AbnDXcFJXytDRa444ciN+JMMdnOwDgS7fmgPROfQsrXWjNUWGy9kVDqCe53eSysY54qrw/vqt+ZXANcW54ncRwAOmFI6Z19De640s+n9SWc9yahkl0oO6jka3GcOaXAO68JwcdEBdDtzWM+K1uO12zsnpnVdn1HRWipmbBDd6q3mOkcXEBpyTxtaScBxaAoztP7QbtpvtD0na7dar1PRTyzGqZS0sUgrgIiQyIucDxNO597t48kBtzdFr+tutHWa40S+qdqS219ZTVUsNv7xkcBDWNLhVMDjlwyOHBIByubUPaTbrXfaizUFqvd+uVKxslVDaKUTeTB27RIXOa0E8w3JOOiAvSKn27tBstx0TcdT0RqZaO3MldV0/d8FRC+JuXxOY4jDx4Egb88bqDi7Y7BI6im9z77HZqyVkEd5koi2iEjjgNL85xnzeINLc/VHmgNmErAOfUql2tn/3W6tP/wBKqfk3Lo0GpqTTPZ1peeppq+smnoaaKCloKZ080z+5acBo2GwJySBtzQF8RU3SevqDUdXcKBtvu1tu9DGJpbdcYBDOYzye0cRa4EjGeLnjOF2LHrm0XfQrtWQulgtccUssrZw1skPdkh7XAHAcOE7ZQFqRUau7SLZR2Cx3F1vu8tVeo+8obXBTd5WSt4Q4ngBwAGkEkuAGea7mkdc27Uz7hTx0lyttyoAHVNvuVP3M7Guzwvxktc04OCCR4oC2otb6X7XbJqWWndbLdfHULo3yVFwfR4paMtYXlksodwhwaOmQCQM5Xf0p2i0+p7hTQ2/T2pI6CqDn09zqKHgpZGhvEHcXFxAO6FzRnI8UBeUREA9JULcqq9QzvFBbKSphABa+SsMbj/5e7P41NFYRPBDWSrWGauqNQTvvFO2krBThrIWEOaWcW7g8Hffphvt5q1BQv7r/AOY/4imeitJ5ZWJnKZUNXalsdBUGCuvFupZm845qljHDryJz1yuH9OWmftjs/wAdi/KUbJehO5epPr5J5qC/Tlpj7YrN8dj/ACl1q7VGlqyklp5NTWtjJBwuMdfG12D4Hi29nipUJeg3r1OTbUVcetnpn+yqlafwsaR/5nDwG9iwOirtPqzSlNBHFBf7LHFG0NYxtbEA0AYA5qTtl5tt14vcy40dYG++8nmbJj14JUyT9CItepILKwUVC5nksexdaurIKGlkqayaOGCMZfJI7ha0ekqou7UNItJHuo8kHGW0k5HPxDMK0YSn4Vkq5xXVl3ULpP8AY2o/l1Z/aZFB/RS0h9k5fiU/5C6tB2jaMooXxQXKYNdLJKc0c/vnvL3fUeLir9jUx4X+CjqR9TYCKk/RS0j9k5fiU/5Cx9FLSP2Tl+JT/kKOwqfS/wAE9rD1LwnNR1nu1FeaJtXbaqKpp3HAew8jzweoO42O6kVjaaeGXTz0MHdYIBG/IrKg9Q6ps+nuD3XrmQOf71ga57yPHhaCcenCRi28JBtJcnFRuNjr2UD/ANjqlx8keeUTuZhz4dW+jLegBsAdla/uPaJoq4UklNU3GUxv8KOcFpB2cDwbEHcHpgL7o+07S0VLCyovEs0rWgOk8gnbxHxxwdfBZnRqPna/wY1UivMvyKk/RT0h9k5fiU/5CfRT0j9k5fiU/wCQo7Cp9L/BPaw9S7LKpMXadpKWRrG3QtLjgF9LMwfCWgD1q4QysniZJE9r43gOa5pyHA8iD1CpKEoeJYLRlGXRnNuiYRVLBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREBo6lvsHZV2iasGrI56bTl+qm19HdGwukhbKWgSRycIJa7PLIxgdFNao1zZNZ9mWtzp19XVUtPaZy6rdSyRwuc6N/mMc8DicMAnAI84c8ra2ByWcBAaT1jqO5af0H2eiC5S2WzVcMEVzu8NOJnUrBC3hAyHBvE7zeIg4KjeyKooKztoustpvd6v1CbCwR191HnSYqCD3Z4GcTAdsgYJzuVv3A6JgIDzpQ6it1o1Db6fsxvF18trbk0Vukq2ke5kDXuzM8BzQ6Dh4i4kOLc9CuXW+oLTp/UF7q9D3u7WvWctViTT0lE+WG5yh3DxCIt+rDR+qMcMjfmvQ+EwgNXWp0j/0QFS+oaGSnS0Be0bhrjUuJA9GVxdlP0w9qXP9m3+39RYtq4WcIDzhSUVXP+ht0RW0dJUVrbVcKe41FNAzifLDHO8uAbzOAc7dAtoW7tOsGpIa2HRdW+73OKikqY42U0oYHNb5rXuc0BpJwOHIKv8AhMBAeQtVamg1FoGGW4a01BctTyTU7q2zR03c0tIROzibIwRDhDTgAl2S4Dmtxds9bT2PV3Z5qG6OdDZ6CvqGVVSI3ObF3kPCwuwNgSttYCYQGqLxcqS8drXZlcbbMJ6KqorjLDKGlvGwxRkHB33URp7VFq7N9Y63otazS20XO6OuVFWzQuMVVE9jRwNe0EcTMYIwDv1W78LGPSgNB0UFRV9nHa7qR9PPS26+Cqnoop2GNz4mQFvelpAI48ZGd8D2qV7Tmj/8M0ALRgUNtxty/VIFujCYCAp/awQeyvVY/wDpVT8m5UPWOo7lp/QfZ6ILlLZbNVwwRXO7w04mdSsELeEDIcG8TvN4iDgrdmFjA6IDQXZFUUFZ2z3Wa03u9X2hNiYGV91HnSYqDnu3cDOJgORkAgnO5UPqqgq6PWd57L6aORlu1Zc4brDJGPNipnZfVN25edDgDlh3RelcfgVTh0nntHn1ZV1xncKBtBSUvdcIp2l/E93Fk8TnHHQYAxugNb9sVujtnaJp6/3O53myacjt0tDJcbU3LqWTjDmh/mP4WOG2ccwM4Ck+zGj03W6gu1+0/qfUepKptB5FJWXAcUAZxcYY1/dMDnAjoTjO63Fj0lYwEBqrsU8lp/0P9pdUUb6mlbRVD5qWGPjfM3jkLmhg98524x1yqv2f3+lotZWGx9nOorje9Pyd42utdbTvf7lxNaeFwlc1rowHAMDHF3M43wt+4QgHmgMoiIAhRCgIX92H8x/xF9avq5aDSd6rKZ/DPT0U0sbvBzWEj8IXyPpw/mP+IuLX30i6i+51R8m5XXiRjfhZ5d3Li5xLnOJLnHckk5yTzJzvnxRMovZRiklwcJt5YREVsIgKY0bVTUer7LLTPMcj62GJxb1a+QNcD6CCVDqS0x9NNi+6NN8sxYLmKdKXHkZKbe9Hq7GyLPRYXjzvGs+3qR8ek6Fkb3NbJXsa8A++HdyOwfa0H2LRa3l2/wD0rWv7ot+RlWjV6PSUuxz9zk3v7gREXVNMJ8KIiBtv9D5I8z6gi4j3bW0zw3oCe9BOPHYfAtzLTH6Hz/bNRf7lL+OZbnXkr9YryO1bftoHkvNfaxK+TtEvDZHlzYjFGwH6lvcsdgeAy4n1lelDyXmjtV/bGvv8ZF8hGs+lfv8A+jHe/tlVT8+aIvTHJCIiAL0Z2OSOk7OrW6RxcQ6ZgJ8BM8AeoAABec16L7GP2ubZ/GVHy8i4+sftxf3N6x8bLthEReeOoERFICIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAJhEQBERAEREAREQBCiFAQv7r/wCY/wCIuXVVFLctNXahp8d9VUk0DM7DicwgZ9pXF+6/+Y/4imVZvDTKJZyjx/Vf6HUPp6weT1EZw+OXzHA+BB3XD5RB9ei/pBexeEeCcI8F1lrEksbTSdivU8deUQfXov6QTymD6/F/TC9i8I8EwMZVvfMvpHcV6njvymnxnv4v6QCsGgqKe7autLKBhmEFXFPK5m4Yxjw4lxHLlgeJ257Lb3a1c7pYxabjaZ3R/qr4Xx44myEgEAt67Ndvz35qz6MudfeLDT1tzpIqaWUBzRHJxBzcDfH1PqyVatqFSdHdtWH9ysLaKqYyWAelZ6LG/gsrhnSKd2nabn1PpwU1G5oqaeZtRE15wHkNc0tz02ccenC0x9DzV32Dl+MwfOL0vhYwtu3vqlCO2BgqW8ajyzzV9DzV/wBgpfjMHzifQ81f9gpfjMHzi9LE43XVoK2nr4XS0kzZYxI6Muac4c1xaR8IWf3rcY8jD3Kmec/oeau+wUvxmD5xPoeau+wUvxmD5xelkT3tX+xPcaZrvsi0lXabpa6puobFVVpjb3AcHd21nFjJG2SXnlkYA9IWxVgADksrQqVJVZOUurNqEFBbUYPJaY7T9A3i4aknutlpxWR1YZ3kYkYx0bmtDeTiAW4a3rnPTZbo6L58VajXlQlvh1K1KaqLEjzT9D3Vw3Nil+MwfOIOzzVx5WOXl/8AMwfOK367fqW46kbbxWPks8lUym4qRpjjaXkeZIRuXAHffHLbOw3JGxsbGtaMNaAAB0wulU1GvCKeVyacLWnNtHm36HmrvsFL8Zg+cT6HmrvsFL8Zg+cXpb2osXvav9jJ3KB5pb2d6uc5rTZJG5IBc6pgwPXh+cfCVvbQ1lfp3TFFa5ZRLJEHOe5o24nuc9wHoBcQrBjKLXuLypcJKfkZaVvGk8ozlEwi1TOEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEKIUBC/uw/mP+IplQ37sP5j/AIimVaRVGcJhEVSwwsYWUQHBUU0M5jM8TZDE4PZxDPC7lkendQjv9Q3Lj5Wusly/wp5nfVf7jjz8HHP1RViK4KmCKpglhnYJIpGlr2uGQ4EYIKlS8iGvM5Qcr6UFZ55aKr9yax7nua3jpZnneaIY2J6vbkZ8Rg+KnMo1gJ5MoiKCSK1H5e6yVbLO1pr3sLYi53CGk7cWfRz9io3Y9arzaWV0Va2M2+SR4aRJktljeY3beB4f/SFswjZQ2k/2Mn/l1Z/aZVljUag4Y6mOUMzTJpZwsLKxGQYRFhAZUJd6yeSpbbba/FXK3iklxkU8Z24/S44IaOpGeQK5rzczRRRx08YmrqglkEJOMuxkkno1o3J/vIB+rPbxb4Hccjp6mZ3eTzu5yPPX0AcgOgCsljkq3ng57dRw0NJFTU7eGOPlk5Jyckk9SSSc9Su5hYCyq5ySlgYREQkJhEQBERAEyiweqAZRUfXuvqTTBFJCwVdyc3i7riw2MHq4/wB3M+jmtV1vaVqupk447hFSNOPMgp2YH9MOKrKcY9Wcy61W2tZbZvk9GZWV5p+iDq77OzfFoPm0+iDq37OzfF4Pm1Tt4epqfqK0+56WReafohau+zs3xaD5tPohau+zs3xaD5tO2h6j9RWn3/B6WWF5q+iFq77OzfFoPm0+iDq77Oy/FoPm07aHqP1Daff8HpXKwvN8XaJq2N4JvLpcfUvpocH4GAq86N7VBV1UdHqSKGnfIeFlVFkR5PIOBzw+vJHq5q0akZcJmxb61a15bIvn7m2UXyHZx4L6VzrBERAE3Qr5z1QH0ijLnfLbamcVxrqen2yGveAT6hzKqtb2padpxmGSoqjyxFER/WwsE7inT8UjNTt6tTwxbL6i1XN2w0I4u5tlS/HLie0fDzwuAdscWf2Hf8YH5Kwe8bf6jY923P0G2kK1hB2wWxzsT2+sYP4Ja4/jCm7f2k6bq+AOrXU73bYmjIx6zy/CrwvaE+kjHOxrw6wZdEXVorhS10Pe0dRDURnk+J4cPhC7OcraTT6Gs011PrKLCypICIiALCyeShb5qSz2MD3VuFPTuO4Y52Xkctmjc/AhWUoxWZMmVlUg9qOkgce6cp9Io58fDwLH0U9I/ZKX4lP+Qhh71R+tfku6KkfRT0j9kpfiU/5CfRT0j9kpfiU/5CDvVD61+S7oqR9FPSP2Sl+JT/kJ9FPSP2Sl+JT/AJCDvVD61+S7plUj6KWkfslL8Sn/ACFL2XV9hvUoittzp5ZjnERJY848GuwSPUhaNxSk8KSLDlF8gr6QzBERAEROqALG6xxY57KNul8ttqbxXCtp6fbIbJIAT6hzPsVZSUeWyYxcuiJQLCodb2padpx+oyVFSfCKIj+thRE3bDQji7m2VL8cuJ7W/izha0r6hHrI2o2FxPlQZtNAtSjtjjz+w7/jA/JXbg7YLaXYnt9YwfwS1x/GFRajbv8AyLPTrlf4G0FhUug7SdN1ZYDWOp3uOOGaMtx7eX4VaqKvpK+IS0VTDPEduKJ4cPhC2IV6c/DI1p0KlPxxwdrKysZ5LKzGMIiIAiIgCIiAIiIAiIgCwVlYQEMfpw/mP+IppQE88UGrozPIyLvKMtYXkDiIeCQCeu/4VOhwOCOqs10Ko+soiKiLBERSAmMoiAjrxbm3CmDGvMU8bu8hmHON45H0+BHUEjqviy3E1sL2VDBDWwO7uoiG/C7xHi0ggg+B8chSnRRlTbw+6QV8EhimYDHJtkSx7nhO/QnIPTcdSpT8mVfBJ5RfAcPELPGPEKCdyPo8lCaT/Y2fbH+nVn9pkUzxDxC61BTQ0UL4oNmulklOT9U97nu/C4qfIjKz1O34IvniHiE4m+IUYG5ep9Hkujc6+G3Ub6io4sDAa1oy57icBrR1JJAC7gcPELrT0sE88EssYe+BxdGT9SSMZA8cE7+vxRfcZzwjoWWhmbLJcLkGm4VAwWg5EDOYjafwk9TvywBM43ymFlG8kpYGEREJCIiAIiIAiIgC6d0q47fbKutm/WqeJ8rvU0En8S7gVf7QPpF1Fj7HVHybkRSo8RbR5oq6ye4Vs9ZWP4qioeZZDvjJ6DwA5AeGy4E6ehZGM79Vzajbkz5bXk51HJ+phPYgx7PUsn/oqmLHODHsTos49Kxv+f8AcoGGEWSMf91jcezmgCHfnyWVjrgKYvDCbTyj0J2QXiW7aPibUOL56KQ0jnn6oNALT/Rc3fqrwXADJIWnuyO801g0FebjXF/k8dyDDwN4nEvjha0ADnkuAXZ1lr2iu+m66gt8FzjqZ2cLHiJzMZI34gchb1WvCkviZ9Y0m3rXdCEorJtgbhZVI7OtTuutM633GQG5UzQeLkZ49gH48ejvA77AhTerr7Fp6yT18w4nNHDGzOON55D8+iRrwlT7VPg2altUp1exa5PnVGqLdpymElwl/VHe8hZ5z3+of57LTOpO0m83Zz2Uj/IKY8mxHzyMA7u5j1gDw9Kql3uVTdrhLWV0hkmlOTknb0D0YXTXmrvU6lV4hwj01lpVKit1RZZ9ve57y+R7nOJJJJJ3PX0lYjY6SRsbGue9x4Q1oJJJ22C+V3bLVNorxRVUo4mQzMkcMZ2BH4fBc5fFL4mdOXwRbiuTsVlmdSiZktdQ+VwN4pKYSO42nbIyRwlw6gO6KLHPbfx3ypu5WxgrKmqFdQy0ZcZWPFQ0vkBOQOAeeHYO+RjIPrVh1PcaeWlr20zRU0ErGdw03GIth5cLmQcPG0gbEek8+a2lQjLOeMGl3qUceeSCuOnTRC7g1PH7niE/reO87zHp2xlV/wDP/qrvfq6kmOqjFUwP79tJ3fC8HvOHGceOFSFhrxjHG0z20pST3HPRVlTQztmo55YJm5w+NxacHmNui2NpftUrKaRsN+YKqA5HfRjhkHpxyPs/CtZJvhWoXdWi8xYr2dKusTR6ttNzpLvRR1dBOyeB4yHNP4PQV3cndeZ9FamqdM3Zs0bnOpZCBPEDnib448VtvW+tKq0T2iO0Q0tS2uhln7yV5ADWGMDGOee8/AvSWuoQq03OXGOp5i50ypTrKnDnPQuFwudJbnUwrqiODymUU8PGccchBIaPSQ0/Au7xeBBWg9UX65amjoYLlDSRQU05qP1Fzi4nu3sx0xjjzn0LYPZ1qh1wj9yro/NwgbmOQ8qiPln/AHhkZ+Ec8C9HUKVapsixcaXVoUu0l/s5u0/VjtM2aNtGW+6NWXMhJGeAD3z8HY4yB6yOmV58nlkqJ5J6mR8s8hy+R5y5x9JPP+5bA7dJnv1lSwkngjoGOaPAukfn+qPgWu1kuJtPaj5lr93OVw6SfCGUyiLWyeeyxlMoiZGWMplEQZYTqDyIwQRsRjkR6fSizyCJuPKZeM5ReUzeHZDq+e8wyWm6SmWupmcccrvfSx5xv/CBIGeoI65K2YV5t7KnuZ2h2UMJHG6Zrt+Y7mQ4+FoW/dRXulsFrkuFeX9wxzGnu2cTiXODWgAdSSAuhCWYKTPomi153VtFvl9CUztlM+rC1PrPXtFd9N11BboLnHVTs4WP7osxkj6rOysXZ3qg3WmNuuLwbnTtzxHbv4+j8ePQ+B8AQsUbqlOeyLPQz0+tTpdrKOEXfKgdUaotunKYSV836q73kLN3v9Q/vOyzq6+xaesk1fMAXNHDGzOC955D8+i823e5VN2uEtZXyGSaU5OSdh4D0LV1C/7stsfEZtO093T3S4ii16k7SLzdnOZSP8gpjybF78jbm7mOu4A8PSqW97nvL5Huc4kkkknc9fSV8IvNVripVeZs9TRtqVFYgj6jY6SRscbXPkcQ0NaCSSdth4n8KlKyzOpBOyauojVwNzJTCR3G07ZAJHCXDPIO6LrWWqZQ3ihq5RxRwzskcMZ2BH4cLv3K2MFZU1QrqGWjLjMx4qGl8gJyBwDzw7B3BAGfhSlBSjnqytWo4zSzhf8A37EIOe2/jvlT1w06aIXfNTx+54hP63jvO8x6dsZU9qe408tLXspmipoJWM7hpuMRZDy4SyDh42kDYj0nmuC/V1JN+mkxVMD+/bSd3wvB4+HGceOFndCEFyzWjc1KjWFgpP5/9Vz0dXU0M7Z6OeWCZvJ0bi04PT1LgRaak4vMWdCUVJYksmzdL9qlZSyNgvzBVQcu+jGJG+zkR6Pxrb9pudJdqKOroJ2TwPGzmn+7ovKe+FYtE6nqdM3Zs0bnPpZDieHnxN8ceK69lqk4NQq8o4t9pMJJzo9T0xldG4XOjt76ZtfURwGqlEEPeHHHIQSGg+JAPwKoa31pVWia0R2iGlqW18Ms4kleQOFhjAxgb57z8CoOqb/ctUMoYblBRxQ005n/AFFzi4nu3sx0x7/OfQuxX1ClRym+Tl2ul1bjEukWb6BydsL7CoXZ1ql1wZ7lXN+bhA3LJD/4iMYHF/vDIB+Ec8C+rZo1Y1oKcTRr0JUJunNdAiIspiCIiAIiIAmERAcFTSwVUZjqYY5ozzZIwOHwFdOjsVqoakVFFbKGCcAgSRQMY4Z57gZUmiZIwMIiISCm6FYQDKFdKG5Uk1wnoY52Oq4GtfLED5zWuzg49OD8C7uygNNFT7QNWxaVtYeGCauqCW08R5EjGXH+CMj17DqtA3u/3S+TmS6V88wJyIuIiNvqaNvbzPXKsHa/XPqte1sTzllJHHAwdBloefbl/NU309FrV6rT2o8PrWpVZV3Sg8JHCaaAneGLPpYCseS0/wBYi/oBcyLX7SXqcLvFX6n+Th8lg+sRf0AnksH1iL+gFzInaS9R3ir9TOHyWD6xF/QCeSwfWIv6AXMidpL1Hb1fqZxCngaciGMEbghgyD61YNP6ovGn52Pt1bN3TedPK8vidvn3p2HrGCoRZCmNWUfMyUrytSkpRkz07o7UdNqezR1tKOB4PBNCTkxvA3GevPn1B9in1orsJrZI9U11E1x7iekMzm/wmPaB+CQ/At6+hdBPKTPoen3LubeNR9TKIik3giIgCIiAIiIAVAdoH0iai+59R8m5T5Vf1/8ASLqL7n1HybkXUx1fAzzCPBSWm42TaitkUzGyRvqomua4ZBBcNiOSjQpTS2Bqe0ZP/i4v64XO/wA/9nzGn++s+o1RHHDqO5xRMbHEypka1rAAAOLkANsK434QWCCg8isFDXWZ9OxzqySEvfITnJ7z6gnIxt128BHar0nenXq61wpB5IZpJhKZmAcGSc4znl6MqT0pZ9S2avizND7hk8U73VDHU7oju5wBORt1wPScc8sU9zOpSpSjVnFx6vqQ+kLxC660NufZrTLTzThpfNT95KGucTjiPPGdiR4Li1fdo319wtsFptNLFHUOjbLBTBkmGu8c+jwXxa3Uh7RKZ1BgUnl4MeNhw8eRj0c8ehR2qfpouuNgauUj+mVXc9vUxutNUdra8WC69pNot7qOea00cFNNbZWRzthYGcTJGtLXHHPc4+FdTs0tdCWGuu1NFUMqJ20lNHMwOaXY4nOwfQOfrCkaushb2l3K3VhPkVzhZTSesxjhPrztn0rrtlipNbafsdLJxU1tw1zhyfK4Evd+fI5WTEd243XCl2yr8emPuUO9MbHea+ONrWsbUSANAwAA88gOi6S7193vty/lMn9Yrolar6nnq37j/kvNj/afvR/+uUv9elXB6vZ+firf2M22ku+ibxRXKBtRTSXHLo38jiOEg+wgH2KT1toW0U+mK+ay2jNwYwGEQcReXAjlvzVNRsZXLU4voj7N7M6jTt7SFOa6lGtMVbPfbfHaDi4iTijfjIY0EcbnDbLcHBGRnIxg4KtPbzJKKO0RjPcl8jn+HEAMfgLla9A6Z9wqF01YA651QDp3A5EYHKMegZO/Ukn0Dl7QdPHUWnZaaLAqoz3sJP74dD6xke3KrCynTtJU/Nm5Xv6dS8jNdInm72n4VhclRDJTzyQzMdHLG4tc1wwQR0XGvNSTTw0emjJNZXQJv0/7+pZHqVq0tWNloBaqatfb7hJUCSOYNzHPkYbHJ1xn1jc7LJSp9pLBir1HSjuSyVTI5jr+f5+pPYrddbfSNstkpxQztuEss0Uj4nNc4ubIAcjhHH1wOIY8TzXOywUMElBVGKQxi4spZIHVkM/EHciSwYacjdpz4Z6rN3SeepgV7BLlclL3WFcZrLb6qvvlRHG+KnpKjuWwGrihDnF7t+NzQ1rQBs3BOeqr9+ooKC4uhpZ2TwFoc0tka8tyPekt2yOWyx1LeVNbmZKV1CrLalhkciIsBtDbB8Fdrk+c2PR1RUMmfG2lrYuIRl2P1WLhBx6G7egKu6ds1VfbtDQ0bCXvPnOxkMaObj+e/JembVQQ2+201FCP1KCMRtzzwBhdjTbR1oyzwmcbUb6NvODXLR58bVRl7Iz3rXPdwt443NyQCcAkc8An2K1dn9hmu94guBL4rfQS8YkaSO9lAI4W/wAEZw7H+7++CvOvdLyamgtUEE7aZtNWCaV+PO7vupGEN/heeNzsOfoNit1DT26jhpaSIRQQsDWMb0AW9b6VGjV3t5SNO51ntaG2KxJ9TRvbl9PUX3Ph+UmVA6LYHbl9PcX3Ph+UmWv+XrW5c+M+N6185I7U9BVQUVLWSxcNNUlwifkedwnB258/HC57PZrheZZGW2mdOY28TzxBrW+snA9imr/j9Iml8DrU+n6tY01cre+w19jus0lHHVSNlZUsZx8LhjZw5kbD4SqbFuwYlbU1WUJPjB0TpW9e6AoDQu8p7vvQ0yMALcgEh2cHcjb0rtu0HqUNybdgePfxflLqams9dZ/JG1FU2rontcaWaKUuYW53xvt0O23hyUjqc/8AsTpQdA2fH9MKVCKzlGSNCit++LyiIg07dZ6Ksq4aQyU1I9zJnNew8JaPOHPJ28FFNaXODWglxOAAOp2WxtF3Rtp0dDPN/s77uIpwdwWOhAOfQNj7F0bXYGWvWdc+sb/q+1B1USeT27mMZ5ZOx9hCl0lxjzLSsIyjCVPz6lYvVjuNmfC26U5gMwJYC9rsgYzyPpCjRzV37QqySvtWnKuc5lmgkkfjYZLgeXoyqSOSxziovCNO8oxo1XGHQs/Zdv2jWLP1yb5CVbe7YfpHn5f7VSf2mJah7Ld+0Wxfxk3yEq9DXi2Ud2oJKO4wMnppOEujeNjhwI+AgH2LahHfR2nt/ZefZ0IzfkzQfq9n5+K7NnirZ77b2Wg4uIk4o34yGNB89zgObcHBGRnIxg4KvOttC2in0zXzWW0ZuDGAwiHiL+IEYxvzU3oHTPuFQOmqw11zqQHTOByGAcowfAb79SSfQONQ0mdOsm3wfQ6+sUp0G4rl8YKp28ySijtEYz3JfI5/hxADH43LT3tPwr0j2g6e/TFp2Wmjx5VGe9hJ/fDofWMj25XnGohkp55IZmOjljcWua4YII6LX1alKNXc+jGjVoyo9n5o40RZHqXJOz5GN+n/AH9SZHMdfz/P1K16WrBNQNtVNWvt9fLUiSOYNzHPkYbHIeeM+sbnZfV1t9I2y2SnFDO24SyzQyOic1zi5sgByOEcfXA4hjxPNbSt247os03d7ZuEkVL2LO6ukdgoaeSgqzFIWC4spZIHVkM/EDyJLAQ05G7TnbIz1XxNZKCrrr5UxRvipqOo7lsBq4oQ5xc7fjc0Na0AbNwT6eqnukyvfqfXHBTUUjfaKCguL4aSds8Ja17S2Rry3IzwktyMjlso5a0o7Xhm5CanFTXRhMDdFJ6es1VfbrDQ0bCXPPnPxsxoO7j+e/LqppwlOSUSKk404uUuhY7k+Y2PR9RUMmextLWxcQjc7H6rFwg4/gt29AXTbVRl7Iz3rXPdwt443NyQCcAkc8An2L0Ha6GG322mooB+pQRtjbnntsq/r3TEupoLVBBO2mbT1onmfjzu77qRhDemfPG55bnfGF6GvpSqpSzzg8/a6xGm+zceM9Sjdn1hmu95gryXRUFBLxiRpx3so24Wn96MniI9Lf3wW5107dRU9uooaWjibFBE0NYxvQBdxdK1t1b01BHKvbt3VVzfTyMoiLZNQIiIAiIgCIiAIiIAiIgMHktXai19dqLUdzt9DR0ToaORkXHK93E4mJj84A5efj2LaJWg9YudBrnUPexTASVMb2EROcCPJ4hsQCOYI9a0NQq1KVLdS6nU0mjSrVnGquMHE68XQ6nk1BiCKvw1ojjceB7RzY4nodvgBxst2advFNfbTDW0ZPBIMOY/30bhza4eIP8A02Wgm1cTmyEOdmPHE0tcDkgY2O+d9hjfp6dv9mun57NbJamu4mVdaWyPhztEAMNB6cWMZPs3AC0dLr1qk5KfQ6GtUKEKcZR4fl/Bp3tQ/bFvwP1yH+zxqrq0dqP7Yt+/jIvkI1V+S3q/jZ8Q1T5up/IGMnKzjHPl1KtV/AGhdLHrmp/A9d7QfE+yXSOyuhj1EXNMJk4eJ0WxIZxbZ2OenL0ER2acsERs06qpt+WSjetM7Y2x09Ks9x1DqChurJKoCkuUcRidIYGh8jc53BGDuNiAFY9QasvUGldPVUNZwVFS2bvn90w8eHADYtwOfTCKEefsXha0mp7pP4fsa1HLPt2WfXlbR0PWQP0lMy48MjLldH00znDfz4s59fFj4VULFp90mr/c2tGIaSRzqhxGwjYdyc9Dt8KOl0a8xOwaUJQed39FcQ9Ffe0+tbcaWw1jGcDJopHtbgbNLhgevGFQiqyjtlg1rqh2FR008mwOw36epvudL8pEt/LQPYb9Pcv3Ol+UiW/it+n4Ee60L5OIREVzsBERAEREAREQAqv6/wDpF1F9z6j5NysBVf1/9IuovufUfJuRdTHV8DPMIRAi5k/Ez5XUfxtj4fhTx8Pz/wAkU7bNKXu5Uraijt73wu965zmsz6QHEEolJ9DJTjVqvEMsguexT8/auxXUdRb6uSlrInQzxnDmOxkfBsfXnC4OWc/9lGHkxtSi9rMevH402OwUxPp+qiuVuoXPg76ujjkiIceEB5IGds9FHVtM+jrqillLTJDI6NxbnBLTjIz02UtNcsvOnUgsyOE5WMoc7ooMWecm7uwH6WLp90HfIxLZ+AtYdgP0s3T7oH5GJbQK6a6H03TvlofwYwExscrKFSbpStb6EodR5qIz5LccYEzRs/0OH4M8/wAS0pqHS12sErm19I8RA4EzBmN2Tjn0PwL08fSvh8bHtLXtDmkYIK5t1ptOv8S4Z0rTU6lt8PVHkj8/+qkaK811FEyKnlaAwucwuhY90ZPMscRlvsx/et93fs/05ceIvom08hGA+nPBj04G2fYqBetBadopcDVdJSH63UPYXH0Dzh+Jcp6RcweafJ1lrVrNYrcFDhvVfDBFCyfzYZDNG50bXPY4uDiQ4jI3G++/tX3VX+41MTYnzMbG2UVAbFCyP9UHJ/mgbru1Fht0ee71Na3jPm4bMSfY1hx+FddlmpHOx7v2pueRc2oaPhMWFHuzUPoY96aZnPaRON1+uDqmed0kLnzgCVpp4+CTDs5czh4Sc9cZXQqZn1MxlkEbXEYxHG2Nvrw0AK023SlsqpcS6tssTc4H6oQ4n1O4fQrtaeyi1PY2We7SVbDuHQcLWn8f41D0y8lxOP5LR1Wwg805Jv7Gm9+nNW/S2gLxfHtfJC6ipNj30zcE/wC63qt1WbR9js7mvorfCJRuJH+e4H0E5I9inw0NAAGAtyhoyTzVZp3GtykttFYIPS2maDTdH3FBH5zjmSV273keJ/uU8R4J4LK7UIKC2xOHKbm90nyYxhFlFcqaA7cvp7i+50XykyoO3Xmr925fT3F9zovlJlQFpXHjPnWtfOSLdT3bT9Xpu2W+8Muomou8w6kEfCeJ+fqj4Y6DqurFLpMmWOenvAiDuKKZj4+8c3A81zTsN88lXMgIN3HOyx9o/NGu72TxmK4+xYNS3qlraOgtlrglht1EHCMzEGR5duXHw3zt/wBhxXe7QVunrLQxMkbLRCTvC4DB4iDtv6PAKEHPHT4E2z/0UOo2zHO6nNyb8yZF1hGjnWgsk8o8t8p4sDg4eADHPOfYpW96uFx0rTW5sL21pDI6qcgfqjGZLBnnnJB9eQqjkYyhxyHNFUkWje1YravTBOX67QXC1WWlhZI2SihMcheAATkct89FBBZPTHNYHVVbbeWa9Wo6r3SLT2W/tjWL+Ml+QlXpRea+y39saxfxkvyEq9KLfo/to9z7O/Kf7Y4QmAsosp3jGNt1Stb6EodRh1RGfJrhjAma3Z/ocOvhnn+JXXosdOSxVaUasds0Xp1Z0pboPB5h1Dpa7WCVza+keIgcCZnnRuycDccj8Cg/z/6r1u+Nr2lr2hzTsQd8qqXjs/05cuJxom00hGA+nPBj04Hm59i4lbRnnNJneoa4ksVkaEorzXUULI6eVoEbnOjLomOdGTzLHEZb7MJDeq+GnihZP5sMhljc6NrnMcSHEhxBI3G++/Xmr5etBadopcfprpKQ/W6h7C4+rzh+JVOpsNvjz3epbXIM+bhkxJ9Yaw4/Ctf3XfLiMXgz+9tObzKSR06m/wBxqYmxPmY2NsoqA2KFkf6oPq/NA85HX64OqZ5nSQl04AmaaePgkweLLmcPCSD1xlcjLPSOdj9MFqbnq5tQ0fCYsKVtulLbVS4l1bZYm5wP1QhxPoDuHPRPduoLlwZPvPTMYU4lXqZ31ExlkEbXEYxHG2Nvrw0AdFxb9Oa3Jaeyi1PjbJUXaWsYdw6Dha0j8Ku1l0hYrO5r6K3wiVpyJHjjcD6CckK0NIrSeanBE9aoQjiksmlNL6BvF7kY+WJ1FSEZ76ZuCc/vW9duR2C3bpbTNBpuiNPQR+c4gySv3e8+JP4gp0AAbbLK7FrYU7fldTiXWoVbl4l0GEAA5LKLeNIxhY4l9Bah/RI9oNboTRsJtDu7udwkMMUuM92GgFxHTO4Ht+CUskNm3M4WV+alLrzVNNdhcor7cBWZyZO+dk/9F7q7D9aT677P6O7VrQ2tY91PUFo4Q97QDxAdMgj8KOPmDYOVji5eteM/0Snate6nWFdpyz101FbKB3cythdwumkHMuPUZ6ctlU+xzta1BpXVNBFU19RWWiaVsU9NM8vHC48255OB3B/uyFO0ZPfSxnbdM+K8Xfole1O+Vmta7T1rrZqK129/cvZC7hMsgzxFx5n0Dlt486pZB7RByi8Vfobe1G+W/W9vsFxrZq21XGRsAjldxGJ52aWnmN8A9ML2Vdq1lttVZXSgujpYXzOA5kNaSfxKWgmdriWc8vSvzs152n6l1XqCeunudRDEJCYIIXljImg7BoHLb/qvRX6FDtHuuqKevsF9qH1k9DGJoKiQ+d3fEG8Lj13Ix4D2YOIyeicpkZVC7a9Zy6F0BXXalaHVhIgp+IZDZHZwSPQASvCdbr3VNZdjcqi+17q0u4hJ3xGPUOWPQij6g/SgHIWHMaeYC0/+ho7Qa7Xejqht4d3txtsjYZJ8YMrXAlpP8LY5Pq9a3FhQ15EpspQ0a2o15Pfa18clMGRmngAziRoPnu6bZ29eeYGLoB4Jjqsqqgo9EZKlWVTG59DzX2o/ti37+Mi+QjVXI9XJWftR/bFv38ZF8hGqutKv42fMNU+bqP7l4dQwXnRlihju9qpZqUz95HVVIjcA5+RtuenXCjINNRSOkYzUFoZUxPGeOfhjIIBDmvxuenLoq0sk8lXem02hK7pyabhyvuXDXNwgmtlot5r2XKupGv7+rjPE08W4aHfVdN/R4k46uoamCXSWm4IZ43zQtm7xjXAuZlwxkZyFWfanP2I6mclKl45uTx14LKyrhZ2dup2zsbVi596Iw8cYb3YHEBnOPSp/UV9t0um33CjliN5usMdNUta4F0YbkPPoDsAY8MLXfPcrB35/jUqq1wjJHUJxjtx5YLRqurp57DpqKGeOSSGmc2RrXhxYduY6Kr9Fk4xsVjoqN7nk1K1Z1pbmbA7Dfp7l+50vykS38VoHsN+nuX7nS/KRLfxXQp+BHvdC+TiERFc7AREQBERAEREAKr+v/pF1F9z6j5NysBVf1/8ASLqL7n1HybkXUx1fAzzCOqIOqLmT8TPldTxs7lmp2Vd4oaabIjmnjjdvjZzgD+NTfaNUyzarq4HkthpSIYIxsGNDRyHLfn/2CrUb3Rva9hLXtOWkcwcq4V9509fpGVd8pblBceEMldRFhZJjkfO39H+avB5jg3beUXRlT3YeTh0XZHagqK2prGTVraOJrhTiXhdM4ghreI8hgEbHwUhqTTMcenKi4i0SWioppGgxmp75srHEDOeYIJH58ou3Xy2UVXcKaKiqBZa2IQyR94DLtyfvtxbnb/LC6F2dYBTFtojuT53OHn1TmAMA32DeefwK+YqJtRlRjQ2vDZarif8A210h/JKT8ZVP1Kf/AGjuo/8A5cv9cqWqtQ002oLHXtjn7qghhjkaWjiJYSTjf0qCu9SysulbVRhwjnnfI0O54c4nf/uq1GmuDBeVYTg1F+f/AA6f4k6ogWLzOZ5m7uwH6Wbp90HfIxLZ61h2AnGmLp90XfIxK1651GdL2ZlwEAqGCeON7AcHhccEj045Lqxi5YSPpuntK2hn0LIi6dsr6e5UEFXRStmp5mcbHt6j8+ihtb6ni01axKWiWsmJZTwk++I3JPg0dT6hzIUwpynLZFcmzUqxpwdST4RIXy+2+xUgqLpUshaThrdy558GtG5PqWsL92n3Cr4o7LSsoYiCO+qMPk36hvvQfa5Ue4VtTcq59ZcJnVFS/Yvd0Gc8I6Bvo5c1wY9PReotNFpwW6ty/Q8PqHtNUqPZb8L1Ozc7jcLsXC6V9TVtduWSyeZ97HmfgXTZHGz9bY1vqA3/AD2VkpW0ztFv8qlljaLhsY4xIc93ywXDbn/l1UpVRgyBtPPK2MWEu4uFoLxk7EHOM9cfCtnvEKL2xguGcuVOtcLfOpnOCmxQvlyIWOe4N4iGAnYDn6liKKSVxETHPIBceAZwBzKtlto6OhqZYP8ASHVZtr5S/Le7y6MnAGM8iN88+iW2joqGplg/0h1YbbJKXkju8ujJwBjPIjfPMclLv4rOEVhp1R7cyKgQCMEDGPWsQN8mm76kL6eb65A4xv8AhG6tEtgpIqeNks/BPJTCcSvqYmtDiOIN7s+djpnx6L59yLeWsha6r8qdQeV8Zc3gB4OLhxjPj1z61bvdCa5WSqtbmk8xlj/Z92bXmobW9gdVC4U4AHdVTfOx6HjBz6TxLZOlu0C13yRlNLmgr37NhmcC2Q7e8dyPPls70YWjgfH1I9rXtLXAFruh/wAliudIoVlmHws3rL2iubd7anxI9RAr6ytSdm2tJY6mC0XiV0kUpDKaokOS09I3E889Dzztvsr/AHnVFlslUymutxgpp3s7xrHk5LckZx6wfgXlq9tUoVHTkuT3dpe0rukqtN8E5lFqTUfaXT0errXNaqxtbaDC5lXFH0JcMOHXiAHLqMjqMbUpKmGspYammkbLBKwPY9hyHNIyCPYsdSlOmk5LqbMZqXCNE9uX09Rfc+L5SZa/V/7cvp7i+50XykyoC51z4j57rfzkgrFYrJSS2qe73qolgt0b+6Y2EAyTPIzwtzt+Z5Yyq7zV1qo31/Zfb3Uo4xQ1MnlDW7luSSHerBHwrHTSfPoa1lBScm1nC6EJeI7AaMSWaa4idrwHRVjW+cMbkFowN/FdJtquDqXyltBWGm4eLvRC7hx45xjC+7JTNnudGaiNxojUxsmeRhoDnAYJ9S2NeL9bbXq6QSVOou8hkDfJY+DuCMYDQ3PIj/Pnurxgp8vg2aVtG4zUqfCUi2W+mqNJ3uulj4qmmfCIn8RHDxOweWx22S92+mpdO2Cqgj4Z6pkpldxE8XC4AbchzKlIHwS6V1e+mjdFA6pidHG8YLQZTgEZ28F1tTfShpX+Ln/C8I0lEmpShGk8en/Sq7oEQLCcgtPZd+2LYv4yX5CVelAV5q7Lv2xbFj65N8hKvShK36HgR732d+U/2zKZVa1zqM6XszLgKfyhvfxxvYDg8LjgkenHJTNtrqe5UEFZQytmp5mh7Ht6j+71dFm2tLd5HcUk3g7iir5fbfYqQVF0qWQtJw1u7nPPg0Dc+xR2ttTxaatne8IlrJssp4c++PUnwaOp9Q6haKuFZU3KufWXCd09S/m93QZzgeDfQPT4ro2GnSunufETjatrVOwW1cyLvfe0+4VfFHZaVtDERjvpwHy+sN96D6y5Um53GvuxcLpX1NYHblksnmf0B5n4PSusMDx9SsNKymdox/lUssbRcNjHGJN+75YLht1/u6r0KtKFpFNRzyeNnqd1fSalPC/BW2Rxs941rc+AwuaKKSUkRRue4N4iGDOwHMq5VUbXSAQTytjFhLg7haC8AnYg5x7PhXBbaOjoKmWAeUOqzbXyl+RwZdGTgDHLB55552Ud+iov4TF3GrKeHLgqkMUkrnCJjnkAuPC3OAOZXGWgjBAxywrdbKOjoamWAeUOqzbZJS8kd3l0ZOAMZ5Eb559FwS2Ckip42yzhk76bvxK+pia0PI4g3uz52OmfHorq/p7sNcFfd9VRzF8lXhb5NP31I59PN0kgcY3/AAjBVps2vdQWx7A6rFwpwMd1VN87A8HjfPpPEuM2e3lrIWuqvKXUHlnGXN4AeDi4cYz49cqudNvUrKlb3aacSVd3li01M3hpftAtl8kZTTA0Fc7ZsMzhwyHb3juR58tnejCuIOd+i8uOa17Sx4BDuhGR8C2V2ba0liqYLPeJXSRSngpqiQ5LXdGOJ5gnYHnk432XDv8ASXRXaUuUeq0n2hVzJUq/EjbaA+Kg7zqmzWWqZTXW4QU072CRrHk5LckZ5egj2Kgai7S6ej1dbJrVWtrbQYXMq4o+hLhhw68QA5dRkdduTTozqdEellUjDqbc8VQe2Ts9p+0bSj7a+byatid3tNUEZDH+BH709fYrtSVMNZSxVFNI2WCVgex7TkOaRsfVhc+BzWLoy/VHh2m/Q066fcxBLHQxUvFjyo1DSMZ58I872YXrns30hSaG0lR2OgcZGRDiklcMGSQ++cfz6BWk46pgdFO4YPNvb12DV+qr/LqHSb4TV1GPKaSVwZxPG3G1x23HPPXl6K72RfocrxS6lpblrUwU9JSPEraSKVsjpnDkCW7Bvj18PEetMIAp3DDAGAvNPb52D3HU+opNQ6QMT6ip/wBqpJXhmXY9+1x2weoPX8HpfmEwoTwMHmPsG7BLnpzUsOoNX9zHNSHipqWN4f5/R7iNtum/RelqqCKqppaeoYHwysMb2Hk5pBBHwLlxusqG8g8Za5/Q16mg1BUO0sKevtcry+MyTNjfED9S7OM+sehbt/Q+9k7uzq2VNTc5Y5r1WtAl7s+bEwb8APXfmeWy2/hAMdSpchgrXaJpOj1tpOusde50bKhuWStxmN45OHjg9OoyvIdZ+hp11Fc3QU8dDPSB2BVCoa0EePCTxfgXuJYwilgYNfdjPZ3B2caW9z2zCprp3d7VzgYDn42DR+9G+FsDPNZWju0HXOobVrK6UFtr2wUtO6NrGdwx2MxMcclwzzcVlo0Z3E9sepjqVFSWWbcgvdDNfqm0MmxXwRNmdG4Yy13UePTPrClF5Wl1Fdpr/He5qsuuUZbwyhobs3OxAxkb7+jYr0PofU1Nqmzsq4MMqGYZUQ53jf8A5HmD1HpyFmurKduk2Y6NxGq8I0d2o/ti33+Mi+QjVWVp7Uv2xr9/GRfIRqrLiV/Gz57qvzdT+TJyVK2Sx1d5751P3UVPCMy1E7+COMekqJ5FXK4Duuy+19wMMmrJHVBHVw4g3J9QHwKkEnyzFaUozcpT6JZIi8aflttI2pFbbq2HjEbnUc4k4SRkZGB4FQvL1rs2yl8tudJScXB38rY+L97xEDP/AEWz5rfbKO6mglj0qy1Rnun97MBVAYwSXcw7O/55V1De8rgz0rNXWZw+FI11RWk1VhuNyE3D5G6Nvd8OePjOOedlm52jyK0Wqu7/ALwVzXu4OHHBwkDnnfmp2mhip9Jaugp5BLDHUQsZI0hwc0SHB225BdfU30oaV/i6j+uEcUkWnbwjTb88f9KqiIsJzDYHYb9Pcv3Ol+UiW/itA9hv09y/c6X5SJb+K6VPwI+haF8nEIiK52AiIgCIiAIiIB0Vf1/k6G1CBzNuqPk3KfXWr6aOtoailnGYpo3RvHiCMH8aIpNZi0eS0XauVuqLRcKi31reGop3cDue/gRnoRuur1XNqJqTyfLbiEqdRxkE/uRFQwjPoHwJ6URAEREGRndERxGMnZSllrBaKcmkiyRTzQdmL+4mlh49QcLu7cW5HkmcEjpkD4FW5Zppm8M080jQQeGSQuGR132W3qTs7ra/s5t9v71lHXvrfdGYStJAyxzA3G2DwlufSCqhrDs/r9L2j3Qqq6lmiEjYy1jXA5d1ydvYvX6bXpQgoz6nvoUKsaUF9iT7H9VTW27x2Wp7yWirJMRgAkxSb77DPCevQc/HP12o1j6rXFXC8ngo444WNJ8Wh5I9PnDPjhWzsi0YbVSNvNzhLbhO39SjeN4Iz4+Dj18BttuFDdsFnkpb5Fd42/6NWMEUjh9TK0bZ9bcD/wAvqVrerRd/ldP+mDV6VV6fJR6/8KDsiIvVfc+bnP5XN5F5Jx/6P3ne8GB77GPWuYXSsHKb/wAP5L70frX73l+HmunnbCmTbaeKx09bIyql74P4pYiO7icDhrXDHU46jmMZWtW7KmlvWcs2qKq1M7X0Ouy9XBlOIWzjg7vuc920uLNxwl2MkbnZGXq4Mp+4E44O67kkxtLizGOEuxkjfkuevsNTSuomxOjndVRNeGsewlpOegdu3b33JZtunqmquNNTTPjhjqA5zZg9kjSB4EOwTnmOY58gsG6127sIz7bvdt5Om651UkDIZHRSMYzu2mSFjnBu+wcRkcz12XyLlViUSd754h8nB4R+t4xw8vDrzXLDaK2bi7tkJa1wj4+/j4C4jOA7iw4+ge1dGSN8T3RytcyRh4XNdzB/zWamqEm1FIw1HXgszbPjmgROmVsmqYlDnNPCS1+PNcNi09CFKdp1c653ey18oDZKqy0szh0Bc6U4/CunRUM90roKCjbmpqXCNu2eHxcfQACT6j6Fum7dnViurqN1Uypa6kpmUkfdSlo7tmcA49ZXC1O4p0q8JPy6ntPZqhUnQn6M8546ezx5f3LcvYPW3OWlraV7OO0wnMcjj7yQ7lgHUb5Ph7VB6m7OWM1dbLTYROyKogdLPLK4vEbWuA4s+O+AOpxy3W6LJbKWzWynoKCLu6eFvC0dT4knqSdyuff3lOrTUYrl/wBHpba3lCeWaR7cvp6i+58Xyky1+thducb260ppSDwPoI2tPpbJJkf+oLXq8zceM8Vra/8AZIKQtN3r7PMZrbVPge4YOMEH1g7H1qPwUWBNrlHMhOUHmLwS931HdrxC2K41j5YmnIaGtYM+kNAyuyzWN/jpmwC5Sd21vCC5jS4D/eIzn05yq/zRTvlnOTJ3mrnO47bLjVMo6qlbOe4qi10zSA7jIORknfmsVNwqaqlpaaeTigpQ4Qt4QOHiIJ9J5BdVFGX0KdrLGMhPWiKDGi0dl37Yti/jJvkJVsrt3nmg0tbu4mki469rXd28tyO6lODg8sgfAtb9ljC7tEsvCMhrpXO9A7mQfjIC3J2k6XqNWWalo6Wpip5IakT8UjSQQGPbjA/3s+xdaycYODl0PdaGm7HC9TzjLNNM3hmnmkaCDwySFwyOu+y2F2P6qmtt3jstT3ktFWSYjABJikPXA34T16Dn45i9Ydn9fpe0e6FVXUs0QkbGWRtcD53Xf8S2F2RaMNqpG3i5wltwnbiKN43gYfR0cevgNttwu/d3FvOg9pu0KdSNTkqfajWSVeuKuF5Pd0cccLGnlu0PJH9IA+OFVAr92wWd9JfYrtG3/RqxgikI+plbyz627cvqfUqF1XW0qUZW0VDyPCa/TnG9m5+YHXIXN5VN5F5Jx4p+873hwPfAYXCcJ05rfcVLqjjRk10O6LpWdJv/AA/kvvR+tfveX4ea5GXq4Rwdy2ccPd9ySY2lxZy4S4jJG525LsG2U8Vip657aqXvg/iliI7uFwOGtcMdT6RzGMrFfYKmldRNifFO6pja8NY9hLSc9A7du3vuS0VO2k8OKRv9ncwjuT4OBl6r2U4hbOODuu5yY2lxZjHCXEZI3Oy4n3OqfAyCV0UrGM7tpkhY9wbvgBxGQNz12Xdt2n6mquNNTSujhjnDnNmEjHtIb4EOwTnGRzHPkF1obPWzcfAyIta8R8ffx8BcRnAdxYccdB+BTutM4WCHG7xl5OL3SqxIJO988Q+Tg8I/W8Y4fgXTX3JG6KR8crHMkYeFzXbEFfC3IQilmKNOcpPiTCxIHOa7hJa/mHDYg9CD47Z9az1XPRUM9zroKCjBNTUvEbdve88uPiAN/Zjmq15RjBuXQyWsJzqxjDrk7fadXPud3stfKA2SpslLM4dAXOlOPhKqOOns8eX9y9GXfs6sd2dRuqm1LXUtMykj7uUtxGzOAcf7xWvdTdnLGautlpsAnbFUQOlnlmcXiJrXY4vXvjHU45DJXm7O/pQjsaPptS2qP4ib7B6y5y0lbSSM47RCQY5HH3khOSxviN8nfbPpW210LJbKWzWynoKCIR08LeFo6+JJPiTuT4qQXGr1FUqOSWMnRpQcIpMIiLEZAiIgCIiAIiIAiIgCYREBhaN7QNDaiuutLpcLdQCekqHRuY/v2NziJjSME55tPRbzXyQs1CvKhLfDqY6lNVFhnlSo0/daa+x2aejcLnJw8MLXB2eIHfIPLAOfAb8uXobQumKfS1nbTRAPqpcPqZh9W7wGfqRyA9JPMknvw2Ghi1DU3oMc6vnibCXuOQ1reg8M9fUpZZ7q9ncRUX5GKjbxpNtHmztR/bFvv8ZF8hGqsrT2pAjtFvmer4SPvEaqy4dfxs+e6r83U/kdMqesOoDb6Ce31lHFcLZOeN0D3cJDvFrunIf3KB6+hOR2Kxxk4vKNSlWlSlugyduF4oHQsZZ7PHb5GyNk7/vnSvBG44SRtuu5VaktldO6rr9OxS3BwBklbUvaxzhyJYNj6vwqrLJ/ApVSSM/fan5JWmvJhst0t/k7cVz43lwdgR8LuLluSOnNfNyuxrrRa6Ew92KFrwH8WePjIPLG2MfhUWNuSepRvbWDE7ibW3IREVTCbA7DPp7l+50vykS38tA9hn09S/c6X5SJb+XSp+BH0PQ/k4hERXOuEREAREQBERAFjCyiAqetdFW/VMTXVBdBXRtLY6mMecB4OB983O+PXgjJWsKjsi1CyZwpqu1zRD3r3yyMJ9bQwj8JW+kUOKfVGhc6bb3L3VI8mgfoSan+u2f4xJ80n0JNT/XbP8Zk+aW/kUbIehre4rP6TQP0JNT4/XbP8Yk+aT6Emp/rtn+MyfNK49p3aAy095abLK11zIxLKD+sA749L/R0G/gDU+zbtCfZ5WW++1DpLe93mTyOy6Bx8cnPD19GfDONqOnylTdVR4MD0qxjLY48nF9CTU/12z/GJPmk+hJqf69Z/jMnzS33G8SMa9hDmuGQ4HIOV9rV7OHoZ/cVn9JoH6Emp/r1n+MSfNq7aK7MqWy1cVddJxX1kZDo2BvDFGfHG5c4HOCfXgHdbHQAZyijFdEZqGk2tCW6MeRwjoupc7dR3OGKKvgbPFHK2ZrX8uJpyD7Cu6EVjonyAupcqCmuVFNS1sLZaeVvC5jhsfz556Hku4eSx6+SJtPKIaTWGaI1boe4WEyT0zH1ttAz3jBmSMZ+rA9fvhtsScKpNc1w4mEEHlghepMbbqqX/QVjvDnSmnNJUu5z0pDHE+JHvXH1grvWmtygtlZZR5XUPZmFZudu8P0NEYKm7PcaS25mY6rdK+JzJKdwBikccjJdnlg5xjORzKstz7LLnBk2y401U3OQydhicB/vDIJ9gVeq9GampuLvLPK9rfq4JWPB9Q4g78C6jvbW5STng4Hui/tJZjDJn3Utss9smq4JpDS07aeSIxtc08Idhw333I80jC7Y1FSRVFoc2OV/kT5TI4Qxw8QeMbNacA/j9qgZLReIx59mu2B4UUjvxBfDbVdScCzXf20EwH4W4UdlaN+P+yv/AL0muz/olqS60kFsdQ8Tw1kxljlfRxSkgjBBa44aQRzB5KHr6h1VWTzySOkc9xJe5gYT7ByK71LpnUFUT5PZK8kfXGiL+uQpug7N9RVTmmobR0TOTjLKZHAegN2P9JRGtaW8nNT5LOx1C7ioOnwU71ruWa1118q/JrVTOqJAQHuG0cYJxl7uQ8epI5AraNn7LLbTua+7Vc9wcOcY/Uoz7B53sLiPQr7Q0VNQU7YKKCKnhb71kbA1o9gWnc64sYoo6lj7LPO65f8ApFb0NpCn03AZZHNqLjK3Es+NgP3rfBv4SefQC24WMcllednOVSW6XLPZUaMKEFCmsJGcBYx0WUVTKUXtU0o/Ulnjlomg3GjLnQtJ/XGnHEzngZwDnxHQElefpY3QzPimY6OWN3C9jwWuaR0IO4K9ckbKCv8ApWyX88V1t8U0nLvW5ZJjw42kOx6MrHUpqZw9T0eN498XiR5g3Reh/oW6R+xs3x2f8tPoW6R+xs3x2f8ALWHu33ON+ma31o88IvQ/0LtJfY2X47P+WujLoPQcV0htslPwV8zDIyA18/G5o644/X68HwOCtc+Y/TNVf5o0Mi9DDsu0jn9jZfjs/wCWn0LdJfY2b47P+Wndl6k/pmt9aPPKOIaNzheh/oW6R+xs3x2f8tSNl0Rp2yTie32yNs4OWySvdK5v+6XkkbeClWyzyy0PZmefjnwVTsf0fUWkS3i5xmKqnj7uGFww6OMnJLh0Jw3bmMekgbQWABssrYSSWEeqtreFtTVOHRHSuVtpLnFFFXwNnjjlbM1j9wHtOQfTgru4Cyik2Dp3KgprlQzUlbC2anlbwuY4c/z8eh5LSWrtD3CwOkqKVr622jfvGDMkY/hgdP4QzyJOFvgrBHT8a2rW8qWsswZz7/TaN9DbUXPqeXGuDgC1wLTywUW+L/oKx3hzpTTmkqXbmelIY4nnkjHC4+kgqkXPssucHE62XGmqm5zwTsMTgP8AeGQT7AvR0NaozWJ8M8ZdezFzTeaXxIrVnuNJbeKZjqt0r4nRyU7gDFI4gjd2eWN8YzkcyuU3S2zTWyarhle6lp208kRja5p4Q7DgcjPMeaRhYqtF6mpeIyWeWRg5PglY8H2cQd+BRslovDB59muwAPSikP4gsn/kqPdv5/k1XQv6SUHT4X2J4ahpIai0OZHLIKJ8plIhjh4uIY2a04B/PqurSXWjgthoeJ4YyYyxyuo4pSQ4YILHHDTkbEHkoltrurjhtlu/q8gmH/8Ayu3S6Z1BVH9QsleSPrjBF/XIVXTtEsb/AO0WitQlLKp/0dG4TuqqyaeSR0rpHEl7mBhPsHJdfG25/Crhb+zfUVU5vlDaSiZyd3kpkcB6A0YP9JW2zdlltp3B92qp7g4c4x+pRH2DzvhcQpnq1tRjti84L0fZ68uJZmtpq6zWutvdZ5Naqd1RICA93/w48nGXO5AdepONgVurQ2kKfTdOZZC2e5StAlnxyH71vgPwk8+gFjoaKmoadsFFBFTwt5RxMDQPYF2cLgXupVLr4eiPXaZotGx+LrIzhMBZCLnHaMcKyiIAiIgCIiAIiIAiIgCIiAIiIAsYWUQDCwSsoUBpLtu0/LFcor7BGXU8zGw1JaPePB81x9BB4fRgDO61cOS9b1NPFUwSQ1EbZYXtLXseA4OB5ghayvXZBQ1M5ks9xloATkxSR9+wDbYbhw5eJWGrR38o8vquiTuKna0fM0qmy2v9Bep+2GH4gfnU+gvVfbFD8QPzqwd2kcf9P3np/ZqhFtf6C9T9sUXxA/Or4HY7KZnRDUtOZWgOcwUJyGknBx3ucHB39Cd2mPcF56L8mq0W1/oL1Ofphh+IH51PoL1P2xQ/ED86ndpD9P3np/ZqjZM8jlbX+gvU/bDD8QPzqmtO9k1toKhk92qpLm9hyI+DuojyO7cknlyJIOTspVs88syU/Z66lJKXCOn2IaelpIKi91cZY6qYIqcOGCYuZd6nHGOWzc9QtsL5a0NADQAB0C+1tpYWEe0tbeNtSVKPkERFJsBERAEREAREQBERAEREAWFlOqA1L28UdNDZ7dNFTwsmkrPPkawBzsscdz1VQ7G6aCr1p3VTDFNH5JIeGRgcAeJnj13PwlXbt+/YG1/yz/Dcqf2IfTz/ADST+sxdqi33GRzqiXeEb/YxsbWsYA1rRhoGwA5YXIhRcU6IREQDCIiAJhEQDCYREAwmERAY4VnAREAwFjCyiAYTCIgGEREATCIgGEwiIBhPYiFAQ+paq5UdomlstvNwrzhscRkaxoP75xcRsPAbnltzHmCuuFdPc5q+tnmNwEvG+Xi88PbtsRyxjpttjkvWpGxXkS6f7XX/AMdL/WK62lbW5ZXkaN5lYwelNAXC83CxxO1DQGmqmhvDKHsIqGkZD8NJ4T4g48R4C0qO0+P9RW7H/wAvH/VCkVy543PBuQXAwmERVLDCIiAIiIAiIgGEwiIDGAmFlEAwPBYwFlEBjAWcIiAYREQBERAEREAREQBERAEREAREQBERAEREAREQBMIiAYWMLKIBhYWUwgKzrrUFTp+yvqKG3VVdVvyI2wwPkYzA3e8tGzR4bE8ttyPPdv1LdqTUPu3DVPluEjvPLgXCUE44C0fU+AGCNsej0zqAf6iuP8nk/qleU7X/ALXQfxsX9YLr6bCEoT3LyNC6bUo4Z6m01dnXm0wVktDV0ErvNfT1UTmPY4cx5wGR4H/qFLr5YMAL6XIfU3l0CYREJGEREAREQBERAEREAREQBEKxlAZRRFdqSx0ExirrzbaaXJ8yaqYw/ASuv+nLTP2xWb49F/mmGRlE+sZUD+nLTP2xWb49F/mn6ctM/bFZvj0X+aYGSl9vv7A2v+Wf4blT+xD6eh/I5P6zFeO0KXTerbdS0rNX2WjdBN3vGZ45c+aRjHGPHPPoobQdq07pe+m4ya4slWO5dEI2yRx8y05z3rv3vL0rp0rmEbV0n1ZpzpN1lM3HlZwoD9OOmPtjs3x6L/NP05aZ+2KzfHov81zMM3Mk/hMKA/Tlpn7YrN8ei/zWRrHTTiANQ2ck7YFbFv8A+pMMZRPIuGCeOoiZLTyMlid717HcQPqI5rmQkIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIDB5FeQ7p/tdf/HS/wBYr14eRXkO6/7XXj/96Uf+o/5LraT4p/waN70WD1ZYP2Ct38nj/qhSPRR1g/YO3fyeP+qFI9FypdWbkfCgiIoLBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREBH6g/YK4/yaT+qV5Stf+1UP8dF/WavVuoP2CuP8mk/qleUrXjyuh3/+LF/Wauxpfgn/AAaF34onrwdFkLA6epZC45vLoEREJCIiAIiIAiIgCIiAIiIAoLU00nBbqGKR8TrhVCndIwkODQx8jsHmCWxuGeYztvup1V/Un7MaW+6T/wCyVKldSr6ErRUFLQwiKjp4oIxtwxtDeXqXawEULTahpJ9WVunmMnFbSUsVZI8gd2WSOc0AHOc5YcjHhuoyycImsBMBMpndBgxwhOELOfQmUGBgLOB4KD0zqOk1Ey5Ooo52C3101vl70AZkiIDi3BPm77cj6F37pcaS00FRXXOpipaOnaXyzSu4WtHiShODuYHgsFoLSCMg9FWNBa1t2t6KvqrRDWxQ0dW+jcaqHunPc1rXcQaTkNIeCOIA+ICtPRORgrVTBFZ9QWt1DG2GK4yvpp4WDDHPET5GyY6OAjIz1B3zgYsqgNR/sxpf7pP/ALHUqfUsqgiIoLBERAEREAREQBERAEREAREQBN0R3JAQ1dfI4Kp9NS0tXX1Me0kdKwHuydwHOcWtBx0JzuDyK4fd24fatefvtJ8+saFxJpG01Lv1yqp2VUh8Xyjjcfhcp9T0KkD7u3D7Vrz99pPn093bh9q15++0nz6nvgRMk4ZAm+3D7Vrz99pPn1BSW21yyOfJ2bvfI4lznOp6AknxJMu6ve6xuiljoQ456lfjvNbGxrI9K3hjGgNDRJSAADp+v8l9+7tw+1a8/faT59ToKymScMgfd24fatefvtJ8+nu7cPtWvP32k+fU7nZPWmSMMg4tQtErGXK311t43cLXVLWFhJOAOON7mgnpkjJwOeyncrr1tLFWUc1NUND4ZmOje075BGCo/R9TLW6TstVUPL556KCWRx+qc6NpJ+Eo+hK4JnKIigkIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCFEygON8jWNLnkNaBkknAAUNLq3TkMjo5b/aWSN5tdWRgj2ZXDWxtuupjbqkcdFSU0dS+F3vZXyPe1nEOob3Tjjlkg8wMT8cUcUYZGxrGN2DWjACngryQn6ctM/bFZvj0X5Sfpy0z9sVm+PRflKd4QmBlOByQX6ctM/bFZvj0X5Sfpy0z9sVm+PRflKd4QhA8E4HJXKvVWlqqkmp5NR2fglY6N2K6LOCMeK11S6P7PqeeGRutGu7p7Xta65UuPNORnDQei3RgeAThCvCrKHEXgrKCk8sgv046Z+2KzfHovyln9OOmftjs3x6L8pTmAs4HgqcF+SC/Tlpn7YrN8ei/KT9OWmftis3x6L8pTvCPQnCPQnBHJBfpy0z9sVm+PRflINY6acQG6is5J2AFbFufhU7gLBaCMEZBTgnk44KiKohbLBIyWJ3vXsdkH1Ec1zKtVMEVn1Ba3ULGwxXGZ9NPCwYY54ifI1+OjgIiM9Qd84GLKoYQREQkIiIAiIgCgNR/svpb7pP8A7JUKe6KA1J+y+lz0Fyfn4pUKV1IfQsHRarq9+1bW+Ptapv69QtqdFEyaetj7rW3J1NmtraZtHUS947z4mlxDcZwN3u3AzvzUEmh6CyW+09hGkdTUNO2PUMDrfLHcSc1AD5mRuj7wni7rge5vd54eE4wrc2zWy+9vOpaW8wx1lKy0UcnkM+HwSu45AHviOQ8tzsSDw8WeeMbAdpCxu0tTacND/qal7sQ0/fSeb3bg5nnZ4ti0Hn61Up9Cm69rV5vN2opW0D7fTQ0lbT1boJmyNMneBr4ntkaMOGdwHDxwgKLO98WiK+gikL7VatcU9HRZcXCKBtRCRGDn3rXOcweGAByV91H3FR236Xppu6kDrNcBJC/Dssc+Ebt8Dg8/Aq1N0hYBpV2mxbIBZXM7t1NvuM54i73xdnzuPPFxednO64bVoiwWq8Q3ako5XXWKN8IrJ6qWeZzHYy1z5HFzwOEY4ieHfGMlAUvsP07ZKSbVVZSWe3QVlPqCvpoZ46ZjXxxB4Aja4DIaP3vJbPrKSmrYDDW08VRBxNeY5mB7ctILTg7ZBAIPQjKjrfpe1W291l1oIZ6errHGSobHVSiGR5ABeYeLu+Mhoy7hz6dypC50EVyoJaSofUsikwC6mqJKeQYIOz43NcOXQjw5ICgdjX+39oX/APaKr5OJbLKq2ndC2PTlfNWWhtyimnkdNMJLrVTMlkcMF72PkLXOIA84gnYK0lAQGo/2Y0v90n/2OpU/1Vf1Gf8AXGlvui/+yVKsHVSyqCIigsEREAREQBERAEREAREQBERAFh3JZWHck8yGQOgfpG079zqf5Jq6vadc6yydnmo7nbJu4rqShmmgk4Q7ge1pIOCCD7crt6B+kbTv3Op/kmqN7XKSpruzDVNJQ08tTUzW6aOOGJhe97iwgANG5J6AI+oRwVmuqa111jtFRRXO4Xi5UJqoWUkLXd4W8PE0kuDWnzs5dwsGDlwOAfs9oVsZpe+3qopLhTusbnx3ChkYzyiFzQHYwHljstc0ghxBB57EKFbba76KujazyOp8lgsNRDNMYncEchMWGudjAJwdjvsqxruhqqbTvbXUVNLPFBUshdBLJG5rZQKVjSWk7OAIIyOowhJsKj1/QVNba2SW+6UtBdXd3b7lURMbT1Ty0ua0AOMjS4AlvGxodjYnIzKWrU1HdJ79DBHUMdZqk0s/eBvnuEbX5ZgnIw4c8bgrXtwqazWNFoi0UtjutLUUdfS3CvkqqZzIaaODzsNmI7uUucWhojLjgknGCuzST12mr7r2kdZLrXVN1qhW240tM90NQHwNZwGYDu4i10ZB7wt2IIzlAY1Z2h177PoW7aYt1zdRXu40/HgU3FJE9rz3BD5PNe7AORhox74cjs+gqJaqjinnpJ6OR4y6CcsL4z4Esc5ufUSPStMQ2i6UnZL2YPltle6WzV9DVVtNHTvdPFG1rmOd3QHGSC8ZaATjJwtz0NVHW0cVTC2ZkcrQ5omhfC8A/vmPAc0+ggFAUvtAhvtssV8v9Jqqai8hp31MFIKSA03DG3i4ZS9jpHFxBBLXt2IwARk2jStwmu+mrTcaqHyeoq6SKeSH6257Q4t9hOFrTW90k1HrF9ivVuvlPo+3OZLU9zZ6uf3VlG4jDoo3N7hpwTk5cRgbbra1uq4q2ihqaeOZkUjeJrZoHwvA8Cx4Dmn0EAoDtHkoHQP0i6d+5tN8k1Tx5KB0F9IunPudTfJNU+RHmT6LCyoJCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAsLKdEBAUX093f7nUfytUp9QFH9Pd3+5tF8rVKfUshFL7Q79drTW6ZobG+hinu9wNG+asp3zNjb3T35DWyMJOWAc+qXaputrpLSL5qaipqie6wRNkpLU8MqGu5U5a6SQtLiD+qZAHo5qO7VbLPervoeKOnr5KWO7mSoko5JYnQs7iUcRkjIcwZIGcjnhcWu9OupaDS1NaIbnWRxaio6qUzVE9a9jATxOLpHOcGAY64Geigkm7l2iaYt09dDU3CTvKCXuqzu6SaUUpwDmUtYRG0h2z3ENODgnhOJK9aotVnMQq5aiV8sfetZRUk1W/ux9WWwtcQz+EcDKpNHaawWPtXikoKjjrqyqNO0wuzUNNHE0Fox5wLgQMZ3BC6sldqGjZpu3Tv1BbbW2xwu76020VU8tZhoMMvHFIIgBy4g0Ek5d5pQFguuqJJ9X6CZZK+OWzXltVJIYw1wnY2DjYQSMjfwI8D4KT17qCr08ywOoo4ZDX3emoJe9aTwxyE5LcEedttnI9C1voax3WkZ2TtqbZXQmgNyFV3sJBp+Jrw3vCBhudscgenRXftUoquui0oKKmnqDDqCjnl7qMv7uNpdxPdgbNGdydggJS8a4sFouVTbqyrmdcKeNs8tLTUk1RK2J2cScEbHHg2wXYwMgEjIz9V+ttO0NutddU3SFtJdP9ikY1zxUHg4g1gAJLiBs3mTgAEnCjLDQ1Mfatq2tlpZm0s1Bb44Z3RkMkLTPxBrsYJGW5xyyPQtcUYqbBY+yUXC01z5aS4VhfSNgPfNb3VQeIRnfZuHBo3IGGgnAQG1hrrTvuFcru+tfDR2w4rRNTSxzU52xxwuaJBkEEebuDkbKTuF9oLfXUlFVTPFXVxTTwRRwPkdIyIAvwGg5IDm7cznYFap1NYbnq629pNwtlDWww3Sgp6Shhq4HU8lQ+Djc53dyAObkuDGlwBOCeWCpyO4VGoe0nR9fR2a9QW6jo61k9TW0MlMGyPbFiPheA7Pm88cJ5NLiDgCf7Pdc02sqWokhoK+jlhnniImpZ2MLY5XMBEj42t4jjJYCXN3B5FXFa/wCyfvrdSXWy11FX01bDcq2oLpaSVsUkclQ97HMlI7t2Q8bBxPPI2WwPFAQGo/2Y0t90n/2SpU/1UBqP9mNK/dF/9kqVP9UIXUIiISEREAREQBRl7oHV1KwQyCOpgkbNDIRkNe3xHgQSDjfBPJSaYTOCMFe92bnF5kumrhJI33zqeendGT/BL5Gux62hfXu7cPtWvP32k+fU/hYU5GGQPu7cPtXvP32k+fT3duH2rXn77SfPqeRMjDIH3duH2r3n77SfPp7u3D7V7z99pPn1PImRhkD7u3D7V7z99pPn093bh9q95++0nz6nkTIwyB93bh9q15++0nz6e7lw+1a8/faT59TyKMjDIGkp6u43KmrrjT+SR0od5PTF4e/jcMF7y3LQeHIABPviSdwBPrGN1lTnISCIigkIiIAiIgCIiAIiIAiIgCIiAI7kUQoQyv6C20VYoz76Kihid6HNYGkfCCrAoGW011PNLJZK+GlZM8yPhqKbvow8ndzQHMIJ3J3IJ3wCTnHk2p/stZv+Fy/8wpaIyTyiNV2Gl1Ppu42Wvkmjpa6EwyPhIa8A9WkggH2FcHk2p/stZv8Ahcv/ADCeTan+y1m/4XL/AMwowTkl6SnZS0kFPGSWRMEYLtzgDHw7LnUD5Nqf7LWb/hcv/MJ5Nqf7LWb/AIXL/wAwmBknvamFA+Tan+y1m/4XL/zCeTan+y1m/wCFy/8AMKcDJPIoHybU/wBl7N/wuT/mE8l1P9l7N/wuT/mEwMk1LI2OJ73kNY0FxJ5ABQ2hmOj0Vp9kgw5tvp2uB8RG1cb7TdK5vc3q5081I738NJSGDvR1a5zpHnhPXGM8uRIVgAAAwnkQZWURQWCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAsLKHkgK/R/T1dj422ix99qv8wp/KhrpSVMdwiuluYyWobGYZYXu4O+jzkYdjZwOcZ2PEQcZyOH3cuA2Ol7wehxLSY+XU9SM4J9PUoH3duH2rXn77SfPp7u3D7Vrz98pPn0wNyJ5FA+7tw+1a8/fKT59Pd24fatefvlJ8+mBuRPIoH3duH2rXn75SfPp7u3D7Vrz98pPn0wNyJ5R9faKG4XC3VtXB3lTbpHTUr+Jw7t7mOY44BwfNcRvnn44XR93bh9q15++Unz6e7tw+1a8/fKT59MDcieRQPu7cPtWvP3yk+fT3duH2rXn75SfPpgbkT2PWs+lQHu7cPtXvP32j+fT3duH2rXn77R/PqMDJjUZzeNLbf/AKi/+yVCsCgKSnrLjcqauuNP5JHShxp6YvD38bhgveW+bnhyAAT74kncAT6lhBERQSEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREATCIgGEwiIBhMIiAYTCIgGEwiIBhERAMIiIBhERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAMJhEQDCIiAIiIAiIgCIiAIiIAmERAYxusoiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiA//2Q=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Slide24.jpg](attachment:Slide24.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SentimentNet(nn.Module):\n",
    "    def __init__(\n",
    "        self,\n",
    "        vocab_size,\n",
    "        output_size,\n",
    "        embedding_dim,\n",
    "        hidden_dim,\n",
    "        n_layers,\n",
    "        drop_prob=0.5,\n",
    "    ):\n",
    "        super(SentimentNet, self).__init__()\n",
    "        self.output_size = output_size\n",
    "        self.n_layers = n_layers\n",
    "        self.hidden_dim = hidden_dim\n",
    "\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
    "        self.lstm = nn.LSTM(\n",
    "            embedding_dim, hidden_dim, n_layers, dropout=drop_prob, batch_first=True\n",
    "        )\n",
    "        self.dropout = nn.Dropout(drop_prob)\n",
    "        self.fc = nn.Linear(hidden_dim, output_size)\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "\n",
    "    def forward(self, x, hidden):\n",
    "        batch_size = x.size(0)\n",
    "\n",
    "        x = x.long()  # (100, 200)\n",
    "\n",
    "        embeds = self.embedding(x)  # (100, 200, 400)\n",
    "\n",
    "        # lstm_out: (100, 200, 512)\n",
    "        lstm_out, hidden = self.lstm(embeds, hidden)\n",
    "\n",
    "        lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)  # (20000, 512)\n",
    "        out = self.dropout(lstm_out)\n",
    "        out = self.fc(out)\n",
    "        out = self.sigmoid(out)  # (20000, 1)\n",
    "        out = out.view(batch_size, -1)  # (100, 200)\n",
    "        # get last timestamp output of every instances in the batch\n",
    "        out = out[:, -1]  # (100, 1)\n",
    "        return out, hidden\n",
    "\n",
    "    def init_hidden(self, batch_size):\n",
    "        weight = next(self.parameters()).data\n",
    "        hidden = (\n",
    "            weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device),\n",
    "            weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device),\n",
    "        )\n",
    "        return hidden"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please note that we can actually load pre-trained word embeddings such as GloVe or fastText which can increase the model's accuracy and decrease training time.\n",
    "\n",
    "With this, we can instantiate our model after defining the arguments. The output dimension will only be 1 as it only needs to output 1 or 0. The learning rate, loss function and optimizer are defined as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SentimentNet(\n",
      "  (embedding): Embedding(57231, 400)\n",
      "  (lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)\n",
      "  (dropout): Dropout(p=0.5, inplace=False)\n",
      "  (fc): Linear(in_features=512, out_features=1, bias=True)\n",
      "  (sigmoid): Sigmoid()\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "vocab_size = len(word2idx) + 1\n",
    "output_size = 1\n",
    "embedding_dim = 400\n",
    "hidden_dim = 512\n",
    "n_layers = 2\n",
    "\n",
    "model = SentimentNet(vocab_size, output_size, embedding_dim, hidden_dim, n_layers)\n",
    "model.to(device)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "57231"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.005\n",
    "# Binary Cross Entropy\n",
    "criterion = nn.BCELoss()\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can start training the model. For every 1000 steps, we’ll be checking the output of our model against the validation dataset and saving the model if it performed better than the previous time.\n",
    "The `state_dict` is the model's weights in PyTorch and can be loaded into a model with the same architecture at a separate time or script altogether."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run if you haven't created the models subdirectory\n",
    "# !mkdir models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1/5... Step: 100... Loss: 0.407154... Val Loss: 0.340196\n",
      "Validation loss decreased (inf --> 0.340196).  Saving model ...\n",
      "Epoch: 1/5... Step: 200... Loss: 0.201936... Val Loss: 0.312401\n",
      "Validation loss decreased (0.340196 --> 0.312401).  Saving model ...\n",
      "Epoch: 1/5... Step: 300... Loss: 0.263954... Val Loss: 0.261425\n",
      "Validation loss decreased (0.312401 --> 0.261425).  Saving model ...\n",
      "Epoch: 1/5... Step: 400... Loss: 0.237902... Val Loss: 0.246571\n",
      "Validation loss decreased (0.261425 --> 0.246571).  Saving model ...\n",
      "Epoch: 1/5... Step: 500... Loss: 0.288367... Val Loss: 0.235317\n",
      "Validation loss decreased (0.246571 --> 0.235317).  Saving model ...\n",
      "Epoch: 1/5... Step: 600... Loss: 0.215628... Val Loss: 0.274795\n",
      "Epoch: 1/5... Step: 700... Loss: 0.199744... Val Loss: 0.237176\n",
      "Epoch: 1/5... Step: 800... Loss: 0.140447... Val Loss: 0.218970\n",
      "Validation loss decreased (0.235317 --> 0.218970).  Saving model ...\n",
      "Epoch: 2/5... Step: 900... Loss: 0.147644... Val Loss: 0.233868\n",
      "Epoch: 2/5... Step: 1000... Loss: 0.190990... Val Loss: 0.243832\n",
      "Epoch: 2/5... Step: 1100... Loss: 0.301946... Val Loss: 0.236099\n",
      "Epoch: 2/5... Step: 1200... Loss: 0.156925... Val Loss: 0.233492\n",
      "Epoch: 2/5... Step: 1300... Loss: 0.150212... Val Loss: 0.225436\n",
      "Epoch: 2/5... Step: 1400... Loss: 0.190430... Val Loss: 0.228162\n",
      "Epoch: 2/5... Step: 1500... Loss: 0.142075... Val Loss: 0.216548\n",
      "Validation loss decreased (0.218970 --> 0.216548).  Saving model ...\n",
      "Epoch: 2/5... Step: 1600... Loss: 0.202155... Val Loss: 0.225895\n",
      "Epoch: 3/5... Step: 1700... Loss: 0.084862... Val Loss: 0.232592\n",
      "Epoch: 3/5... Step: 1800... Loss: 0.175746... Val Loss: 0.245497\n",
      "Epoch: 3/5... Step: 1900... Loss: 0.080264... Val Loss: 0.246931\n",
      "Epoch: 3/5... Step: 2000... Loss: 0.075748... Val Loss: 0.253358\n",
      "Epoch: 3/5... Step: 2100... Loss: 0.136083... Val Loss: 0.237236\n",
      "Epoch: 3/5... Step: 2200... Loss: 0.158798... Val Loss: 0.231686\n",
      "Epoch: 3/5... Step: 2300... Loss: 0.177989... Val Loss: 0.228364\n",
      "Epoch: 3/5... Step: 2400... Loss: 0.167863... Val Loss: 0.235899\n",
      "Epoch: 4/5... Step: 2500... Loss: 0.044877... Val Loss: 0.303135\n",
      "Epoch: 4/5... Step: 2600... Loss: 0.115482... Val Loss: 0.263831\n",
      "Epoch: 4/5... Step: 2700... Loss: 0.204895... Val Loss: 0.260528\n",
      "Epoch: 4/5... Step: 2800... Loss: 0.043700... Val Loss: 0.267417\n",
      "Epoch: 4/5... Step: 2900... Loss: 0.178460... Val Loss: 0.243706\n",
      "Epoch: 4/5... Step: 3000... Loss: 0.141615... Val Loss: 0.244334\n"
     ]
    }
   ],
   "source": [
    "epochs = 5\n",
    "print_every = 100\n",
    "clip = 5\n",
    "valid_loss_min = np.Inf\n",
    "\n",
    "counter = 0\n",
    "\n",
    "model.train()\n",
    "for i in range(epochs):\n",
    "    h = model.init_hidden(batch_size)\n",
    "\n",
    "    for inputs, labels in train_loader:\n",
    "        counter += 1\n",
    "        h = tuple([e.data for e in h])\n",
    "        inputs, labels = inputs.to(device), labels.to(device)\n",
    "        model.zero_grad()\n",
    "        # inputs.shape: (100, 200), h[0].shape: (2, 100, 512)\n",
    "        output, h = model(inputs, h)\n",
    "        # output.shape: torch.Size([100]), output.squeeze().shape: torch.Size([100]), labels.float().shape: torch.Size([100])\n",
    "        loss = criterion(output.squeeze(), labels.float())\n",
    "        loss.backward()\n",
    "        nn.utils.clip_grad_norm_(model.parameters(), clip)\n",
    "        optimizer.step()\n",
    "\n",
    "        if counter % print_every == 0:\n",
    "            val_h = model.init_hidden(batch_size)\n",
    "            val_losses = []\n",
    "            model.eval()\n",
    "            for inp, lab in val_loader:\n",
    "                val_h = tuple([each.data for each in val_h])\n",
    "                inp, lab = inp.to(device), lab.to(device)\n",
    "                out, val_h = model(inp, val_h)\n",
    "                val_loss = criterion(out.squeeze(), lab.float())\n",
    "                val_losses.append(val_loss.item())\n",
    "\n",
    "            model.train()\n",
    "            print(\n",
    "                \"Epoch: {}/{}...\".format(i + 1, epochs),\n",
    "                \"Step: {}...\".format(counter),\n",
    "                \"Loss: {:.6f}...\".format(loss.item()),\n",
    "                \"Val Loss: {:.6f}\".format(np.mean(val_losses)),\n",
    "            )\n",
    "            if np.mean(val_losses) <= valid_loss_min:\n",
    "                torch.save(model.state_dict(), \"./models/state_dict.pt\")\n",
    "                print(\n",
    "                    \"Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...\".format(\n",
    "                        valid_loss_min, np.mean(val_losses)\n",
    "                    )\n",
    "                )\n",
    "                valid_loss_min = np.mean(val_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we’re done training, it's time to test our model on a dataset it has never seen before - our test dataset.\n",
    "We'll first load the model weights from the point where the validation loss is the lowest.\n",
    "We can calculate the accuracy of the model to see how accurate our model’s predictions are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Loading the best model\n",
    "model.load_state_dict(torch.load(\"./models/state_dict.pt\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.215\n",
      "Test accuracy: 91.770%\n"
     ]
    }
   ],
   "source": [
    "test_losses = []\n",
    "num_correct = 0\n",
    "h = model.init_hidden(batch_size)\n",
    "\n",
    "model.eval()\n",
    "for inputs, labels in test_loader:\n",
    "    h = tuple([each.data for each in h])\n",
    "    inputs, labels = inputs.to(device), labels.to(device)\n",
    "    output, h = model(inputs, h)\n",
    "    test_loss = criterion(output.squeeze(), labels.float())\n",
    "    test_losses.append(test_loss.item())\n",
    "    pred = torch.round(output.squeeze())  # rounds the output to 0/1\n",
    "    correct_tensor = pred.eq(labels.float().view_as(pred))\n",
    "    correct = np.squeeze(correct_tensor.cpu().numpy())\n",
    "    num_correct += np.sum(correct)\n",
    "\n",
    "print(\"Test loss: {:.3f}\".format(np.mean(test_losses)))\n",
    "test_acc = num_correct / len(test_loader.dataset)\n",
    "print(\"Test accuracy: {:.3f}%\".format(test_acc * 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We managed to achieve an accuracy of **91.77%** with this simple LSTM model! This shows the effectiveness of LSTM in handling such sequential tasks.\n",
    "\n",
    "This result was achieved with just a few simple layers and without any hyperparameter tuning. There are so many other improvements that can be made to increase the model's effectiveness.\n",
    "\n",
    "Some improvement suggestions are as follows:\n",
    "- Running a hyperparameter search to optimize the configurations. \n",
    "- Increasing the model complexity\n",
    "    - E.g. Adding more layers/using bidirectional LSTMs\n",
    "- Using pre-trained word embeddings such as [GloVe](https://nlp.stanford.edu/projects/glove/) embeddings\n",
    "\n",
    "### Beyond LSTMs\n",
    "For many years, LSTMs has been state-of-the-art when it comes to NLP tasks. However, recent advancements in Attention-based models and Transformers have produced even better results. With the release of pre-trained transformer models such as Google’s BERT and OpenAI’s GPT, the use of LSTM has been declining. Nevertheless, understanding the concepts behind RNNs and LSTMs is definitely still useful, and who knows, maybe one day the LSTM will make its comeback?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}