cf_approach.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Business Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are _Best Film Recommenders (BFR)_, a company located in NYC and founded by two Columbia Students in 2014. Since our inception, we have been recognized by our users for our accurate and personalized movie recommendations. However, our industry is always evolving, requiring constant improvements to remain competitive and profitable.\n",
    "\n",
    "A week ago we received a letter from _Wes Anderson Productions (WAP)_, the company which owns the rights for every movie directed by Wes Anderson. Within the letter they claim that despite the high quality of Wes Anderson's films, most of his work is still unknown by most consumers. The only exceptions are _The Grand Budapest Hotel (2014)_ (which received critical acclaim, including nine Oscars nominations), and _Isle of Dogs (2018)_, which popularity was boosted by the former's success.\n",
    "\n",
    "Due to WAP films having high ratings yet low viewership, the company believes that six of Wes Anderson's films are unpopular due to a lack of publicity when they were released. With the Christmas season approaching, WAP believe that target advertising can bring in new viewers that never knew the movies existed, increasing revenue from movie sales. For this reason, WAP are willing to pay a significant sum of money if we can reach 1,000 users, representing a significant potential income for our company.\n",
    "\n",
    "While we could recommend the six movies to all our users, our company's mission has always been focussed on our users, and we don't want to make recommendations they can find irrelevant. Because of this, we would like to find the movie which is most likely to be enjoyed by each user among the following films: \n",
    "\n",
    "* _Rushmore (1998)_\n",
    "* _The Royal Tenenbaums (2001)_\n",
    "* _The Life Aquatic with Steve Zissou (2004)_\n",
    "* _The Darjeeling Limited (2007)_\n",
    "* _Fantastic Mr. Fox (2009)_\n",
    "* _Moonrise Kingdom (2012)_\n",
    "\n",
    "Therefore, our goal will be to build a new _Wes Anderson Recommendation System (WARS)_, to find the ideal movie per user. While we have no _accuracy cutoff_, we want our recommendations to be as targeted as possible to keep a high availability of advertising space, should new clients wish to use our services."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Phase 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this initial phase our goal is to play with the data so that we have a clearer idea of which route to take in building a robust final product.\n",
    "\n",
    "To maximize the breadth of our training, we will design datasets with different characteristics, try different kinds of recommendation techniques, and fit the models by shifting several hyperparameters. All with the intention to find what works and what doesn't for the first release of _WARS_."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to our previous recommendation systems, we are going to use subsets of the full data set for education and development from [MovieLens](https://grouplens.org/datasets/movielens/). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from datetime import datetime\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_ratings = pd.read_csv('movie_data/ratings.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From previous experience we know that people can change their preferences and opinions over time. Therefore, to ensure the quality of our data, we will use only ratings given since 2015."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_year(ts):\n",
    "    return datetime.fromtimestamp(int(ts)).year\n",
    "\n",
    "df_ratings['year'] = df_ratings['timestamp'].apply(get_year)\n",
    "min_year = 2015\n",
    "df_ratings_since = df_ratings[df_ratings.year > min_year][['userId', 'movieId', 'rating']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we start, let's confirm or deny the _WAP_'s hypothesis: Are the six proposed movies truly unpopular, but good?\n",
    "\n",
    "To do that, let's count the number of total opinions, and the average rating of all the movies in the dataset, and then compare them with the _WAP_ movies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_movies = df_ratings_since.groupby(['movieId']).agg({'rating':['count', np.mean]}).rating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "wap_movies = np.array([2395, 4979, 30810, 55269, 72226, 94959])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3Xt0XNd92PvvnjPvGQCDN0ESICRKJGhKhExRIhUqamintew4rJQ0ct1bLjdupJStbxT13rYr11nLzY297mqyYto3vYuVtZzYZWqFtGM5tFM5dlwltGiJIikJEGlCD1J4EMRrAAzm/Tjn7PvH4RzhRQIk8SLw+2hpkQMMZjbAwe/s+e3f/m2ltUYIIcTq51nuAQghhFgaEvCFEGKNkIAvhBBrhAR8IYRYIyTgCyHEGiEBXwgh1ggJ+EIIsUZIwBdCiDVCAr4QQqwR3uUewGR1dXW6tbV1uYchhBC3jbNnz8a11vXzue+KCvitra2cOXNmuYchhBC3DaVUz3zvKykdIYRYIyTgCyHEGiEBXwgh1ggJ+EIIsUZIwBdCiDViUat0lFLdQAqwAFNrvWsxn08IIcS1LUVZ5j6tdXwJnkcIIcR1rKg6/Juh/kDN+Jj+ghzbKIQQ0y12Dl8DP1JKnVVKPbXIzyWEEOI6FnuGv1drfUUp1QD8WCnVpbU+MfkOVy8ETwG0tLQs8nCEEGLtWtQZvtb6ytU/h4EXgAdnuc/XtNa7tNa76uvn1Q5CCCHETVi0gK+UiiilKsp/B/4JcG6xnk8IIcT1LWZKpxF4QSlVfp5vaa1/uNBPkvq9FEc6jtAV76Ktro0D7QcW+imEEGJVWLSAr7W+BLQv1uOXHek4wqn+UzRFmzjVfwqAgw8cXOynFUKI285tX5b50+6f8oN3f0DWzBL2himWihLwhRBiFrd9wH/x4oukSikAUqUUL158cZlHJIQQK9Nt30snVXSCvefqt1K+LYQQYqrbPuCHvCEUCo/yoFCEvKHlHpIQQqxIt31KxyyZaDSmNt3bQgghZrrtZ/h58te9LYQQwnHbB3whhBDzIwFfCCHWCAn4QgixRkjAF0KINUICvhBCrBES8IUQYo2QgC+EEGvEbR/wqwPV170thBDCcdsH/Kf3PE1NsIagEaQmWMPTe55e7iEJIcSKpLTWyz0G165du/SZM2du6GvUH6gZH9NfuPb3lC6mZxyYEvVHb3isQgixEiilzmqtd83nvrd9L50bNdeBKXJBEEKsVrd9SudGdcW7aIo2EfAGaIo20RXvmvL58gUh7Atzqv8URzqOLNNIhRBiYa25gN9W18ZAeoCCWWAgPUBbXduUz891QRBCiNvVmgv4B9oPsHvDbhKFBAWzQOdQJ4dPHyZdTANzXxCEEOJ2teYCftQf5eADB9nRsIOAN0AsGJuSuilfELKlLLs37OZA+4FlHrEQQiyMVbloe/j04TkXW7viXdSGaukc6mQoPURPosf9GjkEXQixFJa6SGRVBvyvvPoVfnLpJ+zdtJfu8e5Zf5BtdW18s+ObZIoZALymlyMdRyTYCyGWzFxVgwttVQb8olXk5b6XuZy6zL7WfbP+IA+0H+DY+WOYlkljtJG2urYlX6CVElAh1ralLhJZlTn8dCmN3/BjWuY1f5BRf5Qntj/BPY33sKNxB6O50SVfoJUSUCHWtqUuElmVM/xCqcCGyAa8htf9Qe7esHvG/coLsl3xrmVZoJUSUCHWtqWOQasy4OfMHO+MvUNVsIq/uvBXfPbDn531B7ncC7RtdW1u/u5aF6WVbnpa6vFtj/PChRckTcXKStmtpLGIDyx1DFqVAd/UJqZlogqKTCnDd37+HT734OeWdUyz/cIt9zuMhTB90elEzwn3HctSLEKtZEu9IHe7jOVWycXr5q3KgF+WNbP4lZ83B99k57M7OfjAQZ7c+eQtvTiu9WKba6abN/N0DHXM+IW7XX/pyqanpV5890U+fvfHJU3FykrZraSx3KrVdPFaaqs64AMUdRG/x89IZoQv/sMX+ZOTf8IvbvpFDj16iHXRdTf8eNd6sc010+1J9LBn457b+hdu8kWtNdYKCjqHOsmZOfY272U0N8q2+m0MpAeoDdVysu8kIW9oXvsiVqOVlLJbSWO5Vavp4rXUVmWVznQRX4SSVSJVTGFqk9NXTvPMD5+Zcp/B9CCf/s6nue/wfXz6O59mMD0462Nd68U2/eMXRi5MuQ3c9i0bJlcVHT1/lKPnjrKzaScArw+8zu4Nuzn06CF2b9jN6wOvA7CzaeeqqUBKF9McPn2Yp198eko7jmtZSbu2V9JYbtVCVrbc6L/p7W7Vz/DBKdMs2SW8youtbaqD1VwYufDB54tpPvmtT/Lu2LvEAjF+fOnHPPLnj/DMnmfcX4zyzHY4M4xHeWipapkyU5o+gyrPdMu397ftJ2gEr5mvTxfTPPf6cxzvOg7A/q37efL+J2dNFy30bHm+jz/5omZaJgAVgQr2te4jW8q6b6sPPnCQrngXYV+YgDcAwLHzx+Y1/tneRUzePAfc0M9iIX92N5pKWO6igMmWayzzTYHeyL/LQq59rbX00JoI+CW7BDiLuYlcgpJVojXWyr5v7MO0TZKFJOdHzuPBQ6FUwPAYDJgD/Olrf8qJnhM8uPFBOgad/LuhDCxtzZgpTX8RTs7hT75f+UV+pOPIlBf5kY4jHD131N35e/T8UYLeIAfaD/Dk8Sd5c+hNWipbiGfjwK2/KCf/wk2+iF3vRd8aa+X5t55nMDNIIp+gJljDWG6M1/pfm5G6aatr4+Xel4ln43QMdRDyhtzZ/vXGP/kX8Oj5owBTNs/lrTxHzx3FtExO9Z8ib+V5Zs8zsz5Wuphe0J/dYqQSFvKCtBIXM+ebAi1/fD4W8uK11tJDayLglykUJV0iUUhwbvgcSikCRoCcmUNrTYkSaMCGgBHgcvIy3YluTl85zT/d+k8JeAM0VzWTLWX56se/OuWxZ3sRTr99+PTha77Iu+JdmJZJVbAKgHwpT1e8i+def46Xul8CDalCiq21W2/qRTk9GExeRO4Y6qC5spm7a++e8aKf/HX9qX6GM8NkihkivggazQ/f+yE1oZoZwfzxbY/z9Te+zqWxS2iliQVidMW72NG447rjn+1dxORfxs6hTjLFDFXBKibyExzvOn7NgH+k4wgdQx1UBaoYygy5j3+zFiMP/tzrz837AjaXlThbnW8KdLkC7VKtbayUi/GayOGXaT44+tDUJiW7RLqUxtIWGo3ig+MSLduiaBXx4GE4M8zJvpNuzrC1unVeeb/p+cHOoc5rvsjb6trwGl4m8hNM5CfwGl7a6to43nUcj/JgaYvx/DivXH6F4cww6WJ6xuMPpgevOa7pu3qPv33cHUtzZTO9yd5Zc6KTv+6tobdIFpIEjABhX5iaUA1hb5h9rfuoCFRM+Z5euPACmVKGLXVbiPgipIophtJDc+ZcJ+dnvYYXr+ElVUjxUvdLdA510p/qx8ae1793V7yL5spm8maeoDdIb7L3lvK9i5EHP951nEwxQ9AXJFPMuCm9m7FSguhk18q3r5Q25Eu1trFSdtUv+gxfKWUAZ4B+rfUnF/v5boVHefB6vJSsEpa2CBth/IafgDdAyBtye+h/9+ffpWSX2Nu815mVXQ0o06/e02dcBbNAzszNOps40H6AvJWfksMv9/upDlUzlB7CtEwivgiGMtwXzHxr4Kd3B00Wk/RN9NFc1Ux9uJ6KQAWvXn4VgPbGdvdicez8MVKFFCOZERSKrJmlaBUZzY0S9Uf5xU2/yEvdL2FaJl7Dy6e2f8p9vpbKFoYyQzRFm+hL9lERqJjzl2pyauxT2z9FwSrwZ2/8GclCku3124n6ooznxwkZISL+CPu37r/mY7XVtTGaHQWgL9nHfY333dIv9ErKyc9m+my1vbGdw6cPL8ms8loz2Gvl2+fKwy/VjHip/k1XysV4KVI6TwMXgMoleK6bEjJCgJPrrwxUkivlyJQypItpAkbACSxXF11P9Z8iXUwzlBliIDVA2BfmRxd/REtVi3sBAOcF/fxbz9M90Y1Hedhcs5k7YnewvX47x992gnr7Oiewui/kyWevX32zsX/rfo6eP8qIGqE2XMu9DffSXNVMV7yLolVkIDXAhfgFYoEYV5JX+OTWT17zHcTk7qBVgSp3LWJvy94ZKZ7yBSVn5ihYBXonehnLj1EZqCRbyhIwAniUh/fH3mcoO4Tf8NMYbXTH3VbXxkBqgL6JPnoyPWyu3szz/+z5OUthp/8CHj59GIUi5A3ROdxJTbDG/cUBKFgFDr1yiO7EzK6ok4PKo3c9Cgo+/5PPr5j8Nnzw75sv5ee8gE02n418eSt/SymeGwm610onXSugzhVoV2J66laslLLYRQ34SqmNwK8AXwL+/WI+160wlIHf6yeiIgQ8AQqeAj7lo6RLoCBXynGy5yT1kXpqQ7UMZYacmaMGpZQb5KoCVexs2ukuyvYl+0gWkpSsEv3Jfl7zvkZjtJHqUDUPbXyIo+eOcrzrOE9sf4K8medbb32L/lQ/BbNAd6IbYyjOb3zhL3nq7R5+vt7H7x/cyj0N97gvmBM9J7g4fpG6cB0Xxy8SC8bom+hjJDtCX7LPnamXZ1rPn3ueK4UrePBQE66hP9lPQ6QBgHfG3pl1BrK3eS9d8S6G0kPEgjEUijur7yRv5imaRfpSfWyp3ULezFMTrKF7vBtwgu2JnhOg4MH1D1IfrueFCy9c85f2WsGlK96FR3mYKEwQ9AYZzAxSEahgS2wLTdEmvtf1PYAZXVFnrFlYeXfhfa4Acquzyxv5+ifvf3LGu8P5uFZAnPw9Pf3i07c0q7yRoLvQM9iVMiNeKCtlV/1iz/C/AvxHoOJad1BKPQU8BdDS0rLIw5ld2kzjM31UBCoYKgxh2zZKKbzKi0d52FS1iZf7XqYyUEnJKpEr5fAaXnKlHD7lI2AEyJayXIhfoKmiid0bdtM53EnJLpE385SsEhpN3swzkBogW8ryd5f+jrHcGJlihgvxC0R9UQpWAdM2CflCJAtJHvr3X2ZddwGvrflwt81Xvn6F//Zlk/bGdvJWnnMj5/B7nK6gm6s301zVTKaYoTfZS0tlCx7lcXv8R/1RGiINXIhfwMDg7JWz1IZq2duy94N0U2lqumlyRYzX8PIvd/xLjr99nLdH36Yp0kTJLlEfrudy8jKmZTKYHuTRux8FcJ+vvNidKqSuWZpZrqYpLx6X0zAHHzhIW10bP3zvh1QFqkgWkoSMEO8n3qc2VMv6ivWzLuzCzGDVk+jh3sZ7+f4732c4PczP+n7G49sen/Udx2yB7kD7AZ47+5z77mx/2/4Zu7bLgf7Y+WPuZrS5AuXNphTmExBvdVZ5I0F3oWewK2VGvFBWSjpw0RZtlVKfBIa11mevdz+t9de01ru01rvq6+sXazhzKlFiojCB1hoLy+nHo00Uisupy+TMHOO5cUZzo2itqQ5WO5UqWuP3+ClZJZKFJO3r2jnQfoCRzAiJvFMCamOjUET8Ebwep4Nn30QfE4UJAEzbZDgzzEh2hGQ+yUR+Ap/ho63fCfYAXlvT2pPgqx//KkFvkI7BDtZH11O0i9SGa2mqaOL+pvuJhWK0VLaQyCeIZ+N0Dne63+NQegifx4fyKOf71Jb7yxwLxmYuXumpP6POwU7aatt4cP2DoKAqWEXEH8HWNiW7RMgbmvI1kxfmTvadJGfmZl20OtJxhDeH3qQyUMmV1BVeH3idZ888y+HTh3l82+PsbNpJ0BekMdJI2B+mMdzIxfGLnOo/xWhulMupy7zS9wp9E33u4t/0YAXw/be/T3+yH60047lxPvc3n+PQq4fY94197PvGPg69cojB9CDHzh/j3NA5Ooc6qQ3Vuu/Yjp4/SjwbJ56Nc/Tc0RkLb+ULRaqQIlPMuGNYjNnpfBY9b3VBcj7PUS4c6BzqpGAWSBQSCzKDXU0bxVaSxZzh7wX2K6U+AQSBSqXUX2it/+UiPuctsbCmBCyNxrZtErkEpjbxBXxEfBEsbVHpr2QsOwYKLG0RC8ZYX7GeoBEk6o8ylB4i6o9i2iZFq4hGo22NRlO0iuTNvHOx8PlRKAyPgQ8fNja2tvEbfn6+3sfOniI+G0oe6N4UYxsfBLP1FesBJ5A/1vaYW7M/Oc1TE6pxvx+vx0ttqJaqYBXvj7+PaZukCim3BcKOxh186aNfcmet3Ylu9rXuI+ANMJod5UjnEfyGn4ZIAx+946OYtumkXPC4h8h0J7rd5ysvRH/359+lK95Fhb+CNwbfYHv99ilBcPICb7qYZjw/zu4Nu93Z8XP7n+NIxxGePfMszVXN3Nt4L28NvcX5kfM0RZuoDFTSm+ylJlTjBobpM8T9W/fzJz/7EwLeABWBCmKBGK9efpXLqctT9j681v/alHWL0dwon2n/zDXLZicr/7uMZEboneh1qpIiU2enC7UYOZ8Uwa3OKufzHJPfDeXMHDsadizITHalzIhXm0UL+Frr3wN+D0Ap9UvA/7mSg/21+L1+MqUMtm2TN/MYPsP5hIKHmh/ireG3CHqD3NtwL211bXQOdXL49GHeH38fQxm0N7bTO9HrLHR6AygUsWCMeCZOxsyQt/KgIOgNOjNZb9CtovnfP7uOr3z9CvdcMTm/wcsXn2yh6fhv8aOLPyJVSNEUbaKtvo1t9ducmv2zz9Ez0UPRKjKYHmRz9WZiwZj7vXxs88d49uyzDGWGCHlDPLDugRktEOCD9MPkoHn87ePkSjkKZoFUIUWqkOJ39/wuiXyCwcwgpm3SN9GH4TF4+sWn3WAWNIKU7BIRX4R0MU3HYAdvx9+mNdbKoVcOgYI3Bt6gZ6IH27YZzY/SXNnM/evvR2tNV7xryi//qf5T+Dw+miqaKFpFtz9RwSyQLWVnXbAtB6vX+l/j9JXT1IZqiWfjBL1BrqSuYFpOGs3r8XJh5AIfvfOj7rpFyBviQPsBN7BN5J13ZRF/ZMaMt/zzaqtrYzQ3SsgbmhEoF2oxcikC4nyeY7Xl2le7NbXx6mZMFCYIGkEsj0W+lKdgFogFYwSMgLNgi9OHp3I8x+///o/YPmByfr2P5s9u5F1/koH0AJtim3hs62Mcf+c4F0Yu8N7Ye3g9XoK+IBV+Z3ljz8Y9BIyA07IhMkBPoochNcQnf7uCoC/IWG6MJs8o7/X8lKG0s4noUuISqWKKRzY94va3GUoPURVwZqHJQpJEPsHBvzlIIpegZ6IHgA0VGwh4A/yjO/4R3ePdbgsEv+F3L1hd8S5aq1tpb2ynO9HNRGGCiC9C3sxTsAsMZ4YpWAUMZdBS2UJfso9R3yhttW1u2gY+2FDWUtXCYHqQ/lQ/4FREffmVL7s7l8fz41QHqmmMNFIZrERrPWfpam2olt6JXlqqWuid6MXW9pSLzfRgdejRQzzzw2e4MHKBB9Y/QM7McbLvJFprsrkstrbZs3EPo7lRdjTucGfn5UXvvJmfksOfPuOdfJH5TPtnZp29lwOkUoqB1ADPnnnW/dqVUDV0o1Zbrn21U1rrue+1RHbt2qXPnDlzQ1+j/kDNfadbpFBTNm158BD2hinpEiWrhKEM/u7rJR66jJt+OdXs4df+XS0+j48ttVuoDdfyvy79L2ed4Op/ASPAvQ33sqlqE8PZYQbTg6yLrOPxbY+DgsOvHSaejZMqptBa0xhtJFlIYtomhscA7ZQlfuSOj3D/+vv50Xs/omQ7ewiG08OgnOqVRD7BxfGLFMwClrbwG3621m7lzpo72dGwg5d7X2YwPcj5kfPY2mZTbBOPtDzCaG6U3Rt2c/CBg2z50y30TfRhact5fmXQEGngnoZ73Nn45NbI5XxuIpfgpe6X3MA+mht1N24l8gl8Hh9VgSqKdpGoP8qvb/t1Xh94nR2NO2ZNeUzerVweT0OkgeHMMIYyaK5qdgNPeWZ+rfTJwR8c5J3Rd7g4ftH5vqs28e0nvr2oB7iUxz+QGuDi+EU2V292F/rnM2NfKTs2V+p41iKl1Fmt9a753Fdm+HNoSMO3j2nuG4Q318FvPAHDUZuM6eR9FQobm/sGnWAPzp87BmxShRTrouvYs3EP33v7exStIobHcHL5SoN23kH0TvSSM52Kn75kH8ffPs7+tv1E/BF6k70oFCZOFYxHOevsBauARuPBw9ujbwPgNbzuhqjGaCMjWWfheDQ3imVbjGRG8Hg8RP1RN79/oP0AP7n0E05fOY1lOzuO+yb63DYIncPOjL8uXMel8UvY2saDB0MZZEtZ3h17l76JPrTSRHwRd8ZdXuyzbZtYIEZvshetnfv4PD78hh/btsGAkC/kzrBHc6M8sf2JGcGvHFiePfMsjdFG1lesn9Lm4ukXn3bfqZRTC3OlT3Y07iBn5nhww4PuRWJddN2ipkrK7wKePfMsm6s3T0ldzcdKq0+XXPvtZU21VrgZ3z4GD/VBZdH589vHnI9PbsWgtebNdVC6+majpKBjnXMxqPBVoJRCaUXRLqK1pjZt8r++bjH8xQLf+PL7FK/0Ec/FGcoOMZGfIFVI0THYQYW/gpA3hK1t98JiaQtb2+7z+wwf+VKevmQfBdNJtVxOXaYx2sj2+u30JfvIFDOMZkfxeXygnbYR5fx+1B9lNOd8rjpUjaEMUoUP2iAkcglO9Z/ioY0PEfaFCXmdXDfK+f58Hh+D6UFaKlvYXL0ZW9tuZUUsGCOei5MupakMVOIznNJXv+EnV8pRGaikJlRD0S5iaxuvx0u6kHbTSrO1hmiMOhU6Z6+cpXeil+HMME+/+DTDmWF6J6a2h5gtfTL5cZejEqQcIH9712/TVNHkpq7m21pAcubiVsgMfw73DYLvajbHp53b02k0n37C4C+/bdM+oN13Ajkrx8/jP+fSq5fImTn3AnHs6kXEp+GBXpM//fNB/vFvBTC10zqhMdpIbaiWn2V+RsEquGWdhjLwerwUrSJe5cXSFiWrxEh2BNM2UR5FXbiO8dw4/cl+djbtpCHSwLnhc2ys3Oj0DbKdFEhTRRM7Gne434PP8DGRn6BkOZvNwv6ws59gqNNZs/AGeGD9A5y6fIqSXXJTXGO5MR7Y8AAPNT/kLpyWG8sdPn2Y7174LplSBsu28Hl8+Dw+NlVvAuBjd32MM/1nOB8/z4fqPsRYbozLqcvcXXv3rK0hplcm1YRqMJRB2BeetYtp+SIxOX0yn12gC20hj7eUnLm4FRLw5/Dmug+Cc0k5twFnw5U3QMl2cvjpGi8f+60SeSs/Jd9vYbnpHwMDhZp6EbGhfRBnxqwVtrZpq2vjZN9JYoEYqlLx7ti7Ts7/apWPaTt7BOCDhnDJQpKKQAWZkvNcfck+nrr/KZ68/0mOdByZkqfPlDLufgFwFiDff+V9knYSG5uAJ0BDuGFK0GyKNjmblNQHYy1ZJXJmjvpwvTuznr7I+oW//wJFq0jIG6JoFRnLjfEffuE/uM/9q9/6VSJeZ3+CaZlora/ZGqI8jrpwHTWhGi6MXKAx2khLrGXWLqaT0yebqpyLzLmhc/QkeuZ92PpC5Kjnsyt2vlbKjk1xe5KAP4ffeMJJ40zO4QOUdAmf9gGQN/OA03xtcrCfzsICrl5Eygu8Vy8ipm3i9Tg7e7//zvdJF9Jsq9/GI62P8P13vk/fRB8+5SNZTE7p6gkQ8DjvDrKlLBpNyBuiNlhL0Bt0K0xO9JxwG2rVh+sJGkHAmYWfHz5P1szi8XgIe8Jsjm3mfPw8RzqO8Pi2xznRc4IX332Ru2ruwrRNN48PzgK2pS1efPdFttVvcxacr4r6o7TVtnFx/CLpkpNGUVp9cM7Ahgen1Lyni2kao43XvHiAE+hsbWMow03vAO7C52STZ/CT+wh5TS/P/PCZeR22vhA584VMw0jOXNwKCfhzGI7CP/rs1I85C7lw32DWvQiMRJUbBOcy/SLyz5/wOLtVrRKV/kpsbRMLxuiZ6MHr8bKraRcV/gouJy9jlAxsPbU9sI1N1B/Ftp3c/obKDXzkjo+4geVabQ7KLQDKve211ng9XoayQ2yv387ZK2f5+htf5+L4RZoiTVwav+S2Mii3KC7v1v343R9nID0wo1/O49se5+j5o1wavwQaQv4QlYFK3hx6k8H04JRePdVV1Xzirk/wtxf/FpjaXG5yoCsv0LbEnFYckzeezabcddS0THeD2E8u/WReh60vRLCWNIxYKSTg34RvT8rBlxdy931WuWmWuUy/iPg8BobWVAachqI1wRoGMgOkC2kGU4P8UusvEc/EGc2OzvocXuWlOlBNxsygtaZgFjh67igV/gr6k/3UR+tJ5BJu2eLfd/893YlusqUsfsNPZaCSO6ru4OL4RbTWeJSHunAdbw69yXtj7xH2hRlMD7pVQZP70Rse47oBsdwc7Nj5Y1yIX2BjxUbyZp6Wyha3zfLkmneATbFNTtfOwQ63F9BkkwNoU0WT0wmTa3fCjPqjPLH9ieseQXmt3bCzlXveKEnDiJVCAv48fDCjd2bkHx6YfSH3eumc6zFtZ7E24o8Q8UV4f+J9ilYRhaIyWMnP+n5GspB0U0JlhjIIeoOsr1hPVbCKQCFAd6Kb0dwoHjykCikG0gNsrNoIQIW/gopgBX3JPnemniqmyJay+Dw+7lt3H5lShpA3xMMtD9M53El1sJqcmSPoC5LIJwj6gtjaqRbyeZzGcX/99l/TXOn01d/bsnfKGMsz8+lHNdaF69i1YdeMc34//5PPzzmjvpk2wPM9ghKmpnHKh8/cSiWPpGHESiEBfx6mz+izfif3Pnkhd76nMM3GozxOR02t2duylxM9JxjPjVMZqKQh0sCZK2fQOE3ainYRcEoiDQxM2yRTzJAr5ciWspi26W7sKv+9P9nPh+o+RG+ylwp/BUWriInpVv5oNDXhGrbUbmFH4w53hnyi5wSvXXbOqx3Pj7O1diubYpt49fKr+A0/HjzUhmqpi9TN6GUzXdQfdXviXG8BdD7pj+kBdD5tgOdzBGXZ5DROS1XLrEdaCnE7koA/D9NLMz0aXmmeuZB7PR48BL1BsmZ2xucMDJRSpIopvvLoV/ivr/1X/vjkH9Of6ieejbubrQyPgWEbWFh4lRetNF7lJeqP0p/qJ2/m3QWBNYsoAAAgAElEQVRdG5vyLmqlFVfSV/AoZ9ZfFaiiP9VPwAjg8/jcw1u+9NEvcaTjiJsa+eJHvsh/+rv/xKuXX6Uh0sCntn+Kf/Xhf+XOjDuHOtnZtJOKQAU7zZ1TetnMZj4z3ZtJf0hrXiHmRwL+PEwvzXxj3cyF3NmUWzKUZ9IFs+B+bnIu3NQmTqNOzXNnn+PMlTMopZwZvVWkNlSLx+NxjhlUirAR5o7qOxhIDWAog5yZoypQhWk5LReU5RzW7vP4nDSTct5FVPgryFt5qgJVjOXG8BpemiJN3NtwL63VrbP2pP/oHR8l6o86M+fRrimLsuU2AX7Dv2CB8WbSHwudI5ecu1itpJfOPEzP4TvtFeYY19X/DGVQFaxiPDeOVhqfx4dlO/32y5uFPDhn6VaHqtlWt43x3Dh+w894fpxUIYXH4+EXmn+B8yPnaalsYSw3Rjwbp2A5G53CvjABI0BduI7KQCWv9L2C1pqgL0jEFyHoC9IUbeJy8jIhX8jJ/fuC3FV9Fw2RBudEKDPP117/GlWBKvJmnsZIo7v7s9yyYPrGKumjIsTyk146C2y20szZ+Dw+dwOR4TEIeUNonOqbykAl/cl+t3QyYASo9Fc6PfevlmGWa+O31W/j797/O7R28vDVvmpGs6M8tvUxt6zy6PmjNIYb3Zl8hb+Cx7c9zvG3j9MYbSRv5cmVctjYVIeqaYw28n7ifayixba6bdzTcA+mbbrB++kXn3Z70ge9QfqSfTzW9hjANdMbc83GV9oFYaWNR4ilJgF/AXmUh7tid3Fh9AIFq4BSCtu2uVy87MyQ7Q9SOrbl5Nh9hg+f4aNoFcmUMmxr2MYXP/JFzv6PsyTyCTZWbmRf6z46Bjt4qfsl97jBPRv3uKmWciA++MBBuhPd7GzayV9d+Cu0rd2yy3ND56jwVzCQdg4Wn15R01bXRjwbB6A32cv2+u3kzTzvjL5z0ycZPff6c+4Riaf6T5G38jyz55mF+4HfoKVqPCYXFrFSSfO0BaJQFKwCb8XfwtROdUzOzFGwCxR10d1pWqbRxPNxIv4I4/lxJgoTbG/YTtAI8qOLP+KZPc/wibs/wa9u+VVSxRQ1oRoGUgO8Pfo2Px/5OXkzT3tj+4xywdZYKyf7TmLZFulSGtM2OT98Ho2zqao6WM1AZgBLW1OC94H2Azzc8jBtdW38zoO/w96WvXQMdbh9dMonGd1I4DredZxMMUPQFyRTzLh97JfLUjUeK19YZjvOUYjlJDP8G2RgzKiHh7lr8K/1+XQxTcgbIl1Mc3HsIl7lpXO4kz/+x38MfLBw+Py5590DzjWaM1fO8Ct3/8rMckE19fkUyi2rvLP6TgyPwZbaLTREGmZsTrpWqWNtqHbWA8hvt5nsUlXfSEdLsVJJwL9BSqkZh3vfivH8uNuEbTg9TKaUoSJQMSMAHzt/jLyZJ+RzmpD5Df+UQFIOvt9845s0RhsxIyYbKzei0Tzc/DDff+f7TBQmaKlsIRaIuW2FrxWoW6tb3XTMaG6U+nD9lJOsDj5wcM4Uyf6t+zl6/ij5Up6IP8L+rftnjHcpLxZLVX0jZZ1ipZKAf4Pm2z4BnJz+9L430xWsgtMlEo1hGHjxukcYTra/bT/diW6ShSRBb5B1kXVTeqiXO2IWrSKv9r9KyAgR9oe5u+ZuRnOjHHzgoLurdTgzjEd5CPvC/H333/O1s19jJDNC2B/msx/+LJ978HNTLmo5M0dFoMKdsZb71U8+jOR6bRUmB/XJ413qgzyWaserlHWKlUoC/iKqDlQzlh+7ZjrH7/GjtaakS3iVF4/HAwrngJFpntz5JODkxU3bpDZcy/NvPc+x88fYv3U/74y9QzwbxzAMaoI1jOfH2RjayJ3Vd07ZPQtw8G8OcmnsEu+NvUfPRA+JXIKKYAWZTIZnzzxLVaCK7kQ3+1r3EfAGeKXvFfeAlfJJVpMPI4G5u1VOt5rTHtJKQaxUsmi7iMbz48T8sSldNMs98Q0MIr4IJV0CnFSRZVuki2lqw7U8/eLTU05nivqjPLPnGV76Vy/xL+79FwymBxnPjxPPxjl6/qhzSHmih3QhTbqYpinaxAMbHuDwJw/PWGxN5JwzbpVSjOXGADAtk5JdIp6J0zncSVtdmxvc68J1UxaIY6EYTdEm7l9/P5urNzOUHrrhmezkx7+RE5+EEDdPZviLSCnl7qY1lIHP46Mh0kDOzGHZFrlSzr2vrW28ykvIFyJoBGfkyyfrindhWiZVwSoA8qW8c1xhIMrA2AA1wRqnlDKXmHVcsWCMzdWbSRQS1ARriOfi5MycWyaayCWmpCUebnl4yjuEyQeJN1U08VjbY8u+O1YIMTcJ+Auk3EZhMq012VIWw2Ng2RYKxUB6gMZII0WK5K28e19LO7tvKwOVNEYbZ011lBc6O4c6Gc2NEjJDKKXIFrN0xbsIeUPsatpFupQmFogRC8ZmHWv58O6dTTu5WHmRl7pfIm/m8Rt+djTucM+6vVYQX4hgLWkPIZaeBPwFoFCEvWFsbbtn0AaNoHs+bPlikLNyBIwAE/kJ8mZ+xgUi4otQHazmZN9J9rXum1HhcaTjCCd7T6LRjGZHKdklov4ozVXN7Gzaycm+k5TsEh9p/QgD6QF2NO6YcpFI5BPEQjG21G6hvbHdydPfsY+HWh6iY7DDrSqZfNbtbCRYC3F7koB/izx4MDwGKKj0V1KySwS8AbbVbeNnfT+bMos3cE6rKlpFN3c/WVNFE3s27uHc8LlZ+693xbsYyY7w3th7+L1+fNqH4TGoD9dTEahgb/NeXh94fc6DvHOlHLs37J61J46kV4RYvSTg34KAClDSJSK+iHMmreGlLlKHrW3eHXsXS0/doKXRKK0o6MKMxyov7J6+cprPtH9m1hl0W10bP770Y4pWkYJZcM/QLbdTGM2N8sT2J6Z8bbka5kL8AnXhOhL5BDubdk5JFc13xn67bbQSQkw1Z8BXSv3aLB+eAN7SWg8v/JBuHwVdIGgEaYg0MJAaIJ6NE/VFifgiJHUStHP8oK1td/G2qIuzPlbYFybsDRPyhmbMsCenZcK+MLlSDtM28SgPBatAzsxds9dNW10bL/e+TCqfoj/dz/roevom+tw+OjcSxJejdl4IsXDmM8P/18BDwEtXb/8S8CqwRSn1f2ut13SjkLyV552xd9wWBhfHLxL1R9lat5XB9CCWdhZrPTgbnfJmfsbmrfLhKPc03sPuDbtnBNzJgbatto1ELsFYfoyiVaQx3EjQF3R73UyWLqbJW3lOXzlNIp9gXWQdWuspfXRuJIiv5tp5IdaC+QR8G9imtR4CUEo1AoeB3cAJYE0H/DKNdnevZkoZLo5ddFM6Go2BwaaqTfSn+kkUZpZLKqWcvvRW3m15UD53dfKO1uaqZvpT/fi9ThfMcu/6rnjXjNl63spztv8seTOPZVv4DB8f2/wxTNt0Lyo3EsSlZYAQt7f5BPzWcrC/ahjYorUeU0rNXHlcw0pWiaJdxIOHsdyYe9ygoQyigSjj+XF8hm9KAzYPHurD9Tyy6RE6BjvcwHuq/xQnek4Q8AZm7Gjd37af1y6/NuVA8La6timz9Zd7X+b0ldPkzTzJQhJDGQykBjjZd5LPtH/GHfONBHGpnRfi9jafgP9TpdQPgG9fvf3rwAmlVASYfWfPKuTFS2WgkqyZnVJ5M5mbvvF4MO0PDgk3PAYBI0DQG2QkOeLO+MuHozRXNTNRmOBK8gqf3PpJN+i/+O6LfPzuj7O+Yr3T5njkPEWrSHtjO4cePeSeLVvOvX/+J593Z+vxbJxkIUnJcto2lGv8p68R3EgQl3JMIW5v8wn4/w4nyO/Fab7734G/0s7ZiPsWcWwrSsAXwOPxUBGogAIzgn5tsBav4cXrcRZpE7mEW7nj9XjZs2EPr/W/Rsn+4E1RObcfz8Z5Z+wdqgJVXBq7RKKQoDfZS8QXoXeil5aqFjKlDBsrN7Jn4x46hjoIeoMzgu/k2Xr5EJOhzBADqQEqA5X8Yssv8nDLw9dtiyyEWL3mDPhXA/t3rv6/JvmUj7DXyc3b2p4yQ1fKWaxtiDbQ3tiOoQwao438tOenJAoJNlRs4GObP8aZgTMMZ2cWNeWtPIkJ542SZVlciF/A1KbTxjgYw9Y22VKWkDfEzqad1821T56t39d4H4YyuG/dfZzsO0nIG3JbJAgh1qb5lmX+F6ABZ4avcK4DlYs8thWjpEvEc3E8yqmV19oJ9LWhWj687sPUhevcYFpeNP3ND/+mW+J4+PRhzo+cdw8tL+f2AbJm1v17zszRM9HDv33g37qHhicKCdrq2ugc6uRk30n2Nu9lNDc6a6598mx98gLuZ9o/IzXzQoh5pXT+CPhVrfWFxR7MSqbRbtWNgXNA+SMtj7ChcsOU+vXHtz3OMz98hn/o/gdO9Jzg0KOH6BzqRGmF1+OlaBfdlgrTF281mqJVZCA94C6illsRl1snvD7wOk9sf2LOmbqkaoQQ080n4A/dTLBXSgVxyjYDV5/nO1rrL9zo46xEjRWNbKnZwlBmiNHcKJ1DneStPJ++59N88luf5L2x96gKVPHjiz/m4a8/TNEuksgmCBgBLG1RtIp4PV4UTktkuJrPVx63r3x5EbVzqNM9V3Zf6z6ypawEciHETZlPwD+jlDoKfA9wewJorb87x9cVgI9ordNKKR/wslLqRa31qzc/3OXn8/i4p/4exnJjzs7aq2mSo+eO8trl17g4fpGwL8x4fhzbthnKOBWtNjYls4Tf48e2bQxlEAvGGMmMYGMT8oWoDdfy1P1PTQnok1sRS+27EOJWzCfgVwJZ4J9M+pgGrhvwry72pq/e9F39fwFPg10elf5KcqUcIV+IWDBGxB8BIFPM8NPen6JQJHIJ5yQru4TX8GIoA9M2CXqD5Mwchsdw2hqXstRH6on4IvzKll+ZcQwgzCybfHzb4xw+fVj62Qghbth8qnR+82YfXCllAGeBu4D/T2t96mYfa6mUF1RnO5bQwMBn+Hhn7B02Vm7E5/ExkZ/Asi2GMkOU7BIhbwjLcE6u8nq81IfrGUwP4vf4KVpFKvwVhHwhbG1j2iabYpv41PZP8cxDz8w6nnIuvrwI++nvfJqcmWNv817pZyOEuCHXDPhKqf+otf4jpdSfMsvMXGv9O3M9uNbaAu5TSsWAF5RS92itz017nqeApwBaWlpudPwLTqPd+vigESRrOVU0PuXD7/GTLqYJeoP0TvQS9UdJF9NOD3zbZnNsM/F8nIA3QNQfpa2+jfHsOBOFCQpmgepQNUFvkM3Vm8mUMoS8oXktwMIHPW9ShRQFq0BXvIsdjTukn40QYt6uN8MvL9SeudUn0VonlFJ/DzwKnJv2ua8BXwPYtWvXsqd8yrtjbW27B5aYlolSipyVQ9mKklWiNlyLaTu7V4tWEcu2iOfjbKzcSLKQpL2xnYA3gFd5SRQSBI0glraoCFSwpXbLjIPF51LueTOSGaF3opeh9BADkQ9y+tK6WAgxl2sGfK3196/+Nau1/vbkzymlfmOuB1ZK1QOlq8E+BPwyTj3/ilYuvyzP9EtWyWmMdrVffbnVcVWgiqyZpS5cx2B6kGggSrqUZqIwwX2N97mtD5498yx319zN/evvd488LB88ciPKu2jb6tqc4w29oSmtEKR1sRBiLvNZtP09Puijc72PTdcEfPNqHt8DHNNa/+DGh7j0yvn7cqC3cfLt6yLrGM+PA1AZrKTJ10TPRI+bommMNropmqg/yoH2A5zoOUHHUAcA9eF6tw/9jZq8eDvbRippXSyEmMv1cvgfBz4BbFBK/b+TPlUJmLN/1Qe01p3Ah295hMugfAat3+N3FnFtQEGmlKEmVMOm2CYO7DjA2YGzjOXG6Ip3EfFH+OyHPzslEB/pOIJHeWiubKZ7opvR3CgVwQoOnz485X7zScfMtZFKWhcLIeZyvRn+FZz8/X6cSpuyFDB7SckqYmDgUR7WRdeRN/NkzSy2bWNpi/XR9e6M+9zwOWpCNVja4tkzz/K37/2tO8vvinfRUtXC3bV3A9CX7CMWiM1IuSxEOkZaFwsh5nK9HH4H0KGU+pbWs5y4vUqVc/UVgQpaY60UrAJ3Vt/Je2PvUbAK7mJseUZuWiZVwSoGUgMkC0lShZQbtMvHC8azcV678hpNkSaUUjNSLguRjpFWCkKIuczrABSl1P8DfAgIlj+otb5z0Ua1TMqpHEMZAOxo3MGejXvoHO6kL9lHlb+K/lQ/g+lBTl85TWOkkdHcKGErTCKfcA8raYo20TnUyZa6LZy+cppkIUlDuIGCWeDslbNu+4QySccIIZaCZx73+XOcIw1NnP73/51VeqyhvvqfpS1SxRQvvvciAFtqtlATqiGej3M5eZnuRDfj+XH8hp+6cB0Fs4DhMbBsi7yZp3eil0Q+QcdgB5X+SurD9dxRfQd31d7FUHpoRsrlQPsBdm/YTbaUlXSMEGLRzGeGH9Ja/0QppbTWPcB/Vkr9FFgVjdBmo9EEvAG01pzqP0VPooe9zXsZTA9i2zZZM8sdVXeQLqX55Tt/mVcvv8qGig2MZEfoS/bREGkgFooRC8Sm1M0HvAGqQ9Uznk/SMUKIpTCfgJ9XSnmAd5VSnwP6cXrjrwrlNM50trapj9TTFG2iJ9HDaG6UrbVbuTh+kepQNWP5MTZXb2YgPQBAc1Uzd9XexU5zJ9lSltbqVo6eO0rBLJAupjGUQcAbYGfTTqmTF0Isi/kE/N8FwsDvAH+Ik9b5zHW/4jYyPdiX2yoYyqA+XM9AeoD9bfsJGkE6hzqpCdUQ8UfIFDPEgjF2NO6gra6N73V9D9NyjjT81D2fcptRlE/A8nl8PNzyMAFvAL/hlzp5IcSSu27Av7pp6gmt9X/A6Xx5043UbhdRX5RfaPkFtNZT8u3Xa1Nw6JVDUz+goTvRzb7Wfe7JVa9efnXKwSayMCuEWGrXDfhaa0spdf/V/P2y97lZCnkzz/b67YzmRnms7bF5pV2mB/fuRPeMypvyuwSpkxdCLJf5pHTeAP5aKfVtIFP+4DwOQLntGBjEQjGypSwFs0DncOeMXbGzma2scraNUNLMTAixnOYT8GuAUeAjkz425wEotyOvx8vnHvwc9eF6TvWfmnVX7GwmB/f2de3kzTyf/8nnaatr40sf/ZIEeiHEirCoB6DcLjx48Bk+Ir4Ifo//hne+Ti6rnHwkoVTjCCFWkvlsvFr1NJqIL4Ktbb786pd5Y+ANXup+iVQhxUB6gLa6tnk/Vle8i9pQLZ1DnZwbOsex88dIF9Nzf6EQQiyyNR/wy3X4mWKGolkkWUjiM3wAvD7wunuO7KFXD7HvG/vY9419HHrl0DWDeFtdGyf7TtI70UvBKpAzcxzpWJUbk4UQt5n55PBXtXLAL9gFDAzWVazj4vhF6kJ14HXy80c6jnD03FEyRWfN+uj5owS9QTdVM7m9cWt1Kz6Pj4Dh9NVpq2uTmnshxIowZ8BXSv37WT48AZzVWr+58ENaegqFR3nwKA/j+XGqAlUUrAJe0zujKyZAvpSfEsQntzfuGOxgY+VGN/8vNfdCiJViPimdXcC/ATZc/f8p4JeA55RS/3HxhrbwFIqAESBoBKnwVVAXqnNOtVJXfwwaiqUiCoWpTQqlAsfOH6O1uhWv4WUiP8FEfgKv4Z2S15++yBsLxaQZmhBixZlPSqcW2Km1TgMopb4AfAd4BOdglD9avOEtLK/y4jf8pItpLG0R8oXwerxYtoXP46NoF/EqLxOFCSxtUR+uJ2fmQMOn7vkUx7uOA7B/6/4pQXy2OnypzBFCrDTzCfgtQHHS7RKwSWudU0oVFmdYC8+Dh8pAJUop8INpm+RKOULeEJWBSgbSA257ZK2dRdz7m+6nra6N7kQ3X/34V3lmz+wHfclpU0KI28F8Av63gFeVUn999favAs8rpSLAzxdtZIugaBeJBWNsrdsKOLn4ZDHpHEyucZumeT1eQr4QOxp3zCsHL+2NhRC3g/lsvPpDpdT/BB4GFPBvtNZnrn76f1vMwS0kG5u8mWc8N07YGyZVTGFpiz0b93Bu6By5Ug7Ltoj6o3iUh4dbHpYcvBBiVZlPlc5XgaNa668uwXgWlW3b2B6bnokevB4vDZEGXul7BY1ma81WYqEYV9JXuK/xPp7b/5y0RBBCrCrzqdJ5Hfh9pdR7Sqk/VkrtWuxBLRYbG601UX+UikAFo7lRbGws22IsP4atbZ7a+RQPbnyQz//k8xw+fVh2yQohVo35pHS+CXxTKVUD/DrwX5RSLVrruxd9dAusvChbtIukiils26YyWEksGKM6WM2Oxh0EvUG34uZk70lO9JygIdJAa3Wr2+e+ra5Nul8KIW47N7LT9i6gDWjlNlusLfMpH37DjwcPASNAxs5gWiZBI+jW1pdr6pVSnB04y2BmkAfXP8jP+n6G4THY17pPmqIJIW5L88nh/xfg14CLwDHgD7XWicUe2EJTKEL+ELlSDg8eWqtbqQ/X8/74+zREG9jftp/Htz3OMz98ho6hDrTW9Kf6qQ3VMpQZYiw3RkO4Yd4dNGczuQWDvEsQQiy1+czw3wce0lrHF3swi0mjSRaSRL1RDMMgWUgSC8b4/COfn9La2KM8NFc289qV1wh7w0T9UYLeIEWriNfwUjALN90uYXILBnmXIIRYavPJ4f83pVS1UupBIDjp4ycWdWSLQKHwGT6qQ9WkiilC3tCUksuueBctVS3cXessT3QnullfsZ6+ZB8PtzzM3ua9dCe6b7pU80b77AshxEKaT0rnt4CngY3Am8Ae4BWmnoC14qmr/yWLSTZUbqA+Us8T25+YklKZ3CKhLlxHTaiGhkgDj7U9tiDpl9laMAghxFKZT0rnaeAB4FWt9T6lVBvwB4s7rIWn0SgUtrYZy42xtW4rj297fMp9JrdIeLjl4QXPsUsLBiHEcppPwM9rrfNKKZRSAa11l1Jq66KPbBF4lIdKfyX3NNxD1B/lhQsvTMmhL3aLBGnBIIRYTvMJ+JeVUjHge8CPlVLjwJXFHdbC8SkfJV3Cp5xDSaKBKG8OOm38O4c6l3l0QgixdOazaFvOe/xnpdRLQBXww0Ud1QIq6RIKBQosbTGUGaIx2sjF8YvUhGqWe3hCCLFkbuiIQ631PyzWQBaTRmPZFnggaARBw+bqzcRCseUemhBCLJlVfaatX/mxsLC1TTQQxbItKgIV1IfraapoYkfDjin3l41RQojVbD7N026KUqpZKfWSUuqCUuq8UurpxXqu64wBtHPSVb6Ux+/xM54fJ1lM0r6ufUaVTHljVNgX5lT/KY50HFnqIQshxKJZzBm+CfwfWuvXlVIVwFml1I+11kvWh6dgF5wDyq/2zjG1yaaqTdzfdD9BIzhj9i4bo4QQq9miBXyt9QAwcPXvKaXUBZxD0Je08ZpGE/FHaIo20VzVzP3r70drPWswl41RQojVbEly+EqpVuDDwKmleL7JAp4AKGhf107AG0Brfc1gLhujhBCr2aIHfKVUFPgr4He11slZPv8U8BRAS0vLgj+/qU2aIk3EQjF2NOy4bjCXjVFCiNVsUQO+UsqHE+z/h9b6u7PdR2v9NeBrALt27dILPQa/x889Dfewo2GHBHMhxJq2aAFfKaWArwMXtNZfXqznuRYPHqoCVdzTcA/77tgn6RkhxJq3mDP8vcAB4C2l1JtXP/Z/aa3/5yI+pyvsC/PIpkf4i1//C6mlF0IIFrdK52VALdbjX4/f46e5spmJwoQEeyGEuGpV7rQt2kXiuTiN0UZAdtAKIQQs4k7b5TaWG2P/1v2A7KAVQghYxQFfa82n7/00IDtohRACVnHArwpU8cKFFwBorW7lpe6X+MHbP+Cl7pdorW5d3sEJIcQyWLUB/0P1H/rggJPp1f0LXu0vhBAr36pctPXgYSA1QHNVMwDdiW72te4j4A1QMAt0J7qXd4BCCLEMVuUMXymF3+t3Dzhpq2tjID1AwSwwkB6gra5tmUcohBBLb1XO8JsqmlBKuQecSFM0IYRYpQG/YBbYvWE3eTPP0y8+LbX3QgjBKk3pxIIxdq3fRcdQh9TeCyHEVasy4CcLSf7sjT+jNlQrtfdCCHHVqgz4LZUtJAtJTvadlIVaIYS4alXm8NOlNNvrt2Nrm2wpKwu1QgjBKp3hXxq/RKqQ4mN3fYy2uja64l0c6ThCuphe7qEJIcSyWZUz/IJVwOPxcObKGaL+KE3RJk71O8fpyqlXQoi1alXO8D142Ne6j/dG35OmaUIIcdWqDPjVwWoG0gNsq98mO2yFEOKqVRnwN9duZveG3Rx69BC7N+yWhVshhGCV5vDHc+N0DjudMmWHrRBCOFZnwM+Pc2nsErlSDpCFWiGEgFWa0rmj6g4S+YQs1AohxCSrMuC/n3ifqD8qC7VCCDHJqgz4UX+Ukl2ShVohhJhkVebw//k9/5xsKSu5eyGEmGRVzvDloHIhhJhpVQZ827bloHIhhJhmVQb86lC1HFQuhBDTrMqA35fsk+ocIYSYZlUG/PbGdqnOEUKIaVZlwH9w44PLPQQhhFhxVmXA7xjskEPLhRBimlUZ8KWlghBCzLQqA760VBBCiJlWZcCXlgpCCDHTqmytIC0VhBBipkWb4Sul/kwpNayUOrdYzyGEEGL+FjOl8w3g0UV8fAAU6rq3hRBCOBYt4GutTwBji/X47vNMa5oz/bYQQgjHqly0FUIIMdOyB3yl1FNKqTNKqTMjIyPLPRwhhFi1lj3ga62/prXepbXeVV9fv9zDEUKIVWvZA74QQoilsZhlmc8DrwBblVKXlVL/ejGeJ2AErntbCCGEY9E2XmmtP71Yjz1ZwSpc97YQQgiHpHSEEGKNkIAvhBBrhAR8IYRYIyTgCyHEGiEBXwgh1ggJ+EIIsUZIwBdCiDVCAr4QQqwREvCFEGKNkIAvhBBrhAR8IYRYI277gL+xYhb+11wAAAZxSURBVON1bwshhHAsWvO0pdIYbWQiP0E0ECVdSNMYbVzuIQkhxIp028/wt9dvJ+wLk8glCPvCbK/fvtxDEkKIFem2n+EXrSLRQJTW6lbi2ThFq7jcQxJCiBXptg/4f3n+LwG4OH7R/fP5f/b8cg5JCCFWpNs+pSOEEGJ+JOALIcQaIQFfCCHWCAn4QgixRtz2i7b6C3q5hyCEELcFmeELIcQaIQFfCCHWCAn4QgixRkjAF0KINUICvhBCrBES8IUQYo1QWq+cskal1AjQc5NfXgfEF3A4q438fK5Pfj7X9/+3d28hVlVxHMe/PzR96KrdEIvUsEAoyiyMQqiHSYWwoAeflAq6kF0eejB86bWiHqIoukgZ0T1JKjEpSYjSLpjXzLGCTNHCsojo+u9h/4e2w5yT00yzZ+/9+8DhrFln7zNr/Wft/+yzzj7rOD7dVRmfMyLi5CPZcFQl/KGQ9FFEzKq6HaOV49Od49Od49NdXeLjKR0zs5Zwwjcza4kmJfzHqm7AKOf4dOf4dOf4dFeL+DRmDt/MzLpr0hm+mZl1UfuEL2mupJ2SeiUtrbo9I0nSV5K2SNok6aOsmyhpraRdeT8h6yXpwYzTZkkzS8+zOLffJWlxVf0ZKknLJR2QtLVUN2zxkHRBxrs399XI9nBoOsTnbknf5BjaJGl+6bG7sq87JV1Rqh/wmJM0VdKGjNsLksaNXO+GTtLpktZJ2iFpm6Tbs745YygiansDxgC7gWnAOOBTYEbV7RrB/n8FnNSv7l5gaZaXAvdkeT6wGhAwG9iQ9ROBL/J+QpYnVN23/xiPOcBMYOv/EQ9gI3Bx7rMamFd1n4chPncDdw6w7Yw8nsYDU/M4G9PtmANeBBZm+VHg5qr7PMj4TAJmZvlY4POMQ2PGUN3P8C8CeiPii4j4DXgeWFBxm6q2AHg6y08DV5XqV0ThA+AESZOAK4C1EXEwIr4H1gJzR7rRwyEi1gMH+1UPSzzyseMi4v0ojtwVpeeqhQ7x6WQB8HxE/BoRXwK9FMfbgMdcnqleDryc+5djXQsRsS8iPsnyT8AOYDINGkN1T/iTga9LP+/JurYI4C1JH0u6IetOjYh9UAxg4JSs7xSrpsdwuOIxOcv965tgSU5JLO+brmDw8TkR+CEi/uhXX0uSpgDnAxto0Biqe8IfaP6rTZcdXRIRM4F5wC2S5nTZtlOs2hrDwcajqXF6BDgTOA/YB9yf9a2Nj6RjgFeAOyLix26bDlA3qmNU94S/Bzi99PNpwN6K2jLiImJv3h8AVlK83N6fLx3J+wO5eadYNT2GwxWPPVnuX19rEbE/Iv6MiL+AxynGEAw+Pt9RTGmM7VdfK5KOokj2z0bEq1ndmDFU94T/ITA9rw4YBywEVlXcphEh6WhJx/aVgR5gK0X/+64KWAy8luVVwKK8smA2cChfnq4BeiRNyJfzPVnXFMMSj3zsJ0mzc756Uem5aqsvkaWrKcYQFPFZKGm8pKnAdIo3HAc85nJOeh1wTe5fjnUt5N/1SWBHRDxQeqg5Y6jqd8aHeqN4p/xziisHllXdnhHs9zSKKyQ+Bbb19Z1iLvVtYFfeT8x6AQ9nnLYAs0rPdR3Fm3K9wLVV920IMXmOYlrid4qzqeuHMx7ALIqEuBt4iPzgYl1uHeLzTPZ/M0UCm1Tafln2dSelq0k6HXM5Jjdm3F4Cxlfd50HG51KKKZbNwKa8zW/SGPInbc3MWqLuUzpmZnaEnPDNzFrCCd/MrCWc8M3MWsIJ38ysJZzwrfUk3ZYrJD5bdVvM/k++LNNaT9JnFNeZf1mqGxv/rAtj1gg+w7dWk/QoxQeGVkk6JOkxSW8BKySNkXSfpA9zcbEbcx9JekjSdklvSHpT0jVdf5HZKDD23zcxa66IuEnSXOAyYAlwJXBpRPySK5AeiogLJY0H3st/BucDZwPnAKcC24Hl1fTA7Mg54ZsdblVE/JLlHuDc0tn78RRryswBnouIP4G9kt6poJ1mg+aEb3a4n0tlAbdGxGGLyan4GkC/+WW14zl8s87WADfnkrlIOitXJl1PsZLkmFxt8rIqG2l2pHyGb9bZE8AU4JNczvZbiq+kW0nxdX5bKFaNfLeqBpoNhi/LNBsiSU8Br0fEy/+2rVmVPKVjZtYSPsM3M2sJn+GbmbWEE76ZWUs44ZuZtYQTvplZSzjhm5m1hBO+mVlL/A0X0Q+aq/xsQAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = df_movies['count']\n",
    "y = df_movies['mean']\n",
    "x_wap = df_movies[df_movies.index.isin(wap_movies)]['count']\n",
    "y_wap = df_movies[df_movies.index.isin(wap_movies)]['mean']\n",
    "\n",
    "plt.scatter(x, y, c=\"green\", alpha=0.5, s=15)\n",
    "plt.scatter(x_wap, y_wap, c=\"red\", s=15)\n",
    "plt.xlabel(\"freq\")\n",
    "plt.ylabel(\"avg rating\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Watching the red points (_WAP_ movies), they are up to the left. In other words, the six proposed movies seem to be unpopular, but good, confirming the hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### WAP ratings dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first idea for the model dataset is to use as items only the six _WAP_ proposed movies. Since now, we will call this, the `wap_ratings` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_wap_ratings = df_ratings_since[df_ratings_since.movieId.isin(wap_movies)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Target ratings dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, working with only six items could be insufficient for a recommendation system. Then, we can add other movies with the same unpopular-but-good situation, build a more general recommendation system, and test if it improves our results. Since now, we will call this kind of films, _target movies_.\n",
    "\n",
    "To be more specific, we will define a _target movie_ as one with between 1500 and 4000 opinions since 2015, and an average rating greater or equal than 3.6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3Xt0XOd53/vvu/fcMQNgQNwBEhBJCJRIyqRESaEoX5jaEWXLuTh2K9ZtndSqnZUmTZujNI7XiWWvrOrYVdbpaW23rh2dxM6RxVSOndZKTFup6VCkaFEkRQkURQi84UJcB5gB5n7Z+z1/bMwId4IkQALg89HiIgccYDYh8rffefazn1dprRFCCLH2Gbf6AIQQQtwcEvhCCHGbkMAXQojbhAS+EELcJiTwhRDiNiGBL4QQtwkJfCGEuE1I4AshxG1CAl8IIW4Trlt9AFNVV1fr1tbWW30YQgixapw8eTKita5ZzHNXVOC3trZy4sSJW30YQgixaiiluhf7XCnpCCHEbUICXwghbhMS+EIIcZuQwBdCiNuEBL4QQtwmlrVLRyl1GYgDFlDQWu9aztcTQggxv5vRlrlXax25Ca8jhBBiASuqD/96qS+p0q/1U7JloxBCzGW5a/ga+IlS6qRS6jPL8QJTw36ux0IIIRzLvcLfo7XuV0rVAi8ppc5prQ9PfcLkieAzABs2bFjmwxFCiNvXsq7wtdb9kz8PAz8AHpjjOd/UWu/SWu+qqVnUOAghhBDXYdkCXylVppQKFX8N/BJwZqlfZ2bNXmr4Qggxt+Us6dQBP1BKFV/nu1rrg8vxQuf+9Tm6xrpoq2pbji8vhBBrwrIFvtb6IvCe5fr6RZ2RTp5++WlMw8SyLT7/3s/TXt2+3C8rhBCrzqpvy+wa66JjsINIOkK1v5qusS4JfCGEmMOqH63w0oWXOD18mt54L6eHT/PShZdu9SEJIcSKtOoD/2jvUTQaU5loNEd7j97qQxJCiBVp1Qd++7p2FApb2ygU7euknCOEEHNZ9YH/hfd/AT3lvy+8/wu3+pCEEGJFWvWBv+XrWxZ8LIQQwrHqA18IIcTiSOALIcRtYtUH/kfbPrrgYyGEEI5VH/hnR84u+FgIIYRj1Qd+dVn1go+FEEI4Vn3gR5KRBR8LIYRwrPrAn5zGOe9jIYQQjlUf+I9segSFE/IKxSObHrnFRySEECvTqg/8333wd9E4m55oNL/74O/e4iMSQoiVadUH/vXcadsZ6eTFd16kM9K5XIclhBArzqqfh3+tFrNhSmeks7SDlszWF0KsFbdd4HeNdWEaJhsqNtAz3jNrwxTZQUsIsVat+pLONx/75oKPZ2qrasOyLXrGe7Bsa9Y+uFNPCKZh0jXWteTHLIQQt8KqX+E/c/SZWY//1X3/at7nt1e38/n3ft4JdvVuoBdX8Vc7IQghxGq16gP/QvTCgo/nUgz3uUo3U08IUsMXQqwlqz7wbexZjzsjnbRXty948bVYugm4A5yLnOPQ5UOl5xSD//vfh58O3bQ/ylXV1cHHPnarj0IIsZRuZpPIqg/8ufz23/42j297nCM9R+a9+NpW1cZYaozDI4dBw4/P/5i9rXunPWdoCJqbb8WfYG59fbf6CIQQS+lmN4ms+ou2czk/dp6vvvpVErnEvBdf26vb2de2jzsq72Bf2z7C/vAtuUDbN9HHa1eO0zchaS7E7eZmN4msyRW+YRjUB+uJZWILXnzd27qXY73HSOVTt+QCbd9EHy+89QKGMrC1zSe2foLm8hX0lkIIsaxudpPImgz8sCdMQ6iBx7c9jqWteWtjt/oC7UC8H0MZ1JTVMJIcYSDeL4EvxG3kZmfQmgz8S+OX2Fa3jWN9x9jdvHvBb2LxAu2t0BBqxNY2I8kRbG3TEGq8Jcdxo2ZedJI7ld+1kr4XK+lYxLtuZgatycBP5pJ898x3Wedfx7dOfovfvv+32dmw85b/Re+b6GMg3k9DqJHm8maay5v5xNZPTPvYajPzotPj2x7nwJkDcqcyK+uu7ZV0LEtBTl7XZ00Gfl7nAYikIrhNN18+8mX2rN9DQ6hhSf6izwzu+T4+9TEwZ72++GO1mjmq4ljfsQVHV9xOrjbG43Y9lhu11k5eN9OaDPwiG5uclcNreuka6yKZT/LFn32Rf/Gef8GjbY9e19ec70LrzI+/r+V9HO4+XHq8pXrLmqjXT11ZgfP9GEuNAWDZFrubd3PgzAF6xnuIpqP0TfSV7ou43ayku7ZX0rHcqKU+ed1O7xbWdOADuA03PrePZC7JqYFT9E30cfzKcb724a/NCv0fdf2IY33HWF++noZQA0OJXTRTP+05811onfnxc5Fz0x4Dq75eP3VlNZYaQylF2B9GKcUDTQ+U7mPYGN7IocuHONh1kONXjnOs99iaWYVdSzjc6qaAlXosN2opT16327uF2yLw68vqGUuP4Xf5qQvWEU1HOdZ3bFrgf+vkt/jjn/4xhjKI5+Lsbt7NxPl/Sl3jB6eVZwxlzhncMy/AbqnewuHuw6XH2+u2s71u+4L1+r6JPjqGOgDYXrd9VrmoY2iMzoh3Wf5CLibIZq6sFIr31L+H8cw4/fH+0vPaq9vpGuuiKlBVeu6hy4cWHTYz30XM/LxrXZEt1QruesLhVjYFzHSrjmW+7//1/n9ZypPXWip1LcaaD/xMIcNwcpislSVTyNAx1EF1oJr15ev5xolvAGAqky/+wxeJZWN4TA8aTTQTJZ6JlQJ4ZrnG1ta04J7rAmxdsG7OgB+YDMeZgf7t09+mO9YNCk71n+JTOz9Fc3kzJ/tP8v23v4+Z2EDy5VeWbBVS/AdnKnNRF1qLdyefi5wjmUtSU1ZDx1AHpwdPA85souLnFldhHUMdXJm4Qs94DxsqNlw1KOd7F1H8PIDP/f3nSOQSBD1BvvzBLy/4vfhR14945ugzhP1hgp7gDX3vlisclrKksNLKE/OdJG90Zb1UJ6+1VOpajDUf+BYWYxmnxuwxPFjaojHUyNMvP00sE8Pn8pEupMkUMuSsHDkrh6lMLscuk4738aOui0TT0WnlGVtb3N/0wKzXmnkBdubjhW60Goj3kylk8Lv9AKStdOnE8PyZ5xlJjuBJutmYS1x30MxcORf/wXXHuqn0VbK9bvu8ewQUTwyJfIJL0Ut4TA/VgWrqg/XsqN8x63Pbq9t5eMPDfO341zCUQSQaYUv1FlL51ILHP9+7iOLXLr4LCnlDXIpemjYDaa4/75++8qf0TvQymhllc3jzDYX0coRDZ6Tzmk5gV/taK608Md9JcqWsrG9mqWslnIzXfOBPVbALmIbJqYFTWNpCo/HkPSgUSikM5UyaKPeWY2mLglVgKDnIz/t+Tl2wDqBUxpmvU2emqc9b6EarhlAjPpePocQQKKgN1NIQaqRjqIOx1Bg5K0cyHeXM8BlMZZa+/mJ74GeGwe71u0v/4MYz40TT0TmDbOrnXY5eJpqJ4nf78ZpelFJsr9tOIpeY9bmdkU6e73ieVD6FoQzydp7OSCeNocYFg3JqqAbdQZRS0y4ADyYGmdyz/t2f59E11kWlr5JIKkIqlyKajt5QSC9HOBy6fGjRJ7CrWSkhOtV8J8mVtLK+GaWulXIyXvbAV0qZwAngitb6seV+vYXY2Nj29OmaOTuHS7lQKExlUlNWg9twM5oaxTAMDGXidXm5t+FeqgPrMJRJx1AHpwZOEfQESyt1YNYJYK7Onfku3DaXN/OpHZ+aVcPvGOrAZboIGmWMYhNwBzhw5gAbwxsBFt0DP3U6aGekk8H44LvB6gnyxL1P0D3ePet7dujyIa7Er7Clegsuw0VfvI9ULoWNTaWvkr2te2mpaJl1k1vXWBdhf5jRzCipXIqGUAO/fvevzxpQN9PMUAV4vuN5zg6f5aULL2HZFq2VrSRzSbyml5aKlnm/VltVG0FPkM1Vm4llYjz50JM3/I9sWcJhkSewq5krRG/mqnKu15rvJLmYk+dKWBEvlZVyMr4ZK/zfA94Gym/Ca10zU5nsrN9JPBcnXUhja5tMIQMKbNvGpQy8Li/b67YDTi1/ND1K/0Q/reFWClaBH3b+kLH02LQTQDGs++P9hLwh3IYbW1t8YusnSqG+GNvrtnOq/xRDqSGCnjL2bNhTKosAJHIJvC4v2UJ2wR74tqo2oukohy8fBgVl7jI+u+uzpdETQOlkUeyqAWeK6OXoZS7HLlPtr+bO8J10j3eTs3NorTncfZgDZw5Q6aukO9bNxvDGUg0/6AlSF6hj0B7k0zs/veDGNFNNDdXOSCcv97zMQGKAodQQbeE2Ht7wMC93v0ylr5IDZw4AzDlCY66Tx4vvvLiiAmRv614Odh0kkU/QUtHC3ta9i/7cmYE415/3RleViw3dhVaw850kFzp5rpQV8VJZKe9oljXwlVLNwEeA/wD8/nK+1vUyMLgcu4zP7eOX23+ZU/2nGEwOki1kyaIp95aTKWQYSgxhawtDGXhND+O5cd4eeZu8nWcg3o+lbR5ofoBsIVuqvR/tOUr3eDe2tnEbbtymm8ZQI5eilwh6gpwaOMW9DfeWTibffv3bXIhdoGAVONp7lN/a9Vv4Tr7JJ48n6L5jPS9VBEnlXyv9hbkYvcjpwdOldxD7Nu+jO9ZNx1AHsUxsWumnvbqdRzY/wlByiCp/lVO7H+8uvRuZawUCEPaH2de2j85IJ3vW7+Hlnpfxu/2s86yj3FvO145/jVQ+RSQVYXPVuzXy9up2Ht/2OH/6yp/Stq6NIz1HeF/L+64aGjPDpXjtYCI7gaUt0rk0D61/iJbKFjZUbKBjqINnjj5Da7h1VjDMd81iMQFyo6vLxX5+e3U7X/nQV675teYLxKkh+uI7L97QqvJaQnepV7ArZUW8VFZKW+xyr/D/H+DfA6H5nqCU+gzwGYANGzYs8+HMeG0UeZ1nLD1GmDBo6E/0M5AYwMTEtjXpvLPqP3DmADsbdjKSGqEn1oOBgbY1WmvyukAqn+LC2AUaQ42len3WyuJzeUlkk2TsDK/0vOLUv11e7q65mwtjF+gd7+V/X/zf3Nt4L7FsjHQ+DUDveC8XDj7P7gNHqS5YNL5q4v74p9lz1/pSgB3rO8am8Caqy6rJFrKlgXHFrpRi6af4l6ulooW+iT4uxy6j0cQyMVoqW0rloLlWINF0lN7xXoKeIPu376epvImvvvpV6gJ1FOwCdWV19Ez0MJIawe/2T1u5WNoqBfPVWjN/1PUj/vSVP6XSVzmtm6atqg1LW5R7yzEMg8ayRoZTw0TTUQBimRhhf3hWMMx3zSJbyPLWyFs83/E8X9z7xTn/XizUWXLo8iGAeUtTxef8+PyPp3UXXS30rzUAFhOIN7qqvJbQXeoV7EpZES+lldCiu2yBr5R6DBjWWp9USn1gvudprb8JfBNg165dermOZ87Xxnk5C4tIOsLfdv0tE9kJCnbBKelgUdAF/MrPUHKIn/f9HDRU+iops8uYyE6QL+Sd1bvhpq6srlTOGUoMMZQcIpFNYmkL0zBLq3zLtrgwdoGJ3ASmMolmxsjZOXIFp0vIZbgwDZPQ2fN4ChYAnoKF+U4Xj935+6VASuQSXIheIOQNEfQEaatqK9XOvS4viRkdPd3j3aDBZ/pKJaziKvlY37FZ00U7I51ordE4J7aL0Ysc6TlC27o2YpkY+7fv5++6/o5ULoXWmjJ32bTv79R/tN2xbt4eeZvGUOOs9sjOSCfPHH2G3oleIqkIdWV1PNfxHJ/c/knaq9t58qEneeboM7gMFxeiF6hP1KO15oGmB0p39nYMdZS+BzA7rAB6xnt4te9VlFI8+/qzPNj8YOkmMXg3xA9dPkR/vJ/26vZp5bM/fOkPOTNyprRhzsyOmuL/lyvxK1yOXmZf276rdiVdr8UE4o2uKhcbusV3M1ebTnstVsqKeK1ZzhX+HuCXlVIfBnxAuVLq/9Na/7NlfM0bMhAfwO/2Y2ubvM7jVi6qfFXkrfzkRUIPlrap8lVR6atEa025r9z5vck6etFwchiXchH0lDGRi+NWbic8tabCW0GZp4yx1CgAhjJRKDymB7/bj9aaxlAjw20esm/047Ugaypirc5dv1MDCWBr7dZSOM4s80wt6wB4XB5CnhCWtrBte1offXese9aF3qpAFTsqdtAz3sN33vwOZ0fOsq1mGy2VLTSEGnhk8yPEc/E5Wy6L/2ifP/M8r/S8QrqQJlVIzWqPnHqBN5aOMZwcJuQNlfr6H217lI3hjTzX8Rwhb6jUAgpwpOcIlb5KoukoT9z7xLRrFlPDam/rXt4cepOzw2dprmgmmo7yw3d+yEB8wLmmouBg10E+u+uzHOw6yKXYJS5FL7G9bnvpRJrIJwh5nDerM0+mxT+HaZhsqd7C5djlebuSluJi5GID8UZWlYu9sLpctfaVsCJea5Yt8LXWfwT8EcDkCv/JlRz2AHk7TzbrhLZGg1LcXXMXr/W/hkYzkY1T5i6jbV0bveO9tFS2MJoeJegOsqlq07T6/eXYZVAQ9lfhMt3cVXMXIXeI82PnCbgD9E704jY9pAtpgp4gbtPNOv867q69m76JPu6uuZuT5iXOfSDIlr4sA5tqabznHr5x4hv8xem/oDvWTWekk/sa72N38+7SKrR7vJu6YB1V/iq8phdLW6U/397Wvbzw1gtE0k69/cmHnuRY3zGAOXvwZ67Q3xx+k0Qmwfmx8/xC0y/QVtWGqUxs22YgPkDQE8RU5qwLo4cvHyZdSDORnQCg39VfmrEDTjdTLBMj4AqQcWVoX9c+Z1//J7d/kqdffroU4gCmYZaeO/XPOldYffTOj/Lj8z8mmo5ia5vaQC2nB09jY+M1vCTyCY71HaMqUMW+zfs4FznHI5sfKf05gu4gl3OXQUNrZeusIC9+v1L5FNtqtrGvbd+s0s9SBuTNCMSrvcZaq7WvdbdVH/7VzNwQPWdleGvkLTL5DC7lIm/nMZQxeTJwLmhG01FG06OoMcVdFycwD57lpXrFwJ21aK3xmB42VW5i/7b9dAx18Hbkbfom+sgUMrRWtlKwC2wKb+T+pgc43H2YbCHLOv86wr4w3UY35zZV8PqGNBVeDxd6XubU6b/kzPAZgp4gGStDU3lTqbtmLDVGIp9gODHMcHKYbTXbSgFsKucirVKK2kAtQU+QjeGNbAxvnBaiM1v5iqH53Y7vku3POnOJ8snSDWIHzhwg7A8Ty8TYt3nfrLbQ4uq9MlMJOLuRjaXH+M4b3+GFt15AKUUqn+JS7BJ3VNzB+or1lHnK5iwjFEO8WIJpqWjhWO+x0nNnnmymvoMAeLTtUb724a+VWkgBnn39Wcaz42itaSpvKpWIUvkUTaGmUtdM8eLqQjX8xayIpwZkx1DHtNLVarQWa+1r2U0JfK31z4Cf3YzXWlqK3vFe51dKYWubiewEhy79FK/Lx+XYZVK5FD6Xj3DXO/zToxZeC+424b9nJ8jc1czG8B20VLYylBjip5d+WrpgCpreiV7uWncX9zc9wHBymLqyOsL+cKlr50jPEXJWDqUU6XwKlYtTabrJFDIoFJa2SqMiiqtcheLB5gd5a+QtmsubOXDmAIlcgtODp6kL1tE33sc99fdgKIOusS4eu/OxUogOJgZ5/szzvDH4xrQLjo/d+RgnrpzAsi0ydgbLtjg7cpb/8up/IZFLlF67d6J31mrPVCbRdJS6QB1+0080E2UgMcBwahillXOtwhsimUtS5a+iNljLA00P0FzePG9oHus9VmofLdaN5xoPAbM7cx5te7Q0Q+nFd17kgSans2o0PcrH7/54qXw0V2gvZkV9tedMHTkx10iKxVhJ/elSa19dZIW/AIXCxub+Ps2OQThdD681O100hjLJWTnAuXnrPQMa72RFwWdBW2+K401X8Lg8ZK0cR3qOkMglcBturMn2Tr/Lxx3hOzjYdZDuCeeCam2wFnBKLO9reR/feeM7ZAtZLG3hSvTjTQ7jc/lwGS4yuQy9E72lDdCD7iCJfIJX+14FBSf6T9BU3oTX5aVgF4hn44xlxjg1cAq/yz+tvv/CWy9wduQsBauA3+Pnl9t/edoFy/pQPbVltfTH+/G6vAwmBnml9xUmshPEs3EKdoHdzbvpjnVPW3EfOHOAbCHLxcRF9qzfw6XYJYaTwygUBbuAy3ThNb1orRlNj7IusG7BDpjnOp6bdpKxtMVjdz42ZwsisGC5oXivQIWvovS6sLylkmJAPtfxXOn/87WUQlZif7rU2lcPCfwFaDT392n+4BUnxD90EZ55CF5rptRNk7Wy2JbNqXqbf3QRvBZkTDjb5KHcW45CUVNWw2hqlJyVY7LnhXt7Czw4Mk4h3sXrNQm0bYOidDHyXOQcW6q3sM6/jv54v9P+aeeZyE7QHGrmYvQiWmsiqQjba7aztXYr68vX8zfn/oaQN8SeDXsYiA+U+vHH0mOk8k43TVtVG9WB6lLN+9DlQ5yNnCVrZbFsC1VQpQuOpjJLAVPuLSeZT2Iog/HsOBXeCmoCNQwlh9hctZkjPUemdWp0jXUxEB/g7OhZLNviJxd+wuaqzZR7y8lZOdrXtRNwB0gX0nhMD49sfoQHmx4shfVcte/iuxVg2jUDU5lzlhaKH5trNv+tWp3OdT1isaUQqZmLGyGBvwBTmewYdMIenJ93DDqBDxDyhiDrhP9bdwR51u9l5yAcrprgZBPY6RHyOk88E2cg2U+5t5wqf5j2CxP8ztE0PitD9p0TnHmvj4tNNhpNyBOiubyZbCFLNB0lZ+VKc38UzoXloDeI3+XH0haZfIbL45cZS43x56//OYZyauQ1gRoaQg08ce8TpQuzHpeHjqEOPKZnWgsjgFu5SdkpclaO+mB9aQzC1IAZz4wDcDl6mXwhz9mRs2wMb2Rz1eZZK+6iwcQguUIOt+nGY3rY2bCTT+34FODUwS9GL/LM0WdoqGrgjcE3OD1wmqpA1byjIYrlrq21W0v19qmjJWa2BRbLVfPN5r9Zq9Or3RW72GOQmrm4ERL4C9BoTtc7K3vf5Mr9dD0oDKr8YfyuAMlcEtMwCXrK6Gh1c6wxQTxnY1tO6Wc4OcQwQwDkrDw+l497BuzSScRrwd39Od68o5y8VaDCW0G2kCWRS5DIJqgtqyWSjoANLsNdWq2nrTR529nKcSQ1wt+d/zvGs+O0VLaQyqdwm+5SsG0Mb6Q71k0il6Ah2MCe9XvYv31/KWT2tu7le2e/xxuDb2AaJj6Xj5aKltLvT525s612G52jnbhNNxO5CTZVbcJluOa9yPqJrZ/g6ZefJm/liVnOu42pJZtDlw9hGAYNoQan7x9dagOdr2Mo6Amyu3k3x/qOzVnemaq9+t3Z/AF3gHORc6UBZdcyNuBG77q92l2xiyU1c3EjJPAXYGub15qdMs67NXwAm0whi8twRiX0TfQykZmggDNhU8/o9imydIFsIcvrdfC+rndPIq/Xa1L5ND6Xjw2VG6gpq8FjeohlYmyp2gLg7Jpl16CVQTQTxWN4yNvOTV9a61JJ4/zYeVzKKTUVtVe/O+agqbyJS7FLpd8rhtmW6i30TfSRK+RI5VM8c/SZ0l26j297vNTZ8vXjXy+NkC7elDX192cG0M6Gnbx3w3uJZqJciV/h5MBJfvtvf5snH3qSjeGN02b1tFa2EnQHF+zQmTq/f2Z5Z77VbnGO/+GRw6WbploqWha1B8BS1MyXugwjNXNxvSTwr0Jh8FqzXSrjFLVfiLFzcJzTDYqeDR5s26bMGyBuJ7D03IEPzknk5HoP//lh2Nqf52yTl9caC2AXSOaSDCeG6Yx00hBsYCAxADjtn3s27OFvT5wmVkgRz8aZvEkYhULj3OW6ed1msoUs9zXch9flnRYsc405KI4AMJTB8SvHS5vENIYaCfvDdI11cbj7MF999avUB+t5c/BNzkbOotEUdAFDGShUKThnjnIAJ2wbQg3Y2AwlhhhLj2HZFs8cfYZ/vO0fT5vVUywjFVsfZyoGXfEC7dTyzkKtje3V7exr20cynyzdPbvYzdaXIqylDCNWCgn8BTkrWI/hxdY2Be2UUO7vgydfAZ+l+UcXNf+JPK81GySySXwun3NxdJ5VvkaTsdIcbYSjjWCoAqYy8Zpe3KYb0zAxlIHLcBFwB8haWX5p0y/ROdpJNBNFadBal+4ZcBtuNldt5lfv+lUiyQgnBk4wkhwhnotzouKEs/KfbFucegHz26e/zWBikKyV5d6Ge/G5fGwMb+SdyDuEfc7uUAPxAb5w6Ask80l64720VrQC4FIu52tiotELBuLU3vm/OP0XjKZGCXgChP1hgNKNSo2hxlKXzNS2y7lW1DPLO5/c/klg4UmYe1v3cqz3GKl8atZm63OF8NRNX240rKUMI1YKCfyr8BgeCnaBXX0W75ks68y8kLttwOJ4s9NT3hhsIGvn6Jvow55y52fRrBOBdkK7NlBLspDEbbjJWTnOR8+TyWeI5+IcOHOAvok+ktlKyMYAZ8qnz+WjqaKJX73rV3lj8A06hjoYz46TyqfwmB4uRC/w30/8d7bXbqdgF9i/fT8NoQYOdh3k7y/+PRpNupDm7MhZFIrqQDVV66t4ZPMj7G3dy3Mdz+ExPdjadsYi5FOEvCE8pgdDGYS8Idb513F25CzjmfF5yyrFlXlLRcu07Qb3tu4tXRguBuFiJjzODFC4+iTMuUJ3vn77mWWcpZgRI2UYsRJI4F9Fzs5OWdE7F3D/Z7tTe596IdfSFpZlcSF2EY/hmTPsFc6OWlND38bCwmIkPcJH2z/KpvBGusbO8/O+n5N35/GYHkaSI2QLWWe1iVPGcZtubG2Ddur7Q4khbGwSuYQzIsLK4jW9FKwCZyNn8bv8PPv6s/zGjt9gLDPmlGXsAgpFtpDlix/4Ig2hhmmhtrt5N8+9+Rw+lw9DGTz50JM0lzfzpX/4EoYynBER471zzrKZy3w3Nc0sAS1mRT01QBc7Bnhm6M4XwjPLOHNdDBZiNZLAX4SZK/pAfq4LuU5R3dYWGStT+lxDmdjaJuAKoLHJ2wUKM3bdchkuMgXnJqrHtz1OQ6iR1wded/axNT34XX7iuTgaZ2Xvd/udwLdthlPDnB48zaXxS4ylxpwWTu0cS95ySlC5Qo6MlAaQAAAgAElEQVSgO0h/vJ/vnf0eWuvS1M6AK8DWmq00hBp47M7H6Ix0lkojxVEEL77zIjVlNaV59sXQ7pvo4/iV49OC8WquttK9nvKHjOYVYnEk8BdhrtbM15qZdSHX2aNOT/5wtkz0u/zcEb6DoeQQ6XwarcFWNmWuMrJ2lryVJ1PIYGmLC6MX6JvoYygxRCQVwTRMClaBD7R9gJP9JxnMmNSva+ODGz/IqYFTdMe6iefi9Mf7qS+rx+/yk7NypHIp4vk4QXcQ0zRpCDZQ5imjkHTm1w8lh/jw5g9zZvgMdWV1pZX9XDPpN4Y3ksglSBfS00YAFNsap86yWapgvNbyx1LXyKXmLtYqCfxFmLs1c7qAu6zUHpkqpPCZPrS2qfBVMJIcIVPIOCt9TwA7Z5OzczSGGknlU4ylxvCZPnJ2ztm0PD2Gx/RQ5Q8TSY3SN9FHpbcSysrYsm4LH2n7CG+PvF2am29rG6/Lyz0V92AaphPkqo66YB05K8cvbfol6oP1vPDWC7x65VXQEPKE+PIHvzxti8OpM+mLu1fB/OMJFhOMN2vuy1LXyKXmLtYiCfyFlA3DRBMAr5U7PwCYmP40hUE41MREdgK/20dAg8orXKabeGQCv8uPH5jITJA2wC6U4zHdYNdyV2ULp0ZfJ+D2Y2UsosMBqv3rSY+dYSyfRhPmfDpNbVkt92yqIBiowtLWtE1BCnaBP9jzB6XNPMaz47wz+g4n+08S9od5Y/ANHrzvQbZUbyGdT3Nv471Ot8qU2vSL77w4bdPxWCY2azzBXKv4hYJxJc59WUmDx4S42STwF3L3D2Z9qNj3PpWBiT98B7nMOI3lTXTHunG73DSWt3B5/DJZK4/f7acKg6HkkFPrdvlprNvB3Q3vwTX6jrNrFJp/8f6HeLTt/VT/zMvfnPsRW2u2kin0EsvEsHyVdMecu1XnuwDaNdbFhooN+EwfpwZPcU+dMxmzeHK4FHP20y2WcYqKg8Q2hzcTTUd58qEnS1/vertUusa6pm2yfqvnvtzME5CcWMRKJIF/DUxMKvwVjKXHpn3cxmYwMUiqkCKei5OxMhhZg0gygmEYVPmqGE+PY01umQiQKqQ4PXSaMk+ZcxctsLlqc+nmpf3b9nMpegnTMHEZLvZt3sezp54la2X50s++BDhdL3P1qI+lxhhNj6K1Ziw9hq1tyj3lDCWH8JgeBhOD/Lvd/27Ofvnr3fR7zu+XMhfcfetmu1mDx1biOxshQAL/mpiGSYV3duBrtDM1U9ulDp3iVExlKyKpyKzNVcDpouke7yaejZO386yvWA85ps2pLwbwocuHGEgMMJGbwLItvvQPX5p1V2uRUgqlVOlxmbuMidwEqXyKSn8lm8Ob5+yoWajVcb4NyBdayVraYkf9jtIKfzFdPMvpZnXfyERLsVJJ4F8DjWadfx0D8YFprZdAaZDZ1OeCs/qfWQIqKugCkWSECn8F0XiUV3pfIegJ8sS9TwDTA/jQ5UPOZiiT3T/FDUxmhu9zHc9hKIPWylZGkiO0VrY6M3oqNnD48uHSTU9Td4cCZoV2W1Ub0XSU3vFeNJqx1NisKZZXW8kWy0TFDdyXY2/Xa3Gzum+krVOsVBL4i2Qqk4JdYCQ1Utr4ZCEK5cyPzyYpUJin9m+QsTJUUIGtbTyGB7SzL+1Me1v38r3a7/HW8Ft4TA+1gdppQVIM34H4AG8Ovcmm8CZsbZMtZHEbbvZv28/+bfunDR8zDWc3qonsRGlW/lMfeKq0I1RxOFoim6DCWzFrc5GZm5HMN1ZhMXez3qyyx83ovpG2TrFSSeAvUrEcMVcYz8XE6cH3ml4iqUgp7A2M0q99Lh8BT4Dx9DgKZ29Xr8s759drr27n6x/++rT9XIsD0IrjCRK5BEOpIbwuZ0Puzz38uVl3zxbHFxQvpl6OXaZvog9LWxTsQqlUVBwpvKNiBx1DHcQysWk7Wc21GclCYxVmWutlD2nrFCuRBP4ysXCC8QN3fIBX+16ld7wXhSLsD5fCvabM2S1Ka43LdDZJr/RV0lLRMucgsKk3PH3u7z9Hx1AHKDjYdZDP7vos0XSUWDqGoYzSDVVzjQSYejE1lolRsAvY2sa27VI3zcwBZU/c+8S0nayuZVrlXKTsIcTNJ4G/RHyGj4w9va4/nh2nJ9pDJBlxRisojc/loyHUgEJxfux86YYsv8tP0BPkfS3vu+qc9uJqPuQNAZDIJ7C0xf7t+/njQ3+M1/RyIXph3q6YqRdTI8kI3ePd9Ez0YGAwmh7FVOZVyxIzp1XKRh5CrHwS+DeoWKKxsUtjg4vDzdaH1tM90U3Wyjq7YrmCKBQfafsIg4lBLkYvYmCAcrZJrA3Wsi6wjmQ+OW+pozPSSd9EHxrtzMVXUOWvKm1kvmf9nqt2xUy9mNoQauCh9Q/xkws/ocpfhdflLX3efGWJpQprKXsIcXNJ4M/DrdygZnffzKXcW47H8NBa2Up/op+claMt3EbnWCfJXBKNxrItEvkE1YFqvv7a16nyVTGSGinV801lUumt5I3BN0rDz2aWOqZu5D2SGmFLzRZaK1u5MnGF41eOM5YaQymFz+Wb1hUzbbb7ZFlmZs/9pdil0ruKxZRXJKyFWH0k8OdQ3Nhjnm7KkjJXGaZhUuWvIl1IM5gYxNIWPtPHzsadjKXHSOQSeAwPmUIGv8tP93g3Wmui6Sjw7jsEt+Hm7pq78bq8PND0AM3lzbNWz8VSzlsjb9E/0c9wYphIMkJTeRMbKjYAzPrcqSeJ04On2VG/ozQYbWp9X8orQqx9EvhzsLAwtDHnzVJFbuUm4A6QLqSp8FYQ8oQo85RxR+UdvHrlVU71nyKSiTiz6XXW6Z83TLJWlqyVLa3siz97TA+vXnmVbTXbpm3yPVVbVRuxTIyJ7ARKvfv1pnbQzPzc4gVWr8uLoQy8Li+mYc7ZQrmYoJeRAUKsXlcNfKXUx+b48DjQobUeXvpDWhkWCnsAt+lmPDtOhbeiNDagP95P0B1kW802msqbGEwOOi2XSmFpZ/Z81srCjNK6gcG9Dfc6IxTa9i04dfLxbY/ztde+xnhmnFgmRraQ5WN3fYydDTvnDOFiN0wkGSGeixNJRmbN0VlsiMvIACFWN2MRz/k08GfAJyd/fAv4feCoUuqfL+OxrWipQoqcnWM0PcqFsQu8HXmbVC7F8f7jtFS2lFoWixdAPaaHh1sf5r6G+5wLtVO4DBdjmbFp+7pOVQza77/9fY70HOHXtvwad667kwpvBQF3gEOXDi0Y1ndU3sFEboJ7au+hYBd4fNvjpedO/dpPv/w0nZHOef/MU3vni+8ShBCrx2IC3wbu0lr/utb614G7gSzwIPCHy3lwq4GNTbKQLG0sXrALfOeN7/CXb/wlw8nh0rWAkCfEL7b+onOX7rtjbvAYHrwuL+/d8F4+/97PA84cm2LwFsclJHKJUtDWB+up8lcRcAeo9FcS9odL4Vvcsaoz0lkK859c/Ak94z3UlNXQGm6d1r1zLSEuvfNCrG6LqeG3aq2HpjweBu7UWo8ppa7ewrIGqMmELs6en2s2TtbOYmWcm61MZfLWyFuk82kUCo/Lw/ry9ZwePM2V+BU8pod0IY3X9OIxPWyv2U48G+di9OK0HvzHtz3OgTMHZt3Rurd176wNwduq2maVXO4I30HXWBeD8UESuQQ/vfhTfmH9L8wai7zYEJfeeSFWt8UE/stKqReBFyYf/zpwWClVBsSW7chWEAODsD/MpvAm7lx3Jz849wMS+cS059QEasgWsmityVk5LG1hqMlNy7XG6/JSU1ZDIpdAa03AFaDSV0ltoJaWcAuJXIJjfcemjRsoPi6Wh+qD9aVfzzUPf+qEy46hDg6eP0h3rJtkPkl1oJq6sjoe2fzIoufdzEXaMYVYvRYT+P8aJ+T34BQjvgP8tXaaxWcXnNcYE5N1Zev4xN2fAOBE/wmay5s5P3a+NNseoC3c5oS8YbDOv46f9/2cgl3AbbppCDbw1Puf4kT/CeLZeOnzas1aRjOjHO05StbKsqN+B5Zt0THUQTQdZXfzbrpj3fSM92BrmysTV0gX0hzrPTZtb9nSMUxZrUfTUZpCTWwKb+Knl35KfbCetqq2Oa8RSIgLcXtQxZt8VoJdu3bpEydOXNPnqC+pqz/pBigUO+p24DE9XIheIJFNsK7MuRs2X8jjdXlpKW/hsS2Psbt5d6kkM5YaY0f9DupD9ext3cvF6EWe+OETDMYHSyWhmkANTaEmLsYuYtkWG8Mb+Z0HfocDZw6UNhIv7jbVN9HH8SvHS6v/j931sTnn5Ey9yWrqRMxHNj8yb7unEGL1Ukqd1FrvWsxzF9uW+RWgFmeFrwCttS5f8BPXCI3mUuwSHtNDKp+ioAuMJEdoKm+iNlBLU3nTtHkyc2072Bnp5Jmjz5DOp0vtnsWRCp2jnaQKKed5o5389PJPaalsKQV78c7Yvok+xlLOxisL1dqnrtbnOhYhxO1rMSWd/wh8VGv99nIfzEoVy8ZK8+wNDAzDIOQN8dQHnpq112t7dTsXoxd5ruM5djfv5tG2R+ka68JluAi4AyRzSaf0g0E6lyZvOde9FYq8nSeeieMJeGaNIjYNE6UUDzQ9sOiVupRqhBBTLSbwh64n7JVSPuAw4J18ne9prZ+61q+zUky9I7alsoXN4c2l2fjFwWXt1e186+S3+MKhL+AxPfz563/Op+/9NKYyeXPoTQxlEPKE2BjeyPnoeTymM3JBoTCUgdtws6txF/u37y+tzKe2TQI0lzdLiAshrstiAv+EUuqvgL/B6b8HQGv9/at8Xhb4Ra11QinlBo4opX6ktf759R/uraVQ1AXr2FqzFVvbvPDWC07oa/jx+R/zmfs+w1df/SrJfJKclSNTyHCg4wCxbMwZoKYtNpRvwO/2kyk4o5RNw6TKW0Wlv5Lm8mb2b98/a2Uuve9CiKWwmMAvB1LAL035mAYWDPzJLp5i76J78sfKuUI8j6mlG1OZ5PW7txr4TT+P3fkY22q3AfC9s98j5JmcSZ9LlNoiAdL5NJa2cBtubNsmVXC2EHx79G1aK1sxlEGZu4yQJ8Rv7vxNdjXumrPWPlfbpMyzEUJcj6sGvtb6N6/3iyulTOAksBn4utb61ev9WjeL1/Ri2RZVgSoMDIaSQyilnD1nXR5+2PlDTvSf4NM7P03QHeRy7jJoZ7vCIz1HiKajBD1BDAyydpaJ3AQT2QnKveUopXAZLjaFN5HMJQkHwtxRcQf7t+1fMLin7nT1jRPf4GDXwVkbigshxNXMG/hKqX+vtf6PSqmvMsfKXGv9b672xbXWFrBDKVUJ/EAptU1rfWbG63wG+AzAhg0brvX4l5RC4XV5CbqDjKZHMZSBoQwaQ43Oat/KE01HyVt5nj31LPs276O5opmx1BhvjbxFKp8i4AlQ5a9iZ/1OBhODeF1ezo2cI5qJ0hhqLH3svsb7rqlVsngXbX+8n0uxS+zbvI9UPrXm9oIVQiyfhVb4xQu119YYPwetdUwp9TNgH3Bmxu99E/gmOH34N/paN0KjKdgFPC4PtrZBO+MUTGUSz8cZT49jY1PuK2cgMcBPLvyEoeQQdcE6YpkYPrcPy7bwu5zST3Eswnh2nLaqNgp2Yc6NxRejePG2vbqdS9FLnIucoynUdF1TL4UQt6d5A19r/cPJX6a01i9M/T2l1Ceu9oWVUjVAfjLs/cAHcfr5VzRb21i2VdrpysBwRiXYFmWeMpL5JB7Tg4FBld/ZtarKX8VwYph1/nX4TB9PPvRkafTBcx3PAbC9bjs94z3zbix+NcW7aFP5FNvrts96dyCji4UQV7OYi7Z/xLtzdBb62EwNwLcn6/gG8D+01i9e+yHeXOlCmv54Py7DhdYa0zCJpCJopSlzlzk7U1Xfjdt0YygDW9t4Te+cIdxe3c7u5t0c7TlKx1BHacjZ9bjazJup7Ztz7YUrhBAL1fAfBT4MNCml/suU3yoHCnN/1ru01m8CO2/4CG8BpRQe04PH8JC1srgMF1k7W+qqeaz9Mfa27qVrrIuB+ACnB09TU1Yzqx7fGeksjUmIpqPs27yvNH54Zhgvphyz0I1UMrpYCHE1C63w+3Hq97+M02lTFAf+3XIe1K2kUKChNlCL23STKWTwuXzEs3HKPGVU+ippqWgpBe+fnfwzzoycAQ1Heo7w8bs/Xgr+4qp7e912OoY6OHDmAC2VLbNKLktRjpHRxUKIq1mohv8G8IZS6rta69ti7r0LF8pQpRus/uXOf1laoY9nx0lkEzSFmjhw5kBpTk0inyDkCZEpZDg7cpa/PvvXpWmWxVV3x1AHXaNd1Afr5yy5LFU5RkYpCCEWsqgNUJRS/xfOTle+4ge11huX7ahuApdyvdtfb3gI+UIorchaWUzDJFPI0BBq4L9+5L/SNdbFif4THOk5gs/l40r8Cs+feZ76YD1aa+K5OMlsErfLTXt1e6ldsq2qjTsq7+DghYPUB+u5EL1AyBuaVcuXcowQ4mZYTOD/OfAU8J9w5t//JtM26Vt9infTFoeT5a08E5kJbG2Ts3MYGCRzSQ6eP0jbA220VbXx7KlneSfyDq9eeZWgO8g7o+9wf+P9lHvL+Wf3/DMiqQgn+08yEB8g6AmWhp71x/uJpCLsatxFyBtia+3W0mTNIinHCCFuhsUEvl9r/b+VUkpr3Q18USn1Ms5JYFVyGS4s28KjPGS1Mx6oYBUwlTMWIegJYmmLIz1HiGfj7F6/m7A/zD1193Bq8BRNoSZimRhelxefy0d9sJ5L0Us0hhqJZWI8ce8TWNqa1Tcf8oRoDDXOeUxSjhFCLLfFbGKeUUoZQJdS6neUUr+GMxt/VTIw8JpevKYXl8tV2q8WKG3unbfzaDRba7aWZuNYtoXX5cXv8hP2hbG1TbaQxbKdzylenG2pbCmNTI6mo3RGOmkNt/LwhofRWnP8ynGefvnp0iblQghxsyxmhf9vgQDwb4A/wSnrfGo5D2o52dil/WiVpaZtSO41nE3FQ94QlrbIFDK4DBd7W/eW2jBNZTqr98mfi/X2H5//Mb3jvdPq81prNJqgO0h9sJ6qQJX0yQshbpkFA3/ypql/rLX+A5zJl9c9SG0l0jNGBOXtPFWBKn7trl9jID4wq94+X0B3RjpL4V7cMrJrrIuqQBU7KnbQM94DyJhjIcSttWDga60tpdR9k/X7FT/a+EYpFF7TSyqfmrZt4dXMDPfixdepAT/1XYJcmBVC3AqLKem8DvxPpdQLQLL4wUVsgLKqKBQu08Vjdz7GI5sfwVTmvHfFzjRXW+V8nTcS9EKIW2UxgV8FjAK/OOVjV90AZbUwMPCYHpRSrC9fz+888DsA13Tn68xwB3jxnRdpq2q7rkFpQgixHJZ1A5TVwDRMfC4fKAj7w8D13fk6dZMSmVophFiJFtOWuWa5DTduw03OyhH2hemOdfPV41+lb6KPaDp6XRdYiyeLgDvAlfgVDl0+tIx/AiGEWLzFlHTWLK2dDU8KdoHh5DBaa/7qzF8x0DKA1poHmh4oDULrjHSWwnuhXaraqtoYS41xeORwaXPzxe5qJYQQy+m2DHw1+Z+pTEzDRGtNtpCl3FdOwB0gW8iStZw7cIth/7m//xwdQx2g4GDXQb7yoa/MGoVcrOHva9tHMp+cNldHAl8IcatdNfCVUr8/x4fHgZNa69NLf0jLK+AKkLWyeEwPBavgbGNourC1jW07s3TeHHoTj8vDwa6DpVbKRC5ByBsCIJFPTAvxmXX7x7c9TmOokVQ+JT33QogVYzEr/F2TP4pbHn4EeA34LaXUC1rr/7hcB7eUDAxQEPKEsDM26/zr8JgehpJDmJi4XC4qfBU0lzeTt/JsDG9kKDnEocuH2Nu6l6AnyKXoJVDQUtEyLcRnXuS1tCXD0IQQK85iAn8dcK/WOgGglHoK+B7wPpyNUVZF4HtMDy0VLfjdflL5FMl80mnFrFjPhdELmLbJeHYcgFQ+xZX4FTyGp1SD//IHvzxvDX++PnwJeiHESrKYwN8A5KY8zgMtWuu0Uiq7PIe1tAwM1gXW8aFNH+JE/wnKPGUU7AJhX5ittVtJ5VKkC2lGU6NMZCYwDIOAO8AHN36wVIN/7M7HFtx6UFb0QoiVbjGB/13g50qp/zn5+KPA80qpMuDssh3ZErKxiaQiHO4+TJmnjPpgPQDrAuu4c92dvNz9MuPZcTSajJWh3FVOpbfymmrwsqIXQqx0i7nx6k+UUn8HPIyz8clvaa1PTP72J5fz4JaSx/AQz8YBSOaS5O08XtPLg00PAnCg4wCJfAJb22yo2MBT73+qNA1TglwIsRYspkvnPwN/pbX+zzfheJZNPB8nn8hT7itnS80Wesd7KfeW86WffYn7m+5nS/UWTMMkmo7yB3v+gEfbHr3VhyyEEEtqMSWdU8D/qZS6E/gBTvifuMrnrEhuw82ViStoNNF0lHguTiQVoXu8m621W/n43R9nb+te4N1ZOLK6F0KsFYsp6Xwb+LZSqgr4deArSqkNWutV11yeLqSxsanyVdET68HWNqYyqfRVolA0lzcDzOqpn7rRiVyYFUKsVtdyp+1mYAvQyiq5WFtU3MZwS/UWIqkI5yLn8Ll8ZKwMfpefvJUv7VQ1taf+lZ5X+MOX/pC2dW3Y2kZrTVWgSoaiCSFWpcXU8L8CfAy4APwP4E+01rHlPrCl5FIuTMPEZbjwurwYymB3824yhQx1wTruqbun1Ft/MXqR7lg3feN9dAx34DE9nI+ep8xVRsATmLbJyfUE/tQRDHLCEELcTItZ4V8CdmutI8t9MMtFa41C0Z/ox8AgkU/wcvfL3Nd4H7/34O9NG5Fw4MwBKn2VdI12sbFyI4l8glQuRcAVIOgO3tAWhTI6WQhxKy2mhv8NpVRYKfUA4Jvy8cPLemRLqEABy7IopAs0lzdT6avE4/LwyOZHpgVusZyzvW47ALFMjPpQPbFMjCcfepKN4Y03tDq/njn7QgixVBZT0nkC+D2gGTgN/AJwjOk7YK1YCoVG43P5yFk5IskItcFa2ivaSx05RVNHJAQ9QZ6494lZvfg3EtBzjWAQQoibZTElnd8D7gd+rrXeq5TaAnxpeQ9r6Wicvde11rgMF1uqt3B/0/1zjkpY7hEJMoJBCHErLSbwM1rrjFIKpZRXa31OKbVqksrAQKPJWTkqfBUMJAYYTAxy4MwBNoY3zhn6yxnEMoJBCHGrLGaLwz6lVCXwN8BLkzN1+pf3sJaGgYGhDNyGmzJ3GZvCmzCUQSQVIZFzZtoLIcTtYjEXbX9t8pdfVEodAiqAg8t6VEtEobC0hc/lw+v2ksgmiKSdZqO+iT5MZd7iIxRCiJvnmrY41Fr/w3IdyFJzKzfVZdVEU1HK3GWlbQvLPeVsr9uO1/RiaetWH6YQQtw0iynprEqGMoilY+R1nvHsOHmdJ2fn8JgesoVs6c7amTojnbz4zot0RjpvwVELIcTyWbZNzJVS64HvAPWADXzzZk7cVEpR7itnIjOBpS0MbWDbNmWeMvas38P+7ftnXTyVG6OEEGvZcq7wC8D/obW+C6d3/18rpe5extebJmNlGEmO4DbdlHvLaa9qR6NpqWzhUuzSnJ8z9cYo0zDloq4QYk1ZthW+1noAGJj8dVwp9TbQxE0cvKbRFOwCv7HjN6jwVfDW8Ftsr9s+712ucmOUEGItW7bAn0op1QrsBF69Ga9XpNForTl06RBf+dBX6I51LxjmcmOUEGItW/bAV0oFgb8G/q3WemKO3/8M8BmADRs2LMfrUx+sx9LWosJcbowSQqxVyxr4Sik3Ttg/p7X+/lzP0Vp/E/gmwK5du/SSvfbkfxsqNtAQaiiFvIS5EOJ2tZxdOgp4Fnhba/1/L9frzCfkCXFXzV38xo7fKM26F0KI29lyrvD3AP8c6FBKnZ782Oe11n+3jK8JOGHfGm7lqfc/JZuRCyHEpOXs0jkCk3sL3mQBd4BkNkn3ePeteHkhhFiR1uSdtpFUhLydn/YxuYNWCHG7uyltmTebpS0MZZQ2OJE7aIUQYo2u8AGqA9WlUJc7aIUQYg0Hvstwlco3bVVtRNNRjvYcJZqOyh20Qojb0poM/Fp/LY2hxmkrea116c5bIYS4Ha3JwM9YGU4Pni5tcNI11kVVoIqHNzxMVaBKSjpCiNvSmgx8l3KxKbyptMGJDEUTQog12qUT9Aax9LvBLkPRhBBijQZ+Y6iRJx96clqwyxwdIcTtbk0G/iObH+HRtkfpjHTKql4IISatycA/2HWQplATR3qOyM1WQggxaU1etB1Nj/K1418jkUvIzVZCCDFpTQZ+wB2grqyOaDoqnTlCCDFpTZZ0claOgDvAZ3d9ViZmCiHEpDW5wh9JjuDsvwLHeo9x/Mpxnn75aZmUKYS4ra3JwC/zlBH2hznWd0yGpgkhxKQ1GfjN5c1YtsXu5t1yh60QQkxakzX8P3rvH5V67zeGN0ovvhBCsEYDfyA+UPq13GErhBCONRn4Xzj0BR5a/xBBT1BuuBJCiElrsobvMT14XV65UCuEEFOsycC3tU22kJULtUIIMcWaLOm0V7fzoU0fYm/rXinnCCHEpDW5wt9QsYHm8mYJeyGEmGJNBr5sVC6EELOtycCXjcqFEGK2NRn40p0jhBCzrcnAj2ViUtIRQogZ1mTgz9zPVgghxBoN/I3hjbf6EIQQYsVZk4Evs++FEGK2NRn4ctFWCCFmW5OBLyMVhBBitjUZ+DIhUwghZluTgS9hL4QQsy1b4Cul/l+l1LBS6sxyvQZAY7BxwcdCCCEcy7nC/wtg3zJ+fQDCvvCCj4UQQjiWLfC11oeBseX6+kX/ZNs/WfCxEEIIx6qv4X/hZ19Y8LEQQgjHLQ98pdRnlFInlFInRkZGbvXhCCHEmnXLA19r/U2t9S6t9a6amppbfThCCLFm3fLAv1H/7SP/bcHHQgghHMlRZ8EAAAc1SURBVMvZlvk8cAxoV0r1KaU+vRyv83L3yws+FkII4Vi2Tcy11vuX62tP9dLFlxZ8LIQQwrHqSzqJbGLBx0IIIRyrPvDTVnrBx0IIIRyrPvCFEEIszqoPfNeMyxAzHwshhHCs+sD/X//0fy34WAghhGPVB/6Hv/vhBR8LIYRwrPrAF0IIsTgS+EIIcZtY9YHfWt664GMhhBCOVR/4g4nBBR8LIYRwrPrAv7P6zgUfCyGEcKz6wP+VLb+Cx/AA4DE8/MqWX7nFRySEECvTqr9LaX35ejymB6/Li9aa9eXrb/UhCSHEirTqA78h1MA9tfcwnh2nwltBQ6jhVh+SEEKsSKs+8E1l8v+3d38xVpxlHMe/vy7ChVaFtjaEEqGmbdKkxiIaTBuSekGBhKBJL7gqaU36J61/Lryg6YXcqrEXpkZCW9JiCFVRIlEJEG1sYiwtEsrfUrbbGhEoJQg2pFGLjxfzbJxd90x3u4ednZnfJzk573nPnMP7Przz7Jx35rznzKUzXKWrOHPpDAMaqLtJZmbTUuMT/uW4zNCFoRGPzczs/zX+pO2qrasqH5uZWaHxCd/MzMbHCd/MrCMan/DjO1H52MzMCo0/aQtO8mZm49H4I3wzMxsfJ3wzs45wwjcz6wgnfDOzjnDCNzPrCCd8M7OOUMT0uaRR0jvAXz7ky68FzvWxOW3j+FRzfKo5PtXqjM+nI+K68Ww4rRL+ZEjaFxGL627HdOX4VHN8qjk+1ZoSH0/pmJl1hBO+mVlHtCnhb6y7AdOc41PN8anm+FRrRHxaM4dvZmbV2nSEb2ZmFRqf8CUtl3Rc0qCkdXW3ZypJekvSIUkHJO3LujmS9kg6kfezs16SfphxOihpUel91ub2JyStras/kyVpk6Szkg6X6voWD0mfz3gP5ms1tT2cnB7xWS/pbzmGDkhaWXrusezrcUl3l+rH3OckLZS0N+P2U0kzp653kydpvqQXJB2TdETSN7O+PWMoIhp7AwaAN4AbgZnAq8CtdbdrCvv/FnDtqLrvAeuyvA74bpZXAjsBAUuAvVk/BxjK+9lZnl133z5kPJYCi4DDVyIewMvAl/I1O4EVdfe5D/FZD3x7jG1vzf1pFrAw97OBqn0O+BmwJssbgIfr7vME4zMXWJTlq4HXMw6tGUNNP8L/IjAYEUMR8S/geWB1zW2q22rguSw/B3ylVL85Ci8Bn5Q0F7gb2BMR5yPi78AeYPlUN7ofIuJF4Pyo6r7EI5/7eET8KYo9d3PpvRqhR3x6WQ08HxH/jIg3gUGK/W3MfS6PVL8MbMvXl2PdCBFxOiL2Z/ld4BgwjxaNoaYn/HnAX0uPT2ZdVwSwW9KfJT2QdddHxGkoBjDwqazvFau2x7Bf8ZiX5dH1bfBoTklsGp6uYOLxuQa4EBHvj6pvJEkLgNuBvbRoDDU94Y81/9Wly47uiIhFwArgEUlLK7btFauuxnCi8WhrnH4MfAb4HHAa+EHWdzY+kj4G/AL4VkT8o2rTMeqmdYyanvBPAvNLj28ATtXUlikXEafy/iywneLj9tv50ZG8P5ub94pV22PYr3iczPLo+kaLiLcj4nJE/Ad4imIMwcTjc45iSmPGqPpGkfQRimS/JSJ+mdWtGUNNT/ivADfl1QEzgTXAjprbNCUkfVTS1cNlYBlwmKL/w1cFrAV+leUdwL15ZcES4GJ+PN0FLJM0Oz/OL8u6tuhLPPK5dyUtyfnqe0vv1VjDiSx9lWIMQRGfNZJmSVoI3ERxwnHMfS7npF8A7snXl2PdCPn/+gxwLCKeKD3VnjFU95nxyd4ozpS/TnHlwON1t2cK+30jxRUSrwJHhvtOMZf6O+BE3s/JegE/yjgdAhaX3ut+ipNyg8B9dfdtEjHZSjEt8W+Ko6mv9TMewGKKhPgG8CT5xcWm3HrE5yfZ/4MUCWxuafvHs6/HKV1N0mufyzH5csbt58Csuvs8wfjcSTHFchA4kLeVbRpD/qatmVlHNH1Kx8zMxskJ38ysI5zwzcw6wgnfzKwjnPDNzDrCCd86T9I3coXELXW3xexK8mWZ1nmSXqO4zvzNUt2M+N+6MGat4CN86zRJGyi+MLRD0kVJGyXtBjZLGpD0fUmv5OJiD+ZrJOlJSUcl/UbSbyXdU/kPmU0DMz54E7P2ioiHJC0H7gIeBVYBd0bEe7kC6cWI+IKkWcAf84/B7cAtwG3A9cBRYFM9PTAbPyd8s5F2RMR7WV4GfLZ09P4JijVllgJbI+IycErS72top9mEOeGbjXSpVBbw9YgYsZicip8B9MkvaxzP4Zv1tgt4OJfMRdLNuTLpixQrSQ7kapN31dlIs/HyEb5Zb08DC4D9uZztOxQ/Sbed4uf8DlGsGvmHuhpoNhG+LNNskiQ9C/w6IrZ90LZmdfKUjplZR/gI38ysI3yEb2bWEU74ZmYd4YRvZtYRTvhmZh3hhG9m1hFO+GZmHfFfxt7aP/Ndj5YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib.patches import Rectangle\n",
    "plt.scatter(x, y, c=\"green\", alpha=0.5, s=10)\n",
    "plt.scatter(x_wap, y_wap, c=\"red\", s=10)\n",
    "ax = plt.gca()\n",
    "ax.add_patch(Rectangle((1500, 3.6), 2500, 0.8, color='b', alpha=0.3))\n",
    "plt.xlabel(\"freq\")\n",
    "plt.ylabel(\"avg rating\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned before, the idea for _Phase 1_ is to test different approaches. Then, we want to keep the development dataset compact to reduce computational complexity. Because of that, we will use 40 random _target_ movies for the `target_ratings` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(2018)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "similar_movies_filter = (df_movies['count'] >= 1500) & (df_movies['count'] <= 4000) &\\\n",
    "    (df_movies['mean'] >= 3.6) & (~df_movies.index.isin(wap_movies))\n",
    "similar_movies = df_movies[similar_movies_filter].index\n",
    "m_sample_size = 40\n",
    "similar_movies_sampled = np.random.choice(similar_movies, m_sample_size)\n",
    "target_movies = np.concatenate((wap_movies, similar_movies_sampled))\n",
    "df_target_ratings = df_ratings_since[df_ratings_since.movieId.isin(target_movies)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mixed ratings dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Until now, we have two datasets: `wap_ratings`, with the six _WAP_ proposed films, and `target_ratings`, that contains all the movies from the former, adding more random unpopular-but-good movies. But, what if we are missing relevant information from other kinds of films? \n",
    "\n",
    "Let's build a third dataset: `mixed_ratings`. It will add to `target_ratings` films at least as popular as the previous ones (more than 1500 opinions since 2015), but which are not considered _target_, maybe because they have more than 4000 opinions, or their average rating is lower than 3.6.  \n",
    "\n",
    "Again, to keep the dataset simple, we will add only 50 films."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_target_movies = np.setdiff1d(df_movies[df_movies['count'] >= 1500].index, target_movies)\n",
    "m_sample_size = 50\n",
    "not_target_movies_sampled = np.random.choice(not_target_movies, m_sample_size)\n",
    "mixed_movies = np.concatenate((target_movies, not_target_movies_sampled))\n",
    "df_mixed_ratings = df_ratings_since[df_ratings_since.movieId.isin(mixed_movies)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Users selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned before, the idea for this phase is to test several different approaches and compare them, to know which one we can go deeper into. Then, to have a fair comparison, we will work with the same users for the three datasets. Since the `wap_ratings` is a subset of the other two, we are going to select users who have rated at least one of the _WAP_ films, assuming the cost of the bias in which we are incurring. Therefore, in the three datasets, all the users will have rated at least two of the WAP films."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_wap_ratings = df_ratings_since[df_ratings_since.movieId.isin(wap_movies)]\n",
    "counts = df_wap_ratings['userId'].value_counts()\n",
    "user_subset = df_wap_ratings[df_wap_ratings.userId.isin(counts.index[counts > 1])].userId.unique().tolist()\n",
    "df_wap_ratings = df_wap_ratings[df_wap_ratings.userId.isin(user_subset)]\n",
    "df_target_ratings = df_target_ratings[df_target_ratings.userId.isin(user_subset)]\n",
    "df_mixed_ratings = df_mixed_ratings[df_mixed_ratings.userId.isin(user_subset)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of ratings (WAP): 11156\n",
      "Number of ratings (Target): 38513\n",
      "Number of ratings (Mixed): 76380\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of ratings (WAP): \" + str(len(df_wap_ratings)))\n",
    "print(\"Number of ratings (Target): \" + str(len(df_target_ratings)))\n",
    "print(\"Number of ratings (Mixed): \" + str(len(df_mixed_ratings)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train/Test Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Describe split train_test "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test_set = df_wap_ratings.sample(frac = 1.0).groupby('userId').head(1).sample(frac = 0.40)\n",
    "df_wap_ratings_train = pd.concat([df_wap_ratings, df_test_set]).drop_duplicates(keep=False)\n",
    "df_target_ratings_train = pd.concat([df_target_ratings, df_test_set]).drop_duplicates(keep=False)\n",
    "df_mixed_ratings_train = pd.concat([df_mixed_ratings, df_test_set]).drop_duplicates(keep=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Surprise Library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Brief description of the library's objectives, specs, developers, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from surprise import Reader, Dataset, model_selection\n",
    "from surprise.model_selection import train_test_split\n",
    "\n",
    "def create_surprise_ratings(df_ratings):\n",
    "    reader = Reader(rating_scale=(0.5, 5.0))\n",
    "    sp_ratings = Dataset.load_from_df(df_ratings[['userId', 'movieId', 'rating']], reader)\n",
    "    return sp_ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "sp_wap_ratings_train = create_surprise_ratings(df_wap_ratings_train)\n",
    "sp_target_ratings_train = create_surprise_ratings(df_target_ratings_train)\n",
    "sp_mixed_ratings_train = create_surprise_ratings(df_mixed_ratings_train)\n",
    "sp_test_set = create_surprise_ratings(df_test_set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accuracy Metrics "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fraction of Concordant Pairs (FCP) is an error measure proposed in Collaborative Filtering on Ordinal User Feedback by Koren and Sill, and which is already implemented in surprise.\n",
    "\n",
    "Given a test set, the number of concordant pairs ($n_{c}^{u}$) for user $u$ is defined as the amount of pairs ranked correctly by rating predictor $\\hat{r}_u$. In mathematical terms,\n",
    "\n",
    "$$n_{c}^{u} = |\\{(i, j) | \\hat{r}_{ui} > \\hat{r}_{uj} \\text{ and } r_{ui} > r_{uj}\\}|.$$\n",
    "On the other side, the number of discordant pairs are the ones which were misranked:\n",
    "\n",
    "$$n_{d}^{u} = |\\{(i, j) | \\hat{r}_{ui} \\leq \\hat{r}_{uj} \\text{ and } r_{ui} > r_{uj}\\}|.$$\n",
    "To get the total of concordant and discordant pairs, we just sum among every user.\n",
    "\n",
    "$$n_c = \\sum_{u}n_{c}^{u}$$\n",
    "$$n_d = \\sum_{u}n_{d}^{u}$$\n",
    "\n",
    "Finally, the $FCP$ error measure is defined as $$FCP = \\frac{n_c}{n_c + n_d},$$\n",
    "\n",
    "where 1.0 means perfect order, and 0.0 means completely wrong order. Then, the higher is $FCP$, the better is the model.\n",
    "\n",
    "This measure is useful because it tests the accurate order of the values, more than the specific scale. As our problem is an ordinal one, we are going to take this as our __primary accuracy metric__.\n",
    "\n",
    "Also, we will use the classic RMSE as our __secondary accuracy metric__."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grid Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we will be determining the best hyperparameters and models to use in our analysis. Firstly, we will use GridSearchCV to find the best hyperparameters (this can be found within the `surprise` model_selection package. After that is complete, we will compare the models to determine the best from each of the baseline, neighborhood and model based techniques. Within neighborhood we tested basic, mean and zscore knn implementations and for model based, we used svd and svd++. Finally, we will use our held out testing set to compare the baseline, neighbourhood and model based approaches, selecting whichever model has the highest relative accuracy (using FCP).\n",
    "\n",
    "We will also be testing the above process using three different datasets. The `WAP`, `Target` and `Mixed` sets outlined earlier in the report. The goal of this is to determine if adding more information about other films results in signficant performance enhancements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from surprise import BaselineOnly, KNNBasic, KNNWithMeans, KNNWithZScore, SVD, SVDpp\n",
    "cv = 5\n",
    "measures = ['fcp', 'rmse']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_models(sp_ratings, cv, measures):\n",
    "    np.random.seed(2018)\n",
    "    models = {}\n",
    "    models['baseline'] = fit_grid_Baseline(sp_ratings, cv, measures)\n",
    "    models['knn_basic'] = fit_grid_KNN(sp_ratings, cv, measures, 'basic')\n",
    "    models['knn_means'] = fit_grid_KNN(sp_ratings, cv, measures, 'means')\n",
    "    models['knn_zscore'] = fit_grid_KNN(sp_ratings, cv, measures, 'zscore')\n",
    "    models['svd'] = fit_grid_SVD(sp_ratings, cv, measures, 'svd')\n",
    "    models['svdpp'] = fit_grid_SVD(sp_ratings, cv, measures, 'svdpp')\n",
    "    return(models)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_Baseline(sp_ratings, cv, measures, reg_u=[0], reg_i=[0]):\n",
    "    param_grid = {'bsl_options': {'reg_u': reg_u, 'reg_i': reg_i}}\n",
    "    gs_Baseline = model_selection.GridSearchCV(BaselineOnly, param_grid, measures=['fcp', 'rmse'], cv=cv)\n",
    "    gs_Baseline.fit(sp_ratings)\n",
    "    return(gs_Baseline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_KNN(\n",
    "    sp_ratings, cv, measures, knn,\n",
    "    user_based=[False], \n",
    "    k=[5, 10, 15],\n",
    "    reg=[0.0001, 0.001, 0.01], \n",
    "    sim_name=['pearson', 'pearson_baseline', 'cosine'],\n",
    "    sim_min_support=[2, 4],\n",
    "    method=['als']\n",
    "):\n",
    "    param_grid = {'bsl_options': {'method': method,\n",
    "                              'reg': reg},\n",
    "              'k': k,\n",
    "              'sim_options': {'name': sim_name,\n",
    "                              'min_support': sim_min_support,\n",
    "                              'user_based': user_based}\n",
    "              }\n",
    "    knn_methods = {\n",
    "        'basic': KNNBasic,\n",
    "        'means': KNNWithMeans,\n",
    "        'zscore': KNNWithZScore\n",
    "    }\n",
    "    gs_KNN = model_selection.GridSearchCV(knn_methods[knn], param_grid, measures=measures, cv=cv)\n",
    "    gs_KNN.fit(sp_ratings)\n",
    "    return(gs_KNN)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_SVD(\n",
    "    sp_ratings, cv, measures, svd,\n",
    "    n_factors=[5, 10, 20],\n",
    "    lr_all=[0.001, 0.01, 0.1],\n",
    "    reg_all=[0.001, 0.01, 0.1]\n",
    "):\n",
    "    param_grid = {'n_factors': n_factors, 'lr_all': lr_all, 'reg_all': reg_all}\n",
    "    svd_methods = {\n",
    "        'svd': SVD,\n",
    "        'svdpp': SVDpp\n",
    "    }\n",
    "    gs_SVD = model_selection.GridSearchCV(svd_methods[svd], param_grid, measures=measures, cv=cv)\n",
    "    gs_SVD.fit(sp_ratings)\n",
    "    return(gs_SVD)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def print_grid_models_accuracy(models):\n",
    "    print('FCP metric')\n",
    "    for model in models.keys():\n",
    "        print(str(model) + \" accuracy: \" + str(round(models[model].best_score['fcp'], 4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grid Models Fitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we test the accuracy of each model on each dataset after tuning the hyperparameters. This stage is crucial as a preliminary stage between creating a singular baseline model and determining the final model type, parameters, and data subset to use. By testing small chunks of data we can speed up this process and focus the bulk of our time on only the top models and data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "wap_models = fit_grid_models(sp_wap_ratings_train, cv, measures)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FCP metric\n",
      "baseline accuracy: 0.5126\n",
      "knn_basic accuracy: 0.4846\n",
      "knn_means accuracy: 0.506\n",
      "knn_zscore accuracy: 0.5017\n",
      "svd accuracy: 0.5144\n",
      "svdpp accuracy: 0.5128\n"
     ]
    }
   ],
   "source": [
    "print_grid_models_accuracy(wap_models)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "target_models = fit_grid_models(sp_target_ratings_train, cv, measures)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FCP metric\n",
      "baseline accuracy: 0.5725\n",
      "knn_basic accuracy: 0.5689\n",
      "knn_means accuracy: 0.5879\n",
      "knn_zscore accuracy: 0.5905\n",
      "svd accuracy: 0.5692\n",
      "svdpp accuracy: 0.5838\n"
     ]
    }
   ],
   "source": [
    "print_grid_models_accuracy(target_models)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "mixed_models = fit_grid_models(sp_mixed_ratings_train, cv, measures)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FCP metric\n",
      "baseline accuracy: 0.638\n",
      "knn_basic accuracy: 0.6417\n",
      "knn_means accuracy: 0.6602\n",
      "knn_zscore accuracy: 0.6648\n",
      "svd accuracy: 0.6399\n",
      "svdpp accuracy: 0.6515\n"
     ]
    }
   ],
   "source": [
    "print_grid_models_accuracy(mixed_models)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from surprise import accuracy\n",
    "from collections import defaultdict\n",
    "from six import iteritems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After finding the correspondent hyperparameters, we will take the neighborhood-based and model-based techniques with the best scores for each `train dataset`, and test them with the real `test set`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compared with the hyperparameters section, where for simplicity we just used the `surprise` implemented methods, in this final model selection we are going to make some changes so that we are able to conduct a fair comparison between the datasets.\n",
    "\n",
    "Firstly, instead of doing cross-validation, we are going to use a dataset `test_set` which is entirely different from any of the data within the training set.\n",
    "\n",
    "Secondly, we are not going to calculate the accuracy metric for the entire dataset. Conversely, even when we train the model with several films, we will figure the FCP just taking into account the WAP movies, as this is the core business function of our new recommendation system.\n",
    "\n",
    "Finally, we were suspicious about the FCP values we were getting from the `surprise` method. After inspecting the original code, we realized it is not implemented exactly as the paper defined it. We believe the implemented method is useful to choose the hyperparameters because the main difference is located within the handling of scale. However, for this section, we decided to code ours with some modifications so that it can be interpreted as the one proposed on the paper and that we described previously in this document.\n",
    "\n",
    "To achieve this changes, we propose the following functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fcp_wap(predictions_train, predictions_test):\n",
    "    \"\"\"Modified FCP function for the WAP movies, and implementing the paper's definition.\n",
    "    Args:\n",
    "        predictions_train (:obj:`list` of :obj:`Prediction\\\n",
    "            <surprise.prediction_algorithms.predictions.Prediction>`):\n",
    "            A list of predictions, as returned by the :meth:`test()\n",
    "            <surprise.prediction_algorithms.algo_base.AlgoBase.test>` method.\n",
    "        predictions_test (:obj:`list` of :obj:`Prediction\\\n",
    "            <surprise.prediction_algorithms.predictions.Prediction>`):\n",
    "            A list of predictions, as returned by the :meth:`test()\n",
    "            <surprise.prediction_algorithms.algo_base.AlgoBase.test>` method.\n",
    "        verbose: If True, will print computed value. Default is ``True``.\n",
    "    Returns:\n",
    "        The Fraction of Concordant Pairs.\n",
    "    Raises:\n",
    "        ValueError: When ``predictions`` is empty.\n",
    "    \"\"\"\n",
    "\n",
    "    if not predictions_test:\n",
    "        raise ValueError('Prediction list is empty.')\n",
    "\n",
    "    predictions_u = defaultdict(list)\n",
    "    nc_u = defaultdict(int)\n",
    "    nd_u = defaultdict(int)\n",
    "    fcp_u = defaultdict(float)\n",
    "    wap_movies = np.array([2395, 4979, 30810, 55269, 72226, 94959])\n",
    "\n",
    "    for u0, m, r0, est, _ in predictions_train:\n",
    "        if m in wap_movies:\n",
    "            predictions_u[u0].append((r0, r0, 'train'))\n",
    "    \n",
    "    for u0, m, r0, est, _ in predictions_test:\n",
    "        if m in wap_movies:\n",
    "            predictions_u[u0].append((r0, est, 'test'))\n",
    "\n",
    "    for u0, preds in iteritems(predictions_u):\n",
    "        for r0i, esti, kindi in preds:\n",
    "            if kindi == 'test':\n",
    "                for r0j, estj, kindj in preds:\n",
    "                    if ((esti > estj and r0i > r0j) or (esti < estj and r0i < r0j)):\n",
    "                        nc_u[u0] += 1\n",
    "                    if ((esti <= estj and r0i > r0j) or (esti >= estj and r0i < r0j)):\n",
    "                        nd_u[u0] += 1\n",
    "        try:\n",
    "            fcp_u[u0] = nc_u[u0] / (nc_u[u0] + nd_u[u0])\n",
    "        except:\n",
    "            pass\n",
    "\n",
    "    fcp = round(np.mean(list(fcp_u.values())), 4)\n",
    "\n",
    "    return fcp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_test_accuracy(sp_test, sp_train, models, technique):\n",
    "    algo = models[technique].best_estimator['fcp']\n",
    "    trainset = sp_train.build_full_trainset()\n",
    "    testset = sp_test.build_full_trainset()\n",
    "    algo.fit(trainset)\n",
    "    predictions_train = algo.test(trainset.build_testset())\n",
    "    predictions_test = algo.test(testset.build_testset())\n",
    "    accuracy = fcp_wap(predictions_train, predictions_test)\n",
    "    return(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, the three datasets are run through our custom FCP accuracy algorithm using the tuned baseline, knn and svd++ algorithms. Please note that the fcp accuracies have been customized for this given project and should only be used for comparison purposes. They do not represent the overall accuracy in predicting all movies over the entire dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "wap_baseline_fcp = get_test_accuracy(sp_test_set, sp_wap_ratings_train, wap_models, 'baseline')\n",
    "wap_knn_fcp = get_test_accuracy(sp_test_set, sp_wap_ratings_train, wap_models, 'knn_means')\n",
    "wap_svd_fcp = get_test_accuracy(sp_test_set, sp_wap_ratings_train, wap_models, 'svdpp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "baseline test accuracy: 0.7461\n",
      "knn_zscore test accuracy: 0.7724\n",
      "svdpp accuracy: 0.733\n"
     ]
    }
   ],
   "source": [
    "print('baseline test accuracy: ' + str(wap_baseline_fcp))\n",
    "print('knn_zscore test accuracy: ' + str(wap_knn_fcp))\n",
    "print('svdpp accuracy: ' + str(wap_svd_fcp))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "target_baseline_fcp = get_test_accuracy(sp_test_set, sp_target_ratings_train, target_models, 'baseline')\n",
    "target_knn_fcp = get_test_accuracy(sp_test_set, sp_target_ratings_train, target_models, 'knn_means')\n",
    "target_svd_fcp = get_test_accuracy(sp_test_set, sp_target_ratings_train, target_models, 'svdpp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "baseline test accuracy: 0.7929\n",
      "knn_means test accuracy: 0.8003\n",
      "svdpp accuracy: 0.8077\n"
     ]
    }
   ],
   "source": [
    "print('baseline test accuracy: ' + str(target_baseline_fcp))\n",
    "print('knn_means test accuracy: ' + str(target_knn_fcp))\n",
    "print('svdpp accuracy: ' + str(target_svd_fcp))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "mixed_baseline_fcp = get_test_accuracy(sp_test_set, sp_mixed_ratings_train, mixed_models, 'baseline')\n",
    "mixed_knn_fcp = get_test_accuracy(sp_test_set, sp_mixed_ratings_train, mixed_models, 'knn_zscore')\n",
    "mixed_svd_fcp = get_test_accuracy(sp_test_set, sp_mixed_ratings_train, mixed_models, 'svdpp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "baseline test accuracy: 0.7922\n",
      "knn_means test accuracy: 0.8118\n",
      "svdpp accuracy: 0.7911\n"
     ]
    }
   ],
   "source": [
    "print('baseline test accuracy: ' + str(mixed_baseline_fcp))\n",
    "print('knn_means test accuracy: ' + str(mixed_knn_fcp))\n",
    "print('svdpp accuracy: ' + str(mixed_svd_fcp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even when there was a bias by construction, which we expect helped the `wap ratings` dataset (we selected only users with 2 or more _WAP_ movies ratings, the dataset with the best performance was the `mixed ratings` one. Particularly, we got the best results when we used the svd++ technique."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Film Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What follows from here is to find which is the _WAP_ film with the greatest estimated rating for each user, and send her/him the recommendation. Also, it can be unrealistic to send a recommendation for everybody, because there will be users for which all the _WAP_ movies have low ratings. Then, for further phases of this project, we must set a threshold. Maybe a direct scale rating one, or maybe a minimum quantile among all the estimated movies per customer so that we take into account the scale differences among users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After completing `Phase 1` of our project our team has reached the conclusion that using a larger subset of movies (even those not related to WAP's movies) would be extremely beneficial to the overall accuracy of our model. Also, we have found that the SVD++ model consistently has the highest FCP accuracy of WAP movies as the dataset gains movies. Though our model is still in its infancy the results we have been able to obtain show promise in the potential success that we can obtain by targetting advertisments on behalf of Wes Anderson Productions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Future Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to the time contraints associated with training and testing our models, we were unable to complete certain visualizations that we believe are important to show the client. Therefore, the following will be included in `Phase 2` of our project, when we present our findings with Wes Anderson Productions.\n",
    "* Create plots of how the error measure changes when we fix all the hyperparameters, except one.\n",
    "* Get the time it takes to fit each model (on a standard performance VM [ex: aws ec2 t3.large]).\n",
    "* Since we decided that we will use the complete dataset, we know can recommend _WAP_ movies, even if the user hasn't seen any of them. Then, we can use datasets that reflect this advantage, and we must investigate how this will change the _WARS_ performance.\n",
    "* Try with recommendation techniques outside the `surprise` package, like "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}