11.3_Hypothesis_Testing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "i2RUFW7TFcuh"
   },
   "source": [
    "# Springboard Apps project - Tier 3 - Complete\n",
    "\n",
    "Welcome to the Apps project! To give you a taste of your future career, we're going to walk through exactly the kind of notebook that you'd write as a data scientist. In the process, we'll be sure to signpost the general framework for our investigation - the Data Science Pipeline - as well as give reasons for why we're doing what we're doing. We're also going to apply some of the skills and knowledge you've built up in the previous unit when reading Professor Spiegelhalter's *The Art of Statistics* (hereinafter *AoS*). \n",
    "\n",
    "So let's get cracking!\n",
    "\n",
    "**Brief**\n",
    "\n",
    "Did Apple Store apps receive better reviews than Google Play apps?\n",
    "\n",
    "## Stages of the project\n",
    "\n",
    "1. Sourcing and loading \n",
    "    * Load the two datasets\n",
    "    * Pick the columns that we are going to work with \n",
    "    * Subsetting the data on this basis \n",
    " \n",
    " \n",
    "2. Cleaning, transforming and visualizing\n",
    "    * Check the data types and fix them\n",
    "    * Add a `platform` column to both the `Apple` and the `Google` dataframes\n",
    "    * Changing the column names to prepare for a join \n",
    "    * Join the two data sets\n",
    "    * Eliminate the `NaN` values\n",
    "    * Filter only those apps that have been reviewed at least once\n",
    "    * Summarize the data visually and analytically (by the column `platform`)  \n",
    "  \n",
    "  \n",
    "3. Modelling \n",
    "    * Hypothesis formulation\n",
    "    * Getting the distribution of the data\n",
    "    * Permutation test \n",
    "\n",
    "\n",
    "4. Evaluating and concluding \n",
    "    * What is our conclusion?\n",
    "    * What is our decision?\n",
    "    * Other models we could have used. \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QLUAXtl1Fcuk"
   },
   "source": [
    "## Importing the libraries\n",
    "\n",
    "In this case we are going to import pandas, numpy, scipy, random and matplotlib.pyplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XeXJOlGiFcum"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "# scipi is a library for statistical tests and visualizations \n",
    "from scipy import stats\n",
    "# random enables us to generate random numbers\n",
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zVD7NlPAFcup"
   },
   "source": [
    "## Stage 1 -  Sourcing and loading data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KU8bdbImFcuq"
   },
   "source": [
    "### 1a. Source and load the data\n",
    "Let's download the data from Kaggle. Kaggle is a fantastic resource: a kind of social medium for data scientists, it boasts projects, datasets and news on the freshest libraries and technologies all in one place. The data from the Apple Store can be found [here](https://www.kaggle.com/ramamet4/app-store-apple-data-set-10k-apps) and the data from Google Store can be found [here](https://www.kaggle.com/lava18/google-play-store-apps).\n",
    "Download the datasets and save them in your working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "laM1PB76Fcur"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>App</th>\n",
       "      <th>Category</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Reviews</th>\n",
       "      <th>Size</th>\n",
       "      <th>Installs</th>\n",
       "      <th>Type</th>\n",
       "      <th>Price</th>\n",
       "      <th>Content Rating</th>\n",
       "      <th>Genres</th>\n",
       "      <th>Last Updated</th>\n",
       "      <th>Current Ver</th>\n",
       "      <th>Android Ver</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Photo Editor &amp; Candy Camera &amp; Grid &amp; ScrapBook</td>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>4.1</td>\n",
       "      <td>159</td>\n",
       "      <td>19M</td>\n",
       "      <td>10,000+</td>\n",
       "      <td>Free</td>\n",
       "      <td>0</td>\n",
       "      <td>Everyone</td>\n",
       "      <td>Art &amp; Design</td>\n",
       "      <td>January 7, 2018</td>\n",
       "      <td>1.0.0</td>\n",
       "      <td>4.0.3 and up</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Coloring book moana</td>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>3.9</td>\n",
       "      <td>967</td>\n",
       "      <td>14M</td>\n",
       "      <td>500,000+</td>\n",
       "      <td>Free</td>\n",
       "      <td>0</td>\n",
       "      <td>Everyone</td>\n",
       "      <td>Art &amp; Design;Pretend Play</td>\n",
       "      <td>January 15, 2018</td>\n",
       "      <td>2.0.0</td>\n",
       "      <td>4.0.3 and up</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>U Launcher Lite – FREE Live Cool Themes, Hide ...</td>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>4.7</td>\n",
       "      <td>87510</td>\n",
       "      <td>8.7M</td>\n",
       "      <td>5,000,000+</td>\n",
       "      <td>Free</td>\n",
       "      <td>0</td>\n",
       "      <td>Everyone</td>\n",
       "      <td>Art &amp; Design</td>\n",
       "      <td>August 1, 2018</td>\n",
       "      <td>1.2.4</td>\n",
       "      <td>4.0.3 and up</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 App        Category  Rating  \\\n",
       "0     Photo Editor & Candy Camera & Grid & ScrapBook  ART_AND_DESIGN     4.1   \n",
       "1                                Coloring book moana  ART_AND_DESIGN     3.9   \n",
       "2  U Launcher Lite – FREE Live Cool Themes, Hide ...  ART_AND_DESIGN     4.7   \n",
       "\n",
       "  Reviews  Size    Installs  Type Price Content Rating  \\\n",
       "0     159   19M     10,000+  Free     0       Everyone   \n",
       "1     967   14M    500,000+  Free     0       Everyone   \n",
       "2   87510  8.7M  5,000,000+  Free     0       Everyone   \n",
       "\n",
       "                      Genres      Last Updated Current Ver   Android Ver  \n",
       "0               Art & Design   January 7, 2018       1.0.0  4.0.3 and up  \n",
       "1  Art & Design;Pretend Play  January 15, 2018       2.0.0  4.0.3 and up  \n",
       "2               Art & Design    August 1, 2018       1.2.4  4.0.3 and up  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now that the files are saved, we want to load them into Python using read_csv and pandas.\n",
    "\n",
    "# Create a variable called google, and store in it the path of the csv file that contains your google dataset. \n",
    "# If your dataset is in the same folder as this notebook, the path will simply be the name of the file. \n",
    "google = 'googleplaystore.csv'\n",
    "\n",
    "# Read the csv file into a data frame called Google using the read_csv() pandas method.\n",
    "Google = pd.read_csv(google)\n",
    "\n",
    "# Using the head() pandas method, observe the first three entries.\n",
    "Google.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3AW8EA1wFcuu"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>id</th>\n",
       "      <th>track_name</th>\n",
       "      <th>size_bytes</th>\n",
       "      <th>currency</th>\n",
       "      <th>price</th>\n",
       "      <th>rating_count_tot</th>\n",
       "      <th>rating_count_ver</th>\n",
       "      <th>user_rating</th>\n",
       "      <th>user_rating_ver</th>\n",
       "      <th>ver</th>\n",
       "      <th>cont_rating</th>\n",
       "      <th>prime_genre</th>\n",
       "      <th>sup_devices.num</th>\n",
       "      <th>ipadSc_urls.num</th>\n",
       "      <th>lang.num</th>\n",
       "      <th>vpp_lic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>281656475</td>\n",
       "      <td>PAC-MAN Premium</td>\n",
       "      <td>100788224</td>\n",
       "      <td>USD</td>\n",
       "      <td>3.99</td>\n",
       "      <td>21292</td>\n",
       "      <td>26</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>6.3.5</td>\n",
       "      <td>4+</td>\n",
       "      <td>Games</td>\n",
       "      <td>38</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>281796108</td>\n",
       "      <td>Evernote - stay organized</td>\n",
       "      <td>158578688</td>\n",
       "      <td>USD</td>\n",
       "      <td>0.00</td>\n",
       "      <td>161065</td>\n",
       "      <td>26</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>8.2.2</td>\n",
       "      <td>4+</td>\n",
       "      <td>Productivity</td>\n",
       "      <td>37</td>\n",
       "      <td>5</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>281940292</td>\n",
       "      <td>WeatherBug - Local Weather, Radar, Maps, Alerts</td>\n",
       "      <td>100524032</td>\n",
       "      <td>USD</td>\n",
       "      <td>0.00</td>\n",
       "      <td>188583</td>\n",
       "      <td>2822</td>\n",
       "      <td>3.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0.0</td>\n",
       "      <td>4+</td>\n",
       "      <td>Weather</td>\n",
       "      <td>37</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0         id                                       track_name  \\\n",
       "0           1  281656475                                  PAC-MAN Premium   \n",
       "1           2  281796108                        Evernote - stay organized   \n",
       "2           3  281940292  WeatherBug - Local Weather, Radar, Maps, Alerts   \n",
       "\n",
       "   size_bytes currency  price  rating_count_tot  rating_count_ver  \\\n",
       "0   100788224      USD   3.99             21292                26   \n",
       "1   158578688      USD   0.00            161065                26   \n",
       "2   100524032      USD   0.00            188583              2822   \n",
       "\n",
       "   user_rating  user_rating_ver    ver cont_rating   prime_genre  \\\n",
       "0          4.0              4.5  6.3.5          4+         Games   \n",
       "1          4.0              3.5  8.2.2          4+  Productivity   \n",
       "2          3.5              4.5  5.0.0          4+       Weather   \n",
       "\n",
       "   sup_devices.num  ipadSc_urls.num  lang.num  vpp_lic  \n",
       "0               38                5        10        1  \n",
       "1               37                5        23        1  \n",
       "2               37                5         3        1  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a variable called apple, and store in it the path of the csv file that contains your apple dataset. \n",
    "apple = 'AppleStore.csv'\n",
    "\n",
    "# Read the csv file into a pandas DataFrame object called Apple.\n",
    "Apple = pd.read_csv(apple)\n",
    "\n",
    "# Observe the first three entries like you did with your other data. \n",
    "Apple.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vljOHKNTFcuw"
   },
   "source": [
    "### 1b. Pick the columns we'll work with\n",
    "\n",
    "From the documentation of these datasets, we can infer that the most appropriate columns to answer the brief are:\n",
    "\n",
    "1. Google:\n",
    "    * `Category` # Do we need this?\n",
    "    * `Rating`\n",
    "    * `Reviews`\n",
    "    * `Price` (maybe)\n",
    "2. Apple:    \n",
    "    * `prime_genre` # Do we need this?\n",
    "    * `user_rating` \n",
    "    * `rating_count_tot`\n",
    "    * `price` (maybe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uf69AeTvFcux"
   },
   "source": [
    "### 1c. Subsetting accordingly\n",
    "\n",
    "Let's select only those columns that we want to work with from both datasets. We'll overwrite the subsets in the original variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LNSDXyjdFcuy"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Category</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Reviews</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>4.1</td>\n",
       "      <td>159</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>3.9</td>\n",
       "      <td>967</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ART_AND_DESIGN</td>\n",
       "      <td>4.7</td>\n",
       "      <td>87510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Category  Rating Reviews Price\n",
       "0  ART_AND_DESIGN     4.1     159     0\n",
       "1  ART_AND_DESIGN     3.9     967     0\n",
       "2  ART_AND_DESIGN     4.7   87510     0"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Subset our DataFrame object Google by selecting just the variables ['Category', 'Rating', 'Reviews', 'Price']\n",
    "Google = Google[['Category', 'Rating', 'Reviews', 'Price']]\n",
    "\n",
    "# Check the first three entries\n",
    "Google.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AVCtENPtFcu0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prime_genre</th>\n",
       "      <th>user_rating</th>\n",
       "      <th>rating_count_tot</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Games</td>\n",
       "      <td>4.0</td>\n",
       "      <td>21292</td>\n",
       "      <td>3.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Productivity</td>\n",
       "      <td>4.0</td>\n",
       "      <td>161065</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Weather</td>\n",
       "      <td>3.5</td>\n",
       "      <td>188583</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    prime_genre  user_rating  rating_count_tot  price\n",
       "0         Games          4.0             21292   3.99\n",
       "1  Productivity          4.0            161065   0.00\n",
       "2       Weather          3.5            188583   0.00"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Do the same with our Apple object, selecting just the variables ['prime_genre', 'user_rating', 'rating_count_tot', 'price']\n",
    "Apple = Apple[['prime_genre', 'user_rating', 'rating_count_tot', 'price']]\n",
    "\n",
    "# Let's check the first three entries\n",
    "Apple.head(3) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wwK2wH-rFcu2"
   },
   "source": [
    "## Stage 2 -  Cleaning, transforming and visualizing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NsoE5x0JFcu3"
   },
   "source": [
    "### 2a. Check the data types for both Apple and Google, and fix them\n",
    "\n",
    "Types are crucial for data science in Python. Let's determine whether the variables we selected in the previous section belong to the types they should do, or whether there are any errors here. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ymw10SaWFcu4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "prime_genre          object\n",
       "user_rating         float64\n",
       "rating_count_tot      int64\n",
       "price               float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using the dtypes feature of pandas DataFrame objects, check out the data types within our Apple dataframe.\n",
    "# Are they what you expect?\n",
    "Apple.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VAiM6jIxFcu6"
   },
   "source": [
    "This is looking healthy. But what about our Google data frame?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kmP8MhqyFcu7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Category     object\n",
       "Rating      float64\n",
       "Reviews      object\n",
       "Price        object\n",
       "dtype: object"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using the same dtypes feature, check out the data types of our Google dataframe. \n",
    "Google.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "O-dxAy0WFcu9"
   },
   "source": [
    "Weird. The data type for the column 'Price' is 'object', not a numeric data type like a float or an integer. Let's investigate the unique values of this column. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Hu08K9y7Fcu9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['0', '$4.99', '$3.99', '$6.99', '$1.49', '$2.99', '$7.99', '$5.99',\n",
       "       '$3.49', '$1.99', '$9.99', '$7.49', '$0.99', '$9.00', '$5.49',\n",
       "       '$10.00', '$24.99', '$11.99', '$79.99', '$16.99', '$14.99',\n",
       "       '$1.00', '$29.99', '$12.99', '$2.49', '$10.99', '$1.50', '$19.99',\n",
       "       '$15.99', '$33.99', '$74.99', '$39.99', '$3.95', '$4.49', '$1.70',\n",
       "       '$8.99', '$2.00', '$3.88', '$25.99', '$399.99', '$17.99',\n",
       "       '$400.00', '$3.02', '$1.76', '$4.84', '$4.77', '$1.61', '$2.50',\n",
       "       '$1.59', '$6.49', '$1.29', '$5.00', '$13.99', '$299.99', '$379.99',\n",
       "       '$37.99', '$18.99', '$389.99', '$19.90', '$8.49', '$1.75',\n",
       "       '$14.00', '$4.85', '$46.99', '$109.99', '$154.99', '$3.08',\n",
       "       '$2.59', '$4.80', '$1.96', '$19.40', '$3.90', '$4.59', '$15.46',\n",
       "       '$3.04', '$4.29', '$2.60', '$3.28', '$4.60', '$28.99', '$2.95',\n",
       "       '$2.90', '$1.97', '$200.00', '$89.99', '$2.56', '$30.99', '$3.61',\n",
       "       '$394.99', '$1.26', 'Everyone', '$1.20', '$1.04'], dtype=object)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Use the unique() pandas method on the Price column to check its unique values. \n",
    "pd.unique(Google['Price'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "w_7nkogFFcu_"
   },
   "source": [
    "Aha! Fascinating. There are actually two issues here. \n",
    "\n",
    "- Firstly, there's a price called `Everyone`. That is a massive mistake! \n",
    "- Secondly, there are dollar symbols everywhere! \n",
    "\n",
    "\n",
    "Let's address the first issue first. Let's check the datapoints that have the price value `Everyone`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "E75WWmhdFcvA"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Category</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Reviews</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10472</th>\n",
       "      <td>1.9</td>\n",
       "      <td>19.0</td>\n",
       "      <td>3.0M</td>\n",
       "      <td>Everyone</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Category  Rating Reviews     Price\n",
       "10472      1.9    19.0    3.0M  Everyone"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check which data points have the value 'Everyone' for the 'Price' column by subsetting our Google dataframe.\n",
    "\n",
    "# Subset the Google dataframe on the price column. \n",
    "# To be sure: you want to pick out just those rows whose value for the 'Price' column is just 'Everyone'. \n",
    "Google[Google['Price'] == 'Everyone']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ezfLshdRFcvC"
   },
   "source": [
    "Thankfully, it's just one row. We've gotta get rid of it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DBmjUQUkFcvD"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['0', '$4.99', '$3.99', '$6.99', '$1.49', '$2.99', '$7.99', '$5.99',\n",
       "       '$3.49', '$1.99', '$9.99', '$7.49', '$0.99', '$9.00', '$5.49',\n",
       "       '$10.00', '$24.99', '$11.99', '$79.99', '$16.99', '$14.99',\n",
       "       '$1.00', '$29.99', '$12.99', '$2.49', '$10.99', '$1.50', '$19.99',\n",
       "       '$15.99', '$33.99', '$74.99', '$39.99', '$3.95', '$4.49', '$1.70',\n",
       "       '$8.99', '$2.00', '$3.88', '$25.99', '$399.99', '$17.99',\n",
       "       '$400.00', '$3.02', '$1.76', '$4.84', '$4.77', '$1.61', '$2.50',\n",
       "       '$1.59', '$6.49', '$1.29', '$5.00', '$13.99', '$299.99', '$379.99',\n",
       "       '$37.99', '$18.99', '$389.99', '$19.90', '$8.49', '$1.75',\n",
       "       '$14.00', '$4.85', '$46.99', '$109.99', '$154.99', '$3.08',\n",
       "       '$2.59', '$4.80', '$1.96', '$19.40', '$3.90', '$4.59', '$15.46',\n",
       "       '$3.04', '$4.29', '$2.60', '$3.28', '$4.60', '$28.99', '$2.95',\n",
       "       '$2.90', '$1.97', '$200.00', '$89.99', '$2.56', '$30.99', '$3.61',\n",
       "       '$394.99', '$1.26', '$1.20', '$1.04'], dtype=object)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's eliminate that row. \n",
    "\n",
    "# Subset our Google dataframe to pick out just those rows whose value for the 'Price' column is NOT 'Everyone'. \n",
    "# Reassign that subset to the Google variable. \n",
    "# You can do this in two lines or one. Your choice! \n",
    "Google = Google[Google['Price'] != 'Everyone']\n",
    "\n",
    "# Check again the unique values of Google\n",
    "pd.unique(Google['Price']) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "T43TB4qFFcvF"
   },
   "source": [
    "Our second problem remains: I'm seeing dollar symbols when I close my eyes! (And not in a good way). \n",
    "\n",
    "This is a problem because Python actually considers these values strings. So we can't do mathematical and statistical operations on them until we've made them into numbers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W87gVS9BFcvG"
   },
   "outputs": [],
   "source": [
    "# Let's create a variable called nosymb.\n",
    "# This variable will take the Price column of Google and apply the str.replace() method. \n",
    "nosymb = Google['Price'].str.replace('$', '')\n",
    "\n",
    "# Now we need to do two things:\n",
    "# i. Make the values in the nosymb variable numeric using the to_numeric() pandas method.\n",
    "# ii. Assign this new set of numeric, dollar-sign-less values to Google['Price']. \n",
    "# You can do this in one line if you wish.\n",
    "Google['Price'] = pd.to_numeric(nosymb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LxpJQ9yyFcvI"
   },
   "source": [
    "Now let's check the data types for our Google dataframe again, to verify that the 'Price' column really is numeric now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Fn5Gt-QbFcvI"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Category     object\n",
       "Rating      float64\n",
       "Reviews      object\n",
       "Price       float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Use the function dtypes. \n",
    "Google.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FgfuiVo0FcvK"
   },
   "source": [
    "Notice that the column `Reviews` is still an object column. We actually need this column to be a numeric column, too. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MiVEWsWBFcvL"
   },
   "outputs": [],
   "source": [
    "# Convert the 'Reviews' column to a numeric data type. \n",
    "Google['Reviews'] = pd.to_numeric(Google['Reviews'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wWJdXXTTFcvN"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Category     object\n",
       "Rating      float64\n",
       "Reviews       int64\n",
       "Price       float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check the data types of Google again\n",
    "Google.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OiLm2GEKFcvP"
   },
   "source": [
    "### 2b. Add a `platform` column to both the `Apple` and the `Google` dataframes\n",
    "Let's add a new column to both dataframe objects called `platform`: all of its values in the Google dataframe will be just 'google', and all of its values for the Apple dataframe will be just 'apple'. \n",
    "\n",
    "The reason we're making this column is so that we can ultimately join our Apple and Google data together, and actually test out some hypotheses to solve the problem in our brief. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mF-sOcx2FcvQ"
   },
   "outputs": [],
   "source": [
    "# Create a column called 'platform' in both the Apple and Google dataframes. \n",
    "# Add the value 'apple' and the value 'google' as appropriate. \n",
    "Apple['platform'] = 'apple'\n",
    "Google['platform'] = 'google'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EJcry17IFcvS"
   },
   "source": [
    "### 2c. Changing the column names to prepare for our join of the two datasets \n",
    "Since the easiest way to join two datasets is if they have both:\n",
    "- the same number of columns\n",
    "- the same column names\n",
    "we need to rename the columns of `Apple` so that they're the same as the ones of `Google`, or vice versa.\n",
    "\n",
    "In this case, we're going to change the `Apple` columns names to the names of the `Google` columns. \n",
    "\n",
    "This is an important step to unify the two datasets!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bdt9XH2KFcvT"
   },
   "outputs": [],
   "source": [
    "# Create a variable called old_names where you'll store the column names of the Apple dataframe. \n",
    "# Use the feature .columns.\n",
    "old_names = Apple.columns\n",
    "\n",
    "# Create a variable called new_names where you'll store the column names of the Google dataframe. \n",
    "new_names = Google.columns\n",
    "\n",
    "# Use the rename() DataFrame method to change the columns names. \n",
    "Apple = Apple.rename(columns = dict(zip(old_names,new_names)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qewPiLrRFcvX"
   },
   "source": [
    "### 2d. Join the two datasets \n",
    "Let's combine the two datasets into a single data frame called `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jq6fGfvyFcvY"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Category</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Reviews</th>\n",
       "      <th>Price</th>\n",
       "      <th>platform</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12875</th>\n",
       "      <td>Shopping</td>\n",
       "      <td>3.5</td>\n",
       "      <td>14682</td>\n",
       "      <td>0.00</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16954</th>\n",
       "      <td>Entertainment</td>\n",
       "      <td>2.5</td>\n",
       "      <td>110</td>\n",
       "      <td>0.00</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5830</th>\n",
       "      <td>FAMILY</td>\n",
       "      <td>4.5</td>\n",
       "      <td>11</td>\n",
       "      <td>0.00</td>\n",
       "      <td>google</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11240</th>\n",
       "      <td>Health &amp; Fitness</td>\n",
       "      <td>4.5</td>\n",
       "      <td>511</td>\n",
       "      <td>3.99</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16491</th>\n",
       "      <td>Utilities</td>\n",
       "      <td>4.0</td>\n",
       "      <td>8873</td>\n",
       "      <td>0.00</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14655</th>\n",
       "      <td>Social Networking</td>\n",
       "      <td>3.5</td>\n",
       "      <td>281</td>\n",
       "      <td>0.00</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6582</th>\n",
       "      <td>HEALTH_AND_FITNESS</td>\n",
       "      <td>2.6</td>\n",
       "      <td>186</td>\n",
       "      <td>0.00</td>\n",
       "      <td>google</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7287</th>\n",
       "      <td>FAMILY</td>\n",
       "      <td>4.7</td>\n",
       "      <td>15</td>\n",
       "      <td>0.00</td>\n",
       "      <td>google</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3739</th>\n",
       "      <td>NEWS_AND_MAGAZINES</td>\n",
       "      <td>4.3</td>\n",
       "      <td>11667403</td>\n",
       "      <td>0.00</td>\n",
       "      <td>google</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12857</th>\n",
       "      <td>Games</td>\n",
       "      <td>4.5</td>\n",
       "      <td>15030</td>\n",
       "      <td>4.99</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15392</th>\n",
       "      <td>Games</td>\n",
       "      <td>4.0</td>\n",
       "      <td>223</td>\n",
       "      <td>0.00</td>\n",
       "      <td>apple</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8635</th>\n",
       "      <td>PRODUCTIVITY</td>\n",
       "      <td>4.3</td>\n",
       "      <td>549900</td>\n",
       "      <td>0.00</td>\n",
       "      <td>google</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Category  Rating   Reviews  Price platform\n",
       "12875            Shopping     3.5     14682   0.00    apple\n",
       "16954       Entertainment     2.5       110   0.00    apple\n",
       "5830               FAMILY     4.5        11   0.00   google\n",
       "11240    Health & Fitness     4.5       511   3.99    apple\n",
       "16491           Utilities     4.0      8873   0.00    apple\n",
       "14655   Social Networking     3.5       281   0.00    apple\n",
       "6582   HEALTH_AND_FITNESS     2.6       186   0.00   google\n",
       "7287               FAMILY     4.7        15   0.00   google\n",
       "3739   NEWS_AND_MAGAZINES     4.3  11667403   0.00   google\n",
       "12857               Games     4.5     15030   4.99    apple\n",
       "15392               Games     4.0       223   0.00    apple\n",
       "8635         PRODUCTIVITY     4.3    549900   0.00   google"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's use the append() method to append Apple to Google. \n",
    "# append() has been deprecated. I used concat() instead\n",
    "df = pd.concat([Google, Apple], ignore_index=True)\n",
    "\n",
    "# Using the sample() method with the number 12 passed to it, check 12 random points of your dataset.\n",
    "df.sample(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FjqsHugWFcvb"
   },
   "source": [
    "### 2e. Eliminate the NaN values\n",
    "\n",
    "As you can see there are some `NaN` values. We want to eliminate all these `NaN` values from the table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Gp2q6GLyFcvb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(18037, 5)\n",
      "(16563, 5)\n"
     ]
    }
   ],
   "source": [
    "# Lets check first the dimesions of df before droping `NaN` values. Use the .shape feature. \n",
    "print(df.shape)\n",
    "\n",
    "# Use the dropna() method to eliminate all the NaN values, and overwrite the same dataframe with the result. \n",
    "df = df.dropna()\n",
    "\n",
    "# Check the new dimesions of our dataframe. \n",
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qtfeQKtAFcvd"
   },
   "source": [
    "### 2f. Filter the data so that we only see whose apps that have been reviewed at least once\n",
    "\n",
    "Apps that haven't been reviewed yet can't help us solve our brief. \n",
    "\n",
    "So let's check to see if any apps have no reviews at all. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ge9OKXLNFcvd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Category    929\n",
       "Rating      929\n",
       "Reviews     929\n",
       "Price       929\n",
       "platform    929\n",
       "dtype: int64"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Subset your df to pick out just those rows whose value for 'Reviews' is equal to 0. \n",
    "# Do a count() on the result. \n",
    "df[df['Reviews'] == 0].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "05f7H0kAFcvg"
   },
   "source": [
    "929 apps do not have reviews, we need to eliminate these points!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pD8vmPjkFcvg"
   },
   "outputs": [],
   "source": [
    "# Eliminate the points that have 0 reviews.\n",
    "df = df[df['Reviews'] != 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ApnKES_bFcvi"
   },
   "source": [
    "### 2g. Summarize the data visually and analytically (by the column `platform`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2smt9UocFcvj"
   },
   "source": [
    "What we need to solve our brief is a summary of the `Rating` column, but separated by the different platforms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Vqckt347Fcvj"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>platform</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>apple</th>\n",
       "      <td>6268.0</td>\n",
       "      <td>4.049697</td>\n",
       "      <td>0.726943</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>google</th>\n",
       "      <td>9366.0</td>\n",
       "      <td>4.191757</td>\n",
       "      <td>0.515219</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.3</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           count      mean       std  min  25%  50%  75%  max\n",
       "platform                                                     \n",
       "apple     6268.0  4.049697  0.726943  1.0  4.0  4.5  4.5  5.0\n",
       "google    9366.0  4.191757  0.515219  1.0  4.0  4.3  4.5  5.0"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To summarize analytically, let's use the groupby() method on our df.\n",
    "df.groupby('platform')['Rating'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8f9TqMtKFcvl"
   },
   "source": [
    "Interesting! Our means of 4.049697 and 4.191757 don't **seem** all that different! Perhaps we've solved our brief already: there's no significant difference between Google Play app reviews and Apple Store app reviews. We have an ***observed difference*** here: which is simply (4.191757 - 4.049697) = 0.14206. This is just the actual difference that we observed between the mean rating for apps from Google Play, and the mean rating for apps from the Apple Store. Let's look at how we're going to use this observed difference to solve our problem using a statistical test. \n",
    "\n",
    "**Outline of our method:**\n",
    "1. We'll assume that platform (i.e, whether the app was Google or Apple) really doesn’t impact on ratings. \n",
    "\n",
    "\n",
    "2. Given this assumption, we should actually be able to get a difference in mean rating for Apple apps and mean rating for Google apps that's pretty similar to the one we actually got (0.14206) just by: \n",
    "a. shuffling the ratings column, \n",
    "b. keeping the platform column the same,\n",
    "c. calculating the difference between the mean rating for Apple and the mean rating for Google. \n",
    "\n",
    "\n",
    "3. We can make the shuffle more useful by doing it many times, each time calculating the mean rating for Apple apps and the mean rating for Google apps, and the difference between these means. \n",
    "\n",
    "\n",
    "4. We can then take the mean of all these differences, and this will be called our permutation difference. This permutation difference will be great indicator of what the difference would be if our initial assumption were true and platform really doesn’t impact on ratings. \n",
    "\n",
    "\n",
    "5. Now we do a comparison. If the observed difference looks just like the permutation difference, then we stick with the claim that actually, platform doesn’t impact on ratings. If instead, however, the permutation difference differs significantly from the observed difference, we'll conclude: something's going on; the platform does in fact impact on ratings. \n",
    "\n",
    "\n",
    "6. As for what the definition of *significantly* is, we'll get to that. But there’s a brief summary of what we're going to do. Exciting!\n",
    "\n",
    "If you want to look more deeply at the statistics behind this project, check out [this resource](https://www.springboard.com/archeio/download/4ea4d453b0b84014bcef287c50f47f00/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "d6EhTxArFcvm"
   },
   "source": [
    "Let's also get a **visual summary** of the `Rating` column, separated by the different platforms. \n",
    "\n",
    "A good tool to use here is the boxplot!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hFKjrlWgFcvm"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAH/CAYAAAB5K/ayAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQZElEQVR4nO3dd3wU1f7/8fekkgQSQjMQQoIU6c2GeIUAgogFRFBExIYFRSwgIIpURUBABbl6xQt4KYp8UQSFK9KRiFSl92oC0gOE9PP7g1/27pIEEhJ2J8nr+Xjk4e7MmdnPRmby3jNnzlrGGCMAAAAb8vJ0AQAAANkhqAAAANsiqAAAANsiqAAAANsiqAAAANsiqAAAANsiqAAAANsiqAAAANsiqAAAANsiqKBAmDJliizLcvkpW7asoqOjNX/+fE+X5xAVFaWnnnoq19slJCRo8ODBWrZsWb7XVFhFR0crOjr6qu0sy1LPnj2vf0E5NHjwYFmWdU3b/vTTTxo8eHCW606dOqXOnTurXLlysixL7du3v/YiARshqKBAmTx5smJiYrR69Wr961//kre3tx544AHNmzfP06XlSUJCgoYMGUJQwRX99NNPGjJkSJbrhg0bpu+++07jxo1TTEyMRo0a5ebqgOvDx9MFALlRp04d3XLLLY7nbdq0UWhoqGbOnKkHHnjAg5UVLAkJCQoMDPR0GchHW7ZsUZUqVfT444/ny/6MMUpMTFRAQEC+7A+4VvSooEArVqyY/Pz85Ovr67L81KlTeumllxQeHi4/Pz/deOONevvtt5WUlCRJSkxMVMOGDVW1alWdPXvWsd3Ro0cVFham6OhopaWlSZKeeuopFS9eXFu3blXLli0VFBSksmXLqmfPnkpISLhqjYcOHVLXrl1Vrlw5+fv7q2bNmhozZozS09MlSQcOHFDZsmUlSUOGDHFc2rraJaStW7eqdevWCgwMVNmyZfXyyy/rxx9/lGVZLj0z0dHRqlOnjlasWKEmTZooMDBQzzzzTI5qk6Rly5Zl2mdG3ZZlacqUKY5lufldGWM0ceJENWjQQAEBAQoNDVXHjh21b9++TO1GjRqlyMhIFStWTI0aNdKCBQuu+nu/3Oeff67q1avL399ftWrV0tdff+3yXnx8fDRixIhM261YsUKWZenbb7/Ndt8Zv6Np06bpjTfeUFhYmAICAtSsWTNt3LjxqrV98803at26tcqXL6+AgADVrFlT/fv314ULFxxtnnrqKX366aeS5HIJNOP/wy+//KLt27c7lmf8/7rasZAh4xLZZ599ppo1a8rf319Tp051XHZdsmSJnnvuOZUuXVrBwcHq1q2bLly4oKNHj+qRRx5RyZIlVb58efXp00cpKSlXfc9AjhmgAJg8ebKRZH777TeTkpJikpOTzeHDh02vXr2Ml5eXWbhwoaPtxYsXTb169UxQUJD58MMPzc8//2wGDhxofHx8TNu2bR3tdu3aZUqUKGE6dOhgjDEmLS3NtGjRwpQrV87ExsY62j355JPGz8/PVKpUybz33nvm559/NoMHDzY+Pj7m/vvvd6kzMjLSPPnkk47nf//9twkPDzdly5Y1n332mVm4cKHp2bOnkWR69OhhjDEmMTHRLFy40Egyzz77rImJiTExMTFmz5492f4+YmNjTenSpU2lSpXMlClTzE8//WSeeOIJExUVZSSZpUuXOto2a9bMlCpVykRERJjx48ebpUuXmuXLl+eoNmOMWbp0aaZ9GmPM/v37jSQzefLka/pdPffcc8bX19f07t3bLFy40MyYMcPUqFHD3HDDDebo0aOOdoMGDXL8bhYsWGD+9a9/mfDwcBMWFmaaNWuW7e8ogyQTERFhatWqZWbOnGl++OEH06ZNGyPJfPvtt452Dz30kKlUqZJJTU112b5Tp06mQoUKJiUlJdvXyPgdRUREmHbt2pl58+aZadOmmapVq5rg4GCzd+/eTO/H2bBhw8y4cePMjz/+aJYtW2Y+++wzU7lyZdO8eXNHmz179piOHTsaSY5/IzExMSYxMdHExMSYhg0bmhtvvNGx/OzZszk+FjJ+T+Hh4aZevXpmxowZZsmSJWbLli2OY69y5cqmd+/e5ueffzYjR4403t7e5rHHHjONGjUyw4cPN4sWLTL9+vUzksyYMWOu+v8FyCmCCgqEjJPl5T/+/v5m4sSJLm0/++wzI8nMmjXLZfnIkSONJPPzzz87ln3zzTdGkvnoo4/Mu+++a7y8vFzWG3Ppj68k8/HHH7ssf++994wks2rVKseyy4NK//79jSSzZs0al2179OhhLMsyO3fuNMYYc/z4cSPJDBo0KEe/jzfffNNYlmW2bt3qsvyee+7JMqhIMosXL3Zpm9PachtUcvK7iomJyfIP2uHDh01AQIDp27evMcaY06dPm2LFipmHHnrIpd2vv/5qJOU4qAQEBLiEn9TUVFOjRg1TtWpVx7KM9/ndd985lv3111/Gx8fHDBky5IqvkbFto0aNTHp6umP5gQMHjK+vr+nevbtjWVZBxVl6erpJSUkxy5cvN5LMH3/84Vj38ssvZ7tts2bNTO3atV2W5eZYkGRCQkLMqVOnXNpmHHuvvPKKy/L27dsbSWbs2LEuyxs0aGAaNWqU7fsDcotLPyhQvvrqK61du1Zr167VggUL9OSTT+rll1/WhAkTHG2WLFmioKAgdezY0WXbjEspixcvdix75JFH1KNHD7355psaPny4BgwYoFatWmX52pdf++/SpYskaenSpdnWu2TJEtWqVUu33XZbplqMMVqyZMnV33QWli9frjp16qhWrVouyx977LEs24eGhqpFixZuqU26+u9q/vz5sixLXbt2VWpqquMnLCxM9evXd1y2iImJUWJiYqb9NWnSRJGRkTmup2XLlrrhhhscz729vfXoo49qz549OnLkiKRLl8jq16/vuLwiSZ999pksy9Lzzz+fo9fp0qWLyx09kZGRatKkyRX/jUjSvn371KVLF4WFhcnb21u+vr5q1qyZJGn79u05fp+Xy82xIEktWrRQaGholvu6//77XZ7XrFlTknTfffdlWn7w4MFrrhm4HINpUaDUrFkz02DagwcPqm/fvuratatKliypkydPKiwsLNMtoOXKlZOPj49OnjzpsvyZZ57RP//5T/n5+alXr15Zvq6Pj49Kly7tsiwsLEySMu3P2cmTJxUVFZVpeYUKFa667ZWcPHlSlStXzrTc+Y+xs/Lly7uttpz8ro4dOyZjTLb13njjjS7tM7bPap85caXtT548qYoVK0qSevXqpe7du2vnzp268cYb9cUXX6hjx445fq3sXuePP/7Idpvz58/rrrvuUrFixTR8+HBVr15dgYGBOnz4sDp06KCLFy/m6LWzkttjIat/JxlKlSrl8tzPzy/b5YmJiddcM3A5ggoKvHr16um///2vdu3apdtuu02lS5fWmjVrZIxxOUH//fffSk1NVZkyZRzLLly4oCeeeELVq1fXsWPH1L17d82dOzfTa6SmpurkyZMuf4CPHj0qSZn+KDsrXbq04uLiMi2PjY2VJJdacqN06dI6duxYpuUZNV0uq3k7clpbsWLFJCnT4MsTJ05k+Vo5+V2VKVNGlmVp5cqV8vf3z7SPjGUZ7bN6X0ePHs0yaGUlu+2dX0O61CPSr18/ffrpp2rcuLGOHj2ql19+OUevcaXXudK/kSVLlig2NlbLli1z9KJI0pkzZ3L8utnJzbEgZf3vBPA0Lv2gwNu0aZMkOe6cadmypc6fP6/vv//epd1XX33lWJ/hxRdf1KFDhzRnzhx9+eWX+uGHHzRu3LgsX2f69Okuz2fMmCFJV5x0rGXLltq2bZs2bNiQqRbLstS8eXNJ//vDnNNPz82aNdOWLVu0bds2l+XOd7JcTU5rywgDf/75p0u7H374Idt9X+13df/998sYo7/++ku33HJLpp+6detKkho3bqxixYpl2t/q1atzdXlh8eLFLsEuLS1N33zzjapUqeLoTZEuhbLnn39eU6dO1dixY9WgQQPdeeedOX6dmTNnyhjjeH7w4EGtXr36iv9GMsLB5YHt888/z9Q2t/9OcnMsAHZFjwoKlC1btig1NVXSpW7tOXPmaNGiRXrooYccl0K6deumTz/9VE8++aQOHDigunXratWqVXr//ffVtm1b3X333ZKkSZMmadq0aZo8ebJq166t2rVrq2fPnurXr5/uvPNOl7Ebfn5+GjNmjM6fP69bb71Vq1ev1vDhw3XvvffqH//4R7b1vv766/rqq6903333aejQoYqMjNSPP/6oiRMnqkePHqpevbokqUSJEoqMjNTcuXPVsmVLlSpVSmXKlMm2x+C1117Tv//9b917770aOnSobrjhBs2YMUM7duyQJHl5Xf0zSE5rCwsL0913360RI0YoNDRUkZGRWrx4sebMmZPlfnPyu7rzzjv1/PPP6+mnn9a6devUtGlTBQUFKS4uTqtWrVLdunXVo0cPhYaGqk+fPho+fLi6d++uTp066fDhwxo8eHCuLv2UKVNGLVq00MCBAxUUFKSJEydqx44dWQa7l156SaNGjdL69es1adKkHL+GdKmn4qGHHtJzzz2ns2fPatCgQSpWrJjeeuutbLdp0qSJQkND9eKLL2rQoEHy9fXV9OnTs7xclBHgRo4cqXvvvVfe3t6qV6+e4zLM5XJ6LAC25sGBvECOZXXXT0hIiGnQoIEZO3asSUxMdGl/8uRJ8+KLL5ry5csbHx8fExkZad566y1Huz///NMEBAS43KFjzKVbhW+++WYTFRVlTp8+bYy5dCdLUFCQ+fPPP010dLQJCAgwpUqVMj169DDnz5932f7yu36MMebgwYOmS5cupnTp0sbX19fcdNNNZvTo0SYtLc2l3S+//GIaNmxo/P39jaRM+7ncli1bzN13322KFStmSpUqZZ599lkzderUTHeKZHU3SG5ri4uLMx07djSlSpUyISEhpmvXrmbdunVZ3vWT09+VMcb8+9//NrfffrsJCgoyAQEBpkqVKqZbt25m3bp1jjbp6elmxIgRJiIiwvj5+Zl69eqZefPmmWbNmuX4rp+XX37ZTJw40VSpUsX4+vqaGjVqmOnTp2e7TXR0tClVqpRJSEi46v6N+d9dP//5z39Mr169TNmyZY2/v7+56667XN6LMVnf9bN69Wpzxx13mMDAQFO2bFnTvXt3s2HDhky/36SkJNO9e3dTtmxZY1mWkWT2799vjMn+//PVjoXLf0+Xyzj21q5dm+X7OH78uMvyjH8DQH6xjHHqpwSQyVNPPaXZs2fr/Pnzni7lqp5//nnNnDlTJ0+ezPZT9vVUkH5X2fn7778VGRmpV155JcfT0C9btkzNmzfXt99+m+kOGwB5w6UfoIAaOnSoKlSooBtvvFHnz5/X/PnzNWnSJL3zzjseCSkF3ZEjR7Rv3z6NHj1aXl5eevXVVz1dEgARVIACy9fXV6NHj9aRI0eUmpqqatWqaezYsfyBvUaTJk3S0KFDFRUVpenTpys8PNzTJQGQxKUfAABgW9yeDAAAbIugAgAAbIugAiBbU6ZMkWVZjh8fHx+VL19enTt31u7du69pn++//36mCcikS3fOWJbl+J4fAJAIKgByYPLkyYqJidEvv/yinj176ocfftA//vEPnT59Otf7yi6oNGrUSDExMWrUqFE+VAygsOCuHwBXVadOHceXQUZHRystLU2DBg3S999/r6effjpfXiM4OFiNGzfOl30BKDzoUQGQaxmhJeP7cxITE9W7d281aNBAISEhKlWqlO64445MX/BoWZYuXLigqVOnOi4nZXwPTlaXfp566ikVL15ce/bsUdu2bVW8eHFFRESod+/emb4k8ciRI+rYsaNKlCihkiVL6vHHH9fatWtlWZamTJly3X4XAK4velQA5Nr+/fslyfF9QElJSTp16pT69Omj8PBwJScn65dfflGHDh00efJkdevWTZIUExOjFi1aqHnz5ho4cKCkSz0pV5KSkqIHH3xQzz77rHr37q0VK1Zo2LBhCgkJ0bvvvivp0rdgN2/eXKdOndLIkSNVtWpVLVy4UI8++uj1+hUAcBOCCoCrSktLU2pqqhITE/Xrr79q+PDhatq0qR588EFJUkhIiCZPnuzSvmXLljp9+rQ++ugjR1Bp3LixvLy8VLZs2Rxf5klOTtaQIUPUqVMnSZe+8XfdunWaMWOGI6hMnTpVe/bs0YIFC9SmTRtJUuvWrZWQkJDltxADKDi49APgqho3bixfX1+VKFFCbdq0UWhoqObOnSsfn/991vn222915513qnjx4vLx8ZGvr6++/PJLbd++PU+vbVmWHnjgAZdl9erV08GDBx3Ply9f7qjN2WOPPZan1wbgeQQVAFf11Vdfae3atVqyZIleeOEFbd++3SUEzJkzR4888ojCw8M1bdo0xcTEaO3atXrmmWeUmJiYp9cODAxUsWLFXJb5+/u77PfkyZO64YYbMm2b1TIABQuXfgBcVc2aNR0DaJs3b660tDRNmjRJs2fPVseOHTVt2jRVrlxZ33zzjSzLcmx3+YDX66V06dL6/fffMy0/evSoW14fwPVDjwqAXBs1apRCQ0P17rvvKj09XZZlyc/PzyWkHD16NNNdP9Kl3pCLFy/maz3NmjXTuXPntGDBApflX3/9db6+DgD3I6gAyLXQ0FC99dZb2r59u2bMmKH7779fO3fu1EsvvaQlS5Zo6tSp+sc//qHy5ctn2rZu3bpatmyZ5s2bp3Xr1mnnzp15rufJJ59U1apV1bVrV/3zn//UokWL9MYbb+i///2vJMnLi1MdUFBx9AK4Jq+88ooqVaqkoUOHqlu3bvrggw+0YMECtW3bViNHjlT//v3VpUuXTNt9/PHHqlatmjp37qxbb71VL7zwQp5rCQoK0pIlSxQdHa2+ffvq4Ycf1qFDhzRx4kRJUsmSJfP8GgA8wzLGGE8XAQDXw/vvv6933nlHhw4dUsWKFT1dDoBrwGBaAIXChAkTJEk1atRQSkqKlixZok8++URdu3YlpAAFGEEFQKEQGBiocePG6cCBA0pKSlKlSpXUr18/vfPOO54uDUAecOkHAADYFoNpAQCAbRFUAACAbRFUAACAbXl8MG16erpiY2NVokQJl1ktAQBA4WWM0blz51ShQoUrTsro8aASGxuriIgIT5cBAAA84PDhw1ecQsDjQaVEiRKSLhUaHBzs4WoAAIA7xMfHKyIiwpEDsuPxoJJxuSc4OJigAgBAEXO1YR8MpgUAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZ1TUElOjpalmVl+7Nw4cL8rhMAABRBeZpH5eGHH1bx4sUzLQ8PD8/LbgEAACTlMah8+OGHioqKyqdSAAAAXDFGBQAA2BZBBQAA2FaeLv18+eWXOnnypLy8vFS9enW1b99elSpVyq/aAABAEZenoDJ8+HCX53369NHAgQM1cODAbLdJSkpSUlKS43l8fHxeSoBNJSQkaMeOHVdtl5iSpiOnE1QxNFDFfL2v2r5GjRoKDAzMjxIBXCcc/8hP1xRUmjZtqu7du6tJkyYqX768Dh8+rNmzZ2v48OF69913FRwcrFdffTXLbUeMGKEhQ4bkqWjY344dO3TzzTfn+37Xr1+vRo0a5ft+AeQfjn/kJ8sYY/JrZz///LPuuecehYSEKC4uTgEBAZnaZNWjEhERobNnzyo4ODi/SoGH5fQT1Z6/z+nVrzfp484NVLVciau25xMVYH8c/8iJ+Ph4hYSEXPXvf54u/VyudevWuuWWW7Ru3Tr99ttvat68eaY2/v7+8vf3z8+XhQ0FBgbm6JOP319n5b/svGrVbaA64SFuqAzA9cbxj/yU73f9VKtWTZIUFxeX37sGAABFTL4HldOnT0tSljPWAgAA5Ea+BpXjx49r5cqVksSAJwAAkGe5Diq//fabli5dqsvH4B44cEAPPfSQLly4oAcffFAVK1bMtyIBAEDRlOvBtDt27NDTTz+t8uXLq3r16goLC9ORI0e0fv16JSYmqnbt2vriiy+uR60AAKCIyXVQuf3229WjRw+tWbNG27Zt06+//qqgoCA1aNBAnTp1Uo8ePbK8LRkAACC3ch1UatasqYkTJ16PWgAAAFzwpYQAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2fDxdAAqm9UcOKvbcsTzv5/CpBHkV+0srDmzU/vjAPO+vQokbdHPFyDzvB0D29p+4oAtJqXnez56/z7v8N6+C/H1UuUxQvuwL9mEZY4wnC4iPj1dISIjOnj2r4OBgT5aCHNp/4oLumfK2/Msu9nQpmSQdb6n/PvUeJyvgOtl/4oKaf7jM02Vka2mfaI7/AiKnf//pUUGuXUhKVcqZ2/XaHe0VUSpvvSBJqen6Oz5R5YKLyd8nb1ciD59K0OjdcfnySQ9A1jKOr48ebaCq5YrnaV+JKWk6cvqiKoYGqJivd572tefv83rtm00c/4UQQQXXxKQGq2lUQ9UJD/F0KQ5b/jqrUakXPF0GUCRULVc8X47/W6LyXgsKNwbTAgAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2+JLCQEAuWL5xGt//E55Fcvbtyfnp/3x52X5xHu6DFwHBBUAQK74llyjAb+/7+kyMvEt2VJSW0+XgXxGUAEA5ErKmds15r4uqlLOPj0qe/8+r17T93q6DFwHBBUAQK6Y1GBVDr5JtUqHeLoUh/TEszKpxz1dBq4DBtMCAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbIqgAAADbynNQOXXqlMqVKyfLslSjRo38qAkAAEBSPgSVN954QydOnMiPWgAAAFzkKagsXrxYU6dO1XPPPZdf9QAAADhcc1C5ePGiXnzxRdWqVUt9+vTJz5oAAAAkST7XuuGQIUO0d+9eLVu2TL6+vvlZEwAAgKRr7FH5888/NWbMGD399NNq2rRpftcEAAAg6Rp6VNLT0/Xcc8+pZMmSGjVqVK5fMCkpSUlJSY7n8fHxud4HPOtiSpokactfZ/O8r8SUNB05fVEVQwNUzNc7T/va8/f5PNcD4Mo4/uFuuQ4q48eP1++//67JkyerdOnSuX7BESNGaMiQIbneDvax9/+fEPrP2ezhSrIW5H/NVzQBXAXHP9zNMsaYnDY+fPiwatWqpZtvvlnLli1zLD9w4IAqV66sm266STt27LjiPrLqUYmIiNDZs2cVHByc+3cAtzt1IVk/bz2qKuWKKyAfPgW99s0mffRoA1UtVzzPtQX5+6hymaA87wdA1jj+kV/i4+MVEhJy1b//uYqeL730kpKTk/XPf/7zmgvz9/eXv7//NW8PzysV5KfOt1XK131WLVdcdcJD8nWfAPIfxz/cLVdBZf78+SpZsqR69OjhsjwxMVGSdOjQIUVHRzvaFi+e94QMAACKrlxfzDtz5oyWL1+e5bqLFy861qWmpuatMgAAUOTl6vZkY0yWP/v375ck3XTTTY5lJUuWvB71AgCAIoRvTwYAALZFUAEAALZFUAEAALaVLzPjREVFKRfTsQAAAOQIPSoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2CCoAAMC2fDxdAAqnhIQE7dix46rt9vx9TklH92jb5uJKPlbiqu1r1KihwMDA/CgRAFAAEFRwXezYsUM333xzjts/OjVn7davX69GjRpdY1UAgIKGoILrokaNGlq/fv1V2yWmpOnI6QRVDA1UMV/vHO0XAFB0EFRwXQQGBtLzAQDIMwbTAgAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2yKoAAAA2+JLCQEA+SohIUE7duy4ars9f59T0tE92ra5uJKPlbhq+xo1aigwMDA/SkQBQlABAOSrHTt26Oabb85x+0en5qzd+vXr+Vb2IoigAgDIVzVq1ND69euv2i4xJU1HTieoYmigivl652i/KHoIKgCAfBUYGEjPB/INg2kBAG737LPPyrIsx8+zzz7r6ZJgU5YxxniygPj4eIWEhOjs2bMKDg72ZCkAADewLCvbdR7+kwQ3yunff3pUAABuc6WQkpP1KHoIKgAAt3C+vDNo0CAZYxw/gwYNyrIdwKUfAIBbOPeWZPWn52rrUbhw6QcAABR4BBUAAGBbBBUAgFs888wzjseDBw92Wef83LkdwBgVAIDb5OSuHsanFA2MUQEA2M7VQgghBZcjqAAAANsiqAAA3IYJ35BbBBUAgFs89thjjscDBgxwmfBtwIABWbYDGEwLAHALJnyDs+s6mHbs2LHq0KGDqlWrppCQEPn7+ysyMlJPPvmktm7des1FAwAAOLumHpUyZcrowoULqlevnsLDwyVJW7du1a5du+Tn56fvv/9e9957b472RY8KABQN9KjA2XXtUZk7d65Onz6tNWvWaM6cOZozZ4527typiRMnKjk5Wd27d1daWto1Fw8AKHw6d+7sePz222+7rHN+7twOyPcxKtWqVdOePXu0detW1apV66rt6VEpurIa3c+nKKBwY8I3ZPDYhG/e3t6SJD8/v/zeNQqR7E5W3JoIFG5M+Ibcyteg8tVXX2nnzp2qXr26brzxxvzcNQoR5lEAii6Of+RWnoLK6NGj9dRTT6lTp06qU6eOnnzySVWoUEEzZsyQlxdTtCCzy09CzvMoXKkdgILP+SaLnj17uhz/PXv2zLIdkKcxKnfffbcWL17seB4REaH//Oc/atasWbbbJCUlKSkpyfE8Pj5eERERjFEpIhj1DxRdHP9w5pYxKr/88ouMMTp9+rRWrFihm266SdHR0Xrvvfey3WbEiBEKCQlx/EREROSlBAAAUIjl610/KSkpuuOOO7RhwwatWbNGt956a6Y29KgUbXyiAooujn8488hdP76+vnr00UdljNG8efOybOPv76/g4GCXHxRNl49DYVwKULi1adPG8fiVV15xWef83Lkd4JPfOyxTpowk6fjx4/m9axQCxhiXQJJdOOHTFFD4LFiwwHHMT5gwQRMmTMi2HZAh32/NWb58uSSpSpUq+b1rFBLMowAUXRz/yK1cB5WVK1fqm2++UWpqqsvylJQUjR8/Xv/5z38UEBCgRx99NN+KROGT3cmIkxRQuDGPCnIr10Fl79696ty5s8qXL682bdro8ccf1z333KPIyEj16tVLfn5+mjJlCnfz4Kqc51DIai4VAIXLyJEjHY+nT5/ucuxPnz49y3ZAru/62b9/vyZNmqTly5dr3759OnHihPz8/BQVFaUWLVqoV69eqlq1ao73x3f9AEDRwF0/cJbTv//5/qWEuUVQAYCiISOIlC5dWidOnMi0PjQ0VGfOnJFEUCkKPPalhAAAXMnJkyezXJ4RUgBnBBUAgFt88MEHjsczZsxwWef83LkdwKUfAIDb5OSuHi77FA1c+gEA2A7zqCC3CCoAALdhHhXkFkEFAOAW48aNczz+9ttvXeZR+fbbb7NsBzBGBQDgFsyjAmeMUQEA2FJYWFiWy0uXLu3mSlAQEFQAAG519OjRLJdnN78KijaCCgDALcaOHet4PHv2bJd1zs+d2wGMUQEAuA3zqCADY1QAALbDPCrILYIKAMBtmEcFuUVQAQC4xY8//uh4vH79epd5VNavX59lO4AxKgAAt2AeFThjjAoAwJZatWqV5fKmTZu6uRIUBPSoAADcgh4VOKNHBQBgK/Pnz3c83rBhg8s65+fO7QB6VAAAbnP5XT1NmzbVihUrXJbRm1I00KMCALCdy0MIIQVXQ1ABAAC2RVABALjN5Zd+Lr8DiAnfcDmCCgDALbKa8O3nn39mwjdcEYNpAQBuwe3JcMZgWgCALTHhG3KDHhUAgFvQowJn9KgAAGyFCd9wLehRgcdkNbqfT1FA4ZaTu3o4DxQN9KjA1rI7WXFrIlC4XS2EEFJwOYIK3O5qYYSwAhReHP/ILYIK3Oryk5AxxvFzpXYACr7ly5c7Hm/dutXl+N+6dWuW7QDGqMCtGPUPFF0c/3DGGBUAgC098sgjWS5/8MEH3VwJCgJ6VOBWfKICii6OfzijRwW2d/k4FMalAIXbsmXLHI+3bdvmss75uXM7gB4VuB3zKABFF8c/MtCjAttiHgWg6OL4R24RVOAR2Z2MOEkBhRvzqCC3CCrwGOc5FLKaSwVA4RITE+N4vHPnTpdjf+fOnVm2AxijAgBwC+76gTPGqAAAbOmpp57Kcnnnzp3dWwgKBHpUAABuQY8KnNGjAgCwldWrVzse79q1y2Wd83PndgA9KgAAt2EeFWSgRwUAYDvMo4LcIqgAANyGeVSQWwQVAIBbrFq1yvF4+/btLvOobN++Pct2AGNUAABuwV0/cMYYFQCALT3++ONZLn/44YfdXAkKAnpUAABuQY8KnNGjAgCwlZUrVzoe79ixw2Wd83PndgA9KgAAt2EeFWSgRwUAYDvMo4LcIqgAANyGeVSQWwQVAIBbbNu2zfH48OHDLvOoHD58OMt2AGNUAABu4e3trfT0dPn6+io5OTnTej8/P6WkpMjLy0tpaWkeqBDuxBgVAICtpKenS5L69++f5frXX3/dpR0g0aMCAHATelTgjB4VAICtbN68WZKUkpKiI0eOuKw7cuSIUlJSXNoBkuTj6QIAAEVDrVq1HI8jIiIkXbrL5/KOfed2AD0qAAC3uTyUXO05QFABALhNUFBQntaj6CGoAADc4ujRo0pISJAknT592mUeldOnT0uSEhISdPToUU+WCZshqAAA3KJBgwaSpMqVK6tkyZIu60qWLKnIyEiXdoBEUAEAuMmZM2ckSaNGjcpy/fvvv+/SDpCuIagkJCTo+++/17PPPqt69eopODhYQUFBql+/voYOHarz589fjzoBAAVcRi9K3759s1w/YMAAl3aAdA0Tvk2aNEnPPfecJKl27dqqVauW4uPjtXr1ap07d041atTQ8uXLVa5cuRztjwnfAKBoOHr0qMqXLy/p0hgV50By5swZhYaGSpLi4uIUFhbmiRLhRtdtwjc/Pz/16NFDu3bt0pYtWzRr1iwtXLhQO3fuVMOGDbVjxw699tpreakdAFAIhYWFKTAwUJIUGhqqqKgozZgxQ1FRUY6QEhgYSEiBi3ydQj8mJkZNmjSRv7+/4uPj5efnd9Vt6FEpuho1aqSNGzc6njds2FAbNmzwYEUA3CEoKMhx94+zwMBAXbhwwQMVwRNy+vc/X2emrV+/viQpKSlJJ0+edHTxAZezLCvTso0bN2Y5SyWAwuXixYu5Wo6iLV/v+tm3b58kydfXV6VKlcrPXaMQySqk5GY9gILLy8vL8WEkMDBQo0ePdlwOMsbIy4ubUeEqX/9FfPzxx5KkNm3ayN/fPz93jUKiUaNGjsedOnVymfCpU6dOWbYDUDjs2bPHEVLi4uJ04cIF9enTRxcuXFBcXJykS2Flz549niwTNpNvY1R++ukn3X///fLx8dHatWsdl4Eul5SUpKSkJMfz+Ph4RUREMEaliHDuLcnqn97V1gMouPz8/JSSkpLtWJSMsSu+vr5KTk72QIVwp+t2109Wtm/frq5du8oYo9GjR2cbUiRpxIgRCgkJcfxkfIMmAKBwS0lJkSQNGTIky/UZ86hktAOkfOhROXLkiO68804dOnRIb7zxhsaMGXPF9vSoFG30qABFFz0qcOaWHpUTJ06oVatWOnTokJ5++ml9+OGHV93G399fwcHBLj8oOho2bOh4/Mgjj7isc37u3A5A4bBt2zZJWX/xoPMXFma0A6Q89KicO3dOLVq00Lp169ShQwfNmjVL3t7eud4P86gUPTm5q4feFKBwcr7rJyuWZSk9Pd2NFcFTrmuPSlJSktq1a6d169bpnnvu0cyZM68ppKBouloIIaQAhdfVQgghBZfLdVBJS0vTY489pqVLl+quu+7SnDlzcjQDLeDMGJPp8k7Dhg0JKUAhxzxKyK1cz0w7YcIEfffdd5KkMmXK6KWXXsqy3YcffqgyZcrkrToUakyXDxQtO3fudDz+66+/VKFCBcfz2NhYhYeHO9rddNNNbq8P9pTroHL69GnH44zAkpXBgwcTVAAADrVr15Z06aYK55AiSRUqVJC/v7+SkpJUu3ZtpaameqJE2FCuL/0MHjzYZTbR7H6ioqKuQ7kAgIIqLS1NkjRw4MAs1/ft29elHSDl87cnXwvu+gGAosHHx0dpaWny9/dXYmJipvXFihVTUlKSvL296VEpAtw6My0AAFezdetWSZfuHI2NjXVZFxsb65gMNKMdIF3DGBUAAK6F8wDZjIGzXl5emW5JZiAtnNGjAgBwm8tHG1weUpiiAJcjqAAA3CYoKChP61H0EFQAAG7h/H0+p0+fdrlTNGPqi6y+BwhFG0EFAOAWDRo0kCRVrlxZJUuWdFlXsmRJRUZGurQDJIIKAMBNzpw5I0kaNWpUluvff/99l3aARFABALhJRi9KxsRulxswYIBLO0AiqAAA3GTTpk2SpP3792fqNTlz5owOHjzo0g6QmEcFAOAmYWFhCgwMVEJCgkJDQ2VZlmMelYzbkgMDAxUWFubhSmEn9KgAANzmwoULjsfGGKWlpbnMneK8HpAIKgAAN3KeJ8WyLHl7e8uyrCzXAxJBBQDgJpfPo5Kenq7U1FSlp6czjwqyRVABALgF86jgWhBUAABuwTwquBYEFQCAWzCPCq4FQQUA4BbMo4JrQVABALhFxjwqkhQaGqqAgABVrFhRAQEBCg0NlcQ8KsiMCd8AAG5z4cIFeXl5yRijxMRE/fXXX451lmUxjwoyoUcFAOA2t912m2OCt2LFisnX11fFihWTdGkCuNtuu82T5cGGCCoAALc4f/681q5dK8uydPHiRV28eFHJycmOx5Zlae3atTp//rynS4WNEFQAAG7xxBNPSJK6du3q6EXJUKxYMXXp0sWlHSARVAAAbrJ3715JUp8+fbJc/8Ybb7i0AySCCgDATapUqSJJ+vDDD7NcP3bsWJd2gCRZxvlrKz0gPj5eISEhOnv2rIKDgz1ZCgDgOjp//rxKlCghy7KUkJDgcvknMTFRgYGBMsbo3LlzKl68uAcrhTvk9O8/PSoAALcoXry4br31VhljFBgYqK5du2rDhg3q2rWrI6TceuuthBS4IKjAY3r37i3Lshw/vXv39nRJAK6z33//3RFWpk+frptvvlnTp093hJTff//d0yXCZggq8AjLshzXozOMHTtWlmV5qCIA7lKhQoVcLUfRRlCB210eRi6/TZGwAhRe7du319y5c+Xn56f+/ftrz5496t+/v/z8/DR37ly1b9/e0yXCZggqcCvnyzvjxo2TMUYXL16UMUbjxo3Lsh2AwuHixYuOkHLu3DmNGDFCVapU0YgRI3Tu3DlHWLl48aKnS4WNEFTgVs6Xe1577TWXdc7PL78sBKDge/PNNyVdmi/Fz8/PZZ2fn5/jHJDRDpAIKvCQyy/3ZPD19XVzJQDcZffu3ZKk7t27Z7n+2WefdWkHSAQVeEhiYmKWy1NSUtxcCQB3qVatmiRp0qRJWa7/8ssvXdoBEkEFbpYxRbYkffTRRy7rnJ87twNQOIwePVrSpUu7ycnJLuuSk5Md54CMdoDEzLTwgMvv6vH19c3Uk+Lhf5YArhPnu35q1aqlwMBAJSQkaNu2bUpOTla7du30/fffe7pMuEFO//4TVOARV7oFmZACFG5hYWE6duxYpuU33HCDjh496oGK4AlMoQ9bM8ZkurzzxhtvEFKAQq59+/Y6duyYfH191ahRI915551q1KiRfH19dezYMeZRQSb0qAAA3OLixYsKDAx0zKPifItycnKySpQooeTkZCUkJCggIMCDlcId6FEBANgK86jgWhBUAABuwTwquBYEFQCAWzCPCq4FY1QAAG7BGBU4Y4wKAMBWAgIC1K5dO0co6devn3bt2qV+/fo5Qkq7du0IKXBBjwoAwK0yJn27HJO9FS05/fvv48aaAADQ999/r4sXL+rNN9/U7t27Va1aNY0ePZqeFGSJoAIAcLuAgABNmDDB02WgAGCMCgAAsC2CCgAAsC2CCgAAsC2CCgAAsC0G0wIA3C4tLU0rV65UXFycypcvr7vuukve3t6eLgs2RI8KAMCt5syZo6pVq6p58+bq0qWLmjdvrqpVq2rOnDmeLg02RFABALjNnDlz1LFjR9WtW1cxMTE6d+6cYmJiVLduXXXs2JGwgkyYmRYA4BZpaWmqWrWq6tatq++//15eXv/7rJyenq727dtry5Yt2r17N5eBigC+6wcAYCsrV67UgQMHNGDAAJeQIkleXl566623tH//fq1cudJDFcKOCCoAALeIi4uTJNWpUyfL9RnLM9oBEkEFAOAm5cuXlyRt2bIly/UZyzPaARJBBQDgJnfddZeioqL0/vvvKz093WVdenq6RowYocqVK+uuu+7yUIWwI4IKAMAtvL29NWbMGM2fP1/t27d3ueunffv2mj9/vj788EMG0sIFE74BANymQ4cOmj17tnr37q0mTZo4lleuXFmzZ89Whw4dPFgd7IjbkwEAbsfMtMjp3396VAAAbuft7a3o6GhPl4ECgDEqAADAtggqAADAtnJ96Wf9+vVatGiRfv/9d61Zs0axsbHy9/dXYmLi9agPAAAUYbkOKsOGDdPcuXOvRy0AAAAucn3p54477tC7776refPm6ejRo9ejJhQRo0ePlmVZjp/Ro0d7uiQAbrJp0yaX43/Tpk2eLgk2lefbky3LytOlH25PLposy8p2nYfvmAdwnXH8Q+Lbk2Fjl5+kypQpc8X1AAoP5+Pby8tLr7/+uss3KXP843IEFbiV8+WdmTNnyhij48ePyxijmTNnZtkOQOHgfHln//79SktL09ixY5WWlqb9+/dn2Q5w+6WfpKQkJSUlOZ7Hx8crIiKCSz9FhPOnpaz+6V1tPYCCK+P49vLyUlpaWqb13t7eji8r5Pgv/Gx76WfEiBEKCQlx/ERERLi7BNjA5Zd7MoSGhrq5EgDu9uqrr2a5/IUXXnBzJSgI6FGBW9GjAhRd9KjAmW17VPz9/RUcHOzyg6Jj1KhRjsdff/21yzrn587tABQOGzdulCSlp6frwIEDLusOHDjgCCkZ7QCJ25PhAZeP6g8NDdXp06ddlvFpCiicLr/r54UXXtDnn3/uCCkSx39RYdseFeDykxAhBSg6nI/v9PR0/fOf/ySk4IoIKvAIY0ymyzujRo3iJAUUAcaYTJd3Nm7cyPGPLHHpBwAAuF1O//7n+ksJf/zxRw0bNsxlWXJysho3bux4PnDgQN1333253TUAAICLXAeV48ePa82aNS7LjDEuy44fP573ygAAQJGX50s/ecWlHwAAih7u+gEAAAUeQQUAANgWQQUAANgWQQUAANgWQQUAANgWQQUAANhWrudRyW8Zd0fHx8d7uBIAAOAuGX/3rzZLiseDyrlz5yRJERERHq4EAAC427lz5xQSEpLteo9P+Jaenq7Y2FiVKFHC5eu/UTTEx8crIiJChw8fZsI/oIjh+C/ajDE6d+6cKlSoIC+v7EeieLxHxcvLSxUrVvR0GfCw4OBgTlRAEcXxX3RdqSclA4NpAQCAbRFUAACAbRFU4FH+/v4aNGiQ/P39PV0KADfj+EdOeHwwLQAAQHboUQEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAALZFUAEAFHjMXVp4EVRQYKSnp3u6BAA2cfn5wLIsD1WC683H0wUAOZGWliZvb29J0rx587Rhwwbt3btXJUqUUJcuXVSlShWFhYV5uEoA7uB8Pli6dKn27dunzZs369Zbb1X9+vVVp04dD1eI/MR3/cD20tPT5eV1qfPv7bff1siRI5Weni5fX1+lpKQoKChIDzzwgPr06aNGjRp5uFoA15Pz+WDQoEEaN26cLly44Lj0ExAQoE8++URdunRRQECAJ0tFPuHSD2wv46Q0YsQIjRgxQs2aNdPPP/+svXv3asGCBWrXrp2+/vpr9e3bV7t37/ZwtQCup4zzwZAhQzRs2DDVrl1bM2bM0H//+1+99957KlWqlJ577jmNHz9e58+f93C1yA9c+kGBsGHDBn366aeqVauWxo0bp3r16kmSKlasqCVLlkiSSpYsqdTUVE+WCcANVqxYoTFjxui2227T559/7jgftGrVSps2bdK3336rX3/9VV27dlXx4sU9XC3yih4VFAgHDx5UbGysevXq5TgpSZc+VY0ePVr33Xef+vfvr5o1a0qSkpOTPVUqgHyU1SD6P//8U+fPn9c777zjcj4YPny4vv32W7Vr105vv/22KlSo4M5ScZ0QVGALv/zyi15//fVse0T27t0rSSpfvrxj2dChQzVkyBC1bdtWw4cP1y233CJJ+uOPP7Ro0SJdvHjx+hcOIN/9+uuv+uijj5SWliYvLy9HWMn477p16yRJUVFRjm2GDh2qd999V23bttXgwYN12223SZJWrVqljRs3uvcNIF9x6Qced/bsWfXu3VubN2+Wt7e3Ro4c6RjRnyFjUNzmzZv1wAMPaOjQoRo8eLDatm2r9957T/Xr13e0/fTTT7VixQotW7aMwXRAAXP8+HE99NBDOnPmjLy8vPTyyy/L29vbZRBtxYoVJf1v7pRBgwZp2LBhWZ4PJk+erM2bN2vRokUKCQlx/xtC3hnABhYvXmwaNWpkLMsyr732mklNTXVZf+DAAVOhQgXTtGlT8+KLLxrLssz9999vNm7c6NLuhx9+MIGBgeaRRx4xZ8+edeM7AJAfkpKSzLRp00yFChVMmTJlzNixYx3ng5SUFGOMMdOmTTOWZZlOnTqZt99+21iWZdq2bWs2bdrksq/vvvvOeHt7m169epnk5GS3vxfkD4IKPCo9Pd3xePny5aZu3bqZwkp6erqJj483vXr1Mj4+PsayLNOiRQuzbt06l338+uuv5rbbbjNlypQxP/30k/vfDIB8kZSUZGbNmmXKli2bKawYY0xCQoLjg41lWaZNmzZm9+7dLvtYuXKlqVevngkLCzNLly518ztAfiKowOOuFFYyPkEZY8yff/5pmjdvbry8vEzTpk3N77//bs6fP2+Sk5PNd999Z+rXr28syzITJ070xNsAkI+uFlY2bNhgqlWrZizLMg899JDLtvPmzTONGzc2lmWZL774wt2lI58RVGALVworzl22a9euNffff7/x8fExISEhpk6dOo62QUFBZvz48Y62aWlpbn0PAPLXlcJKUlKSWbRokSOsVKtWzbRv3940b97ceHt7G19fX/PRRx859uV8jkHBwsy0sIWM0fwZg+VWrFihnj17asuWLXr11Vc1atQo+fr6SpL279+vlStX6ssvv1RcXJyMMWrXrp3uuecetWrVyrG/jH0BsD/nafGdJScna+7cuXr55ZdljNGAAQPUq1cvR9sjR46oX79+2rhxo/766y+VKFFCLVq0UMeOHfXggw9K4nxQ0BFU4BE5OXFcKaxkSEtLU0pKiooVK5arfQOwp4EDB6pWrVp67LHHHMuuFlbS09N19uxZHTt2TKVKlVJoaKjjXMH5oOAjqMDtnD85LV++XH/99Zf279+vO++8U5UqVdKNN97oaHt5WBk9erR8fHwyffoyxvDtqUABlBEk0tPTtWnTJt1yyy0KCAjQV199pYcfftjR7kph5fIwwvmgkPHMFScUVc7XiQcOHGj8/f0dI/f9/PxMrVq1zLRp01y2ye5uIAAFm/OxvHfvXnP06FHTrl074+XlZUqWLGlmz57t0v5KY1Y4LxReBBV4xAcffGAsyzL169c3n3zyienfv79p3769I7R8+OGHLu2dw0rPnj1d7gYCUPA4D3YfNGiQufHGG01AQIBp2LChYxqC4sWLmzlz5rhs5xxWwsLCzOjRowkphRxBBW7hfCK5ePGiqVu3rmnVqpXZsmWLS7vPP//cEVYuv814xYoVpkaNGsayLLN8+XK31A3g+ho+fLjjFuOff/7ZnDt3zsyZM8cxsWNgYGCWYeXbb781wcHBxtvb22zYsMFD1cMdGKMCt5o2bZoqVaqkJ554QhMmTNADDzzg+D6PjGvKkydP1rPPPitJWrRokVq2bOnYfsmSJYqLi9Pjjz/ukfoB5J8tW7aodevW8vLy0oIFC1S3bl2X9e+//77eeecdBQYG6quvvlKHDh0c65KTkzVr1iwlJCTo+eefd3fpcCO+6wdu88UXX+iFF15Q06ZNdebMGQUFBUmSI6RkDIh7+umndeDAAQ0bNkyLFy9Wy5YtlZqaKh8fH7Vo0cKxP0bzAwVbfHy8Tp48qW7dujlCivNxPWDAAF28eFHvvfeenn76aaWlpalTp06SJD8/P3Xp0sXRlvNB4cX/VbhN27Zt1aRJE61YsULnzp3ToUOHJLnOoZLxOCOQLFq0SElJSfLxyZypOSkBBduFCxeUkpKiY8eOKT09XampqY7jOqOz//nnn1fjxo117tw5vfDCC5o7d65je+dzAOeDwov/s3Cb8PBwzZ492zEp26BBg3To0CHH7YXO7rjjDt1www1KTU3NtA5A4VC5cmVVrFhRq1ev1t69e+Xj4+M43jMuBYeHhyssLEzBwcE6c+aMevXqpVWrVnmybLgZQQX5LrthT2lpaQoLC9PUqVN1zz336PDhw+revbsOHDggLy8vx1gVSVq6dKmOHTvmmFOFsAIUTBnnA3Pp5g2XdVWrVtXdd9+tU6dO6cUXX1RsbKzjXGCMcVzOCQ8PV/fu3TVo0CAdPnxY06dPV1JSUrbnGhQuBBXkq7S0NMcnoZMnT+rAgQPat2+fJDkmaAsLC9PkyZPVokUL/fLLL+rQoYM2btyo8+fPS7o0Cdzo0aMlSY8++qgCAgLo1gUKIOfzgWVZOnXqlNLT05WSkuJoM2bMGN1+++1aunSpevbsqdjYWHl7e8uyLHl5eSkmJkb/93//Jx8fH/Xp00cNGjTQ999/rxMnTjCpWxHBYFrkm/T0dEcY+fDDDzVz5kzt2LFDlmUpOjpa999/vx555BGVKlVKYWFhmjZtmp588kktWrRITZs2Vc2aNVW6dGmtWrVK3t7eGjt2rB555BFJzDQJFDTOs0dPmTJFP/74o5YuXarIyEiFhYXp3XffVc2aNRUaGqpPPvlEL774or7//ntt375dAwcO1I033qi4uDiNHj1ap06d0p133qmgoCBFRERo06ZN2rNnj8LDwz38LuEWnrovGoXXgAEDjGVZplSpUubuu+82ERERxs/Pz/j5+ZlOnTqZo0ePOtrGxcWZNm3aGMuyTIUKFUyHDh3MrFmzzKJFixxt+BZkoGBxnoG6b9++xrIsExAQYCpXrmwqVKhgLMsyoaGhZvDgwebQoUPGGGN27txpWrdu7ZhHyfnHeQLI22+/3VSsWNEcPHjQ7e8LnkFQQZ45B4lff/3VBAcHmw4dOjgmYdq3b5+ZMmWKqV+/vrEsy7Rt29b8/fffjm3i4uJMixYtjGVZpn379ubMmTPGmEsnO0IKUHCNHTvWcVxv2LDBJCYmmhMnTph+/fqZyMhIExAQYPr162diY2Md2/zrX/8yvXv3Nk2bNjW9e/d2mext5MiRxrIs8/DDD5v4+HhPvCV4AEEFeeL8yenw4cPm66+/NqGhoWbTpk3GmP+FmLS0NPPnn3+am2++2ViWZV5//XVz4cIFx7ZxcXGmefPmxrIs065dO3P48GGX7QEULHFxcaZBgwamYsWKjvNBxgzVaWlpZurUqeamm24yJUqUMF999VWm7S8/9seMGWPKlStnwsLCzK5du67/G4BtEFSQL9566y3TsGFD88gjj5h69eoZYy6dlJyDjDHGrFq1ylSqVMnUqFHD0XWb0ca5Z+Xhhx82Bw4ccO+bAJBv/vjjD+Pl5WW6d+9ujHH90GLMpeN+woQJxrIsU7lyZXPs2DHH8ox2aWlpZvv27eahhx4yfn5+JioqymzevNkD7waexK0UyLMzZ87o5MmT2rRpkxYuXKjExETHwFpz2e2D9evX1x133KGdO3fqxx9/lPS/+RLCwsI0ffp0tWrVSnPmzNHbb7+t1NRUt78fAHl35swZGWO0b98+nTt3znHnnpeXl2Nw/Msvv6zmzZvrwIEDWrNmjaT/nQ+8vLzk5eWl2NhYbdu2TR07dtQvv/yiOnXqeOw9wTMIKsizkiVLasCAAerfv79SUlK0e/duffzxx5L+d1KSLt0VVLx4cbVr107SpduXL5dx6/IDDzygfv36ZTkjLQD7q127tqpUqaIDBw7or7/+knTpTiDpUhhJTk6WJDVr1kySdODAgSz306JFCy1evFifffaZqlSpcv0Lh+0QVJArxmnyJmeRkZF65pln9Prrr8vX11f//ve/NW/ePEmXTkopKSmOT1THjx+XJN1www1ZvkaFChU0Z86cTF9QBsBeLj8PZEhLS1NgYKDuuusu7d+/X4MHD5YkxyzU6enp8vPzk3TpywX9/PxUo0aNbPcfHh6uEiVKXJ83AdsjqCDHnCdvio2N1fbt27VlyxadPXtW0qVZJp9++mm9+uqr2rFjh4YNG6aZM2dKknx9fSVJv/32m7788kv5+vqqVq1a2b5WxvwLAOzJ+XwgSadPn3ZM2ujt7a2AgAD17t1bFSpU0KxZs9StWzclJiY6LulIl84H06dPV1hYmMLCwjK9BnMnQZIsk10kBpw4T940fvx4TZkyRbt27VJqaqqio6P16KOP6qmnnpIk7d27V5999pk+/vhj+fn5qV27dmrevLmOHj2qr7/+Wtu2bdNHH32kXr16efAdAbhWzueDyZMn67///a+WLVumEiVKqEmTJurcubOaNGmikJAQrV27Vq1bt9bZs2cVHR2te++9Vy1atNC2bdv06aef6vfff9dnn32m559/3sPvCrblqVG8KDicbxPs37+/Y7Kmzp07m86dOxtvb29TrFgxM3DgQEe7PXv2mP79+5uAgADHqP6IiAjTuXNnM2XKlCz3DcD+sprMrUSJEqZmzZomODjYWJZlwsPDzRtvvOG4k2fDhg2mcePGjvNBxk/x4sXNJ598kuW+gQwEFeRYxuRNDzzwgFmzZo1j+Ysvvmi8vLyMZVkuYWXXrl1mwIABxs/Pz7Ro0cLMnj3bZX+EFKDgcj4f/P777yYpKcls3rzZDBw40FSuXNlYlmV69OhhTpw4YYy5NM/Sjz/+aF5//XXTvXt38/HHH5uVK1c69sf5ANkhqCBH1q9fb6pXr24aNmxo1q9fb4wxJikpyYwcOdIUL17chIeHG29vb2NZlnnnnXcc2+3atcv07t3beHt7m6ZNm5qff/7ZsY5PT0DBdOzYMVOvXj1TqVIls2XLFmPM/yZzi4+PN3PnzjXVqlUzgYGBZty4cSYpKemK+yOk4EoYTIscWbVqlXbv3q33339fjRo1UkpKiiZOnKihQ4cqLCxMR44c0ZIlS+Tl5aX33ntPAwYMkCRVq1ZNPXr00Ouvv67Vq1dr+PDhWrx4sSQGygEFVVxcnDZv3qyWLVuqdu3aSk1NdQyQLVGihFq3bq1BgwbJx8dHc+bMcWyXnp6e5f74dnRcCf86kCNt2rRR79691apVKxljtHDhQo0ePVrlypXTypUrJUl169bVQw89JOnStye/9dZbkqQqVarolVde0Wuvvaa1a9eqd+/eWr58ucfeC4CcyypcZCw7c+aMJMnHx8flg0exYsXUunVrVa9eXatWrdLChQslEUhwbfhXgxypXr26hg8fLm9vb1mWpenTp+vixYuaNWuWwsLClJKSotDQUEVEROiGG25QamqqRo4cqd9++02SVKlSJb3yyit68skn9ddffykqKsqzbwjAVRljHOHi6NGjjuWlS5dWQECAli5dqhUrVmTaLi0tTWXLllXHjh0lSRcuXHBPwSiUCCrIMX9/f0nShg0b9H//93+qVauWYy6UjBlkk5OT1bZtWy1evFgff/yxGjdu7Ni+UqVKevvtt7V161ZFRka6/w0AyJWMXpK+ffuqVatW+vPPP5WamqpKlSrp+eef19mzZzVr1iyXEJOamuq4dTmjx6V8+fJurx2FB0EFuVayZEnHLJEZJyTLsvTrr79q3rx58vf3V/PmzfXKK69Icu06rlixosqVK+f+ogHkmPMxm56erv3792vnzp168803tXXrVklShw4dVLNmTU2cOFFjxozR3r17Jf3vQ8tvv/2mWbNmKSoqiqCCPOGLVJBrYWFhatKkiX766Sd98MEHuvfeexUbG6uRI0fq2LFjatOmjUt7rksDBYfzZG6zZ8/W33//rWPHjiksLEyLFi1S//79NXr0aN11110aOnSoevfurTFjxmjjxo1q2bKl2rRpo82bN2vChAnav3+/Pv/8c910000eflcoyJiZFtdkx44duu+++3TgwAFH97AxRh999JGjJwVAwWL+/7caS1L//v01fvx4lSpVSv/4xz+Umpqq+fPnKykpSU2bNtXEiRNVq1Yt/fjjj5owYYKWLFmilJQUx76CgoL0/vvvO84HzvsGcoOggmu2Z88eff7551q0aJFuv/12tW7dWg8//LCkS93F9KQABdMXX3yhF154QQ8++KCGDRvm+ILQTZs2acCAAVq4cKHuuusuTZw4UbVr19bhw4f1559/au7cuUpISNAtt9yiBg0aKDo6WhLnA+QNQQV5lvHtpxk4KQEFV1JSkp544gnNnj1by5cv11133SXpf5eE9u7dq7feekuzZ89Wy5YtNXbsWNWuXVteXl5Z9ppwPkBeEVQAAA4JCQlq1qyZDh8+rD179qh48eKOsJERRHbu3Kl27dpp165datOmjT744APVq1fP06WjkCLmAgBc+Pv76++//9aCBQsk/W9AvGVZMsbopptu0rvvvitJWrhwoV599VXFxsZKutTzAuQnggoAwCEwMFBdunSRt7e3fvnlF128eNFlfUYnfFRUlMqUKaOOHTtq+fLl6tevn6T/TVkA5BeCCgDARbt27VS1alV98cUX+vTTT7Nsc+DAAfn5+emll15SnTp1NH36dM2aNcvNlaIoIKgAAFyEh4fr66+/lre3t/r27auhQ4dqx44dki5dBoqJidGECRNUvnx5RUdHq3fv3rIsSwcPHvRw5SiMmPANAJBJ/fr1tWbNGt19990aPHiwZs2apdtvv12hoaGaNWuWjhw54uhtKV68uIwxBBVcFwQVAECWGjVqpJiYGA0ZMkSzZ8/Wtm3bJEnlypXTJ598oh49ekiS5s+fL8uy9I9//MOT5aKQ4vZkAMAVpaamavfu3dqxY4dKliypMmXKOCaBGz9+vN544w3Vq1dP8+fP53t9kO8IKgCAXDPGaNCgQfr8888lScuXL1eNGjU8XBUKIwbTAgByJONz7eHDh/XYY49p+PDhCgsL07JlywgpuG7oUQEA5Mr58+c1Z84cnTlzRg8//LDCw8M9XRIKMYIKACDX0tPTZYxhgjdcdwQVAABgW4xRAQAAtkVQAQAAtkVQAQAAtkVQAQAAtkVQAQAAtkVQAZArUVFR+uijj/K8n4SEBD388MMKDg6WZVk6c+ZMnvcJoPAhqAC4rqZMmaKSJUtmWj516lStXLlSq1evVlxcnEJCQtxfHADb49uTAXjE3r17VbNmTdWpU+ea95GWlibLsuTlxWcuoLDi6AbgIjo6Wj179lTPnj1VsmRJlS5dWu+8846ymxty7Nixqlu3roKCghQREaGXXnpJ58+flyQtW7ZMTz/9tM6ePSvLsmRZlgYPHqzo6GiNGTNGK1askGVZio6OliSdPn1a3bp1U2hoqAIDA3Xvvfdq9+7djtfK6J2ZP3++atWqJX9/fx08eFBRUVEaPny4unXrpuLFiysyMlJz587V8ePH1a5dOxUvXlx169bVunXrrvvvD0D+IqgAyGTq1Kny8fHRmjVr9Mknn2jcuHGaNGlSlm29vLz0ySefaMuWLZo6daqWLFmivn37SpKaNGmijz76SMHBwYqLi1NcXJz69OmjOXPm6LnnntMdd9yhuLg4zZkzR5L01FNPad26dfrhhx8UExMjY4zatm2rlJQUx+slJCRoxIgRmjRpkrZu3apy5cpJksaNG6c777xTGzdu1H333acnnnhC3bp1U9euXbVhwwZVrVpV3bp1yzZwAbApAwBOmjVrZmrWrGnS09Mdy/r162dq1qxpjDEmMjLSjBs3LtvtZ82aZUqXLu14PnnyZBMSEpKp3auvvmqaNWvmeL5r1y4jyfz666+OZSdOnDABAQFm1qxZjn1JMps2bXLZV2RkpOnatavjeVxcnJFkBg4c6FgWExNjJJm4uLgr/wIA2Ao9KgAyady4sSzLcjy/4447tHv3bqWlpWVqu3TpUrVq1Urh4eEqUaKEunXrppMnT+rChQu5es3t27fLx8dHt99+u2NZ6dKlddNNN2n79u2OZX5+fqpXr16m7Z2X3XDDDZKkunXrZlr2999/56ouAJ5FUAFwzQ4ePKi2bduqTp06+r//+z+tX79en376qSS5XK7JCZPNJRljjEtoCggIcHmewdfX1/E4Y31Wy9LT03NVFwDPIqgAyOS3337L9LxatWry9vZ2Wb5u3TqlpqZqzJgxaty4sapXr67Y2FiXNn5+fln2xFyuVq1aSk1N1Zo1axzLTp48qV27dqlmzZp5eDcACjKCCoBMDh8+rDfeeEM7d+7UzJkzNX78eL366quZ2lWpUkWpqakaP3689u3bp//85z/67LPPXNpERUXp/PnzWrx4sU6cOKGEhIQsX7NatWpq166dnnvuOa1atUp//PGHunbtqvDwcLVr1+66vE8A9kdQAZBJt27ddPHiRd122216+eWX9corr+j555/P1K5BgwYaO3asRo4cqTp16mj69OkaMWKES5smTZroxRdf1KOPPqqyZctq1KhR2b7u5MmTdfPNN+v+++/XHXfcIWOMfvrpJ5dLOACKFstkd2EYQJEUHR2tBg0a5Ms0+QCQV/SoAAAA2yKoAAAA2+LSDwAAsC16VAAAgG0RVAAAgG0RVAAAgG0RVAAAgG0RVAAAgG0RVAAAgG0RVAAAgG0RVAAAgG39PytZQEZTFM4kAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Call the boxplot() method on our df.\n",
    "df.boxplot(by='platform', column=['Rating'], grid=False, rot=45, fontsize=15);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FqTlAjnmFcvo"
   },
   "source": [
    "Here we see the same information as in the analytical summary, but with a boxplot. Can you see how the boxplot is working here? If you need to revise your boxplots, check out this this [link](https://www.kaggle.com/ramamet4/app-store-apple-data-set-10k-apps). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "O8L-s1fgFcvp"
   },
   "source": [
    "## Stage 3 - Modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5fbRq2MZFcvq"
   },
   "source": [
    "### 3a. Hypothesis formulation\n",
    "\n",
    "Our **Null hypothesis** is just:\n",
    "\n",
    "**H<sub>null</sub>**: the observed difference in the mean rating of Apple Store and Google Play apps is due to chance (and thus not due to the platform).\n",
    "\n",
    "The more interesting hypothesis is called the **Alternate hypothesis**:\n",
    "\n",
    "**H<sub>alternative</sub>**: the observed difference in the average ratings of apple and google users is not due to chance (and is actually due to platform)\n",
    "\n",
    "We're also going to pick a **significance level** of 0.05. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TjBqkabaFcvq"
   },
   "source": [
    "### 3b. Getting the distribution of the data\n",
    "Now that the hypotheses and significance level are defined, we can select a statistical test to determine which hypothesis to accept. \n",
    "\n",
    "There are many different statistical tests, all with different assumptions. You'll generate an excellent judgement about when to use which statistical tests over the Data Science Career Track course. But in general, one of the most important things to determine is the **distribution of the data**.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bFwVhkeWFcvr"
   },
   "outputs": [],
   "source": [
    "# Create a subset of the column 'Rating' by the different platforms.\n",
    "# Call the subsets 'apple' and 'google' \n",
    "apple = df[df['platform'] == 'apple']['Rating']\n",
    "google = df[df['platform']== 'google']['Rating']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oPPFpwPvFcvu"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NormaltestResult(statistic=1778.9974234584017, pvalue=0.0)\n"
     ]
    }
   ],
   "source": [
    "# Using the stats.normaltest() method, get an indication of whether the apple data are normally distributed\n",
    "# Save the result in a variable called apple_normal, and print it out\n",
    "apple_normal = stats.normaltest(apple)\n",
    "print(apple_normal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rCdh8a8KFcvw"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NormaltestResult(statistic=3678.6157187516856, pvalue=0.0)\n"
     ]
    }
   ],
   "source": [
    "# Do the same with the google data. \n",
    "google_normal = stats.normaltest(google)\n",
    "print(google_normal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9gDqu21yFcv0"
   },
   "source": [
    "Since the null hypothesis of the normaltest() is that the data are normally distributed, the lower the p-value in the result of this test, the more likely the data are to be non-normal. \n",
    "\n",
    "Since the p-values is 0 for both tests, regardless of what we pick for the significance level, our conclusion is that the data are not normally distributed. \n",
    "\n",
    "We can actually also check out the distribution of the data visually with a histogram. A normal distribution has the following visual characteristics:\n",
    "    - symmetric\n",
    "    - unimodal (one hump)\n",
    "As well as a roughly identical mean, median and mode. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "P9tT9RjNFcv1"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGdCAYAAADjWSL8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAieElEQVR4nO3df2xV9f3H8de1vwTWnlGwvb2hss4hoq3EFNOWqKDFAlvp/JGAI2lwY8AmFLtCVPQPu8XQaSa4pRlDR0QRh8m06iLrqBHqCFSB0QgMGQacEHopsnIvsO4W8Hz/8MuJtz+AW6m37/J8JDfh3vu+l89nn2Gfub239bmu6woAAMCYq+K9AAAAgN4gYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGBSYrwX0Fe++OILHTlyRKmpqfL5fPFeDgAAuASu6+rkyZMKBAK66qoLv9YyYCPmyJEjys7OjvcyAABALxw6dEgjRoy44MyAjZjU1FRJX/6PkJaWFufVAACASxEOh5Wdne19Hb+QARsx57+FlJaWRsQAAGDMpbwVhDf2AgAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYlxnsBAABcVtVOvFfQO9WheK/AHF6JAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJsUUMTU1Nbr11luVmpqqjIwM3XPPPdq3b1/UzIMPPiifzxd1KSwsjJqJRCKqqKjQ8OHDNWTIEJWVlenw4cNRM21tbSovL5fjOHIcR+Xl5Tpx4kTvdgkAAAacmCKmsbFR8+fPV1NTkxoaGnT27FmVlJTo9OnTUXNTpkxRS0uLd1m/fn3U/ZWVlaqrq9O6deu0efNmnTp1SqWlpTp37pw3M3PmTDU3N6u+vl719fVqbm5WeXn519gqAAAYSBJjGa6vr4+6/uKLLyojI0M7duzQHXfc4d2ekpIiv9/f7XOEQiGtWrVKa9as0aRJkyRJr7zyirKzs/Xuu+9q8uTJ2rt3r+rr69XU1KSCggJJ0gsvvKCioiLt27dPo0ePjmmTAABg4Pla74kJhUKSpPT09KjbN23apIyMDF1//fWaM2eOWltbvft27NihM2fOqKSkxLstEAgoNzdXW7ZskSRt3bpVjuN4ASNJhYWFchzHm+ksEokoHA5HXQAAwMDV64hxXVdVVVW67bbblJub690+depUrV27Vu+9956effZZbdu2TXfddZcikYgkKRgMKjk5WUOHDo16vszMTAWDQW8mIyOjy9+ZkZHhzXRWU1PjvX/GcRxlZ2f3dmsAAMCAmL6d9FULFizQRx99pM2bN0fdPmPGDO/Pubm5GjdunEaOHKl33nlH9913X4/P57qufD6fd/2rf+5p5quWLFmiqqoq73o4HCZkAAAYwHr1SkxFRYXefvttbdy4USNGjLjgbFZWlkaOHKn9+/dLkvx+vzo6OtTW1hY119raqszMTG/m6NGjXZ7r2LFj3kxnKSkpSktLi7oAAICBK6aIcV1XCxYs0BtvvKH33ntPOTk5F33M8ePHdejQIWVlZUmS8vPzlZSUpIaGBm+mpaVFu3fv1vjx4yVJRUVFCoVC+vDDD72ZDz74QKFQyJsBAABXtpi+nTR//ny9+uqreuutt5Samuq9P8VxHA0aNEinTp1SdXW17r//fmVlZenTTz/V448/ruHDh+vee+/1ZmfPnq1FixZp2LBhSk9P1+LFi5WXl+d9WmnMmDGaMmWK5syZo5UrV0qS5s6dq9LSUj6ZBAAAJMUYMStWrJAkTZw4Mer2F198UQ8++KASEhK0a9cuvfzyyzpx4oSysrJ055136rXXXlNqaqo3v3z5ciUmJmr69Olqb29XcXGxVq9erYSEBG9m7dq1WrhwofcpprKyMtXW1vZ2nwAAYIDxua7rxnsRfSEcDstxHIVCId4fAwBXkmon3ivonepQvFfQL8Ty9ZvfnQQAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTYoqYmpoa3XrrrUpNTVVGRobuuece7du3L2rGdV1VV1crEAho0KBBmjhxovbs2RM1E4lEVFFRoeHDh2vIkCEqKyvT4cOHo2ba2tpUXl4ux3HkOI7Ky8t14sSJ3u0SAAAMODFFTGNjo+bPn6+mpiY1NDTo7NmzKikp0enTp72ZZ555RsuWLVNtba22bdsmv9+vu+++WydPnvRmKisrVVdXp3Xr1mnz5s06deqUSktLde7cOW9m5syZam5uVn19verr69Xc3Kzy8vLLsGUAADAQ+FzXdXv74GPHjikjI0ONjY2644475LquAoGAKisr9eijj0r68lWXzMxMPf3005o3b55CoZCuueYarVmzRjNmzJAkHTlyRNnZ2Vq/fr0mT56svXv36sYbb1RTU5MKCgokSU1NTSoqKtLHH3+s0aNHX3Rt4XBYjuMoFAopLS2tt1sEAFhT7cR7Bb1THYr3CvqFWL5+f633xIRCX/4Pnp6eLkk6ePCggsGgSkpKvJmUlBRNmDBBW7ZskSTt2LFDZ86ciZoJBALKzc31ZrZu3SrHcbyAkaTCwkI5juPNdBaJRBQOh6MuAABg4Op1xLiuq6qqKt12223Kzc2VJAWDQUlSZmZm1GxmZqZ3XzAYVHJysoYOHXrBmYyMjC5/Z0ZGhjfTWU1Njff+GcdxlJ2d3dutAQAAA3odMQsWLNBHH32kP/3pT13u8/l8Uddd1+1yW2edZ7qbv9DzLFmyRKFQyLscOnToUrYBAACM6lXEVFRU6O2339bGjRs1YsQI73a/3y9JXV4taW1t9V6d8fv96ujoUFtb2wVnjh492uXvPXbsWJdXec5LSUlRWlpa1AUAAAxcMUWM67pasGCB3njjDb333nvKycmJuj8nJ0d+v18NDQ3ebR0dHWpsbNT48eMlSfn5+UpKSoqaaWlp0e7du72ZoqIihUIhffjhh97MBx98oFAo5M0AAIArW2Isw/Pnz9err76qt956S6mpqd4rLo7jaNCgQfL5fKqsrNTSpUs1atQojRo1SkuXLtXgwYM1c+ZMb3b27NlatGiRhg0bpvT0dC1evFh5eXmaNGmSJGnMmDGaMmWK5syZo5UrV0qS5s6dq9LS0kv6ZBIAABj4YoqYFStWSJImTpwYdfuLL76oBx98UJL0yCOPqL29XQ899JDa2tpUUFCgDRs2KDU11Ztfvny5EhMTNX36dLW3t6u4uFirV69WQkKCN7N27VotXLjQ+xRTWVmZamtre7NHAAAwAH2tnxPTn/FzYgDgCsXPiTHtG/s5MQAAAPFCxAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJMS470AAEA/Vu3EewVAj3glBgAAmETEAAAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAk4gYAABgEhEDAABMijli3n//fU2bNk2BQEA+n09vvvlm1P0PPvigfD5f1KWwsDBqJhKJqKKiQsOHD9eQIUNUVlamw4cPR820tbWpvLxcjuPIcRyVl5frxIkTMW8QAAAMTDFHzOnTpzV27FjV1tb2ODNlyhS1tLR4l/Xr10fdX1lZqbq6Oq1bt06bN2/WqVOnVFpaqnPnznkzM2fOVHNzs+rr61VfX6/m5maVl5fHulwAADBAJcb6gKlTp2rq1KkXnElJSZHf7+/2vlAopFWrVmnNmjWaNGmSJOmVV15Rdna23n33XU2ePFl79+5VfX29mpqaVFBQIEl64YUXVFRUpH379mn06NGxLhsAAAwwffKemE2bNikjI0PXX3+95syZo9bWVu++HTt26MyZMyopKfFuCwQCys3N1ZYtWyRJW7duleM4XsBIUmFhoRzH8WY6i0QiCofDURcAADBwXfaImTp1qtauXav33ntPzz77rLZt26a77rpLkUhEkhQMBpWcnKyhQ4dGPS4zM1PBYNCbycjI6PLcGRkZ3kxnNTU13vtnHMdRdnb2Zd4ZAADoT2L+dtLFzJgxw/tzbm6uxo0bp5EjR+qdd97Rfffd1+PjXNeVz+fzrn/1zz3NfNWSJUtUVVXlXQ+Hw4QMAAADWJ9/xDorK0sjR47U/v37JUl+v18dHR1qa2uLmmttbVVmZqY3c/To0S7PdezYMW+ms5SUFKWlpUVdAADAwNXnEXP8+HEdOnRIWVlZkqT8/HwlJSWpoaHBm2lpadHu3bs1fvx4SVJRUZFCoZA+/PBDb+aDDz5QKBTyZgAAwJUt5m8nnTp1Sp988ol3/eDBg2publZ6errS09NVXV2t+++/X1lZWfr000/1+OOPa/jw4br33nslSY7jaPbs2Vq0aJGGDRum9PR0LV68WHl5ed6nlcaMGaMpU6Zozpw5WrlypSRp7ty5Ki0t5ZNJAABAUi8iZvv27brzzju96+ffhzJr1iytWLFCu3bt0ssvv6wTJ04oKytLd955p1577TWlpqZ6j1m+fLkSExM1ffp0tbe3q7i4WKtXr1ZCQoI3s3btWi1cuND7FFNZWdkFfzYNAAC4svhc13XjvYi+EA6H5TiOQqEQ748BgN6qduK9gitHdSjeK+gXYvn6ze9OAgAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgEkxR8z777+vadOmKRAIyOfz6c0334y633VdVVdXKxAIaNCgQZo4caL27NkTNROJRFRRUaHhw4dryJAhKisr0+HDh6Nm2traVF5eLsdx5DiOysvLdeLEiZg3CAAABqaYI+b06dMaO3asamtru73/mWee0bJly1RbW6tt27bJ7/fr7rvv1smTJ72ZyspK1dXVad26ddq8ebNOnTql0tJSnTt3zpuZOXOmmpubVV9fr/r6ejU3N6u8vLwXWwQAAAORz3Vdt9cP9vlUV1ene+65R9KXr8IEAgFVVlbq0UcflfTlqy6ZmZl6+umnNW/ePIVCIV1zzTVas2aNZsyYIUk6cuSIsrOztX79ek2ePFl79+7VjTfeqKamJhUUFEiSmpqaVFRUpI8//lijR4++6NrC4bAcx1EoFFJaWlpvtwgAV7ZqJ94ruHJUh+K9gn4hlq/fl/U9MQcPHlQwGFRJSYl3W0pKiiZMmKAtW7ZIknbs2KEzZ85EzQQCAeXm5nozW7duleM4XsBIUmFhoRzH8WY6i0QiCofDURcAADBwXdaICQaDkqTMzMyo2zMzM737gsGgkpOTNXTo0AvOZGRkdHn+jIwMb6azmpoa7/0zjuMoOzv7a+8HAAD0X33y6SSfzxd13XXdLrd11nmmu/kLPc+SJUsUCoW8y6FDh3qxcgAAYMVljRi/3y9JXV4taW1t9V6d8fv96ujoUFtb2wVnjh492uX5jx071uVVnvNSUlKUlpYWdQEAAAPXZY2YnJwc+f1+NTQ0eLd1dHSosbFR48ePlyTl5+crKSkpaqalpUW7d+/2ZoqKihQKhfThhx96Mx988IFCoZA3AwAArmyJsT7g1KlT+uSTT7zrBw8eVHNzs9LT03XttdeqsrJSS5cu1ahRozRq1CgtXbpUgwcP1syZMyVJjuNo9uzZWrRokYYNG6b09HQtXrxYeXl5mjRpkiRpzJgxmjJliubMmaOVK1dKkubOnavS0tJL+mQSAAAY+GKOmO3bt+vOO+/0rldVVUmSZs2apdWrV+uRRx5Re3u7HnroIbW1tamgoEAbNmxQamqq95jly5crMTFR06dPV3t7u4qLi7V69WolJCR4M2vXrtXChQu9TzGVlZX1+LNpAADAledr/ZyY/oyfEwMAlwE/J+abw8+JkRTHnxMDAADwTSFiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYFJivBcAAFeMaifeKwAGFF6JAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEy67BFTXV0tn88XdfH7/d79ruuqurpagUBAgwYN0sSJE7Vnz56o54hEIqqoqNDw4cM1ZMgQlZWV6fDhw5d7qQAAwLA+eSXmpptuUktLi3fZtWuXd98zzzyjZcuWqba2Vtu2bZPf79fdd9+tkydPejOVlZWqq6vTunXrtHnzZp06dUqlpaU6d+5cXywXAAAYlNgnT5qYGPXqy3mu6+q5557TE088ofvuu0+S9NJLLykzM1Ovvvqq5s2bp1AopFWrVmnNmjWaNGmSJOmVV15Rdna23n33XU2ePLkvlgwAAIzpk1di9u/fr0AgoJycHD3wwAM6cOCAJOngwYMKBoMqKSnxZlNSUjRhwgRt2bJFkrRjxw6dOXMmaiYQCCg3N9eb6U4kElE4HI66AACAgeuyR0xBQYFefvll/e1vf9MLL7ygYDCo8ePH6/jx4woGg5KkzMzMqMdkZmZ69wWDQSUnJ2vo0KE9znSnpqZGjuN4l+zs7Mu8MwAA0J9c9m8nTZ061ftzXl6eioqKdN111+mll15SYWGhJMnn80U9xnXdLrd1drGZJUuWqKqqyrseDocJGQCAHdVOvFcQu+pQXP/6Pv+I9ZAhQ5SXl6f9+/d775Pp/IpKa2ur9+qM3+9XR0eH2traepzpTkpKitLS0qIuAABg4OrziIlEItq7d6+ysrKUk5Mjv9+vhoYG7/6Ojg41NjZq/PjxkqT8/HwlJSVFzbS0tGj37t3eDAAAwGX/dtLixYs1bdo0XXvttWptbdVTTz2lcDisWbNmyefzqbKyUkuXLtWoUaM0atQoLV26VIMHD9bMmTMlSY7jaPbs2Vq0aJGGDRum9PR0LV68WHl5ed6nlQAAAC57xBw+fFg/+tGP9Pnnn+uaa65RYWGhmpqaNHLkSEnSI488ovb2dj300ENqa2tTQUGBNmzYoNTUVO85li9frsTERE2fPl3t7e0qLi7W6tWrlZCQcLmXCwAAjPK5ruvGexF9IRwOy3EchUIh3h8DoH+w+MZN4EL64I29sXz95ncnAQAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAky77rx0AgG8EP/0WuOLxSgwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAk4gYAABgEhEDAABMImIAAIBJRAwAADCJiAEAACYRMQAAwCQiBgAAmETEAAAAk4gYAABgEhEDAABMSoz3AoABp9qJ9wpiVx2K9woAIGa8EgMAAEwiYgAAgElEDAAAMImIAQAAJhExAADAJCIGAACYRMQAAACTiBgAAGASEQMAAEwiYgAAgElEDAAAMInfnYT+zeLvIQIAfCN4JQYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiY9YX0n4uDIAYAAhYnqLIAAAIK74dhIAADCp30fM73//e+Xk5Ojqq69Wfn6+/v73v8d7SQAAoB/o1xHz2muvqbKyUk888YR27typ22+/XVOnTtVnn30W76UBAIA469cRs2zZMs2ePVs//elPNWbMGD333HPKzs7WihUr4r00AAAQZ/32jb0dHR3asWOHHnvssajbS0pKtGXLli7zkUhEkUjEux4KhSRJ4XC4bxYYcfvmeYF46Kt/J32Jf4NA/PXBfzvOf9123Yv/G++3EfP555/r3LlzyszMjLo9MzNTwWCwy3xNTY1++ctfdrk9Ozu7z9YIDBi/5tN2AHqhD//bcfLkSTnOhZ+/30bMeT6fL+q667pdbpOkJUuWqKqqyrv+xRdf6D//+Y+GDRvW7fzXEQ6HlZ2drUOHDiktLe2yPnd/wP7sG+h7HOj7kwb+HtmffX21R9d1dfLkSQUCgYvO9tuIGT58uBISErq86tLa2trl1RlJSklJUUpKStRt3/72t/tyiUpLSxuw/+eU2N9AMND3OND3Jw38PbI/+/pijxd7Bea8fvvG3uTkZOXn56uhoSHq9oaGBo0fPz5OqwIAAP1Fv30lRpKqqqpUXl6ucePGqaioSM8//7w+++wz/exnP4v30gAAQJz164iZMWOGjh8/rl/96ldqaWlRbm6u1q9fr5EjR8Z1XSkpKXryySe7fPtqoGB/9g30PQ70/UkDf4/sz77+sEefeymfYQIAAOhn+u17YgAAAC6EiAEAACYRMQAAwCQiBgAAmETEdPL+++9r2rRpCgQC8vl8evPNNy/6mMbGRuXn5+vqq6/Wd7/7Xf3hD3/o+4V+DbHucdOmTfL5fF0uH3/88Tez4BjU1NTo1ltvVWpqqjIyMnTPPfdo3759F32cpTPszR4tneGKFSt08803ez9Aq6ioSH/9618v+BhL5yfFvkdL59edmpoa+Xw+VVZWXnDO2jmedyn7s3aG1dXVXdbq9/sv+Jh4nB8R08np06c1duxY1dbWXtL8wYMH9f3vf1+33367du7cqccff1wLFy7U66+/3scr7b1Y93jevn371NLS4l1GjRrVRyvsvcbGRs2fP19NTU1qaGjQ2bNnVVJSotOnT/f4GGtn2Js9nmfhDEeMGKFf//rX2r59u7Zv36677rpLP/zhD7Vnz55u562dnxT7Hs+zcH6dbdu2Tc8//7xuvvnmC85ZPEfp0vd3nqUzvOmmm6LWumvXrh5n43Z+Lnokya2rq7vgzCOPPOLecMMNUbfNmzfPLSws7MOVXT6XsseNGze6kty2trZvZE2XU2trqyvJbWxs7HHG+hleyh4tn6Hruu7QoUPdP/7xj93eZ/38zrvQHq2e38mTJ91Ro0a5DQ0N7oQJE9yHH364x1mL5xjL/qyd4ZNPPumOHTv2kufjdX68EvM1bd26VSUlJVG3TZ48Wdu3b9eZM2fitKq+ccsttygrK0vFxcXauHFjvJdzSUKhkCQpPT29xxnrZ3gpezzP2hmeO3dO69at0+nTp1VUVNTtjPXzu5Q9nmft/ObPn68f/OAHmjRp0kVnLZ5jLPs7z9IZ7t+/X4FAQDk5OXrggQd04MCBHmfjdX79+if2WhAMBrv8QsrMzEydPXtWn3/+ubKysuK0sssnKytLzz//vPLz8xWJRLRmzRoVFxdr06ZNuuOOO+K9vB65rquqqirddtttys3N7XHO8hle6h6tneGuXbtUVFSk//3vf/rWt76luro63Xjjjd3OWj2/WPZo7fwkad26dfrHP/6hbdu2XdK8tXOMdX/WzrCgoEAvv/yyrr/+eh09elRPPfWUxo8frz179mjYsGFd5uN1fkTMZeDz+aKuu///Q5A7327V6NGjNXr0aO96UVGRDh06pN/85jf98h/feQsWLNBHH32kzZs3X3TW6hle6h6tneHo0aPV3NysEydO6PXXX9esWbPU2NjY4xd5i+cXyx6tnd+hQ4f08MMPa8OGDbr66qsv+XFWzrE3+7N2hlOnTvX+nJeXp6KiIl133XV66aWXVFVV1e1j4nF+fDvpa/L7/QoGg1G3tba2KjExsdtaHSgKCwu1f//+eC+jRxUVFXr77be1ceNGjRgx4oKzVs8wlj12pz+fYXJysr73ve9p3Lhxqqmp0dixY/Xb3/6221mr5xfLHrvTn89vx44dam1tVX5+vhITE5WYmKjGxkb97ne/U2Jios6dO9flMZbOsTf7605/PsPOhgwZory8vB7XG6/z45WYr6moqEh/+ctfom7bsGGDxo0bp6SkpDitqu/t3Lmz3728K31Z/hUVFaqrq9OmTZuUk5Nz0cdYO8Pe7LE7/fUMu+O6riKRSLf3WTu/nlxoj93pz+dXXFzc5ZMsP/7xj3XDDTfo0UcfVUJCQpfHWDrH3uyvO/35DDuLRCLau3evbr/99m7vj9v59enbhg06efKku3PnTnfnzp2uJHfZsmXuzp073X//+9+u67ruY4895paXl3vzBw4ccAcPHuz+4he/cP/5z3+6q1atcpOSktw///nP8drCRcW6x+XLl7t1dXXuv/71L3f37t3uY4895kpyX3/99XhtoUc///nPXcdx3E2bNrktLS3e5b///a83Y/0Me7NHS2e4ZMkS9/3333cPHjzofvTRR+7jjz/uXnXVVe6GDRtc17V/fq4b+x4tnV9POn96ZyCc41ddbH/WznDRokXupk2b3AMHDrhNTU1uaWmpm5qa6n766aeu6/af8yNiOjn/MbjOl1mzZrmu67qzZs1yJ0yYEPWYTZs2ubfccoubnJzsfuc733FXrFjxzS88BrHu8emnn3avu+469+qrr3aHDh3q3nbbbe4777wTn8VfRHf7kuS++OKL3oz1M+zNHi2d4U9+8hN35MiRbnJysnvNNde4xcXF3hd317V/fq4b+x4tnV9POn+RHwjn+FUX25+1M5wxY4ablZXlJiUluYFAwL3vvvvcPXv2ePf3l/Pzue7/v/MGAADAEN7YCwAATCJiAACASUQMAAAwiYgBAAAmETEAAMAkIgYAAJhExAAAAJOIGAAAYBIRAwAATCJiAACASUQMAAAwiYgBAAAm/R9Yz5qFIqKCUgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a histogram of the apple reviews distribution\n",
    "plt.hist(apple);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8dE1uC5iFcv2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGdCAYAAADjWSL8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtEklEQVR4nO3df2xVdZ7/8dcdSitge5a29t7eUJEZawdsIaaY9nYdQVoKrLUqJjDbyQ3usKAjFrtAEJg/ZDazFN0VdNIMiwwRBdyarFPHXfAONUpdAoXSsREQWSaCW2IvRbe9t2W6t9g53z/my4m3LT9uobaf+nwkJ+k9531OP28/kvvK555z67Jt2xYAAIBhvjfUAwAAABgIQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEhxQz2AwfLnP/9ZX3zxhRITE+VyuYZ6OAAA4DrYtq2Ojg55vV5973tXX2sZsSHmiy++UEZGxlAPAwAADEBzc7MmTJhw1ZoRG2ISExMl/eU/QlJS0hCPBgAAXI9wOKyMjAznffxqRmyIufwRUlJSEiEGAADDXM+tINzYCwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGCkuKEeAAAAN9Mda/YM9RAG5OzGB4d6CMZhJQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjHRDIaayslIul0sVFRXOPtu2tX79enm9Xo0ZM0YzZ87UiRMnos6LRCIqLy9Xamqqxo0bp9LSUp07dy6qpq2tTX6/X5ZlybIs+f1+tbe338hwAQDACDLgENPQ0KBXXnlFU6dOjdr/wgsvaNOmTaqqqlJDQ4M8Ho9mz56tjo4Op6aiokI1NTWqrq7WgQMH1NnZqZKSEvX09Dg1ZWVlampqUiAQUCAQUFNTk/x+/0CHCwAARpgBhZjOzk795Cc/0bZt2zR+/Hhnv23beumll/Tzn/9c8+fPV3Z2tl577TX96U9/0htvvCFJCoVC2r59u1588UUVFRXpnnvu0a5du3Ts2DG99957kqSTJ08qEAjoN7/5jXw+n3w+n7Zt26b//M//1KlTp25C2wAAwHQDCjHLli3Tgw8+qKKioqj9Z86cUTAYVHFxsbMvISFBM2bM0MGDByVJjY2NunTpUlSN1+tVdna2U3Po0CFZlqW8vDynJj8/X5ZlOTW9RSIRhcPhqA0AAIxccbGeUF1drT/84Q9qaGjocywYDEqS3G531H63263PP//cqYmPj49awblcc/n8YDCotLS0PtdPS0tzanqrrKzUL37xi1jbAQAAhoppJaa5uVnPPPOMdu3apVtuueWKdS6XK+q1bdt99vXWu6a/+qtdZ+3atQqFQs7W3Nx81d8HAADMFlOIaWxsVGtrq3JzcxUXF6e4uDjV1dXpV7/6leLi4pwVmN6rJa2trc4xj8ej7u5utbW1XbXm/PnzfX7/hQsX+qzyXJaQkKCkpKSoDQAAjFwxhZjCwkIdO3ZMTU1NzjZ9+nT95Cc/UVNTk77//e/L4/GotrbWOae7u1t1dXUqKCiQJOXm5mr06NFRNS0tLTp+/LhT4/P5FAqFdOTIEafm8OHDCoVCTg0AAPhui+memMTERGVnZ0ftGzdunFJSUpz9FRUV2rBhgzIzM5WZmakNGzZo7NixKisrkyRZlqXFixdr5cqVSklJUXJyslatWqWcnBznRuHJkydr7ty5WrJkibZu3SpJWrp0qUpKSpSVlXXDTQMAAPPFfGPvtaxevVpdXV166qmn1NbWpry8PO3bt0+JiYlOzebNmxUXF6cFCxaoq6tLhYWF2rFjh0aNGuXU7N69W8uXL3eeYiotLVVVVdXNHi4AADCUy7Zte6gHMRjC4bAsy1IoFOL+GAD4DrljzZ6hHsKAnN344FAPYViI5f2bv50EAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADBSTCFmy5Ytmjp1qpKSkpSUlCSfz6d3333XOf7444/L5XJFbfn5+VHXiEQiKi8vV2pqqsaNG6fS0lKdO3cuqqatrU1+v1+WZcmyLPn9frW3tw+8SwAAMOLEFGImTJigjRs36ujRozp69KhmzZqlhx9+WCdOnHBq5s6dq5aWFmfbu3dv1DUqKipUU1Oj6upqHThwQJ2dnSopKVFPT49TU1ZWpqamJgUCAQUCATU1Ncnv999gqwAAYCSJi6X4oYceinr9T//0T9qyZYvq6+t19913S5ISEhLk8Xj6PT8UCmn79u3auXOnioqKJEm7du1SRkaG3nvvPc2ZM0cnT55UIBBQfX298vLyJEnbtm2Tz+fTqVOnlJWVFXOTAABg5BnwPTE9PT2qrq7WxYsX5fP5nP379+9XWlqa7rrrLi1ZskStra3OscbGRl26dEnFxcXOPq/Xq+zsbB08eFCSdOjQIVmW5QQYScrPz5dlWU5NfyKRiMLhcNQGAABGrphDzLFjx3TrrbcqISFBTz75pGpqajRlyhRJ0rx587R79269//77evHFF9XQ0KBZs2YpEolIkoLBoOLj4zV+/Pioa7rdbgWDQacmLS2tz+9NS0tzavpTWVnp3ENjWZYyMjJibQ0AABgkpo+TJCkrK0tNTU1qb2/XW2+9pUWLFqmurk5TpkzRwoULnbrs7GxNnz5dEydO1J49ezR//vwrXtO2bblcLuf1N3++Uk1va9eu1YoVK5zX4XCYIAMAwAgWc4iJj4/XnXfeKUmaPn26Ghoa9PLLL2vr1q19atPT0zVx4kSdPn1akuTxeNTd3a22trao1ZjW1lYVFBQ4NefPn+9zrQsXLsjtdl9xXAkJCUpISIi1HQAAYKgb/p4Y27adj4t6++qrr9Tc3Kz09HRJUm5urkaPHq3a2lqnpqWlRcePH3dCjM/nUygU0pEjR5yaw4cPKxQKOTUAAAAxrcSsW7dO8+bNU0ZGhjo6OlRdXa39+/crEAios7NT69ev12OPPab09HSdPXtW69atU2pqqh599FFJkmVZWrx4sVauXKmUlBQlJydr1apVysnJcZ5Wmjx5subOnaslS5Y4qztLly5VSUkJTyYBAABHTCHm/Pnz8vv9amlpkWVZmjp1qgKBgGbPnq2uri4dO3ZMr7/+utrb25Wenq4HHnhAb775phITE51rbN68WXFxcVqwYIG6urpUWFioHTt2aNSoUU7N7t27tXz5cucpptLSUlVVVd2klgEAwEjgsm3bHupBDIZwOCzLshQKhZSUlDTUwwEAfEvuWLNnqIcwIGc3PjjUQxgWYnn/5m8nAQAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMFFOI2bJli6ZOnaqkpCQlJSXJ5/Pp3XffdY7btq3169fL6/VqzJgxmjlzpk6cOBF1jUgkovLycqWmpmrcuHEqLS3VuXPnomra2trk9/tlWZYsy5Lf71d7e/vAuwQAACNOTCFmwoQJ2rhxo44ePaqjR49q1qxZevjhh52g8sILL2jTpk2qqqpSQ0ODPB6PZs+erY6ODucaFRUVqqmpUXV1tQ4cOKDOzk6VlJSop6fHqSkrK1NTU5MCgYACgYCamprk9/tvUssAAGAkcNm2bd/IBZKTk/XP//zP+ulPfyqv16uKigo9++yzkv6y6uJ2u/X888/riSeeUCgU0m233aadO3dq4cKFkqQvvvhCGRkZ2rt3r+bMmaOTJ09qypQpqq+vV15eniSpvr5ePp9Pn376qbKysq5rXOFwWJZlKRQKKSkp6UZaBAAY5I41e4Z6CANyduODQz2EYSGW9+8B3xPT09Oj6upqXbx4UT6fT2fOnFEwGFRxcbFTk5CQoBkzZujgwYOSpMbGRl26dCmqxuv1Kjs726k5dOiQLMtyAowk5efny7Isp6Y/kUhE4XA4agMAACNXzCHm2LFjuvXWW5WQkKAnn3xSNTU1mjJlioLBoCTJ7XZH1bvdbudYMBhUfHy8xo8ff9WatLS0Pr83LS3NqelPZWWlcw+NZVnKyMiItTUAAGCQmENMVlaWmpqaVF9fr5/97GdatGiRPvnkE+e4y+WKqrdtu8++3nrX9Fd/reusXbtWoVDI2Zqbm6+3JQAAYKCYQ0x8fLzuvPNOTZ8+XZWVlZo2bZpefvlleTweSeqzWtLa2uqszng8HnV3d6utre2qNefPn+/zey9cuNBnleebEhISnKemLm8AAGDkuuHvibFtW5FIRJMmTZLH41Ftba1zrLu7W3V1dSooKJAk5ebmavTo0VE1LS0tOn78uFPj8/kUCoV05MgRp+bw4cMKhUJODQAAQFwsxevWrdO8efOUkZGhjo4OVVdXa//+/QoEAnK5XKqoqNCGDRuUmZmpzMxMbdiwQWPHjlVZWZkkybIsLV68WCtXrlRKSoqSk5O1atUq5eTkqKioSJI0efJkzZ07V0uWLNHWrVslSUuXLlVJScl1P5kEAABGvphCzPnz5+X3+9XS0iLLsjR16lQFAgHNnj1bkrR69Wp1dXXpqaeeUltbm/Ly8rRv3z4lJiY619i8ebPi4uK0YMECdXV1qbCwUDt27NCoUaOcmt27d2v58uXOU0ylpaWqqqq6Gf0CAIAR4oa/J2a44ntiAOC7ie+JMdu38j0xAAAAQ4kQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgpJhCTGVlpe69914lJiYqLS1NjzzyiE6dOhVV8/jjj8vlckVt+fn5UTWRSETl5eVKTU3VuHHjVFpaqnPnzkXVtLW1ye/3y7IsWZYlv9+v9vb2gXUJAABGnJhCTF1dnZYtW6b6+nrV1tbq66+/VnFxsS5evBhVN3fuXLW0tDjb3r17o45XVFSopqZG1dXVOnDggDo7O1VSUqKenh6npqysTE1NTQoEAgoEAmpqapLf77+BVgEAwEgSF0txIBCIev3qq68qLS1NjY2Nuv/++539CQkJ8ng8/V4jFApp+/bt2rlzp4qKiiRJu3btUkZGht577z3NmTNHJ0+eVCAQUH19vfLy8iRJ27Ztk8/n06lTp5SVlRVTkwAAYOS5oXtiQqGQJCk5OTlq//79+5WWlqa77rpLS5YsUWtrq3OssbFRly5dUnFxsbPP6/UqOztbBw8elCQdOnRIlmU5AUaS8vPzZVmWU9NbJBJROByO2gAAwMg14BBj27ZWrFih++67T9nZ2c7+efPmaffu3Xr//ff14osvqqGhQbNmzVIkEpEkBYNBxcfHa/z48VHXc7vdCgaDTk1aWlqf35mWlubU9FZZWencP2NZljIyMgbaGgAAMEBMHyd909NPP62PP/5YBw4ciNq/cOFC5+fs7GxNnz5dEydO1J49ezR//vwrXs+2bblcLuf1N3++Us03rV27VitWrHBeh8NhggwAACPYgFZiysvL9c477+iDDz7QhAkTrlqbnp6uiRMn6vTp05Ikj8ej7u5utbW1RdW1trbK7XY7NefPn+9zrQsXLjg1vSUkJCgpKSlqAwAAI1dMIca2bT399NP67W9/q/fff1+TJk265jlfffWVmpublZ6eLknKzc3V6NGjVVtb69S0tLTo+PHjKigokCT5fD6FQiEdOXLEqTl8+LBCoZBTAwAAvtti+jhp2bJleuONN/S73/1OiYmJzv0plmVpzJgx6uzs1Pr16/XYY48pPT1dZ8+e1bp165SamqpHH33UqV28eLFWrlyplJQUJScna9WqVcrJyXGeVpo8ebLmzp2rJUuWaOvWrZKkpUuXqqSkhCeTAACApBhDzJYtWyRJM2fOjNr/6quv6vHHH9eoUaN07Ngxvf7662pvb1d6eroeeOABvfnmm0pMTHTqN2/erLi4OC1YsEBdXV0qLCzUjh07NGrUKKdm9+7dWr58ufMUU2lpqaqqqgbaJwAAGGFctm3bQz2IwRAOh2VZlkKhEPfHAMB3yB1r9gz1EAbk7MYHh3oIw0Is79/87SQAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJFiCjGVlZW69957lZiYqLS0ND3yyCM6depUVI1t21q/fr28Xq/GjBmjmTNn6sSJE1E1kUhE5eXlSk1N1bhx41RaWqpz585F1bS1tcnv98uyLFmWJb/fr/b29oF1CQAARpyYQkxdXZ2WLVum+vp61dbW6uuvv1ZxcbEuXrzo1LzwwgvatGmTqqqq1NDQII/Ho9mzZ6ujo8OpqaioUE1Njaqrq3XgwAF1dnaqpKREPT09Tk1ZWZmampoUCAQUCATU1NQkv99/E1oGAAAjgcu2bXugJ1+4cEFpaWmqq6vT/fffL9u25fV6VVFRoWeffVbSX1Zd3G63nn/+eT3xxBMKhUK67bbbtHPnTi1cuFCS9MUXXygjI0N79+7VnDlzdPLkSU2ZMkX19fXKy8uTJNXX18vn8+nTTz9VVlbWNccWDodlWZZCoZCSkpIG2iIAwDB3rNkz1EMYkLMbHxzqIQwLsbx/39A9MaFQSJKUnJwsSTpz5oyCwaCKi4udmoSEBM2YMUMHDx6UJDU2NurSpUtRNV6vV9nZ2U7NoUOHZFmWE2AkKT8/X5ZlOTUAAOC7LW6gJ9q2rRUrVui+++5Tdna2JCkYDEqS3G53VK3b7dbnn3/u1MTHx2v8+PF9ai6fHwwGlZaW1ud3pqWlOTW9RSIRRSIR53U4HB5gZwAAwAQDXol5+umn9fHHH+vf/u3f+hxzuVxRr23b7rOvt941/dVf7TqVlZXOTcCWZSkjI+N62gAAAIYaUIgpLy/XO++8ow8++EATJkxw9ns8Hknqs1rS2trqrM54PB51d3erra3tqjXnz5/v83svXLjQZ5XnsrVr1yoUCjlbc3PzQFoDAACGiCnE2Latp59+Wr/97W/1/vvva9KkSVHHJ02aJI/Ho9raWmdfd3e36urqVFBQIEnKzc3V6NGjo2paWlp0/Phxp8bn8ykUCunIkSNOzeHDhxUKhZya3hISEpSUlBS1AQCAkSume2KWLVumN954Q7/73e+UmJjorLhYlqUxY8bI5XKpoqJCGzZsUGZmpjIzM7VhwwaNHTtWZWVlTu3ixYu1cuVKpaSkKDk5WatWrVJOTo6KiookSZMnT9bcuXO1ZMkSbd26VZK0dOlSlZSUXNeTSQAAYOSLKcRs2bJFkjRz5syo/a+++qoef/xxSdLq1avV1dWlp556Sm1tbcrLy9O+ffuUmJjo1G/evFlxcXFasGCBurq6VFhYqB07dmjUqFFOze7du7V8+XLnKabS0lJVVVUNpEcAADAC3dD3xAxnfE8MAHw38T0xZvvWvicGAABgqAz4e2IAAMDNY+IK0lCvHrESAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJHihnoAAIDh6441e4Z6CMAVsRIDAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIwUc4j58MMP9dBDD8nr9crlcuntt9+OOv7444/L5XJFbfn5+VE1kUhE5eXlSk1N1bhx41RaWqpz585F1bS1tcnv98uyLFmWJb/fr/b29pgbBAAAI1PMIebixYuaNm2aqqqqrlgzd+5ctbS0ONvevXujjldUVKimpkbV1dU6cOCAOjs7VVJSop6eHqemrKxMTU1NCgQCCgQCampqkt/vj3W4AABghIr5byfNmzdP8+bNu2pNQkKCPB5Pv8dCoZC2b9+unTt3qqioSJK0a9cuZWRk6L333tOcOXN08uRJBQIB1dfXKy8vT5K0bds2+Xw+nTp1SllZWbEOGwAAjDCDck/M/v37lZaWprvuuktLlixRa2urc6yxsVGXLl1ScXGxs8/r9So7O1sHDx6UJB06dEiWZTkBRpLy8/NlWZZT01skElE4HI7aAADAyHXTQ8y8efO0e/duvf/++3rxxRfV0NCgWbNmKRKJSJKCwaDi4+M1fvz4qPPcbreCwaBTk5aW1ufaaWlpTk1vlZWVzv0zlmUpIyPjJncGAACGk5g/TrqWhQsXOj9nZ2dr+vTpmjhxovbs2aP58+df8TzbtuVyuZzX3/z5SjXftHbtWq1YscJ5HQ6HCTIAAIxgg/6IdXp6uiZOnKjTp09Lkjwej7q7u9XW1hZV19raKrfb7dScP3++z7UuXLjg1PSWkJCgpKSkqA0AAIxcgx5ivvrqKzU3Nys9PV2SlJubq9GjR6u2ttapaWlp0fHjx1VQUCBJ8vl8CoVCOnLkiFNz+PBhhUIhpwYAAHy3xfxxUmdnp/74xz86r8+cOaOmpiYlJycrOTlZ69ev12OPPab09HSdPXtW69atU2pqqh599FFJkmVZWrx4sVauXKmUlBQlJydr1apVysnJcZ5Wmjx5subOnaslS5Zo69atkqSlS5eqpKSEJ5MAAICkAYSYo0eP6oEHHnBeX74PZdGiRdqyZYuOHTum119/Xe3t7UpPT9cDDzygN998U4mJic45mzdvVlxcnBYsWKCuri4VFhZqx44dGjVqlFOze/duLV++3HmKqbS09KrfTQMAAL5bXLZt20M9iMEQDodlWZZCoRD3xwDAAN2xZs9QDwHD2NmND970a8by/s3fTgIAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGCnmEPPhhx/qoYcektfrlcvl0ttvvx113LZtrV+/Xl6vV2PGjNHMmTN14sSJqJpIJKLy8nKlpqZq3LhxKi0t1blz56Jq2tra5Pf7ZVmWLMuS3+9Xe3t7zA0CAICRKeYQc/HiRU2bNk1VVVX9Hn/hhRe0adMmVVVVqaGhQR6PR7Nnz1ZHR4dTU1FRoZqaGlVXV+vAgQPq7OxUSUmJenp6nJqysjI1NTUpEAgoEAioqalJfr9/AC0CAICRyGXbtj3gk10u1dTU6JFHHpH0l1UYr9eriooKPfvss5L+suridrv1/PPP64knnlAoFNJtt92mnTt3auHChZKkL774QhkZGdq7d6/mzJmjkydPasqUKaqvr1deXp4kqb6+Xj6fT59++qmysrKuObZwOCzLshQKhZSUlDTQFgHgO+2ONXuGeggYxs5ufPCmXzOW9++bek/MmTNnFAwGVVxc7OxLSEjQjBkzdPDgQUlSY2OjLl26FFXj9XqVnZ3t1Bw6dEiWZTkBRpLy8/NlWZZT01skElE4HI7aAADAyHVTQ0wwGJQkud3uqP1ut9s5FgwGFR8fr/Hjx1+1Ji0trc/109LSnJreKisrnftnLMtSRkbGDfcDAACGr0F5OsnlckW9tm27z77eetf0V3+166xdu1ahUMjZmpubBzByAABgipsaYjwejyT1WS1pbW11Vmc8Ho+6u7vV1tZ21Zrz58/3uf6FCxf6rPJclpCQoKSkpKgNAACMXDc1xEyaNEkej0e1tbXOvu7ubtXV1amgoECSlJubq9GjR0fVtLS06Pjx406Nz+dTKBTSkSNHnJrDhw8rFAo5NQAA4LstLtYTOjs79cc//tF5febMGTU1NSk5OVm33367KioqtGHDBmVmZiozM1MbNmzQ2LFjVVZWJkmyLEuLFy/WypUrlZKSouTkZK1atUo5OTkqKiqSJE2ePFlz587VkiVLtHXrVknS0qVLVVJScl1PJgEAgJEv5hBz9OhRPfDAA87rFStWSJIWLVqkHTt2aPXq1erq6tJTTz2ltrY25eXlad++fUpMTHTO2bx5s+Li4rRgwQJ1dXWpsLBQO3bs0KhRo5ya3bt3a/ny5c5TTKWlpVf8bhoAAPDdc0PfEzOc8T0xAHDj+J4YXM2I+p4YAACAbwshBgAAGIkQAwAAjBTzjb0AgIHh/hLg5mIlBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARrrpIWb9+vVyuVxRm8fjcY7btq3169fL6/VqzJgxmjlzpk6cOBF1jUgkovLycqWmpmrcuHEqLS3VuXPnbvZQAQCAwQZlJebuu+9WS0uLsx07dsw59sILL2jTpk2qqqpSQ0ODPB6PZs+erY6ODqemoqJCNTU1qq6u1oEDB9TZ2amSkhL19PQMxnABAICB4gblonFxUasvl9m2rZdeekk///nPNX/+fEnSa6+9JrfbrTfeeENPPPGEQqGQtm/frp07d6qoqEiStGvXLmVkZOi9997TnDlzBmPIAADAMIOyEnP69Gl5vV5NmjRJP/7xj/XZZ59Jks6cOaNgMKji4mKnNiEhQTNmzNDBgwclSY2Njbp06VJUjdfrVXZ2tlPTn0gkonA4HLUBAICR66aHmLy8PL3++uv6/e9/r23btikYDKqgoEBfffWVgsGgJMntdked43a7nWPBYFDx8fEaP378FWv6U1lZKcuynC0jI+MmdwYAAIaTmx5i5s2bp8cee0w5OTkqKirSnj17JP3lY6PLXC5X1Dm2bffZ19u1atauXatQKORszc3NN9AFAAAY7gb9Eetx48YpJydHp0+fdu6T6b2i0tra6qzOeDwedXd3q62t7Yo1/UlISFBSUlLUBgAARq5BDzGRSEQnT55Uenq6Jk2aJI/Ho9raWud4d3e36urqVFBQIEnKzc3V6NGjo2paWlp0/PhxpwYAAOCmP520atUqPfTQQ7r99tvV2tqqX/7ylwqHw1q0aJFcLpcqKiq0YcMGZWZmKjMzUxs2bNDYsWNVVlYmSbIsS4sXL9bKlSuVkpKi5ORkrVq1yvl4CgAAQBqEEHPu3Dn97d/+rb788kvddtttys/PV319vSZOnChJWr16tbq6uvTUU0+pra1NeXl52rdvnxITE51rbN68WXFxcVqwYIG6urpUWFioHTt2aNSoUTd7uAAMdceaPUM9BABDzGXbtj3UgxgM4XBYlmUpFApxfwwwAhFigKF3duODN/2asbx/87eTAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAI8UN9QAADL071uwZ6iEAQMxYiQEAAEYixAAAACMRYgAAgJEIMQAAwEjc2AvcZNwkCwDfDlZiAACAkYZ9iPn1r3+tSZMm6ZZbblFubq7+67/+a6iHBAAAhoFh/XHSm2++qYqKCv3617/WX//1X2vr1q2aN2+ePvnkE91+++1DPTx8C/hoBgBwJcN6JWbTpk1avHix/v7v/16TJ0/WSy+9pIyMDG3ZsmWohwYAAIbYsF2J6e7uVmNjo9asWRO1v7i4WAcPHuxTH4lEFIlEnNehUEiSFA6HB2V82c/9flCuCwCAKQbjPfbyNW3bvmbtsA0xX375pXp6euR2u6P2u91uBYPBPvWVlZX6xS9+0Wd/RkbGoI0RAIDvMuulwbt2R0eHLMu6as2wDTGXuVyuqNe2bffZJ0lr167VihUrnNd//vOf9b//+79KSUnpt/5GhMNhZWRkqLm5WUlJSTf12sMB/ZlvpPc40vuTRn6P9Ge+werRtm11dHTI6/Ves3bYhpjU1FSNGjWqz6pLa2trn9UZSUpISFBCQkLUvr/6q78azCEqKSlpxP7PKdHfSDDSexzp/Ukjv0f6M99g9HitFZjLhu2NvfHx8crNzVVtbW3U/traWhUUFAzRqAAAwHAxbFdiJGnFihXy+/2aPn26fD6fXnnlFf3P//yPnnzyyaEeGgAAGGLDOsQsXLhQX331lf7xH/9RLS0tys7O1t69ezVx4sQhHVdCQoKee+65Ph9fjRT0Z76R3uNI708a+T3Sn/mGQ48u+3qeYQIAABhmhu09MQAAAFdDiAEAAEYixAAAACMRYgAAgJEIMb18+OGHeuihh+T1euVyufT2229f85y6ujrl5ubqlltu0fe//33967/+6+AP9AbE2uP+/fvlcrn6bJ9++um3M+AYVFZW6t5771ViYqLS0tL0yCOP6NSpU9c8z6Q5HEiPJs3hli1bNHXqVOcLtHw+n959992rnmPS/Emx92jS/PWnsrJSLpdLFRUVV60zbR4vu57+TJvD9evX9xmrx+O56jlDMX+EmF4uXryoadOmqaqq6rrqz5w5o7/5m7/Rj370I3300Udat26dli9frrfeemuQRzpwsfZ42alTp9TS0uJsmZmZgzTCgaurq9OyZctUX1+v2tpaff311youLtbFixeveI5pcziQHi8zYQ4nTJigjRs36ujRozp69KhmzZqlhx9+WCdOnOi33rT5k2Lv8TIT5q+3hoYGvfLKK5o6depV60ycR+n6+7vMpDm8++67o8Z67NixK9YO2fzZuCJJdk1NzVVrVq9ebf/whz+M2vfEE0/Y+fn5gziym+d6evzggw9sSXZbW9u3MqabqbW11ZZk19XVXbHG9Dm8nh5NnkPbtu3x48fbv/nNb/o9Zvr8XXa1Hk2dv46ODjszM9Oura21Z8yYYT/zzDNXrDVxHmPpz7Q5fO655+xp06Zdd/1QzR8rMTfo0KFDKi4ujto3Z84cHT16VJcuXRqiUQ2Oe+65R+np6SosLNQHH3ww1MO5LqFQSJKUnJx8xRrT5/B6erzMtDns6elRdXW1Ll68KJ/P12+N6fN3PT1eZtr8LVu2TA8++KCKioquWWviPMbS32UmzeHp06fl9Xo1adIk/fjHP9Znn312xdqhmr9h/Y29JggGg33+IKXb7dbXX3+tL7/8Uunp6UM0spsnPT1dr7zyinJzcxWJRLRz504VFhZq//79uv/++4d6eFdk27ZWrFih++67T9nZ2VesM3kOr7dH0+bw2LFj8vl8+r//+z/deuutqqmp0ZQpU/qtNXX+YunRtPmTpOrqav3hD39QQ0PDddWbNo+x9mfaHObl5en111/XXXfdpfPnz+uXv/ylCgoKdOLECaWkpPSpH6r5I8TcBC6XK+q1/f+/BLn3flNlZWUpKyvLee3z+dTc3Kx/+Zd/GZb/+C57+umn9fHHH+vAgQPXrDV1Dq+3R9PmMCsrS01NTWpvb9dbb72lRYsWqa6u7opv8ibOXyw9mjZ/zc3NeuaZZ7Rv3z7dcsst132eKfM4kP5Mm8N58+Y5P+fk5Mjn8+kHP/iBXnvtNa1YsaLfc4Zi/vg46QZ5PB4Fg8Gofa2trYqLi+s3rY4U+fn5On369FAP44rKy8v1zjvv6IMPPtCECROuWmvqHMbSY3+G8xzGx8frzjvv1PTp01VZWalp06bp5Zdf7rfW1PmLpcf+DOf5a2xsVGtrq3JzcxUXF6e4uDjV1dXpV7/6leLi4tTT09PnHJPmcSD99Wc4z2Fv48aNU05OzhXHO1Tzx0rMDfL5fPqP//iPqH379u3T9OnTNXr06CEa1eD76KOPht3yrvSX5F9eXq6amhrt379fkyZNuuY5ps3hQHrsz3Cdw/7Ytq1IJNLvMdPm70qu1mN/hvP8FRYW9nmS5e/+7u/0wx/+UM8++6xGjRrV5xyT5nEg/fVnOM9hb5FIRCdPntSPfvSjfo8P2fwN6m3DBuro6LA/+ugj+6OPPrIl2Zs2bbI/+ugj+/PPP7dt27bXrFlj+/1+p/6zzz6zx44da//DP/yD/cknn9jbt2+3R48ebf/7v//7ULVwTbH2uHnzZrumpsb+7//+b/v48eP2mjVrbEn2W2+9NVQtXNHPfvYz27Ise//+/XZLS4uz/elPf3JqTJ/DgfRo0hyuXbvW/vDDD+0zZ87YH3/8sb1u3Tr7e9/7nr1v3z7bts2fP9uOvUeT5u9Kej+9MxLm8Zuu1Z9pc7hy5Up7//799meffWbX19fbJSUldmJion327FnbtofP/BFiern8GFzvbdGiRbZt2/aiRYvsGTNmRJ2zf/9++5577rHj4+PtO+64w96yZcu3P/AYxNrj888/b//gBz+wb7nlFnv8+PH2fffdZ+/Zs2doBn8N/fUlyX711VedGtPncCA9mjSHP/3pT+2JEyfa8fHx9m233WYXFhY6b+62bf782XbsPZo0f1fS+01+JMzjN12rP9PmcOHChXZ6ero9evRo2+v12vPnz7dPnDjhHB8u8+ey7f9/5w0AAIBBuLEXAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACP9Pzk1+HWUaYrgAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a histogram of the google data\n",
    "plt.hist(google);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QhZV89FzFcv5"
   },
   "source": [
    "### 3c. Permutation test\n",
    "Since the data aren't normally distributed, we're using a *non-parametric* test here. This is simply a label for statistical tests used when the data aren't normally distributed. These tests are extraordinarily powerful due to how few assumptions we need to make.  \n",
    "\n",
    "Check out more about permutations [here.](http://rasbt.github.io/mlxtend/user_guide/evaluate/permutation_test/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bxNhQgpqFcv6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>platform</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>apple</th>\n",
       "      <td>6268.0</td>\n",
       "      <td>4.129882</td>\n",
       "      <td>0.608573</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.3</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>google</th>\n",
       "      <td>9366.0</td>\n",
       "      <td>4.138095</td>\n",
       "      <td>0.615877</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.3</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           count      mean       std  min  25%  50%  75%  max\n",
       "platform                                                     \n",
       "apple     6268.0  4.129882  0.608573  1.0  4.0  4.3  4.5  5.0\n",
       "google    9366.0  4.138095  0.615877  1.0  4.0  4.3  4.5  5.0"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a column called `Permutation1`, and assign to it the result of permuting (shuffling) the Rating column\n",
    "# This assignment will use our numpy object's random.permutation() method\n",
    "df['Permutation1'] = np.random.permutation(df['Rating'])\n",
    "\n",
    "# Call the describe() method on our permutation grouped by 'platform'. \n",
    "df.groupby('platform')['Permutation1'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AaGNfPGTFcv7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>platform</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>apple</th>\n",
       "      <td>6268.0</td>\n",
       "      <td>4.049697</td>\n",
       "      <td>0.726943</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>google</th>\n",
       "      <td>9366.0</td>\n",
       "      <td>4.191757</td>\n",
       "      <td>0.515219</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.3</td>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           count      mean       std  min  25%  50%  75%  max\n",
       "platform                                                     \n",
       "apple     6268.0  4.049697  0.726943  1.0  4.0  4.5  4.5  5.0\n",
       "google    9366.0  4.191757  0.515219  1.0  4.0  4.3  4.5  5.0"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lets compare with the previous analytical summary:\n",
    "df.groupby('platform')['Rating'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aJPNpX0JFcv-"
   },
   "outputs": [],
   "source": [
    "# The difference in the means for Permutation1 (0.001103) now looks hugely different to our observed difference of 0.14206. \n",
    "# It's sure starting to look like our observed difference is significant, and that the Null is false; platform does impact on ratings\n",
    "# But to be sure, let's create 10,000 permutations, calculate the mean ratings for Google and Apple apps and the difference between these for each one, \n",
    "# and then take the average of all of these differences.\n",
    "# Let's create a vector with the differences - that will be the distibution of the Null.\n",
    "\n",
    "# First, make a list called difference.\n",
    "difference = []\n",
    "\n",
    "# Now make a for loop that does the following 10,000 times:\n",
    "# 1. makes a permutation of the 'Rating' as you did above\n",
    "# 2. calculates the difference in the mean rating for apple and the mean rating for google. \n",
    "for i in range(10_000):\n",
    "    permutation = np.random.permutation(df['Rating'])\n",
    "    difference.append(np.mean(permutation[df['platform']=='apple']) - np.mean(permutation[df['platform']=='google']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VlzpYTkYFcv_"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGdCAYAAADjWSL8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAl10lEQVR4nO3de1DV953/8dcpN5XAtyDCgYYQNiVeFnpZ7AJuE7UaxCmSNJnRhCxjdlxjf4laVp00NjujvSyY7CZmZm2sZR01xsTsNtJmpg4jaRJSC3hhZINKHG0x1Q2IsXiOuha8fH5/9Of3lyOIAc/h8MHnY+bMeL7nc46fdx3js1/O9xyPMcYIAADAMl8I9wYAAAAGg4gBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYKXIcG8gVK5evapPPvlEcXFx8ng84d4OAAD4HIwxOnfunNLS0vSFL/R/rmXERswnn3yi9PT0cG8DAAAMwokTJ3TnnXf2u2bERkxcXJykv/yPEB8fH+bdAACAz8Pv9ys9Pd39d7w/IzZirv0IKT4+nogBAMAyn+etILyxFwAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAVooM9wYAhF/Olpxwb2HAWua3hHsLAMKMMzEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKw0oIiprKzUN77xDcXFxSk5OVkPPfSQjhw5ErDmiSeekMfjCbjl5+cHrOnu7taSJUuUlJSk2NhYlZSU6OTJkwFrurq6VFZWJsdx5DiOysrKdPbs2cFNCQAARpwBRUxdXZ2efvppNTY2qra2VpcvX1ZhYaEuXLgQsK6oqEjt7e3ubefOnQGPl5eXq7q6Wtu3b9fu3bt1/vx5FRcX68qVK+6a0tJSNTc3q6amRjU1NWpublZZWdktjAoAAEaSyIEsrqmpCbi/adMmJScnq6mpSffff797PCYmRl6vt8/X8Pl82rhxo7Zu3aqZM2dKkl577TWlp6frnXfe0axZs9Ta2qqamho1NjYqLy9PklRVVaWCggIdOXJE48ePH9CQAABg5Lml98T4fD5JUmJiYsDx999/X8nJybr33nu1cOFCdXZ2uo81NTXp0qVLKiwsdI+lpaUpOztb9fX1kqSGhgY5juMGjCTl5+fLcRx3zfW6u7vl9/sDbgAAYOQadMQYY7Rs2TJ985vfVHZ2tnt89uzZ2rZtm9599129+OKL2rdvn771rW+pu7tbktTR0aHo6GglJCQEvF5KSoo6OjrcNcnJyb1+z+TkZHfN9SorK933zziOo/T09MGOBgAALDCgHyd91uLFi/Xhhx9q9+7dAcfnzZvn/jo7O1uTJ09WRkaGfv3rX+vhhx++4esZY+TxeNz7n/31jdZ81sqVK7Vs2TL3vt/vJ2QAABjBBnUmZsmSJXr77bf13nvv6c477+x3bWpqqjIyMnT06FFJktfrVU9Pj7q6ugLWdXZ2KiUlxV1z6tSpXq91+vRpd831YmJiFB8fH3ADAAAj14AixhijxYsXa8eOHXr33XeVmZl50+ecOXNGJ06cUGpqqiQpNzdXUVFRqq2tdde0t7fr4MGDmjJliiSpoKBAPp9Pe/fuddfs2bNHPp/PXQMAAG5vA/px0tNPP63XX39dv/rVrxQXF+e+P8VxHI0ePVrnz5/X6tWr9cgjjyg1NVXHjx/XD37wAyUlJek73/mOu3bBggVavny5xo4dq8TERK1YsUI5OTnu1UoTJ05UUVGRFi5cqA0bNkiSnnzySRUXF3NlEgAAkDTAiFm/fr0kadq0aQHHN23apCeeeEIRERFqaWnRq6++qrNnzyo1NVXTp0/Xm2++qbi4OHf92rVrFRkZqblz5+rixYuaMWOGNm/erIiICHfNtm3btHTpUvcqppKSEq1bt26wcwIAgBHGY4wx4d5EKPj9fjmOI5/Px/tjgJvI2ZIT7i0MWMv8lnBvAUAIDOTfb747CQAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYKTLcGwCAwcjZkhPuLQxYy/yWcG8BGFE4EwMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKwUGe4NACNNzpaccG8BAG4LnIkBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYKUBRUxlZaW+8Y1vKC4uTsnJyXrooYd05MiRgDXGGK1evVppaWkaPXq0pk2bpkOHDgWs6e7u1pIlS5SUlKTY2FiVlJTo5MmTAWu6urpUVlYmx3HkOI7Kysp09uzZwU0JAABGnAFFTF1dnZ5++mk1NjaqtrZWly9fVmFhoS5cuOCueeGFF/TSSy9p3bp12rdvn7xerx544AGdO3fOXVNeXq7q6mpt375du3fv1vnz51VcXKwrV664a0pLS9Xc3KyamhrV1NSoublZZWVlQRgZAACMBB5jjBnsk0+fPq3k5GTV1dXp/vvvlzFGaWlpKi8v1/e//31JfznrkpKSoueff16LFi2Sz+fTuHHjtHXrVs2bN0+S9Mknnyg9PV07d+7UrFmz1NraqkmTJqmxsVF5eXmSpMbGRhUUFOijjz7S+PHjb7o3v98vx3Hk8/kUHx8/2BGBAeMTe3EjLfNbwr0FYNgbyL/ft/SeGJ/PJ0lKTEyUJLW1tamjo0OFhYXumpiYGE2dOlX19fWSpKamJl26dClgTVpamrKzs901DQ0NchzHDRhJys/Pl+M47prrdXd3y+/3B9wAAMDINeiIMcZo2bJl+uY3v6ns7GxJUkdHhyQpJSUlYG1KSor7WEdHh6Kjo5WQkNDvmuTk5F6/Z3JysrvmepWVle77ZxzHUXp6+mBHAwAAFhh0xCxevFgffvih3njjjV6PeTyegPvGmF7Hrnf9mr7W9/c6K1eulM/nc28nTpz4PGMAAABLDSpilixZorffflvvvfee7rzzTve41+uVpF5nSzo7O92zM16vVz09Perq6up3zalTp3r9vqdPn+51lueamJgYxcfHB9wAAMDINaCIMcZo8eLF2rFjh959911lZmYGPJ6ZmSmv16va2lr3WE9Pj+rq6jRlyhRJUm5urqKiogLWtLe36+DBg+6agoIC+Xw+7d27112zZ88e+Xw+dw0AALi9RQ5k8dNPP63XX39dv/rVrxQXF+eecXEcR6NHj5bH41F5ebkqKiqUlZWlrKwsVVRUaMyYMSotLXXXLliwQMuXL9fYsWOVmJioFStWKCcnRzNnzpQkTZw4UUVFRVq4cKE2bNggSXryySdVXFz8ua5MAgAAI9+AImb9+vWSpGnTpgUc37Rpk5544glJ0jPPPKOLFy/qqaeeUldXl/Ly8rRr1y7FxcW569euXavIyEjNnTtXFy9e1IwZM7R582ZFRES4a7Zt26alS5e6VzGVlJRo3bp1g5kRAACMQLf0OTHDGZ8Tg3Dhc2JwI3xODHBzQ/Y5MQAAAOFCxAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsNOGI++OADzZkzR2lpafJ4PPrlL38Z8PgTTzwhj8cTcMvPzw9Y093drSVLligpKUmxsbEqKSnRyZMnA9Z0dXWprKxMjuPIcRyVlZXp7NmzAx4QAACMTAOOmAsXLuirX/2q1q1bd8M1RUVFam9vd287d+4MeLy8vFzV1dXavn27du/erfPnz6u4uFhXrlxx15SWlqq5uVk1NTWqqalRc3OzysrKBrpdAAAwQkUO9AmzZ8/W7Nmz+10TExMjr9fb52M+n08bN27U1q1bNXPmTEnSa6+9pvT0dL3zzjuaNWuWWltbVVNTo8bGRuXl5UmSqqqqVFBQoCNHjmj8+PED3TYAABhhQvKemPfff1/Jycm69957tXDhQnV2drqPNTU16dKlSyosLHSPpaWlKTs7W/X19ZKkhoYGOY7jBowk5efny3Ecd831uru75ff7A24AAGDkCnrEzJ49W9u2bdO7776rF198Ufv27dO3vvUtdXd3S5I6OjoUHR2thISEgOelpKSoo6PDXZOcnNzrtZOTk90116usrHTfP+M4jtLT04M8GQAAGE4G/OOkm5k3b5776+zsbE2ePFkZGRn69a9/rYcffviGzzPGyOPxuPc/++sbrfmslStXatmyZe59v99PyAAAMIKF/BLr1NRUZWRk6OjRo5Ikr9ernp4edXV1Bazr7OxUSkqKu+bUqVO9Xuv06dPumuvFxMQoPj4+4AYAAEaukEfMmTNndOLECaWmpkqScnNzFRUVpdraWndNe3u7Dh48qClTpkiSCgoK5PP5tHfvXnfNnj175PP53DUAAOD2NuAfJ50/f17Hjh1z77e1tam5uVmJiYlKTEzU6tWr9cgjjyg1NVXHjx/XD37wAyUlJek73/mOJMlxHC1YsEDLly/X2LFjlZiYqBUrVignJ8e9WmnixIkqKirSwoULtWHDBknSk08+qeLiYq5MAgAAkgYRMfv379f06dPd+9fehzJ//nytX79eLS0tevXVV3X27FmlpqZq+vTpevPNNxUXF+c+Z+3atYqMjNTcuXN18eJFzZgxQ5s3b1ZERIS7Ztu2bVq6dKl7FVNJSUm/n00DAABuLx5jjAn3JkLB7/fLcRz5fD7eH4MhlbMlJ9xbwDDVMr8l3FsAhr2B/PvNdycBAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsFBnuDQDA7SJnS064tzBgLfNbwr0F4IY4EwMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsNOGI++OADzZkzR2lpafJ4PPrlL38Z8LgxRqtXr1ZaWppGjx6tadOm6dChQwFruru7tWTJEiUlJSk2NlYlJSU6efJkwJquri6VlZXJcRw5jqOysjKdPXt2wAMCAICRacARc+HCBX31q1/VunXr+nz8hRde0EsvvaR169Zp37598nq9euCBB3Tu3Dl3TXl5uaqrq7V9+3bt3r1b58+fV3Fxsa5cueKuKS0tVXNzs2pqalRTU6Pm5maVlZUNYkQAADASeYwxZtBP9nhUXV2thx56SNJfzsKkpaWpvLxc3//+9yX95axLSkqKnn/+eS1atEg+n0/jxo3T1q1bNW/ePEnSJ598ovT0dO3cuVOzZs1Sa2urJk2apMbGRuXl5UmSGhsbVVBQoI8++kjjx4+/6d78fr8cx5HP51N8fPxgRwQGLGdLTri3AARNy/yWcG8Bt5mB/Psd1PfEtLW1qaOjQ4WFhe6xmJgYTZ06VfX19ZKkpqYmXbp0KWBNWlqasrOz3TUNDQ1yHMcNGEnKz8+X4zjumut1d3fL7/cH3AAAwMgV1Ijp6OiQJKWkpAQcT0lJcR/r6OhQdHS0EhIS+l2TnJzc6/WTk5PdNderrKx03z/jOI7S09NveR4AADB8heTqJI/HE3DfGNPr2PWuX9PX+v5eZ+XKlfL5fO7txIkTg9g5AACwRVAjxuv1SlKvsyWdnZ3u2Rmv16uenh51dXX1u+bUqVO9Xv/06dO9zvJcExMTo/j4+IAbAAAYuYIaMZmZmfJ6vaqtrXWP9fT0qK6uTlOmTJEk5ebmKioqKmBNe3u7Dh486K4pKCiQz+fT3r173TV79uyRz+dz1wAAgNtb5ECfcP78eR07dsy939bWpubmZiUmJuquu+5SeXm5KioqlJWVpaysLFVUVGjMmDEqLS2VJDmOowULFmj58uUaO3asEhMTtWLFCuXk5GjmzJmSpIkTJ6qoqEgLFy7Uhg0bJElPPvmkiouLP9eVSQAAYOQbcMTs379f06dPd+8vW7ZMkjR//nxt3rxZzzzzjC5evKinnnpKXV1dysvL065duxQXF+c+Z+3atYqMjNTcuXN18eJFzZgxQ5s3b1ZERIS7Ztu2bVq6dKl7FVNJSckNP5sGAADcfm7pc2KGMz4nBuHC58RgJOFzYjDUwvY5MQAAAEOFiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAVhrw58QAQ4nLlQEAN8KZGAAAYCUiBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYCUiBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWCnrErF69Wh6PJ+Dm9Xrdx40xWr16tdLS0jR69GhNmzZNhw4dCniN7u5uLVmyRElJSYqNjVVJSYlOnjwZ7K0CAACLheRMzF//9V+rvb3dvbW0tLiPvfDCC3rppZe0bt067du3T16vVw888IDOnTvnrikvL1d1dbW2b9+u3bt36/z58youLtaVK1dCsV0AAGChyJC8aGRkwNmXa4wxevnll/Xcc8/p4YcfliRt2bJFKSkpev3117Vo0SL5fD5t3LhRW7du1cyZMyVJr732mtLT0/XOO+9o1qxZodgyAACwTEjOxBw9elRpaWnKzMzUo48+qj/84Q+SpLa2NnV0dKiwsNBdGxMTo6lTp6q+vl6S1NTUpEuXLgWsSUtLU3Z2trsGAAAg6Gdi8vLy9Oqrr+ree+/VqVOn9JOf/ERTpkzRoUOH1NHRIUlKSUkJeE5KSoo+/vhjSVJHR4eio6OVkJDQa8215/elu7tb3d3d7n2/3x+skQAAwDAU9IiZPXu2++ucnBwVFBTonnvu0ZYtW5Sfny9J8ng8Ac8xxvQ6dr2bramsrNQPf/jDW9g5AACwScgvsY6NjVVOTo6OHj3qvk/m+jMqnZ2d7tkZr9ernp4edXV13XBNX1auXCmfz+feTpw4EeRJAADAcBLyiOnu7lZra6tSU1OVmZkpr9er2tpa9/Genh7V1dVpypQpkqTc3FxFRUUFrGlvb9fBgwfdNX2JiYlRfHx8wA0AAIxcQf9x0ooVKzRnzhzddddd6uzs1E9+8hP5/X7Nnz9fHo9H5eXlqqioUFZWlrKyslRRUaExY8aotLRUkuQ4jhYsWKDly5dr7NixSkxM1IoVK5STk+NerQQAABD0iDl58qQee+wxffrppxo3bpzy8/PV2NiojIwMSdIzzzyjixcv6qmnnlJXV5fy8vK0a9cuxcXFua+xdu1aRUZGau7cubp48aJmzJihzZs3KyIiItjbBQAAlvIYY0y4NxEKfr9fjuPI5/PxoyWL5WzJCfcWgNtay/yWmy8Cgmgg/37z3UkAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsFPQvgAQAjBw2fn8Z3/d0++BMDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBLfYn0bsfHbaAEAuBHOxAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALASEQMAAKxExAAAACsRMQAAwEpEDAAAsBIRAwAArETEAAAAKxExAADASkQMAACwEhEDAACsRMQAAAArETEAAMBKRAwAALBSZLg3AABAMOVsyQn3FgalZX5LuLdgHSJmkGz9SwIAwEgx7H+c9MorrygzM1OjRo1Sbm6ufvvb34Z7SwAAYBgY1hHz5ptvqry8XM8995wOHDig++67T7Nnz9Yf//jHcG8NAACE2bCOmJdeekkLFizQP/7jP2rixIl6+eWXlZ6ervXr14d7awAAIMyG7Xtienp61NTUpGeffTbgeGFhoerr63ut7+7uVnd3t3vf5/NJkvx+f0j2d+XilZC8LgDg9jTpZ5PCvYUBayxtDPprXvt32xhz07XDNmI+/fRTXblyRSkpKQHHU1JS1NHR0Wt9ZWWlfvjDH/Y6np6eHrI9AgBwO3P+jxOy1z537pwcp//XH7YRc43H4wm4b4zpdUySVq5cqWXLlrn3r169qj/96U8aO3Zsn+uHO7/fr/T0dJ04cULx8fHh3s6QYGZmHqmYmZlHqlDMbIzRuXPnlJaWdtO1wzZikpKSFBER0eusS2dnZ6+zM5IUExOjmJiYgGNf/OIXQ7nFIREfH3/b/GW4hplvD8x8e2Dm20OwZ77ZGZhrhu0be6Ojo5Wbm6va2tqA47W1tZoyZUqYdgUAAIaLYXsmRpKWLVumsrIyTZ48WQUFBfr5z3+uP/7xj/rud78b7q0BAIAwG9YRM2/ePJ05c0Y/+tGP1N7eruzsbO3cuVMZGRnh3lrIxcTEaNWqVb1+RDaSMfPtgZlvD8x8ewj3zB7zea5hAgAAGGaG7XtiAAAA+kPEAAAAKxExAADASkQMAACwEhETJl1dXSorK5PjOHIcR2VlZTp79my/zzHGaPXq1UpLS9Po0aM1bdo0HTp0KGDNokWLdM8992j06NEaN26cHnzwQX300UchnOTzC8XMf/rTn7RkyRKNHz9eY8aM0V133aWlS5e6350VbqH6c/75z3+uadOmKT4+Xh6P56avGUqvvPKKMjMzNWrUKOXm5uq3v/1tv+vr6uqUm5urUaNG6a/+6q/0s5/9rNeat956S5MmTVJMTIwmTZqk6urqUG1/UII986FDh/TII4/o7rvvlsfj0csvvxzC3Q9OsGeuqqrSfffdp4SEBCUkJGjmzJnau3dvKEcYlGDPvWPHDk2ePFlf/OIXFRsbq6997WvaunVrKEcYsFD8nb5m+/bt8ng8euihh4KzWYOwKCoqMtnZ2aa+vt7U19eb7OxsU1xc3O9z1qxZY+Li4sxbb71lWlpazLx580xqaqrx+/3umg0bNpi6ujrT1tZmmpqazJw5c0x6erq5fPlyqEe6qVDM3NLSYh5++GHz9ttvm2PHjpnf/OY3JisryzzyyCNDMdJNherPee3ataaystJUVlYaSaarqyvEk/Rt+/btJioqylRVVZnDhw+b733veyY2NtZ8/PHHfa7/wx/+YMaMGWO+973vmcOHD5uqqioTFRVlfvGLX7hr6uvrTUREhKmoqDCtra2moqLCREZGmsbGxqEaq1+hmHnv3r1mxYoV5o033jBer9esXbt2iKb5fEIxc2lpqfnpT39qDhw4YFpbW80//MM/GMdxzMmTJ4dqrJsKxdzvvfee2bFjhzl8+LA5duyYefnll01ERISpqakZqrH6FYqZrzl+/Lj50pe+ZO677z7z4IMPBmW/REwYHD582EgK+I9yQ0ODkWQ++uijPp9z9epV4/V6zZo1a9xjf/7zn43jOOZnP/vZDX+v//7v/zaSzLFjx4I3wCAM5cz/+Z//aaKjo82lS5eCN8AgDMXM7733Xlgj5m//9m/Nd7/73YBjEyZMMM8++2yf65955hkzYcKEgGOLFi0y+fn57v25c+eaoqKigDWzZs0yjz76aJB2fWtCMfNnZWRkDLuICfXMxhhz+fJlExcXZ7Zs2XLrGw6SoZjbGGO+/vWvm3/+53++tc0GSahmvnz5svm7v/s78x//8R9m/vz5QYsYfpwUBg0NDXIcR3l5ee6x/Px8OY6j+vr6Pp/T1tamjo4OFRYWusdiYmI0derUGz7nwoUL2rRpkzIzM8P+bd5DNbMk+Xw+xcfHKzIyvJ/lOJQzh0NPT4+ampoC9ipJhYWFN9xrQ0NDr/WzZs3S/v37denSpX7XDIf5QzXzcDZUM//v//6vLl26pMTExOBs/BYNxdzGGP3mN7/RkSNHdP/99wdv84MUypl/9KMfady4cVqwYEFQ90zEhEFHR4eSk5N7HU9OTu71hZeffY6kXl9+mZKS0us5r7zyiu644w7dcccdqqmpUW1traKjo4O0+8EJ9czXnDlzRj/+8Y+1aNGiW9zxrRuqmcPl008/1ZUrVwa0146Ojj7XX758WZ9++mm/a4bD/KGaeTgbqpmfffZZfelLX9LMmTODs/FbFMq5fT6f7rjjDkVHR+vb3/62/v3f/10PPPBA8IcYoFDN/Lvf/U4bN25UVVVV0PdMxATR6tWr5fF4+r3t379fkuTxeHo93xjT5/HPuv7xvp7z+OOP68CBA6qrq1NWVpbmzp2rP//5z7c4Xd+Gy8zSX74S/tvf/rYmTZqkVatW3cJU/RtOMw8HA91rX+uvPz7c5w/FzMNdKGd+4YUX9MYbb2jHjh0aNWpUEHYbPKGYOy4uTs3Nzdq3b5/+5V/+RcuWLdP7778fvE3fomDOfO7cOf393/+9qqqqlJSUFPS9DuvvTrLN4sWL9eijj/a75u6779aHH36oU6dO9Xrs9OnTvYr2Gq/XK+kv1Zuamuoe7+zs7PWca1fCZGVlKT8/XwkJCaqurtZjjz020JFuarjMfO7cORUVFemOO+5QdXW1oqKiBjrK5zZcZg63pKQkRURE9Pp/aP3t1ev19rk+MjJSY8eO7XfNcJg/VDMPZ6Ge+d/+7d9UUVGhd955R1/5yleCu/lbEMq5v/CFL+jLX/6yJOlrX/uaWltbVVlZqWnTpgV3iAEKxcyHDh3S8ePHNWfOHPfxq1evSpIiIyN15MgR3XPPPYPeM2digigpKUkTJkzo9zZq1CgVFBTI5/MFXE64Z88e+Xw+TZkypc/XzszMlNfrVW1trXusp6dHdXV1N3zONcYYdXd3B2fI6wyHmf1+vwoLCxUdHa2333475P9PbjjMPBxER0crNzc3YK+SVFtbe8O9FhQU9Fq/a9cuTZ482Q3PG60ZDvOHaubhLJQz/+u//qt+/OMfq6amRpMnTw7+5m/BUP5Zh/K/0QMRipknTJiglpYWNTc3u7eSkhJNnz5dzc3Nt/5+zaC8PRgDVlRUZL7yla+YhoYG09DQYHJycnpdejt+/HizY8cO9/6aNWuM4zhmx44dpqWlxTz22GMBl97+/ve/NxUVFWb//v3m448/NvX19ebBBx80iYmJ5tSpU0M6X19CMbPf7zd5eXkmJyfHHDt2zLS3t7u34XJZebBnNsaY9vZ2c+DAAVNVVWUkmQ8++MAcOHDAnDlzZshmM+b/X465ceNGc/jwYVNeXm5iY2PN8ePHjTHGPPvss6asrMxdf+1yzH/6p38yhw8fNhs3bux1Oebvfvc7ExERYdasWWNaW1vNmjVrhuUl1sGcubu72xw4cMAcOHDApKammhUrVpgDBw6Yo0ePDvl8fQnFzM8//7yJjo42v/jFLwL+3p47d27I57uRUMxdUVFhdu3aZX7/+9+b1tZW8+KLL5rIyEhTVVU15PP1JRQzXy+YVycRMWFy5swZ8/jjj5u4uDgTFxdnHn/88V6XyUoymzZtcu9fvXrVrFq1yni9XhMTE2Puv/9+09LS4j7+P//zP2b27NkmOTnZREVFmTvvvNOUlpbe8HLeoRaKma9dYtzXra2tbWgG60coZjbGmFWrVvU582dfZ6j89Kc/NRkZGSY6Otr8zd/8jamrq3Mfmz9/vpk6dWrA+vfff998/etfN9HR0ebuu+8269ev7/Wa//Vf/2XGjx9voqKizIQJE8xbb70V6jEGJNgzt7W19fnnef3rhFOwZ87IyOhz5lWrVg3BNJ9fsOd+7rnnzJe//GUzatQok5CQYAoKCsz27duHYpTPLRR/pz8rmBHjMeb/vQMHAADAIrwnBgAAWImIAQAAViJiAACAlYgYAABgJSIGAABYiYgBAABWImIAAICViBgAAGAlIgYAAFiJiAEAAFYiYgAAgJWIGAAAYKX/C7bRI5TZ48cvAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Make a variable called 'histo', and assign to it the result of plotting a histogram of the difference list. \n",
    "histo = plt.hist(difference)\n",
    "histo;\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KhqgDFd3FcwB"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.1420605474512291\n"
     ]
    }
   ],
   "source": [
    "# Now make a variable called obs_difference, and assign it the result of the mean of our 'apple' variable and the mean of our 'google variable'\n",
    "obs_difference = np.mean(apple) - np.mean(google)\n",
    "\n",
    "# Make this difference absolute with the built-in abs() function. \n",
    "obs_difference = abs(obs_difference)\n",
    "\n",
    "# Print out this value; it should be 0.1420605474512291. \n",
    "print(obs_difference)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QUNuuSgDFcwE"
   },
   "source": [
    "## Stage 4 -  Evaluating and concluding\n",
    "### 4a. What is our conclusion?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XMQHnqx1FcwF"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "0.0\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "What do we know? \n",
    "\n",
    "Recall: The p-value of our observed data is just the proportion of the data given the null that's at least as extreme as that observed data.\n",
    "\n",
    "As a result, we're going to count how many of the differences in our difference list are at least as extreme as our observed difference.\n",
    "\n",
    "If less than or equal to 5% of them are, then we will reject the Null. \n",
    "'''\n",
    "\n",
    "extreme_differences = 0\n",
    "\n",
    "for i in range(10_000):\n",
    "    if abs(difference[i]) >= obs_difference:\n",
    "        extreme_differences += 1\n",
    "\n",
    "p = extreme_differences/len(difference)\n",
    "print(extreme_differences)\n",
    "print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4RTofY9wFcwI"
   },
   "source": [
    "### 4b. What is our decision?\n",
    "So actually, zero differences are at least as extreme as our observed difference!\n",
    "\n",
    "So the p-value of our observed data is 0. \n",
    "\n",
    "It doesn't matter which significance level we pick; our observed data is statistically significant, and we reject the Null.\n",
    "\n",
    "We conclude that platform does impact on ratings. Specifically, we should advise our client to integrate **only Google Play** into their operating system interface. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Cu_eh7fTFcwJ"
   },
   "source": [
    "### 4c. Other statistical tests, and next steps\n",
    "The test we used here is the Permutation test. This was appropriate because our data were not normally distributed! \n",
    "\n",
    "As we've seen in Professor Spiegelhalter's book, there are actually many different statistical tests, all with different assumptions. How many of these different statistical tests can you remember? How much do you remember about what the appropriate conditions are under which to use them? \n",
    "\n",
    "Make a note of your answers to these questions, and discuss them with your mentor at your next call. \n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "5fbRq2MZFcvq",
    "4RTofY9wFcwI",
    "Cu_eh7fTFcwJ"
   ],
   "name": "Springboard Apps project - Tier 3 - Complete.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}