NB5_PredictiveModelling+Comarision.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predictive model using Different Algorithms\n",
    "\n",
    "In this notebook we finally begin our path for prediction. We have considered mainly three algothirms for our regression problem mainly:\n",
    "* Linear Regression\n",
    "* Decision Tree Regression\n",
    "* Random Forest Regression\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import sklearn\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.preprocessing import RobustScaler\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "\n",
    "import warnings\n",
    "# ignore warnings\n",
    "warnings.simplefilter(action='ignore', category=FutureWarning)\n",
    "\n",
    "# display all columns\n",
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load data\n",
    "df = pd.read_csv(\"data/data_final.csv\",index_col=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Scaling\n",
    "\n",
    "Feature Scaling is a technique to standardize the independent features present in the data in a fixed range. It is performed during the data pre-processing to handle highly varying magnitudes or values or units. If feature scaling is not done, then a machine learning algorithm tends to weigh greater values, higher and consider smaller values as the lower values, regardless of the unit of the values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create feature (x) and target (y) variables\n",
    "x = df.drop(\"Price\", axis=1)\n",
    "Y = df[\"Price\"].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#feature scaling using robust scaler\n",
    "scaler = RobustScaler()\n",
    "df = scaler.fit_transform(df.astype(np.float64))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test, Y_train, Y_test = train_test_split(x,Y, test_size = .20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us create the instances for all the selected models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create instance\n",
    "tree = DecisionTreeRegressor()\n",
    "forest = RandomForestRegressor(n_jobs=-1)\n",
    "linear = LinearRegression(n_jobs=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We fit our data without any hypertuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(n_jobs=-1)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#fit data into model\n",
    "tree.fit(x_train, Y_train)\n",
    "forest.fit(x_train, Y_train)\n",
    "linear.fit(x_train, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets predict and compare the root mean square error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE : linear regression = 444476.81\n",
      "RMSE : random forest = 185241.68\n",
      "RMSE : decision tree = 112830.35\n"
     ]
    }
   ],
   "source": [
    "models= [('linear regression', linear), ('random forest', forest), ('decision tree', tree)]\n",
    "for i, model in models:    \n",
    "    prediction = model.predict(x_train)\n",
    "    MSE = mean_squared_error(Y_train, prediction)\n",
    "    RMSE = np.sqrt(MSE)\n",
    "    print('RMSE :', i, \"=\",'{}'.format(round(RMSE, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since RMSE (root mean square error) is data dependent, we need a human understandable format to assess our model. Hence we use Mean absolute Percentage Error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy : linear regression = 70.99 %\n",
      "Accuracy : random forest = 89.61 %\n",
      "Accuracy : decision tree = 97.39 %\n"
     ]
    }
   ],
   "source": [
    "for i, model in models:\n",
    "    predictions = model.predict(x_train)\n",
    "    errors = abs(predictions - Y_train)\n",
    "    mape = np.mean(100 * (errors / Y_train))\n",
    "    accuracy = 100 - mape    \n",
    "    print('Accuracy :', i,\"=\",'{}'.format(round(accuracy, 2)),\"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Observations:__\n",
    "\n",
    "After analyzing the above result it seems that decision tree performs the best in terms of accuracy, also the RMSE is the lowest. At this point we have only predicted on the training set so therefor the chances of overfitting is high. We need to compare the models using cross validation to find the true potential."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparing Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "models= [('linear regression', linear), ('random forest', forest), ('decision tree', tree)]\n",
    "scoring = ['neg_mean_squared_error', 'neg_mean_absolute_error', 'r2']\n",
    "\n",
    "results= []\n",
    "metric= []\n",
    "for name, model in models:\n",
    "    for i in scoring:\n",
    "        scores = cross_validate(model, x_train, Y_train, scoring=i, cv=10, return_train_score=True)\n",
    "        results.append(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets prepare a structure for comparision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Linear Regression\n",
    "LR_RMSE_mean = np.sqrt(-results[0]['test_score'].mean())\n",
    "LR_RMSE_std= results[0]['test_score'].std()\n",
    "LR_MAE_mean = -results[1]['test_score'].mean()\n",
    "LR_MAE_std= results[1]['test_score'].std()\n",
    "LR_r2_mean = results[2]['test_score'].mean()\n",
    "LR_r2_std = results[2]['test_score'].std()\n",
    "\n",
    "# Random Forest\n",
    "RF_RMSE_mean = np.sqrt(-results[3]['test_score'].mean())\n",
    "RF_RMSE_std= results[3]['test_score'].std()\n",
    "RF_MAE_mean = -results[4]['test_score'].mean()\n",
    "RF_MAE_std= results[4]['test_score'].std()\n",
    "RF_r2_mean = results[5]['test_score'].mean()\n",
    "RF_r2_std = results[5]['test_score'].std()\n",
    "\n",
    "# Decision Tree\n",
    "DT_RMSE_mean = np.sqrt(-results[6]['test_score'].mean())\n",
    "DT_RMSE_std= results[6]['test_score'].std()\n",
    "DT_MAE_mean = -results[7]['test_score'].mean()\n",
    "DT_MAE_std= results[7]['test_score'].std()\n",
    "DT_r2_mean = results[8]['test_score'].mean()\n",
    "DT_r2_std = results[8]['test_score'].std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>RMSE_mean</th>\n",
       "      <th>RMSE_std</th>\n",
       "      <th>MAE_mean</th>\n",
       "      <th>MAE_std</th>\n",
       "      <th>r2_mean</th>\n",
       "      <th>r2_std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Random Forest</td>\n",
       "      <td>423406.204233</td>\n",
       "      <td>1.409456e+10</td>\n",
       "      <td>249338.318751</td>\n",
       "      <td>6289.304157</td>\n",
       "      <td>0.446402</td>\n",
       "      <td>0.037290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Linear Regression</td>\n",
       "      <td>445427.273617</td>\n",
       "      <td>1.134180e+10</td>\n",
       "      <td>285977.748390</td>\n",
       "      <td>5280.468309</td>\n",
       "      <td>0.386476</td>\n",
       "      <td>0.018763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Decision Trees</td>\n",
       "      <td>551492.715534</td>\n",
       "      <td>1.343569e+10</td>\n",
       "      <td>309986.203475</td>\n",
       "      <td>5432.182574</td>\n",
       "      <td>0.063903</td>\n",
       "      <td>0.062821</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Model      RMSE_mean      RMSE_std       MAE_mean      MAE_std  \\\n",
       "1      Random Forest  423406.204233  1.409456e+10  249338.318751  6289.304157   \n",
       "0  Linear Regression  445427.273617  1.134180e+10  285977.748390  5280.468309   \n",
       "2     Decision Trees  551492.715534  1.343569e+10  309986.203475  5432.182574   \n",
       "\n",
       "    r2_mean    r2_std  \n",
       "1  0.446402  0.037290  \n",
       "0  0.386476  0.018763  \n",
       "2  0.063903  0.062821  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_comparision = pd.DataFrame({\n",
    "    'Model'       : ['Linear Regression', 'Random Forest', 'Decision Trees'],\n",
    "    'RMSE_mean'    : [LR_RMSE_mean, RF_RMSE_mean, DT_RMSE_mean],\n",
    "    'RMSE_std'    : [LR_RMSE_std, RF_RMSE_std, DT_RMSE_std],\n",
    "    'MAE_mean'   : [LR_MAE_mean, RF_MAE_mean, DT_MAE_mean],\n",
    "    'MAE_std'   : [LR_MAE_std, RF_MAE_std, DT_MAE_std],\n",
    "    'r2_mean'      : [LR_r2_mean, RF_r2_mean, DT_r2_mean],\n",
    "    'r2_std'      : [LR_r2_std, RF_r2_std, DT_r2_std],\n",
    "    }, columns = ['Model', 'RMSE_mean', 'RMSE_std', 'MAE_mean', 'MAE_std', 'r2_mean', 'r2_std'])\n",
    "\n",
    "model_comparision.sort_values(by='r2_mean', ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\tfgpu\\lib\\site-packages\\seaborn\\categorical.py:3704: UserWarning: The `factorplot` function has been renamed to `catplot`. The original name will be removed in a future release. Please update your code. Note that the default `kind` in `factorplot` (`'point'`) has changed `'strip'` in `catplot`.\n",
      "  warnings.warn(msg)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x1dedaf7f518>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAFuCAYAAABOYJmxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAepUlEQVR4nO3dfbBkVX3u8e/jDBJ8AQFHQvESiJIX8AVlLmAwiYqBSWLEm0AyJpHRkCJavsTkGktyTUAsLDUm5qKioZQwECMQX1EDOAExV0RgkDeBIFOiOIEro4OIUdHB3/1jr3Npjuec6cNMn8Oa+/1UdfXutfdavbr79NPrrL17d6oKSVJfHrHYHZAkzZ/hLUkdMrwlqUOGtyR1yPCWpA4tXewOPJysWLGiLrzwwsXuhiSNykyFjrxHfPOb31zsLkjSWAxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhz+ctaYsc9s7DFrsLXbrsVZdtUX1H3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA5NPLyTfDXJDUmuTbK2le2SZE2SW9v1ziPbn5BkXZJbkhw5Un5Qa2ddklOTpJVvn+TcVn5Fkn1G6qxq93FrklWTfqyStFAWauT9nKo6sKqWt9uvBy6uqv2Ai9ttkuwPrAQOAFYApyVZ0uq8Bzge2K9dVrTy44C7q+pJwDuAt7a2dgFOBA4BDgZOHP2QkKSeLda0yVHA6ra8GnjhSPk5VXVfVd0GrAMOTrI7sGNVXV5VBZw1rc5UWx8CDm+j8iOBNVW1saruBtbwQOBLUtcWIrwL+HSSq5Mc38p2q6o7Adr1E1r5HsDXR+qub2V7tOXp5Q+qU1WbgHuAXedo60GSHJ9kbZK1GzZseMgPUpIW0tIFuI/DquqOJE8A1iT5jzm2zQxlNUf5Q63zQEHV6cDpAMuXL/+J9ZL0cDTxkXdV3dGu7wI+yjD//I02FUK7vqttvh7Ya6T6nsAdrXzPGcofVCfJUmAnYOMcbUlS9yYa3kkeneSxU8vAEcCXgPOBqaM/VgEfb8vnAyvbEST7MuyYvLJNrdyb5NA2n33stDpTbR0NXNLmxS8Cjkiyc9tReUQrk6TuTXraZDfgo+2ovqXAP1fVhUmuAs5LchxwO3AMQFXdmOQ84CZgE/CKqrq/tfVy4ExgB+CCdgF4P3B2knUMI+6Vra2NSd4EXNW2O7mqNk7ywUrSQskwSBUMc95r165d7G5IXTnsnYctdhe6dNmrLht305n23/kNS0nqkeEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nq0NLF7kCPDvqLsxa7C126+m+OXewuSNsMR96S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHPLeJunX7yU9Z7C50ae+/vmGxu6CtwJG3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1aEHCO8mSJNck+WS7vUuSNUlubdc7j2x7QpJ1SW5JcuRI+UFJbmjrTk2SVr59knNb+RVJ9hmps6rdx61JVi3EY5WkhbBQI+8/BW4euf164OKq2g+4uN0myf7ASuAAYAVwWpIlrc57gOOB/dplRSs/Dri7qp4EvAN4a2trF+BE4BDgYODE0Q8JSerZxMM7yZ7AbwLvGyk+CljdllcDLxwpP6eq7quq24B1wMFJdgd2rKrLq6qAs6bVmWrrQ8DhbVR+JLCmqjZW1d3AGh4IfEnq2kKMvP8eeB3w45Gy3arqToB2/YRWvgfw9ZHt1reyPdry9PIH1amqTcA9wK5ztPUgSY5PsjbJ2g0bNjyEhydJC2+i4Z3k+cBdVXX1uFVmKKs5yh9qnQcKqk6vquVVtXzZsmVjdlOSFtekR96HAS9I8lXgHOC5Sf4J+EabCqFd39W2Xw/sNVJ/T+COVr7nDOUPqpNkKbATsHGOtiSpexMN76o6oar2rKp9GHZEXlJVfwicD0wd/bEK+HhbPh9Y2Y4g2Zdhx+SVbWrl3iSHtvnsY6fVmWrr6HYfBVwEHJFk57aj8ohWJkndW6wfY3gLcF6S44DbgWMAqurGJOcBNwGbgFdU1f2tzsuBM4EdgAvaBeD9wNlJ1jGMuFe2tjYmeRNwVdvu5KraOOkHJkkLYcHCu6ouBS5ty98CDp9lu1OAU2YoXws8eYbyH9DCf4Z1ZwBnPNQ+S9LDld+wlKQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUoeWzmfjJL8E7DNar6rO2sp9kiRtxtjhneRs4InAtcD9rbgAw1uSFth8Rt7Lgf2rqibVGUnSeOYz5/0l4Kcn1RFJ0vjmM/J+PHBTkiuB+6YKq+oFW71XkqQ5zSe8T5pUJyRJ8zN2eFfVZyfZEUnS+Mae805yaJKrknw3yQ+T3J/kO5PsnCRpZvPZYfku4EXArcAOwB+3MknSApvXl3Sqal2SJVV1P/CPST4/oX5JkuYwn/D+XpJHAtcmeRtwJ/DoyXRLkjSX+UybvLht/0rgv4C9gN+ZRKckSXObz9EmX0uyA7B7Vb1xgn2SJG3GfI42+S2G85pc2G4fmOT8CfVLkjSH+UybnAQcDHwboKquZTjDoCRpgc0nvDdV1T0T64kkaWzzOdrkS0l+H1iSZD/g1YCHCkrSIpjPyPtVwAEMJ6X6IPAd4DUT6JMkaTPGDu+q+l5V/c+q+m9Vtbwt/2CuOkl+KsmVSa5LcmOSN7byXZKsSXJru955pM4JSdYluSXJkSPlByW5oa07NUla+fZJzm3lVyTZZ6TOqnYftyZZNY/nRZIe1uZztMnyJB9J8sUk109dNlPtPuC5VfU04EBgRZJDgdcDF1fVfsDF7TZJ9gdWMozwVwCnJVnS2noPcDywX7usaOXHAXdX1ZOAdwBvbW3tApwIHMKwo/XE0Q8JSerZfKZNPgCcyfDFnN8aucyqBt9tN7drlwKOAla38tXAC9vyUcA5VXVfVd0GrAMOTrI7sGNVXd5+yeesaXWm2voQcHgblR8JrKmqjVV1N7CGBwJfkro2nx2WG6pq3sd1t5Hz1cCTgHdX1RVJdquqOwGq6s4kT2ib7wF8YaT6+lb2o7Y8vXyqztdbW5uS3APsOlo+Q53R/h3PMKJn7733nu/Dk6RFMZ/wPjHJ+ximOUZ/Secjc1VqJ7E6MMnjgI8mefIcm2emJuYof6h1Rvt3OnA6wPLly/19TkldmE94vxT4BYapjx+3sgLmDO8pVfXtJJcyTF18I8nubdS9O3BX22w9wzlTpuwJ3NHK95yhfLTO+iRLgZ2Aja382dPqXDpOXyXp4W4+c95Pa0eZrKqql7bLH81VIcmyNuKmnRflecB/AOcDU0d/rAI+3pbPB1a2I0j2ZdgxeWWbYrm3/SBEgGOn1Zlq62jgkjYvfhFwRJKd247KI1qZJHVvPiPvLyTZv6pumked3YHVbd77EcB5VfXJJJcD5yU5DrgdOAagqm5Mch5wE7AJeEWbdgF4OcMO0x2AC9oF4P3A2UnWMYy4V7a2NiZ5E3BV2+7kqto4j75L0sPWfML7WcCqJLcxzHmH4YCSp85WoaquB54+Q/m3gMNnqXMKcMoM5WuBn5gvb8eaHzNLW2cAZ8zWP0nq1XzCe87D7JLs3A7JkyRN2LzO572ZTS4GnrFl3ZEkjWM+Oyw3Z6ZD8yRJE7A1w9tjpCVpgWzN8JYkLRCnTSSpQ5sN7yTPHVned9q63x65OeOhf5KkrW+ckffbR5Y/PG3dG6YW/AKMJC2cccI7syzPdFuStADGCe+aZXmm25KkBTDOl3R+Nsn5DKPsqWXa7X1nryZJmpRxwvuokeW3T1s3/bYkaQFsNryr6rOjt5Nsx3CCqP+sqrtmriVJmqRxDhV8b5ID2vJOwHUMvyF5TZIXTbh/kqQZjLPD8per6sa2/FLgy1X1FOAg4HUT65kkaVbjhPcPR5Z/DfgYQFX9n0l0SJK0eeOE97eTPD/J04HDgAsB2u9F7jDJzkmSZjbO0SZ/ApwK/DTwmpER9+HApybVMUnS7MY52uTLzPArOlV1Ef6gryQtis2Gd5JT51pfVa/eet2RJI1jnGmTlwFfAs4D7sDzmUjSohsnvHdn+HX23wM2AecCH/bHhiVp8Wz2aJOq+lZVvbeqngO8BHgccGOSF0+4b5KkWYz96/FJngG8iOFY7wuAqyfVKUnS3MbZYflG4PnAzcA5wAlVtWnSHZMkzW6ckfdfAV8BntYub04Cw47LqqqnTq57kqSZjBPenrNbkh5mxvmSztdmKk+yBFgJzLhekjQ545wSdsckJyR5V5IjMngVw1TK706+i5Kk6caZNjkbuBu4HPhj4C+ARwJHVdW1k+uaJGk2Y/2GZTt/N0neB3wT2Luq7p1ozyRJsxrnlLA/mlqoqvuB2wxuSVpc44y8n5bkO205wA7t9tShgjtOrHeSpBmNc7TJkoXoiCRpfONMm0iSHmYMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1aKLhnWSvJJ9JcnOSG5P8aSvfJcmaJLe2651H6pyQZF2SW5IcOVJ+UJIb2rpTM/UT9sn2Sc5t5Vck2Wekzqp2H7cmWTXJxypJC2nSI+9NwP+oql8EDgVekWR/4PXAxVW1H3Bxu01btxI4AFgBnNZ+6BjgPcDxwH7tsqKVHwfcXVVPAt4BvLW1tQtwInAIcDBw4uiHhCT1bKLhXVV3VtUX2/K9wM3AHsBRwOq22WrghW35KOCcqrqvqm4D1gEHJ9kd2LGqLq+qAs6aVmeqrQ8Bh7dR+ZHAmqraWFV3A2t4IPAlqWsLNufdpjOeDlwB7FZVd8IQ8MAT2mZ7AF8fqba+le3RlqeXP6hOVW0C7gF2naOt6f06PsnaJGs3bNiwBY9QkhbOgoR3kscAHwZeU1XfmWvTGcpqjvKHWueBgqrTq2p5VS1ftmzZHF2TpIePiYd3ku0YgvsDVfWRVvyNNhVCu76rla8H9hqpvidwRyvfc4byB9VJshTYCdg4R1uS1L1JH20S4P3AzVX1dyOrzgemjv5YBXx8pHxlO4JkX4Ydk1e2qZV7kxza2jx2Wp2pto4GLmnz4hcBRyTZue2oPKKVSVL3xvn1+C1xGPBi4IYk17ayvwTeApyX5DjgduAYgKq6Mcl5wE0MR6q8oqrub/VeDpwJ7ABc0C4wfDicnWQdw4h7ZWtrY5I3AVe17U6uqo0TepyStKAmGt5V9TlmnnsGOHyWOqcAp8xQvhZ48gzlP6CF/wzrzgDOGLe/ktQLv2EpSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6NNHwTnJGkruSfGmkbJcka5Lc2q53Hll3QpJ1SW5JcuRI+UFJbmjrTk2SVr59knNb+RVJ9hmps6rdx61JVk3ycUrSQpv0yPtMYMW0stcDF1fVfsDF7TZJ9gdWAge0OqclWdLqvAc4HtivXabaPA64u6qeBLwDeGtraxfgROAQ4GDgxNEPCUnq3UTDu6r+Hdg4rfgoYHVbXg28cKT8nKq6r6puA9YBByfZHdixqi6vqgLOmlZnqq0PAYe3UfmRwJqq2lhVdwNr+MkPEUnq1mLMee9WVXcCtOsntPI9gK+PbLe+le3RlqeXP6hOVW0C7gF2naOtn5Dk+CRrk6zdsGHDFjwsSVo4D6cdlpmhrOYof6h1HlxYdXpVLa+q5cuWLRuro5K02BYjvL/RpkJo13e18vXAXiPb7Qnc0cr3nKH8QXWSLAV2Ypimma0tSdomLEZ4nw9MHf2xCvj4SPnKdgTJvgw7Jq9sUyv3Jjm0zWcfO63OVFtHA5e0efGLgCOS7Nx2VB7RyiRpm7B0ko0n+SDwbODxSdYzHAHyFuC8JMcBtwPHAFTVjUnOA24CNgGvqKr7W1MvZzhyZQfggnYBeD9wdpJ1DCPula2tjUneBFzVtju5qqbvOJWkbk00vKvqRbOsOnyW7U8BTpmhfC3w5BnKf0AL/xnWnQGcMXZnJakjD6cdlpKkMRnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ4a3JHXI8JakDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUOGtyR1yPCWpA4Z3pLUIcNbkjpkeEtShwxvSeqQ4S1JHTK8JalDhrckdcjwlqQOGd6S1CHDW5I6ZHhLUocMb0nqkOEtSR0yvCWpQ9t8eCdZkeSWJOuSvH6x+yNJW8M2Hd5JlgDvBn4d2B94UZL9F7dXkrTltunwBg4G1lXVV6rqh8A5wFGL3CdJ2mKpqsXuw8QkORpYUVV/3G6/GDikql45ss3xwPHt5s8Dtyx4R7euxwPfXOxOyNfhYWJbeB2+WVUrphcuXYyeLKDMUPagT6uqOh04fWG6M3lJ1lbV8sXux//vfB0eHrbl12FbnzZZD+w1cntP4I5F6oskbTXbenhfBeyXZN8kjwRWAucvcp8kaYtt09MmVbUpySuBi4AlwBlVdeMid2vStpkpoM75Ojw8bLOvwza9w1KStlXb+rSJJG2TDG9J6pDhPYYk352h7GVJjl3gflzavup/XZKrkhy4kPc/lyQvWMzTDyS5P8m1Sb6U5BNJHreV2n1JkndtjbamtTv1Wl7bLkdv7fto97NPkt+fRNtz3OfUa3Fj+1v98yQPKWuSnJzkeXOs3+L3YZKnjLwOG5Pc1pb/bUvanTTnvMeQ5LtV9ZgFvs8wvD4/Him7FHhtVa1N8lLg96vq17bCfS2pqvu3tJ3FNPoaJVkNfLmqTtkK7b4EWD76xa6tYfS1nGe9pVW1aR7bP7vdz/Pn1cEtMO21eALwz8BlVXXiQvXhoUpyJvDJqvrQtPJ5Pe8LwZH3Q5TkpCSvbcuXJnlrkiuTfDnJL7fyJUn+po2Sr0/yJ638MUkuTvLFJDckOaqV75Pk5iSnAV/kwceoT3c5sEer9+gkZ7T7uWakvUclOa/d97lJrkiyvK37bhvVXAE8M8kftv5fm+QfWt+XJDmzjWZvSPJnre6rk9zU2j2nlf2/EWqSn2mP7/p2vXcrPzPJqUk+n+QrkxptTntuDm73d027/vmR/n4kyYVJbk3ytqnKSV7aXsfPAoeNlM/1uN6T5DPtcf1qez1ubmEwliS7JPlYa/8LSZ7ayk9KcnqSTwNnJVmW5MPt9b4qyWFtu18dGUFek+SxwFuAX25lf7alT+x8VdVdDN9gfmUGM74nWv9f1/7OrkvyllZ25tTfSZK3jPzdvb2Vjb4PD2zP2/VJPppk51Y+4/tzc1q9N7e/gz9NclCSzya5OslFSXZv2z2x/R1dneR/J/mFVn5Me+9cl+Tft9qTOqWqvGzmAnx3hrKTGEY0AJcCf9uWfwP4t7Z8PPCGtrw9sBbYl+EQzR1b+eOBdQzfBt0H+DFw6Cz9uJRhFAjwGuDNbfnNwB+25ccBXwYeDbwW+IdW/mRg00j9An63Lf8i8Algu3b7NOBY4CBgzcj9P65d3wFsP63sJcC72vIngFVt+Y+Aj7XlM4F/YRg07M9w3pmt+hoxHBL6LwynRQDYEVjalp8HfHikv18BdgJ+Cvgaw4fl7sDtwDLgkcBlYz6uc9preBTwHeAp7XFeDRw4y2t5C3Btu+wKvBM4sa1/LnDtyN/a1cAO7fY/A89qy3sDN4/077C2/BiGv7NnM4wkF/v9cjewG7O/J34d+DzwqLZul5Hn9mhgl/Z8Tc0WTP3dncQD78PrgV9tyycDfz/X+3OWvp8JHD1S77S2vF3r37J2+/cYDj0GuBjYry0fAlzSlm8A9hjt79a8bNPHeS+wj7TrqxlCGOAI4KkjI8ydgP0Yvvn55iS/whDWezD8YQN8raq+MMf9fCDJoxlC6hkj9/OCqREIQxjtDTwL+F8AVfWlJNePtHM/8OG2fDhDUF+VBGAH4C6GMPjZJO8EPgV8um1/fevHx4CPzdDHZwK/3ZbPBt42su5jNUwF3ZRkt5+o+dDtkORahuf+amBNK98JWJ1kP4YPrO1G6lxcVfcAJLkJ+BmGD9NLq2pDKz8X+LkxHtcnqqqS3AB8o6puaPVvbH26doY+/0GNTJskeRbwOwBVdUmSXZPs1FafX1Xfb8vPA/ZvrxXAjm2UfRnwd0k+AHykqtaPbLPYpjoy23viecA/VtX3AKpq47T63wF+ALwvyaeATz6o8eF5elxVfbYVrWb4EJ8y0/tzHOe2659nGACtac/pEuDOJI8Bfgn4l5Hnevt2fRlwZpLzRu5/qzG8t5772vX9PPC8BnhVVV00umGGedRlwEFV9aMkX2UIXID/2sz9/AFwHcO/w+9mCJMAv1NVDzqpVuZ+5/6gHpjnDrC6qk6YvlGSpwFHAq8AfpdhxPmbwK8ALwD+KskBm+nz6I6V+0aWt2ayfL+qDmxv4k+2/p4KvAn4TFX99yT7MIymZurL6Os27o6gmR7Xj6e1+2PGf5/NdS6e0b+LRwDPHAnzKW9pwfYbwBcyx46+hZTkZxme37uY/T2xgjme9xq+cHcww0BjJfBKhv9OxjXT+3McU897gBur6pmjK5PsCHy7qg6coc8vS3IIw/vl2iQHVtW35nHfc3LOe7IuAl6eZDuAJD/XRs07AXe14H4Ow4hvbFX1I+ANwKFJfrHdz6umwjrJ09umn2MIXDKcx/wpszR5MXB0hp1LU3OvP5Pk8cAjqurDwF8Bz8hw1MBeVfUZ4HUM0zTTd+Z+nuENBsOHzefm8/i2RBtJvxp4bXvedwL+s61+yRhNXAE8u416twOOGVk36cf1763dqR2N36yq78yw3acZwou27YHt+olVdUNVvZVhOuIXgHuBx27lfo4tyTLgvQxTT8Xs74lPA3+U5FGtfJdp7TwG2Kmq/pVhyvDA0fXtdb97ZD77xcBn2XpuAZYleWbrz3ZJDmivz21JjmnlaQOeqdfjiqr6a4YzG861D2veHHmP51FJ1o/c/rsx672P4V+0L7Zg3QC8EPgA8Ikkaxn+nf6P+Xaoqr6f5G8Z5rVfCfw9cH27n68Cz2eYu17dpkuuYZjuuGeGtm5K8gbg0y2cf8Qwcv0+8I954DCvExj+XfynNsIN8I6q+va0Qf6rgTOS/EV7zC+d7+PbElV1TZLrGIL2bQzPwZ8Dl4xR984kJzHs9LyTYcfxkrZ60o/rJIbn+3rge8CqWbZ7NfDutt1ShtB/GfCaNhi4H7gJuIBh5L+pPR9nVtU7tnKfZzI1hbUdw36Ws3ngPTPje6KqLmwfQmuT/BD4V+AvR9p8LPDxJD/F8Hc3087XVcB72wfAV9iKr09V/bBN9Zza/vaXMrznbmT4wH1Pew9tx7D/4zrgb9p0XRgGSNdtrf6Ahwpu0zL8ktB2VfWDJE9k+AP6uRp+mEJSxxx5b9seBXym/Ysa4OUGt7RtcOQtSR1yh6UkdcjwlqQOGd6S1CHDW5pDkkpy9sjtpUk2JPnkXPVmaOer7bj5LdpGmmJ4S3P7L+DJSXZot3+NB770Iy0aw1vavAsYvuIM8CLgg1MrMvuZAHdN8ukMZ/f7B0a++p4ZzuC4kA9G2wbDW9q8c4CV7dt9T2X4Cv2UNwLXVNVTGb4ReFYrPxH4XFU9HTif4URhtNMZ/B7D2f8OZPg25B8sxIPQtsUv6UibUVXXt5NavYjha9ujZjsT4K/QzkBYVZ9KcnfbfrYzOErzYnhL4zkfeDvD+bF3HSmf60yAM30DbtYzOErz4bSJNJ4zgJOnztM9YrYzAY6W/zqwc9t+xjM4Trz32uY48pbGUFXraT9sMc1JzHwmwDcCH0zyRYZTk97e2pntDI5fm+wj0LbGc5tIUoecNpGkDhnektQhw1uSOmR4S1KHDG9J6pDhLUkdMrwlqUP/F/xh/rIRbdvPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot results\n",
    "sns.factorplot(x= 'Model', y= 'RMSE_mean', data= model_comparision, kind='bar', legend='True')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Observations:__\n",
    "\n",
    "We can clearly see that Random Forest outperformed Linear Regression and Decision Trees. The Root Mean Square Error for all all 10 cross validations, Random Forest has the lowest error. Hence we choose Random Forest as our desired model for predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> In the next section we optimise our Random Forest to perform much better in prediction."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}