creditcard_fraud_detection_classification_Tree.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Credit Card Fraud Detection using Scikit-Learn**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "After completing this lab you will be able to:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Perform basic data preprocessing in Python\n",
    "* Model a classification task using the Scikit-Learn and Snap ML Python APIs\n",
    "* Train Suppport Vector Machine and Decision Tree models using Scikit-Learn and Snap ML\n",
    "* Run inference and assess the quality of the trained models\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 10px\">\n",
    "    <ol>\n",
    "        <li><a href=\"#introduction\">Introduction</a></li>\n",
    "        <li><a href=\"#import_libraries\">Import Libraries</a></li>\n",
    "        <li><a href=\"#dataset_analysis\">Dataset Analysis</a></li>\n",
    "        <li><a href=\"#dataset_preprocessing\">Dataset Preprocessing</a></li>\n",
    "        <li><a href=\"#dataset_split\">Dataset Train/Test Split</a></li>\n",
    "        <li><a href=\"#dt_sklearn\">Build a Decision Tree Classifier model with Scikit-Learn</a></li>\n",
    "        <li><a href=\"#dt_snap\">Build a Decision Tree Classifier model with Snap ML</a></li>\n",
    "        <li><a href=\"#dt_sklearn_snap\">Evaluate the Scikit-Learn and Snap ML Decision Tree Classifiers</a></li>\n",
    "        <li><a href=\"#svm_sklearn\">Build a Support Vector Machine model with Scikit-Learn</a></li>\n",
    "        <li><a href=\"#svm_snap\">Build a Support Vector Machine model with Snap ML</a></li>\n",
    "        <li><a href=\"#svm_sklearn_snap\">Evaluate the Scikit-Learn and Snap ML Support Vector Machine Models</a></li>\n",
    "    </ol>\n",
    "</div>\n",
    "<br>\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<div id=\"Introduction\">\n",
    "    <h2>Introduction</h2>\n",
    "    <br>Imagine that you work for a financial institution and part of your job is to build a model that predicts if a credit card transaction is fraudulent or not. You can model the problem as a binary classification problem. A transaction belongs to the positive class (1) if it is a fraud, otherwise it belongs to the negative class (0).\n",
    "    <br>\n",
    "    <br>You have access to transactions that occured over a certain period of time. The majority of the transactions are normally legitimate and only a small fraction are non-legitimate. Thus, typically you have access to a dataset that is highly unbalanced. This is also the case of the current dataset: only 492 transactions out of 284,807 are fraudulent (the positive class - the frauds - accounts for 0.172% of all transactions).\n",
    "    <br>\n",
    "    <br>To train the model you can use part of the input dataset and the remaining data can be used to assess the quality of the trained model. First, let's download the dataset.\n",
    "    <br>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting opendatasets\n",
      "  Downloading opendatasets-0.1.22-py3-none-any.whl (15 kB)\n",
      "Requirement already satisfied: tqdm in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from opendatasets) (4.60.0)\n",
      "Collecting kaggle (from opendatasets)\n",
      "  Downloading kaggle-1.6.4.tar.gz (84 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.6/84.6 kB\u001b[0m \u001b[31m16.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: click in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from opendatasets) (8.1.3)\n",
      "Requirement already satisfied: importlib-metadata in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from click->opendatasets) (4.11.4)\n",
      "Requirement already satisfied: six>=1.10 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (1.16.0)\n",
      "Requirement already satisfied: certifi in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (2023.5.7)\n",
      "Requirement already satisfied: python-dateutil in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (2.8.2)\n",
      "Requirement already satisfied: requests in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (2.29.0)\n",
      "Collecting python-slugify (from kaggle->opendatasets)\n",
      "  Downloading python_slugify-8.0.3-py2.py3-none-any.whl (10 kB)\n",
      "Requirement already satisfied: urllib3 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (1.26.15)\n",
      "Requirement already satisfied: bleach in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from kaggle->opendatasets) (6.0.0)\n",
      "Requirement already satisfied: webencodings in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from bleach->kaggle->opendatasets) (0.5.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from importlib-metadata->click->opendatasets) (3.15.0)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from importlib-metadata->click->opendatasets) (4.5.0)\n",
      "Collecting text-unidecode>=1.3 (from python-slugify->kaggle->opendatasets)\n",
      "  Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.2/78.2 kB\u001b[0m \u001b[31m16.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->kaggle->opendatasets) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from requests->kaggle->opendatasets) (3.4)\n",
      "Building wheels for collected packages: kaggle\n",
      "  Building wheel for kaggle (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for kaggle: filename=kaggle-1.6.4-py3-none-any.whl size=111929 sha256=77d07d515ce22d1b62302297e33bcd53bb0754a114fa065bd06a936fed40fcce\n",
      "  Stored in directory: /home/jupyterlab/.cache/pip/wheels/dd/28/b3/80b05d2bcc90c29c64fa4beec3102485ee3c90075eef8e12a9\n",
      "Successfully built kaggle\n",
      "Installing collected packages: text-unidecode, python-slugify, kaggle, opendatasets\n",
      "Successfully installed kaggle-1.6.4 opendatasets-0.1.22 python-slugify-8.0.3 text-unidecode-1.3\n"
     ]
    }
   ],
   "source": [
    "# install the opendatasets package\n",
    "!pip install opendatasets\n",
    "\n",
    "import opendatasets as od\n",
    "\n",
    "# download the dataset (this is a Kaggle dataset)\n",
    "od.download(\"https://www.kaggle.com/mlg-ulb/creditcardfraud\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Did you know?__ When it comes to Machine Learning, you will most likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](https://ibm.biz/BdPfxf)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"import_libraries\">\n",
    "    <h2>Import Libraries</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting snapml\n",
      "  Downloading snapml-1.14.0-cp37-cp37m-manylinux2014_x86_64.whl (7.4 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m67.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: scikit-learn in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from snapml) (0.20.1)\n",
      "Requirement already satisfied: scipy in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from snapml) (1.7.3)\n",
      "Requirement already satisfied: numpy>=1.18.5 in /home/jupyterlab/conda/envs/python/lib/python3.7/site-packages (from snapml) (1.21.6)\n",
      "Installing collected packages: snapml\n",
      "Successfully installed snapml-1.14.0\n"
     ]
    }
   ],
   "source": [
    "# Snap ML is available on PyPI. To install it simply run the pip command below.\n",
    "!pip install snapml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import the libraries we need to use in this lab\n",
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import normalize, StandardScaler\n",
    "from sklearn.utils.class_weight import compute_sample_weight\n",
    "from sklearn.metrics import roc_auc_score\n",
    "import time\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"dataset_analysis\">\n",
    "    <h2>Dataset Analysis</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section you will read the dataset in a Pandas dataframe and visualize its content. You will also look at some data statistics. \n",
    "\n",
    "Note: A Pandas dataframe is a two-dimensional, size-mutable, potentially heterogeneous tabular data structure. For more information: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# read the input data\n",
    "raw_data = pd.read_csv('creditcard.csv')\n",
    "print(\"There are \" + str(len(raw_data)) + \" observations in the credit card fraud dataset.\")\n",
    "print(\"There are \" + str(len(raw_data.columns)) + \" variables in the dataset.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 284807 observations in the credit card fraud dataset.\n",
      "There are 31 variables in the dataset.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.191857</td>\n",
       "      <td>0.266151</td>\n",
       "      <td>0.166480</td>\n",
       "      <td>0.448154</td>\n",
       "      <td>0.060018</td>\n",
       "      <td>-0.082361</td>\n",
       "      <td>-0.078803</td>\n",
       "      <td>0.085102</td>\n",
       "      <td>-0.255425</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.225775</td>\n",
       "      <td>-0.638672</td>\n",
       "      <td>0.101288</td>\n",
       "      <td>-0.339846</td>\n",
       "      <td>0.167170</td>\n",
       "      <td>0.125895</td>\n",
       "      <td>-0.008983</td>\n",
       "      <td>0.014724</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.358354</td>\n",
       "      <td>-1.340163</td>\n",
       "      <td>1.773209</td>\n",
       "      <td>0.379780</td>\n",
       "      <td>-0.503198</td>\n",
       "      <td>1.800499</td>\n",
       "      <td>0.791461</td>\n",
       "      <td>0.247676</td>\n",
       "      <td>-1.514654</td>\n",
       "      <td>...</td>\n",
       "      <td>0.247998</td>\n",
       "      <td>0.771679</td>\n",
       "      <td>0.909412</td>\n",
       "      <td>-0.689281</td>\n",
       "      <td>-0.327642</td>\n",
       "      <td>-0.139097</td>\n",
       "      <td>-0.055353</td>\n",
       "      <td>-0.059752</td>\n",
       "      <td>378.66</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.966272</td>\n",
       "      <td>-0.185226</td>\n",
       "      <td>1.792993</td>\n",
       "      <td>-0.863291</td>\n",
       "      <td>-0.010309</td>\n",
       "      <td>1.247203</td>\n",
       "      <td>0.237609</td>\n",
       "      <td>0.377436</td>\n",
       "      <td>-1.387024</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.108300</td>\n",
       "      <td>0.005274</td>\n",
       "      <td>-0.190321</td>\n",
       "      <td>-1.175575</td>\n",
       "      <td>0.647376</td>\n",
       "      <td>-0.221929</td>\n",
       "      <td>0.062723</td>\n",
       "      <td>0.061458</td>\n",
       "      <td>123.50</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>-1.158233</td>\n",
       "      <td>0.877737</td>\n",
       "      <td>1.548718</td>\n",
       "      <td>0.403034</td>\n",
       "      <td>-0.407193</td>\n",
       "      <td>0.095921</td>\n",
       "      <td>0.592941</td>\n",
       "      <td>-0.270533</td>\n",
       "      <td>0.817739</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.009431</td>\n",
       "      <td>0.798278</td>\n",
       "      <td>-0.137458</td>\n",
       "      <td>0.141267</td>\n",
       "      <td>-0.206010</td>\n",
       "      <td>0.502292</td>\n",
       "      <td>0.219422</td>\n",
       "      <td>0.215153</td>\n",
       "      <td>69.99</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n",
       "2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n",
       "3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n",
       "4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n",
       "\n",
       "         V8        V9  ...       V21       V22       V23       V24       V25  \\\n",
       "0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "1  0.085102 -0.255425  ... -0.225775 -0.638672  0.101288 -0.339846  0.167170   \n",
       "2  0.247676 -1.514654  ...  0.247998  0.771679  0.909412 -0.689281 -0.327642   \n",
       "3  0.377436 -1.387024  ... -0.108300  0.005274 -0.190321 -1.175575  0.647376   \n",
       "4 -0.270533  0.817739  ... -0.009431  0.798278 -0.137458  0.141267 -0.206010   \n",
       "\n",
       "        V26       V27       V28  Amount  Class  \n",
       "0 -0.189115  0.133558 -0.021053  149.62      0  \n",
       "1  0.125895 -0.008983  0.014724    2.69      0  \n",
       "2 -0.139097 -0.055353 -0.059752  378.66      0  \n",
       "3 -0.221929  0.062723  0.061458  123.50      0  \n",
       "4  0.502292  0.219422  0.215153   69.99      0  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In practice, a financial institution may have access to a much larger dataset of transactions. To simulate such a case, we will inflate the original one 10 times.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 2848070 observations in the inflated credit card fraud dataset.\n",
      "There are 31 variables in the dataset.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "1   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "2   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "3   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "4   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "\n",
       "         V8        V9  ...       V21       V22       V23       V24       V25  \\\n",
       "0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "1  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "2  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "3  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "4  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "\n",
       "        V26       V27       V28  Amount  Class  \n",
       "0 -0.189115  0.133558 -0.021053  149.62    0.0  \n",
       "1 -0.189115  0.133558 -0.021053  149.62    0.0  \n",
       "2 -0.189115  0.133558 -0.021053  149.62    0.0  \n",
       "3 -0.189115  0.133558 -0.021053  149.62    0.0  \n",
       "4 -0.189115  0.133558 -0.021053  149.62    0.0  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_replicas = 10\n",
    "\n",
    "# inflate the original dataset\n",
    "big_raw_data = pd.DataFrame(np.repeat(raw_data.values, n_replicas, axis=0), columns=raw_data.columns)\n",
    "\n",
    "print(\"There are \" + str(len(big_raw_data)) + \" observations in the inflated credit card fraud dataset.\")\n",
    "print(\"There are \" + str(len(big_raw_data.columns)) + \" variables in the dataset.\")\n",
    "\n",
    "# display first rows in the new dataset\n",
    "big_raw_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each row in the dataset represents a credit card transaction. As shown above, each row has 31 variables. One variable (the last variable in the table above) is called Class and represents the target variable. Your objective will be to train a model that uses the other variables to predict the value of the Class variable. Let's first retrieve basic statistics about the target variable.\n",
    "\n",
    "Note: For confidentiality reasons, the original names of most features are anonymized V1, V2 .. V28. The values of these features are the result of a PCA transformation and are numerical. The feature 'Class' is the target variable and it takes two values: 1 in case of fraud and 0 otherwise. For more information about the dataset please visit this webpage: https://www.kaggle.com/mlg-ulb/creditcardfraud.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGZCAYAAABmNy2oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+G0lEQVR4nO3dd3gU1f4G8He2p/cOhIQSeuhIM6FIt1AEETVIVK5YriCIev1dy/UqiAUQFEWK1AsoggWQXhSk99AhBAjpvW+Z3x+BkZAEEthkdmffz/PkCTt7dva7s2HfnXPOzAiiKIogIiICoJK7ACIish0MBSIikjAUiIhIwlAgIiIJQ4GIiCQMBSIikjAUiIhIwlAgIiIJQ4GIiCQMhWoQBKFKP9u3b5e71DLi4uLw3nvvIT4+/q5tBw8eDCcnJ2RlZVXaZtSoUdBqtUhOTr7v2uLj4yEIAhYuXFjtx27fvh2CIOCHH364a9v33nsPgiDcQ4XljR8/HoIg4PTp05W2+de//gVBEHDo0KEqr7d+/foYPXq0FSq8N8nJyXjzzTfRsmVLuLq6wmAwoFGjRvjnP/+Jc+fOyVbXrXbv3o333nvvjn+fdH8YCtWwZ8+eMj8DBgyAk5NTueVt27aVu9Qy4uLi8P7771cpFGJjY1FUVIRly5ZVeH92djZ++uknDBo0CAEBAfddW1BQEPbs2YOBAwfe97pqS2xsLABg/vz5Fd5vsViwaNEitG7d2ub+Fiqzb98+tGzZEvPmzcOwYcOwevVqbNiwARMnTsShQ4fQsWNHuUsEUBoK77//PkOhBmnkLsCePPDAA2Vu+/n5QaVSlVt+rwoKCuDs7GyVdd2r/v37Izg4GPPnz8e4cePK3b98+XIUFhZKH4z3ymw2w2QyQa/XW2371ZYWLVqgY8eOWLx4MT766CNoNGX/G23cuBFXr17F5MmTZaqwenJycvDoo4/CYDBg9+7dqFOnjnRfdHQ0xo4dW6W9MVIG7ilY2ezZs/Hggw/C398fLi4uaNmyJT755BMYjcYy7aKjo9GiRQvs3LkTXbp0gbOzM8aMGQMAuHr1KoYNGwY3Nzd4enpi1KhR2L9/f4XdLAcOHMAjjzwCb29vGAwGtGnTBitXrpTuX7hwIR5//HEAQI8ePaQursq6a9RqNWJiYnDw4EEcP3683P0LFixAUFAQ+vfvj9TUVIwbNw7NmjWDq6sr/P390bNnT+zatavMY252EX3yySf48MMPERYWBr1ej23btlXYfXT+/Hk8++yzaNSoEZydnRESEoKHH364wnoAoKioCBMmTEBgYCCcnJwQFRWFw4cPV9j2ditWrEDnzp3h4uICV1dX9O3bt0qPjY2NRVJSEtavX1/hNtLr9Rg1ahSKiorw+uuvo3Xr1vDw8IC3tzc6d+6MtWvX3vU5Fi5cCEEQyu3h3ew2u72bcvPmzejVqxfc3d3h7OyMrl27YsuWLXd9nrlz5yIpKQmffPJJmUC41bBhw8rc/vnnn9G5c2c4OzvDzc0NDz30EPbs2VOmzejRo1G/fv1y66qoK08QBLz88stYvHgxmjZtCmdnZ0RGRuLXX38t87hJkyYBAMLCwsp1127duhXR0dHw8fGBk5MT6tWrh6FDh6KgoOCu24D+xlCwsgsXLuDJJ5/E4sWL8euvvyI2NhbTpk3D2LFjy7W9fv06nnrqKTz55JNYt24dxo0bh/z8fPTo0QPbtm3D1KlTsXLlSgQEBGDEiBHlHr9t2zZ07doVWVlZmDNnDtauXYvWrVtjxIgR0ofswIED8dFHHwEoDaybXVx36q4ZM2YMBEEo1z0SFxeHffv2ISYmBmq1GhkZGQCAd999F7/99hsWLFiA8PBwREdHVziuMnPmTGzduhWffvop1q9fjyZNmlT4/ImJifDx8cGUKVOwYcMGzJ49GxqNBp06dcKZM2fKtX/77bdx8eJFfPfdd/juu++QmJiI6OhoXLx4sdLXCAAfffQRRo4ciWbNmmHlypVYvHgxcnNz0b17d8TFxd3xsSNHjoSzs3O5bZSZmYm1a9di8ODB8PLyQnFxMTIyMjBx4kSsWbMGy5cvR7du3TBkyBAsWrTojs9RHUuWLEGfPn3g7u6O77//HitXroS3tzf69u1712DYuHEj1Go1Hn744So917Jly/Doo4/C3d0dy5cvx7x585CZmYno6Gj88ccf9/wafvvtN8yaNQsffPABfvzxR3h7e2Pw4MHS+/jcc8/hlVdeAQCsXr26THdtfHw8Bg4cCJ1Oh/nz52PDhg2YMmUKXFxcUFJScs81OSSR7llMTIzo4uJS6f1ms1k0Go3iokWLRLVaLWZkZEj3RUVFiQDELVu2lHnM7NmzRQDi+vXryywfO3asCEBcsGCBtKxJkyZimzZtRKPRWKbtoEGDxKCgINFsNouiKIqrVq0SAYjbtm2r8muLiooSfX19xZKSEmnZ66+/LgIQz549W+FjTCaTaDQaxV69eomDBw+Wll+6dEkEIDZo0KDM+m6979bXVdF6S0pKxEaNGonjx4+Xlm/btk0EILZt21a0WCzS8vj4eFGr1YrPPfectOzdd98Vb/1zT0hIEDUajfjKK6+Uea7c3FwxMDBQHD58eKX13BQTEyNqtVoxOTlZWvbll1+KAMRNmzZV+lqMRqMYGxsrtmnTpsx9oaGhYkxMjHR7wYIFIgDx0qVLZdrdfN0338/8/HzR29tbfPjhh8u0M5vNYmRkpNixY8c7vo4mTZqIgYGBd3m1f68zODhYbNmypfT3JYql283f31/s0qWLtCwmJkYMDQ0tt47b3wtRFEUAYkBAgJiTkyMtS0pKElUqlfjxxx9Ly6ZNm1bhNvnhhx9EAOKRI0eq9DqoctxTsLLDhw/jkUcegY+PD9RqNbRaLZ555hmYzWacPXu2TFsvLy/07NmzzLIdO3bAzc0N/fr1K7N85MiRZW6fP38ep0+fxqhRowAAJpNJ+hkwYACuX79e4bfqqoqNjUVaWhp+/vlnaf1LlixB9+7d0ahRI6ndnDlz0LZtWxgMBmg0Gmi1WmzZsgWnTp0qt85HHnkEWq32rs9tMpnw0UcfoVmzZtDpdNBoNNDpdDh37lyF633yySfLdEeEhoaiS5cu2LZtW6XP8fvvv8NkMuGZZ54ps+0MBgOioqKqNIMsNjYWRqMRixcvlpYtWLAAoaGh6NWrl7Rs1apV6Nq1K1xdXaVtNG/evApfy73YvXs3MjIyEBMTU+a1WCwW9OvXD/v370d+fr5VnuvMmTNITEzE008/DZXq748PV1dXDB06FH/99dc9d9f06NEDbm5u0u2AgAD4+/vj8uXLd31s69atodPp8MILL+D777+/614iVY6hYEUJCQno3r07rl27hhkzZmDXrl3Yv38/Zs+eDQAoLCws0z4oKKjcOtLT0yuc1XP7spvTQSdOnAitVlvm5+YAcVpa2j2/lmHDhsHDwwMLFiwAAKxbtw7JycllBpg///xzvPjii+jUqRN+/PFH/PXXX9i/fz/69etX7rVW9norMmHCBPzf//0fHnvsMfzyyy/Yu3cv9u/fj8jIyArXGxgYWOGy9PT0Sp/j5vbr0KFDue23YsWKKm277t27o3HjxtI2OnbsGA4dOoRnn31WCqnVq1dj+PDhCAkJwZIlS7Bnzx7s378fY8aMQVFRUZW2x93cfC3Dhg0r91qmTp0KURSlrr6K1KtXD6mpqVUKjpvbtKL3Mjg4GBaLBZmZmff0Onx8fMot0+v1Fb7nt2vQoAE2b94Mf39/vPTSS2jQoAEaNGiAGTNm3FMtjoyzj6xozZo1yM/Px+rVqxEaGiotP3LkSIXtK5o37+Pjg3379pVbnpSUVOa2r68vAOCtt97CkCFDKlx/REREVUsvx8nJCSNHjsTcuXNx/fp1zJ8/H25ubtKgNVDajx0dHY2vv/66zGNzc3MrXGdVjxNYsmQJnnnmGWks5Ka0tDR4enqWa3/7trm5rKIPmZtubr8ffvihzHtVXWPGjMGbb76Jffv2YdmyZVCpVGWONViyZAnCwsKwYsWKMq+/uLj4rus2GAwVtr09sG6+li+//LLSmVx3mj7ct29fbNy4Eb/88gueeOKJO9Z0c5tev3693H2JiYlQqVTw8vKS6q/odd7Pl5U76d69O7p37w6z2YwDBw7gyy+/xGuvvYaAgIC7vi76G/cUrOjmf3q9Xi8tE0URc+fOrfI6oqKikJubW25Wy//+978ytyMiItCoUSMcPXoU7du3r/Dn5q74zXqq8o3rVrGxsTCbzZg2bRrWrVuHJ554osyUWUEQyrxWoPTb8u2zUKqrovX+9ttvuHbtWoXtly9fDvGWq8pevnwZu3fvRnR0dKXP0bdvX2g0Gly4cKHS7VcVMTEx0Gg0+Oabb7B06VL06tWrTMgIggCdTlcmEJKSkqo0++jmzJ1jx46VWX6zS++mrl27wtPTE3FxcZW+Fp1OV+nzxMbGIjAwEG+88Ual23j16tUASv/uQkJCsGzZsjLbPD8/Hz/++KM0I+lm/SkpKWUOciwpKcHvv/9+19demar8LavVanTq1EnaQ6/OAYTEPQWreuihh6DT6TBy5Ei88cYbKCoqwtdff12t3emYmBh88cUXeOqpp/Dhhx+iYcOGWL9+vfQf6dZ+3G+++Qb9+/dH3759MXr0aISEhCAjIwOnTp3CoUOHsGrVKgCl8+oB4Ntvv4WbmxsMBgPCwsLu+E0aANq3b49WrVph+vTpEEWx3LEJgwYNwn/+8x+8++67iIqKwpkzZ/DBBx8gLCwMJpOpyq/5doMGDcLChQvRpEkTtGrVCgcPHsS0adMqnS6ZkpKCwYMH4/nnn0d2djbeffddGAwGvPXWW5U+R/369fHBBx/gX//6Fy5evIh+/frBy8sLycnJ2LdvH1xcXPD+++/ftdbAwEAMGDAACxYsqHQbrV69GuPGjcOwYcNw5coV/Oc//0FQUNBdjxLu0KEDIiIiMHHiRJhMJnh5eeGnn34qN8PH1dUVX375JWJiYpCRkYFhw4bB398fqampOHr0KFJTU8vtzd3Kw8MDa9euxaBBg9CmTRu8/PLL6Ny5szSOs2TJEhw9ehRDhgyBSqXCJ598glGjRmHQoEEYO3YsiouLMW3aNGRlZWHKlCnSekeMGIF///vfeOKJJzBp0iQUFRVh5syZMJvNd92ulWnZsiUAYMaMGYiJiYFWq0VERASWLl2KrVu3YuDAgahXrx6KioqkmWG9e/e+5+dzSHKOctu7imYf/fLLL2JkZKRoMBjEkJAQcdKkSeL69evLzf6JiooSmzdvXuF6ExISxCFDhoiurq6im5ubOHToUHHdunUiAHHt2rVl2h49elQcPny46O/vL2q1WjEwMFDs2bOnOGfOnDLtpk+fLoaFhYlqtfqus31uNWPGDBGA2KxZs3L3FRcXixMnThRDQkJEg8Egtm3bVlyzZk25WSc3ZxhNmzat3Doqmn2UmZkpxsbGiv7+/qKzs7PYrVs3cdeuXWJUVJQYFRUltbs5C2fx4sXiq6++Kvr5+Yl6vV7s3r27eODAgTLPU9GMF1EUxTVr1og9evQQ3d3dRb1eL4aGhorDhg0TN2/eXKXtI4qiuHbtWhGA6O3tLRYVFZW7f8qUKWL9+vVFvV4vNm3aVJw7d26F9dw++0gURfHs2bNinz59RHd3d9HPz0985ZVXxN9++63C2WQ7duwQBw4cKHp7e4tarVYMCQkRBw4cKK5atapKryMpKUmcPHmy2Lx5c9HZ2VnU6/Viw4YNxbFjx4rHjx8v03bNmjVip06dRIPBILq4uIi9evUS//zzz3LrXLdundi6dWvRyclJDA8PF2fNmlXp7KOXXnqp3OMr2iZvvfWWGBwcLKpUKmk77NmzRxw8eLAYGhoq6vV60cfHR4yKihJ//vnnKr12+psgirfsA5LN+uijj/DOO+8gISGh0m/MRET3i91HNmjWrFkAgCZNmsBoNGLr1q2YOXMmnnrqKQYCEdUohoINcnZ2xhdffIH4+HgUFxejXr16mDx5Mt555x25SyMihWP3ERERSTgllYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIglDgYiIJAwFIiKSMBSIiEjCUCAiIolG7gKIakKJyYLUvGKk5hYjJadI+nd+sQkmiwiLRYTJIsJ8yw8AqFUCNGoVtGoBGtWN32oBeo0avq56BLjrEeBugL+7Hr4ueqhUgsyvlMi6GApkVywWEZczCnA9uxCpuTc+9KXfRdLt7EIjRLFma9GoBCko/N0Npb/dDH/fdjMg0MMAbxddzRZCZEWCKNb0fx2ie2MyW3AuJQ8nrmXjZGIOTlzLxqnrOcgvMctdWrX4uOjQIsQDLUM80LJO6e9gTye5yyKqEEOBbEKxyYwzSbk4cS0HJxKzcfJaNk4n5aLYZJG7tBrh61oaFK1CPNCyjidahngg0MMgd1lEDAWSx+X0fOw8l4bjV7Nw4loOzqXkwmh27D9FPzd96d7EjZ8O9b3h4ayVuyxyMAwFqhVmi4iDlzOx5XQytpxKwfmUPLlLsnkalYD29b3Qu2kA+jQLRD0fZ7lLIgfAUKAak1tkxI6zqdhyKgXbz6Qgs8Aod0l2rXGAK3o3DUDvZgFoU9cTgsCZT2R9DAWyqsvp+dh8KgVbTiVjf3yGw3cJ1RQ/Nz16NfFH76YB6NbIFwatWu6SSCEYCnTfDl7OwMaTydhymt1CcnDSqtG1oS/6NAtAz6b+8HXVy10S2TGGAt2TtLxi/HDwKlbuv4KLaflyl0M3qASgWyM/PNmxLno3DYBGzZMWUPUwFKjKLBYRO86m4n/7E7D1dAq7hmycn5sej7erg5Ed66GuNwepqWoYCnRXKblFWLY3ASv3X0FidpHc5VA1CQLQraEvRnashz7NuPdAd8ZQoEodvZKFBX9ewrrjSSgxK/MgMkcT7GHA053r48mO9XgMBFWIoUBlGM0WrDt+HQv+jMeRK1lyl0M1xEmrxuC2IRjTtT4a+rvJXQ7ZEIYCASg9zcSSvxLw7c4LSM4plrscqiU3u5ZejGqALg195S6HbABDwcGZLSJ+PHQVMzafw7WsQrnLIRl1a+iLN/s3QYsQD7lLIRkxFBzYhhPX8enGszy2gCSCAAxsGYRJfSMQ6uMidzkkA4aCA9p9IQ2fbDjDMQOqlFYt4IkO9fBqr0bwc+PBcI6EoeBAjl/Nxie/n8auc2lyl0J2wlmnxnPdwvBCVAO46nlNLkfAUHAAF1Lz8NnGM1h/IqnGr0ZGyuTjosNLPRriqQdCodPwOAclYygo2PXsQkzfdA4/HLoqXYOY6H7U9XbChIca49HIEF6fWqEYCgpksYiY98clfLbpDIqMPOiMrK9pkDs+eLQ5OtT3lrsUsjKGgsKcT8nDpB+O4nBCltylkMKpBODZrmGY1DeCp+5WEIaCQpgtIr7deRHTN59V7HWNyTaF+7pg2uORaBfqJXcpZAUMBQU4m5yLST8cw1FOMSWZqAQgtlsYXu/DvQZ7x1CwYyazBd/svIgZW86hhHsHZAMa+Lng08cj0aYe9xrsFUPBTp1OysGkVcdw/Fq23KUQlaFWCXi+ezjGP9QIeg33GuwNQ8HOmMwWfLX9AmZtPc/TWZNNa+Tvis+GR6JVHU+5S6FqYCjYkbjEHEz64ShOJubIXQpRlahVAsY+GI7XejfmQW92gqFgJ5btTcC7P5/gJTDJLjUJdMPMkW3QOIDXbrB1DAUbZzRb8N7PJ7F0b4LcpRDdF1e9BtNHtEbvZgFyl0J3wFCwYel5xXhx6SHsu5QhdylEVqESgIl9IzAuuqHcpVAlGAo26mRiNl5YdJAXviFFerR1MKYObcVjGmwQQ8EG/XosEZNWHUOh0Sx3KUQ1JrKOB759pj0C3A1yl0K3YCjYEFEU8enGM5i97YLcpRDVigB3Pb59uj0i63rKXQrdwFCwEXnFJrz2v8PYfCpF7lKIapVeo8LUoa3wWJsQuUshMBRsQnxaPp5fdADneK1kcmD/iGqAN/pG8DoNMmMoyGzn2VS8svwwsguNcpdCJLteTfwxY2QbXvpTRgwFGa3Yn4C3fzrBq6IR3aKRvyvmj+6Aut7OcpfikBgKMln45yW8/2scr5lMVIFAdwOWPd8J4X6ucpficBgKMvh6+wVM3XBa7jKIbJqfmx5Ln+vEU2PUMoZCLft801nM3HJO7jKI7IK3iw6LYzuiebCH3KU4DIZCLfpo3Sl8u/Oi3GUQ2RUPJy0Wx3bkKbhrCUOhlrz380ks3B0vdxlEdsnNoMHCZzvyOtC1gCc4rwX//S2OgUB0H3KLTBg9fx+vQ14LGAo1bNrvpzF31yW5yyCye7nFJjwzfx/ieJGpGsVQqEHTN5/leYyIrCi70Iin5+3FueRcuUtRLIZCDZm97Tymb+YsIyJrS88vwajv9uJSWr7cpSgSQ6EGfL87HtN+PyN3GUSKlZJbjFFz/8KVjAK5S1EchoKV7Tibig9+jZO7DCLFS8wuQsz8fcgp4nnDrImhYEUXUvPw8rJDPJcRUS25mJaPV5YdhoX/56yGoWAl2QVGPPf9AeQWmeQuhcih7Dibiik8bYzVMBSswGS24KVlhzjwRSSTb3dexOpDV+UuQxEYClbwwa9x+ON8mtxlEDm0t1YfxxEe3HbfGAr3aeney1i057LcZRA5vGKTBWMXH0ByTpHcpdg1hsJ92HMhHe/9fFLuMojohuScYryw+CCKjGa5S7FbDIV7lJBegHFLD8Jo5qwHIlty9EoW3l59XO4y7BZD4R7kFZsQ+/1+ZBZwfjSRLVp9+Bq+2cFTzNwLhkI1WSwiXl1+GOdS8uQuhYjuYOqG09h2JkXuMuwOQ6Gapm08g62n+YdGZOssIvDq8sO4kMovcNXBUKiGfZcyuEtKZEdyi0wYy4HnamEoVFFBiQmTfjgKHk1PZF/Op+Thi01n5S7DbjAUqmjK+tO4nM4zMhLZo+/+uITDCZlyl2EXGApVsPt8Ghb/xQPUiOyV2SJi4qqj7EaqAobCXeQVmzDph2MQ2W1EZNcupOazG6kKGAp38d/f4nAtq1DuMojICtiNdHcMhTvYfiYFy/ddkbsMIrISdiPdHUOhEtmFRrz5Iw+VJ1IadiPdGUOhEh/8Eocknm2RSJHYjVQ5hkIFNscl40desINIsdiNVDmGwm2yCkrw1k/sNiJSOnYjVYyhcJv3f4lDam6x3GUQUS1gN1J5DIVbHL2ShTVHrsldBhHVErNFxJs/HoeF56+RMBRuMWX9aR6kRuRgziTncgzxFgyFG7adScGei+lyl0FEMpi++RyKTRx0BhgKAEovnDN1/Wm5yyAimVzLKsTiPTy/GcBQAACsOXINp5Ny5S6DiGT01fYLyC3iJXYdPhSKTWZ8tpHT0ogcXUZ+CebuvCh3GbJz+FBYvOcyT3hHRABKp6im5Tn2lHSHDoWcIiNmbzsvdxlEZCMKSsz4css5ucuQlUOHwtfbLyCzgH2IRPS3ZfsScCXDca+y6LChkJxThAV/XpK7DCKyMUaziM82npG7DNk4bCh8seksiowWucsgIhu09mgi4hJz5C5DFg4ZCudTcrHqII9gJKKKiSIw7XfHPHbJIUPhi03nYOa5TojoDradScVeBzzLgcOFwpWMAmw4mSR3GURkB77c6nizEx0uFL7fHc+9BCKqkj/Op+FssmOd7cChQiG/2IQVB67IXQYR2ZEFf8bLXUKtcqhQWHngCnKLTHKXQUR2ZM3ha8gqKJG7jFrjMKFgsYhYuDte7jKIyM4UGs34337H6WFwmFDYcjoFl9Md9yhFIrp3i/dcdpixSIcJhUV74uUugYjs1LWsQvzuILMWHSIUEtIL8Mf5NLnLICI7tmxvgtwl1AqHCIXl+xN47WUiui9/XkhDggN0QSs+FIxmC1Yd4CktiOj+iCKw4oDy9xYUHwqb4pId/qIZRGQdqw5chcms7BNpKj4UHKUfkIhqXkpuMbaeTpG7jBql6FBISC/Anxc4wExE1qP0YxYUHQrrTlznADMRWdWOs6lIySmSu4wao+hQ2BSXLHcJRKQwZouIzaeU24Wk2FBIyyvG4YRMucsgIgXackq5XzgVGwpbTiXDQY5KJ6Ja9ueFNBSWmOUuo0YoNhTYdURENaXIaFHsWRIUGQqFJWbFvmFEZBs2K/SLpyJDYee5VBQZlX2ACRHJa+uZFIgKnN6oyFBg1xER1bTU3GIcuZIldxlWp7hQsFhExR9xSES2YYsCp6YqLhQOJmQiI99xLp1HRPLZrMCpqYoLBXYdEVFtOZ2Ui6uZyjqdNkOBiOg+KG0WkqJC4XxKLi6l5ctdBhE5kC0KG8NUVCgo+XwkRGSb9l7MQF6xSe4yrEZRoXDoMs91RES1q8RswW4FHSyrqFA4fi1b7hKIyAEp6bNHMaGQmluM69nKPcc5EdkuhoINOn4tS+4SiMhBnWAo2J6jV5TzphCRfUnLK8H17EK5y7AKxYSCknbfiMj+HL+qjM8gxYTCMYW8IURkn04k5shdglUoIhQSswqRllcsdxlE5MCUMq6giFDgXgIRyU0pXdiKCAXOPCIiuaXmFiM5x/6nxSsiFLinQES2QAmDzYoIBaXsthGRfTuRaP+fRXYfCgnpBcgqMMpdBhGRIgab7T4UuJdARLZCCZ9Hdh8KCRnKuuoREdmv5JxipNv59Hi7DwUljPYTkXLY+4k5GQpERFaUkmvfn0kMBSIiK0rJYfeRrJLt/A0gImVJybXvzyS7DgVRFO1+V42IlMXeP5PsOhTS80tgNItyl0FEJLH33gu7DoUkOx/lJyLlYfeRjOx9N42IlCfVzie/2HUoJGXbdyITkfKk5hVDFO23W9u+Q8HOE5mIlMdoFpFpx+djs+tQSGEoEJENsufjp+w6FLinQES2yJ4Hm+06FOx96hcRKZM992LYdShkF5TIXQIRUTncU5BJidkidwlEROXkFZvkLuGe3VMofPXVVwgLC4PBYEC7du2wa9euO7bfsWMH2rVrB4PBgPDwcMyZM+eeir1diYmhQES2x2yx3ympmuo+YMWKFXjttdfw1VdfoWvXrvjmm2/Qv39/xMXFoV69euXaX7p0CQMGDMDzzz+PJUuW4M8//8S4cePg5+eHoUOH3lfxJpk2vKW4AFm7lqDg3B5YCrKh8w+HV+8XoA9qDAAw52cic/tCFMUfhqUoH/q6zeHdeyy03iF3XG/O/rXIPbIO5pxUqJzc4RzRFV5RMRA0OgBA9p6VKDi7B8aMqxA0OuhDmsIrajS0PnWkdVyeOqjCdXtGPwuPTqXbO2PLXOSf2AJB6wSv6NFwaRYltcs/tQv5J7fCf9i797WNiByZyY5PvyOI1TzKolOnTmjbti2+/vpraVnTpk3x2GOP4eOPPy7XfvLkyfj5559x6tQpadk//vEPHD16FHv27LmP0oFG/1ony7mPUtdOhTH1Mrz7joPa1Rv5J7chZ/9aBD/3FdSuPkhaMhGCSgOvnrFQ6ZyRs38NCi8dRHDs11DpDBWuM+/kNqSvnwnfAf+EPqQpjBnXkL5uOpybdId3r+cBAMkr/w2Xpg9CF9gIEM3I2rkYJanxZdZrzssss97CiweQvn4mgsfOhdYzEAXn9yJ9w5fwH/ouTJmJSF8/AyHjFkLt5A5LUR6uLxqPgCf+C427f81uRLJJuYd+Q/a+1TDnZUDnWw9evZ6HoW6LCtua8jKQuXUeSpLPw5SRCLd2D8O79wtl2iQtexPFV06Ue6xTeHv4P/5e6XMeXofcw+tgyk4GAGh968Gzy0g4NWgvtc/euxo5+1YDADweGAb3Do9J9xUnnkHGxq8Q+MznEFTq+3n5VvNM51B88GjF262qdu7ciWnTpuHgwYO4fv06fvrpJzz22GN3fMyOHTswYcIEnDx5EsHBwXjjjTfwj3/8o1rPW63uo5KSEhw8eBB9+vQps7xPnz7YvXt3hY/Zs2dPufZ9+/bFgQMHYDTe3wEecgSCxViMgjN/wrPHszDUbQGtVzA8u42CxjMAuYfXw5SZiJLEM/DuMw76oMbQ+tSBd58XIZYUIf/UjkrXW5J4GoY6TeHSLBoajwA4hbWFc9MHUZJ0TmoTMPwDuLbsDZ1fKHT+4fAZ8BrMOakoST4vtVG7epX5KTi/F4bQltB6BgIAjOlXYKjbEvqgRnBpFgVB5wxTVhIAIHP7Ari1GchAcFD5p3YiY8tceHQejuDRM6Gv0xwpq96DKSel4geYjVA7u8Oj83Bo/cMqbOI3+F+o89Ji6SdozGxAUMG5STepjdrNB15RMQiKmY6gmOkwhEYiZfWHKEm9DAAoSY1H9h9L4fvIJPg+PBFZOxehJDUeACCaTUj/fTa8+75kM4EAWKcXIz8/H5GRkZg1a1aV2t/slenevTsOHz6Mt99+G6+++ip+/PHHaj1vtUIhLS0NZrMZAQEBZZYHBAQgKSmpwsckJSVV2N5kMiEtLa1axd7KKNcgs8UMiBYIam2ZxYJGh+KrJyGajdJt6T6VGoJag+KrcZWuVh/SDMVJF1CceAYAYMxKQuGFA3Bq0KHyUorzAQAqg2uF95vzM1F4YT9cW/0dyjq/MJQknYe5KA/FSechmoqh8QpG0dWTKEm+ALd2D99lA5BS5exfA9dWD8Etsi+0vnXh3fsFqN18kXt4XYXtNR4B8O49Fq4tekGld66wjdrJrcyXlKL4IxC0ejhH/B0Kzg07walBB2i9Q6D1DoHXg89ApTP8/X8h7Qq0fvXhFBoJp/qtofWrD2P61dKa962GoW5zqevWVpit8IW1f//++PDDDzFkyJAqtZ8zZw7q1auH6dOno2nTpnjuuecwZswYfPrpp9V63mqPKQCAIAhlbouiWG7Z3dpXtLw65BrIUemdoQ9uguzd/4PWpy7ULp7IP7UTJYlnofEOhta7DtTu/sja8T28+70MlVaPnP1rYM7PhDkvo9L1ujSLgrkwB0lLJwMQAYsZrm0GwOOBxytsL4oiMrd+B32dZtD51a+wTd6JLVDpnODcuIu0zCm8HVyaRyPp+/EQNDr4DhwPlVaPjN+/gs/A8aW78od+hdrJHd59X4bOL/R+NhfZCdFsREnSeXg8MKzMcqewNii+dtpqz5N3bCNcmj5YaTeqaDGj4PQfsBiLoA9pAgDQ+dWHKfNa6R6LCJgyrkHnGwpjZiLyjm9GUMx0q9VnLXKMd1bWKzNv3jwYjUZotdpKHllWtULB19cXarW63F5BSkpKub2BmwIDAytsr9Fo4OPjU52nL+M+8uS++Qx6HenrZ+DaVzGAoIIusAFcmkWhJPkCBLUGfoPfRvr6Gbg64wlAUMFQvzUM4e3uuM6ihGPI3rMC3n1ehD44AqbMRGRsnossl+Xw7DqyXPuMTXNQkhKPwFGfVLrOvGOb4dIsusxeCwB4dhsFz26jpNtZfyyFoX5rCCo1svesQPCY2Sg8vw/pv32OoNEzqrl17JcgiNAJIjSCCI3qxr9VIrSCpXSZAGhVFmgFEWrcuE8lQgNL6W/BAjUAjepme0tpuxv/1ggi1LBArRKhgQi1YIEGFqiFm/8u/a2GCNXN+yBCdbMNLFALltLbsECFG8sgQgUzVLBAJYhQi6Vtyv3cWC7cbC/dtiAlMx+RogXfN9yBTvWPlLYRzfgs5AKWX7mOw3U/hQALBNEMoYJhyL76BLRyTce0kMr3/vdfzkVU2mVsjnFHh5CPytx3IjEfPT4/jiKTBa56NVY/1xj9mi8pvTMEmJvrj1k/jgMAfPFoMJ6PXIyBs07i86GBMGe/g/9uuAKNSoVPh9ZHt4YeVvubuFclLr0BRNbqc96tVyYoKKhK66lWKOh0OrRr1w6bNm3C4MGDpeWbNm3Co48+WuFjOnfujF9++aXMso0bN6J9+/ZVTq6KqGVMBa1XEAKfnAJLSREsJQXQuHojde1UaDxK3xB9YEMEP/slLMX5EM0mqJ09cH3RhNIB4kpk7VoC1+Y94RbZF0DptyOLsRgZG2bBo8sICMLfPX0Zm+ag8PxeBDw5BRp33wrXV3TlBEwZV+H66Bt3fC3G9CvIj9uOoNEzkXdsEwx1WkDt7AHnJt2Rvn4GLMUFlXYNKI0oCigWBRQDgFnuamqXKTcdwAaMuf4Y9Kqm0vLszBXIM21DiysTpWVq4UYwqgCdUBqMZ4zv4EJRfZzKHA2tSoT2ZijeEqR7d82DV2AdLA96HauMpW3UQmngqfxKMOaNdBiLC3DqyCE8vXQ3XnrtMdQJDoQKFuiiRbzR4+8gfW33fuQ6ZyE34lH8899f4NO3X0BmVjae+O5HfP/xy3DSqW8E5a3BeSNQhdKAVAsWCOLf4XozIFW3haoglq5HuLlcNEOAeGN56TqEW5eJFtm+tVqjV6ba3UcTJkzA008/jfbt26Nz58749ttvkZCQII1wv/XWW7h27RoWLVoEoHSm0axZszBhwgQ8//zz2LNnD+bNm4fly5dX96nLUKtk3FW4QaUzQKUzwFyUh8JLh+AV/WzZ+/UuAABjxjWUJJ2HZ/enKl2XaCwu94dUGgQiIIqAcKPLaPMcFJzdg4CRH0uDxxXJO7YJusCG0PmHV/6cooj0DbPg1eM5qHROgGiBaLlx0M3N3yKPBXEEamd3QFDBnF929pq5IAtqF8+yy0QVzCIAC5B/Y1mxRYBoUuNCgVOF67cYi3D1yF54dh+F9akVf5EBQktHOdt2g/lsMr765TB8+r1crpW5IBtJaz9HwJNT8faeCzB51sXUvD6ABsgq+QkvHKhTaZdqbRkRWBdTa/k5rdUrU+1QGDFiBNLT0/HBBx/g+vXraNGiBdatW4fQ0NK+5+vXryMhIUFqHxYWhnXr1mH8+PGYPXs2goODMXPmzPs+RkEQBAhC6edlbSu8eBAAoPEOgSnzOjK3z4fWOwSuLXsDAPJP/wG1szvU7v4wpsYjY/O3cG70AJzC2krrSPv1sxuzLkYDAJwadkTO/jXQ+YdDFxwBU+Z1ZO1aAqeGnaRZFRmbvkZ+3A74D3kHKp2zNP1U0DtDpdVL67YUF6DgzB/w6hF7x9eRd/T30r2CRp0AAPqQpsj6YxmKr51G4cWD0PrUq3QQm5RFUGuhC2yIwvgjZcagiuKPwOnG38f9KDj9B0SzES7Ne1TxEaI0aeN2mVvmwq3DY9C4+6Ik6SxE8y27dRYzYJH/i4xOU/sni7BWr8w9DTSPGzcO48aNq/C+hQsXllsWFRWFQ4cO3ctT3ZFaEGCSIRUsxQXI2vk9TLlpUBvc4BzRBZ4PPgNBXbo5zXkZyNz6Hcz5WVC7esG1eU94dH2izDpMOanALV1CHl2eACAga9cSmPPSoXLygFPDjvB68GmpTd6NWSDJy98qsy6fAa9JgQSUTi2EiDIHpd3OnJ+J7D0rEfjUNGmZPjgC7h0HI+WH96Fy9oDvwPHV3zhkt9w7PIa0Xz+HPrAh9MFNkXt0A0w5qXBrPQAAkLljIcy56fAd9Lr0mJLkiwAA0VgEc2F26W21Bjrfsgey5h3bCOdGD0Dt5F7ueTN3fA+n8HbQuPvBUlKI/FM7UZRwAv6Pv1+ubeGlwzBmJsJn0AQAgC6oMUwZV1F44QBMuWmASg3NXQ4SrQ3WCIW8vDycP//3dPNLly7hyJEj8Pb2Rr169WqsV6baB6/ZksbvrOepLoisKPfQb8je+yPM+RnQ+YaWOXgt7bcvYMpORuCTU6T2FR1Br3b3R50X50u3jRnXkDh3LPyH/wdOYW3KtU9bNwNFl4/CnJ8Bld4FOr/6cO80rFxbi7EY1xe+Cr9HJkMX8He3aO7R35G1azEEtRbefcbB+Q7TuGvLi9ENMLlfk/tax/bt29GjR/k9q5iYGCxcuBCjR49GfHw8tm/fLt23Y8cOjB8/Xjp4bfLkydU+eM2uQ6HZvzegoMTBRgSJyOb9s1cjjH/Ito6dqCq7Pkuql7Pu7o2IiGqZHGMK1mK/lQPwdWUoEJHt0TMU5OHnpr97IyKiWuZuuPdjsOTGUCAisrJAj4pP42EP7DsUXBkKRGR7ghgK8vDlngIR2SDuKciEewpEZGtc9Rq4cUxBHhxTICJbY897CQBDgYjIqux5PAFgKBARWVWgO0NBNs46DZx1tnNdViIi7inIjHsLRGRLAj0qvqaEvbD/UOAMJCKyIdxTkBn3FIjIlnD2kczq+TjG9YOJyD5wT0FmzYM95C6BiAgA4KRVw9POT+mvgFAof3k/IiI5hHjZ9yAzoIBQCPd1gQunpRKRDWihgC+pdh8KgiCgaZD9vxFEZP8i63rKXcJ9s/tQANiFRES2gaFgI5qHcLCZiOSlVQuK+IKqjFBQwBtBRPYtItANeo39j28qIhQaB7hBp1bESyEiOxVZx1PuEqxCEZ+kWrUKjQNd5S6DiByYEsYTAIWEAgA0D+K4AhHJpzVDwbY0D+G4AhHJw0WnRkM/ZfRWKCcUeLoLIpJJixAPqFSC3GVYhWJCoWmQGxTynhCRnVFK1xGgoFBw1mnQJJBdSERU+5QyyAwoKBQAoEcTP7lLICIH1KqOcrqvFRUKPZsEyF0CETkYPzc96ngp57ouigqFNnU94eNi3+cyJyL70jPCX+4SrEpRoaBSCYiKYBcSEdWevi2U1UOhqFAAgF7sQiKiWuKiU6NrQ1+5y7AqxYXCg419oVVzbioR1bzoCH9FnATvVooLBTeDFh3qe8tdBhE5gD7NldczobhQAICeTZQ18ENEtkerFtBDgZ81igyF3k2Vl95EZFseCPeBu0ErdxlWp8hQqO/rgnBfF7nLICIF69s8UO4SaoQiQwFgFxIR1RxBAPo0U2aPBEOBiKiaWtf1hL+7Qe4yaoRiQ6FDmDfcDBq5yyAiBerTTJldR4CCQ0GrVuEhhe7eEZG8+ipwKupNig0FABjRvq7cJRCRwjT0d0W4Qq6yVhFFh0KncB+EcRYSEVnRgBbK7ToCFB4KADCcewtEZCUqARjRsZ7cZdQoxYfC0HYh0PA6nURkBT0i/BHi6SR3GTVK8aHg72ZAtMLOd05E8hj1gLL3EgAHCAUAeKIDu5CI6P6EeDohurHyv2A6RCj0aKL8XT4iqlkjO9aFygG6oh0iFNQqAU89ECp3GURkp7RqAcMdpMfBIUIBKO1C0msc5uUSkRX1axEEfzdlntbidg7zKenlosOjrYPlLoOI7NBz3cLkLqHWOEwoAEBMl/pyl0BEdqZDfS9E1vWUu4xa41Ch0DzYAx15qU4iqobYbuFyl1CrHCoUAODZrvXlLoGI7ESoj7Nir5tQGYcLhX4tAtE0yF3uMojIDozpGuYQ01Bv5XChIAgCJvVtLHcZRGTjPJy0eLx9HbnLqHUOFwoA0LNJANqHesldBhHZsBejG8BZ53gX6nLIUACAN/o1kbsEIrJRIZ5OGO2gsxUdNhQ6hnkjOsJP7jKIyAa93qcxDFq13GXIwmFDAQAm9Y2A4FhjSER0F82C3DG4TYjcZcjGoUOhebAHBrQMkrsMIrIhbw9oCsGBvy06dCgAwOsPNeZFeIgIANC9kS+6NfKVuwxZOXwohPu5Ylg7x5t2RkRlqQTgrf5N5S5Ddg4fCgDwz96NeAZVIgc3uE0dNAvmga38JAQQ5OGEp3m9BSKHpdeoMJEHtQJgKEjG9WgIV73jHahCRMCzXcMQ5MGrMwIMBYm3iw4vRjeQuwwiqmVezlqM68H/+zcxFG7xwoPhPFkekYN5uWcjuBu0cpdhMxgKt9CqVZg2rBWnqBI5iHBfF44n3oahcJsWIR74RxR3JYmUTq0S8NnwSOg487AMbo0KvNqrERoHuMpdBhHVoLEPhqNNPZ4t+XYMhQroNCpMGxYJNbuRiBSpSaAbXuvNKagVYShUIrKuJ57rFiZ3GURkZTq1Cl+MaM1uo0pwq9zB+IcaI9zPRe4yiMiK/tm7EWcZ3gFD4Q4MWjWmDWsF9iIRKUObep6cSHIXDIW7aBfqjRgHvQITkZI4adX47HGOFd4NQ6EK3ujbBKE+znKXQUT3YXK/CIT7cVbh3TAUqsBJp8bUoa14lTYiO9WlgQ/3+KuIoVBFD4T7IKZzfbnLIKJqctNrMO3xSIe+mlp1MBSq4V8Dm6JjfW+5yyCiavi/h5shxJNnQK0qhkI1aNUqfPVUWwR7GOQuhYiqoH+LQAxvX1fuMuwKQ6GafF31+PaZ9jBouemIbFmzIHd8NjxS7jLsDj/Z7kGLEA9MHdpK7jKIqBJ+bnrMG90ezjpeOKu6GAr36NHWIRj7YLjcZRDRbfQaFeY+055XUrtHDIX7MLlfEzzY2E/uMojoFtMej0Trup5yl2G3GAr3QaUS8OXINgjz5fmRiGzBq70a4ZHIYLnLsGsMhfvk4aTFt0+3g6uefZdEchrYKgjjezeSuwy7x1CwgkYBbvhiRGse8Uwkk1Z1PPAZD1CzCoaClTzULACv9eJFO4hqW6C7AXOfaQ+DVi13KYrAULCiV3s1RP8WgXKXQeQwnLRqzH2mPQLceUCptTAUrEgQBHwxojW6NvSRuxQixRME4PPhkWhZx0PuUhSFoWBlhhvfXNqH8oLgRDVpUt8I9G8ZJHcZisNQqAHOOg3mP9sBLUP4DYaoJrzasyHGRTeUuwxFYijUEHeDFovGdETjAF7Ug8iaXurRABP6RMhdhmIxFGqQl4sOS57rxIPbiKzkH1ENMKlvE7nLUDSGQg3zdzNg2fMMBqL79cKD4XizPwOhpjEUakGQhxP+98IDCPdjMBDdi9huYXh7QFO5y3AIgiiKotxFOIrU3GI8OfcvnEvJk7sUIrvxUg92GdUmhkItS88rxqjv9uJ0Uq7cpRDZvDf6RXCWUS1jKMggM78Eo77bi7jrOXKXQmSTBAF47+HmiOlSX+5SHA5DQSbZhUa8tPQQ/jifJncpRDZFrRIwZUhLPM5rK8uCoSAjk9mCD36Nw6I9l+Uuhcgm6DQqfDG8NQa24pHKcmEo2IDFf13G+z+fhMnCt4IcV4C7HnOeaoc29XiKGDkxFGzEn+fTMG7pIWQXGuUuhajWtQv1wtdPtYW/G892KjeGgg25lJaP2O/342JqvtylENWakR3r4v1HWkCn4WFTtoChYGOyC414edkh7DrHAWhSNq1awLsPN8dTD4TKXQrdgqFgg8wWEf/5NQ4Ld8fLXQpRjfB11ePrp9qiQ31vuUuh2zAUbNjSvZfx7loOQJOyRNbxwJyn2yHIw0nuUqgCDAUbt/tC6QB0VgEHoMn+DW1bB/8d3ILXU7ZhDAU7EJ+Wj3/+7zCOXs2WuxSie6JRCXh7QFOM6RYmdyl0FwwFO2G2iPhq23nM3HoORjPfMrIfPi46fPlkG3Rp4Ct3KVQFDAU7E5eYgwkrj/CEemQXHo4MxvuPNIe3i07uUqiKGAp2qMRkwYwtZzFnx0WYOQhNNsjfTY8PH2uBPs0D5S6FqomhYMeOXMnC6yuP4AIPdiMb8ni7OnhnUDN4OGnlLoXuAUPBzhUZzfj09zOY/+clcKeB5BTi6YSPh7TEg4395C6F7gNDQSH2XcrAxFVHkZBRIHcp5GAEAXiqUyje7N8ELnqN3OXQfWIoKEhBiQkfrTuFpXsTwHeVakOYrwumDGmJTuE+cpdCVsJQUKA/zqXh3Z9PcKyBaoxKAGK7heH1PhE8EE1hGAoKZbaIWLH/CqZvPouU3GK5yyEFaRLoho+HtOR1DxSKoaBwBSUmfLfrEr7deRF5xSa5yyE7FurjjPG9G+ORyGCoVILc5VANYSg4iPS8Yszccg7L9iXwiGiqlkB3A17p1RAj2teFRs1rHigdQ8HBxKflY9rvZ/Db8etyl0I2zttFhxejGuDpzqEcN3AgDAUHdeRKFj5edwp7L2XIXQrZGDe9BrHdw/Bc93C4coqpw2EoOLgtp5IxdcNpnE3Ok7sUkplBq0JM5/r4R1QDePFcRQ6LoUAwW0SsPnQVc3ddZDg4IK1awIgOdfFqz0bwdzfIXQ7JjKFAZew8m4p5f1zCznOpPABO4Zy0ajzWJgQvRjVAPR9nucshG8FQoAqdS87F/D8vYfWhayg2WeQuh6wo1McZTz8Qisfb1+VJ66gchgLdUUZ+CZbvS8DyfQm4mlkodzl0jwQBeLCRH0Z3qY/oCD8IAo8zoIoxFKhKLBYRO8+lYuneBGw9ncLrONgJPzc9hrQNwRMd6iHM10XucsgOMBSo2pKyi7Bi/xWs2J+AxOwiucuh22hUAqIj/DGiQ130iPDjAWdULQwFumdmi4h9lzKwMS4Jm+KS2b0kswZ+Lni8fV0MaRsCfzfOIqJ7w1Agq4lLzMHGuCRsPJmMuOs5cpejeFq1gI5h3ugR4Y+eTfwR7ucqd0mkAAwFqhFXMwuwKS4ZG08mY398Bkwcg7AKX1c9ekT4oWcTf3Rr5As3A2cPkXUxFKjGZRWUYMupFGyKS8bOc6koKDHLXZLdEASgZYgHekT4o1dTf7QM8eDMIapRDAWqVUVGM/48n4Z98Rk4diUbJ65lI5en9C7DVa9Bt4a+6NnUH9ERfhwfoFrFUCBZiaKIC6n5OHY1C8euZuPo1SzEJeY4zAFz3i46NA92R7MgdzQLdkfzYHeE+7ryegUkG4YC2Ryj2YIzSbk4djUbx65m4ejVbJxLzrX7cYm63k5oFuSO5sEepUEQ7I4gDye5yyIqg6FAdqHIaMbJxGxcSM1HcnYRknKKkJxThOvZpb/T80tkP1eTIACeTlr4uenh66pHkIeT9O2/WbA73DkoTHaAoUCKUGKyIPlGUCTlFCEp+8bPjWVpeSUoNppRYhZhNFtQYrLAaLZUuvchCIBKEKASAHeDFr6uevi66Up/u+qlD35fV51028dFxwPFyO4xFMihWSwijJbS8QuVIEAtCOzPJ4fGUCAiIgn3dYmISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISMJQICIiCUOBiIgkDAUiIpIwFIiISPL/drqtXAUUeTYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# get the set of distinct classes\n",
    "labels = big_raw_data.Class.unique()\n",
    "\n",
    "# get the count of each class\n",
    "sizes = big_raw_data.Class.value_counts().values\n",
    "\n",
    "# plot the class value counts\n",
    "fig, ax = plt.subplots()\n",
    "ax.pie(sizes, labels=labels, autopct='%1.3f%%')\n",
    "ax.set_title('Target Variable Value Counts')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown above, the Class variable has two values: 0 (the credit card transaction is legitimate) and 1 (the credit card transaction is fraudulent). Thus, you need to model a binary classification problem. Moreover, the dataset is highly unbalanced, the target variable classes are not represented equally. This case requires special attention when training or when evaluating the quality of a model. One way of handing this case at train time is to bias the model to pay more attention to the samples in the minority class. The models under the current study will be configured to take into account the class weights of the samples at train/fit time.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Determined the range of amounts in the credit card transactions by identifying the minimum and maximum values. Additionally, computed the 90th percentile of the amount values and present the outcome."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGsCAYAAAAPJKchAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfl0lEQVR4nO3df2yU933A8c+FHwZR+1KgBrs4xIkaFmHKqMkCWchCkZxAw4bKpmzqEhqtk5iANHGRWpJJNN00MimJULT8UDpChtImUWWSsYE6XIUfaSHbANMkDWG0YZgRexSa2IS2NoRnf1TcdMX8OAfzBfN6SY/EPfd97r735ZH81t1zdi7LsiwAABK5IvUEAIDLmxgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSuqRiZPPmzTF79uyorq6OXC4Xr7zySsmPkWVZPPLII3HddddFWVlZ1NTUxN/93d+d/8kCAOdkYOoJlOLo0aMxceLEuOeee2Lu3Lm9eoyvfvWrsX79+njkkUdiwoQJ0dHREYcOHTrPMwUAzlXuUv1DeblcLl5++eWYM2dOYV93d3f89V//dXznO9+JDz74IOrq6uLv//7v49Zbb42IiF27dsVnP/vZeOutt2LcuHFpJg4AFLmkPqY5m3vuuSd+9KMfxYsvvhhvvPFG/Mmf/EncfvvtsWfPnoiI+Jd/+Ze45ppr4l//9V+jtrY2rr766vjKV74Sv/jFLxLPHAAuX/0mRn72s5/FCy+8EN/73vdi2rRpce2118bixYvj5ptvjpUrV0ZExLvvvhv79u2L733ve7Fq1ap47rnnYvv27fHHf/zHiWcPAJevS+qakTPZsWNHZFkW1113XdH+rq6uGDFiREREnDhxIrq6umLVqlWFcStWrIj6+vrYvXu3j24AIIF+EyMnTpyIAQMGxPbt22PAgAFF933iE5+IiIiqqqoYOHBgUbBcf/31ERHR2toqRgAggX4TI5MmTYqPPvooDh48GNOmTetxzO///u/H8ePH42c/+1lce+21ERHxX//1XxERMXbs2As2VwDg/11S36b58MMP46c//WlE/CY+HnvssZg+fXoMHz48rrrqqvjzP//z+NGPfhSPPvpoTJo0KQ4dOhSvvvpqTJgwIWbNmhUnTpyIG264IT7xiU/E8uXL48SJE7FgwYKoqKiI9evXJ351AHB5uqRiZOPGjTF9+vRT9s+bNy+ee+65OHbsWPzt3/5trFq1Kg4cOBAjRoyIqVOnxkMPPRQTJkyIiIj33nsvFi1aFOvXr49hw4bFzJkz49FHH43hw4df6JcDAMQlFiMAQP/Tb77aCwBcmsQIAJDUJfFtmhMnTsR7770X5eXlkcvlUk8HADgHWZbFkSNHorq6Oq644vTvf1wSMfLee+9FTU1N6mkAAL2wf//+GDNmzGnvvyRipLy8PCJ+82IqKioSzwYAOBednZ1RU1NT+Dl+OpdEjJz8aKaiokKMAMAl5myXWLiAFQBISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQ1MDUE0gt99CZ/6wxxbKlWeopANDPeGcEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJBUSTGybNmyuOGGG6K8vDwqKytjzpw5sXv37jMes3Hjxsjlcqds77zzzseaOADQP5QUI5s2bYoFCxbE66+/Hs3NzXH8+PFoaGiIo0ePnvXY3bt3R1tbW2H7zGc+0+tJAwD9x8BSBn//+98vur1y5cqorKyM7du3xy233HLGYysrK+PKK68seYIAQP/2sa4Z6ejoiIiI4cOHn3XspEmToqqqKmbMmBEbNmw449iurq7o7Ows2gCA/qnXMZJlWTQ2NsbNN98cdXV1px1XVVUVzzzzTDQ1NcXq1atj3LhxMWPGjNi8efNpj1m2bFnk8/nCVlNT09tpAgAXuVyWZVlvDlywYEGsXbs2fvjDH8aYMWNKOnb27NmRy+VizZo1Pd7f1dUVXV1dhdudnZ1RU1MTHR0dUVFR0Zvpnlbuodx5fbz+Llvaq9MFgMtQZ2dn5PP5s/787tU7I4sWLYo1a9bEhg0bSg6RiIgpU6bEnj17Tnt/WVlZVFRUFG0AQP9U0gWsWZbFokWL4uWXX46NGzdGbW1tr560paUlqqqqenUsANC/lBQjCxYsiO9+97vxz//8z1FeXh7t7e0REZHP52Po0KEREbFkyZI4cOBArFq1KiIili9fHldffXWMHz8+uru74/nnn4+mpqZoamo6zy8FALgUlRQjTz31VERE3HrrrUX7V65cGV/+8pcjIqKtrS1aW1sL93V3d8fixYvjwIEDMXTo0Bg/fnysXbs2Zs2a9fFmDgD0C72+gPVCOtcLYHrDBaylcQErAOeqTy9gBQA4X8QIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkiopRpYtWxY33HBDlJeXR2VlZcyZMyd279591uM2bdoU9fX1MWTIkLjmmmvi6aef7vWEAYD+paQY2bRpUyxYsCBef/31aG5ujuPHj0dDQ0McPXr0tMfs3bs3Zs2aFdOmTYuWlpZ44IEH4t57742mpqaPPXkA4NKXy7Is6+3BP//5z6OysjI2bdoUt9xyS49jvv71r8eaNWti165dhX3z58+PH//4x7F169Zzep7Ozs7I5/PR0dERFRUVvZ1uj3IP5c7r4/V32dJeny4AXGbO9ef3x7pmpKOjIyIihg8fftoxW7dujYaGhqJ9t912W2zbti2OHTvW4zFdXV3R2dlZtAEA/VOvYyTLsmhsbIybb7456urqTjuuvb09Ro0aVbRv1KhRcfz48Th06FCPxyxbtizy+Xxhq6mp6e00AYCLXK9jZOHChfHGG2/ECy+8cNaxuVzxRyEnPxn67f0nLVmyJDo6Ogrb/v37eztNAOAiN7A3By1atCjWrFkTmzdvjjFjxpxx7OjRo6O9vb1o38GDB2PgwIExYsSIHo8pKyuLsrKy3kwNALjElPTOSJZlsXDhwli9enW8+uqrUVtbe9Zjpk6dGs3NzUX71q9fH5MnT45BgwaVNlsAoN8pKUYWLFgQzz//fHz3u9+N8vLyaG9vj/b29vjVr35VGLNkyZK4++67C7fnz58f+/bti8bGxti1a1c8++yzsWLFili8ePH5exUAwCWrpBh56qmnoqOjI2699daoqqoqbC+99FJhTFtbW7S2thZu19bWxrp162Ljxo3xu7/7u/E3f/M38fjjj8fcuXPP36sAAC5ZH+v3jFwofs/IxcPvGQHgXF2Q3zMCAPBxiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkVXKMbN68OWbPnh3V1dWRy+XilVdeOeP4jRs3Ri6XO2V75513ejtnAKAfGVjqAUePHo2JEyfGPffcE3Pnzj3n43bv3h0VFRWF25/61KdKfWoAoB8qOUZmzpwZM2fOLPmJKisr48orryz5OACgf7tg14xMmjQpqqqqYsaMGbFhw4Yzju3q6orOzs6iDQDon/o8RqqqquKZZ56JpqamWL16dYwbNy5mzJgRmzdvPu0xy5Yti3w+X9hqamr6epoAQCK5LMuyXh+cy8XLL78cc+bMKem42bNnRy6XizVr1vR4f1dXV3R1dRVud3Z2Rk1NTXR0dBRdd3I+5B7KndfH6++ypb0+XQC4zHR2dkY+nz/rz+8kX+2dMmVK7Nmz57T3l5WVRUVFRdEGAPRPSWKkpaUlqqqqUjw1AHCRKfnbNB9++GH89Kc/Ldzeu3dv7Ny5M4YPHx5XXXVVLFmyJA4cOBCrVq2KiIjly5fH1VdfHePHj4/u7u54/vnno6mpKZqams7fqwAALlklx8i2bdti+vTphduNjY0RETFv3rx47rnnoq2tLVpbWwv3d3d3x+LFi+PAgQMxdOjQGD9+fKxduzZmzZp1HqYPAFzqPtYFrBfKuV4A0xsuYC2NC1gBOFcX9QWsAAAniREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkVXKMbN68OWbPnh3V1dWRy+XilVdeOesxmzZtivr6+hgyZEhcc8018fTTT/dmrgBAP1RyjBw9ejQmTpwY//AP/3BO4/fu3RuzZs2KadOmRUtLSzzwwANx7733RlNTU8mTBQD6n4GlHjBz5syYOXPmOY9/+umn46qrrorly5dHRMT1118f27Zti0ceeSTmzp1b6tMDAP1Mn18zsnXr1mhoaCjad9ttt8W2bdvi2LFjPR7T1dUVnZ2dRRsA0D/1eYy0t7fHqFGjivaNGjUqjh8/HocOHerxmGXLlkU+ny9sNTU1fT1NACCRC/JtmlwuV3Q7y7Ie95+0ZMmS6OjoKGz79+/v8zkCAGmUfM1IqUaPHh3t7e1F+w4ePBgDBw6MESNG9HhMWVlZlJWV9fXUAICLQJ+/MzJ16tRobm4u2rd+/fqYPHlyDBo0qK+fHgC4yJUcIx9++GHs3Lkzdu7cGRG/+eruzp07o7W1NSJ+8xHL3XffXRg/f/782LdvXzQ2NsauXbvi2WefjRUrVsTixYvPzysAAC5pJX9Ms23btpg+fXrhdmNjY0REzJs3L5577rloa2srhElERG1tbaxbty7uv//+eOKJJ6K6ujoef/xxX+sFACIiIpedvJr0ItbZ2Rn5fD46OjqioqLivD527qGeL6KlZ9nSi/50AeAica4/v/1tGgAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACTVqxh58skno7a2NoYMGRL19fXx2muvnXbsxo0bI5fLnbK98847vZ40ANB/lBwjL730Utx3333x4IMPRktLS0ybNi1mzpwZra2tZzxu9+7d0dbWVtg+85nP9HrSAED/UXKMPPbYY/EXf/EX8ZWvfCWuv/76WL58edTU1MRTTz11xuMqKytj9OjRhW3AgAG9njQA0H+UFCPd3d2xffv2aGhoKNrf0NAQW7ZsOeOxkyZNiqqqqpgxY0Zs2LDhjGO7urqis7OzaAMA+qeSYuTQoUPx0UcfxahRo4r2jxo1Ktrb23s8pqqqKp555ploamqK1atXx7hx42LGjBmxefPm0z7PsmXLIp/PF7aamppSpgkAXEIG9uagXC5XdDvLslP2nTRu3LgYN25c4fbUqVNj//798cgjj8Qtt9zS4zFLliyJxsbGwu3Ozk5BAgD9VEnvjIwcOTIGDBhwyrsgBw8ePOXdkjOZMmVK7Nmz57T3l5WVRUVFRdEGAPRPJcXI4MGDo76+Ppqbm4v2Nzc3x0033XTOj9PS0hJVVVWlPDUA0E+V/DFNY2Nj3HXXXTF58uSYOnVqPPPMM9Ha2hrz58+PiN98xHLgwIFYtWpVREQsX748rr766hg/fnx0d3fH888/H01NTdHU1HR+XwkAcEkqOUbuvPPOOHz4cHzrW9+Ktra2qKuri3Xr1sXYsWMjIqKtra3od450d3fH4sWL48CBAzF06NAYP358rF27NmbNmnX+XgUAcMnKZVmWpZ7E2XR2dkY+n4+Ojo7zfv1I7qGeL7ylZ9nSi/50AeAica4/v/1tGgAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApMQIAJCUGAEAkhIjAEBSYgQASEqMAABJiREAICkxAgAkJUYAgKTECACQlBgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABISowAAEmJEQAgKTECACQlRgCApHoVI08++WTU1tbGkCFDor6+Pl577bUzjt+0aVPU19fHkCFD4pprromnn366V5MFAPqfkmPkpZdeivvuuy8efPDBaGlpiWnTpsXMmTOjtbW1x/F79+6NWbNmxbRp06KlpSUeeOCBuPfee6OpqeljTx4AuPTlsizLSjngxhtvjM997nPx1FNPFfZdf/31MWfOnFi2bNkp47/+9a/HmjVrYteuXYV98+fPjx//+MexdevWc3rOzs7OyOfz0dHRERUVFaVM96xyD+XO6+P1d9nSkk4XAC5j5/rze2ApD9rd3R3bt2+Pb3zjG0X7GxoaYsuWLT0es3Xr1mhoaCjad9ttt8WKFSvi2LFjMWjQoFOO6erqiq6ursLtjo6OiPjNizrvfn3+H7I/65P/AwD6pZM/M872vkdJMXLo0KH46KOPYtSoUUX7R40aFe3t7T0e097e3uP448ePx6FDh6KqquqUY5YtWxYPPfTQKftrampKmS59IP9wPvUUALjEHDlyJPL50//8KClGTsrlij/ayLLslH1nG9/T/pOWLFkSjY2NhdsnTpyIX/ziFzFixIgzPk+pOjs7o6amJvbv33/eP/7B+vY169u3rG/fsbZ962Ja3yzL4siRI1FdXX3GcSXFyMiRI2PAgAGnvAty8ODBU979OGn06NE9jh84cGCMGDGix2PKysqirKysaN+VV15ZylRLUlFRkfw/rD+zvn3L+vYt69t3rG3fuljW90zviJxU0rdpBg8eHPX19dHc3Fy0v7m5OW666aYej5k6deop49evXx+TJ0/u8XoRAODyUvJXexsbG+Mf//Ef49lnn41du3bF/fffH62trTF//vyI+M1HLHfffXdh/Pz582Pfvn3R2NgYu3btimeffTZWrFgRixcvPn+vAgC4ZJV8zcidd94Zhw8fjm9961vR1tYWdXV1sW7duhg7dmxERLS1tRX9zpHa2tpYt25d3H///fHEE09EdXV1PP744zF37tzz9yp6qaysLJYuXXrKR0KcH9a3b1nfvmV9+4617VuX4vqW/HtGAADOJ3+bBgBISowAAEmJEQAgKTECACR1WcfIk08+GbW1tTFkyJCor6+P1157LfWULirf/OY3I5fLFW2jR48u3J9lWXzzm9+M6urqGDp0aNx6663xk5/8pOgxurq6YtGiRTFy5MgYNmxY/OEf/mH8z//8T9GY999/P+66667I5/ORz+fjrrvuig8++OBCvMQLavPmzTF79uyorq6OXC4Xr7zyStH9F3I9W1tbY/bs2TFs2LAYOXJk3HvvvdHd3d0XL/uCOdv6fvnLXz7lfJ4yZUrRGOvbs2XLlsUNN9wQ5eXlUVlZGXPmzIndu3cXjXH+9t65rG+/P3+zy9SLL76YDRo0KPv2t7+dvf3229lXv/rVbNiwYdm+fftST+2isXTp0mz8+PFZW1tbYTt48GDh/ocffjgrLy/PmpqasjfffDO78847s6qqqqyzs7MwZv78+dmnP/3prLm5OduxY0c2ffr0bOLEidnx48cLY26//fasrq4u27JlS7Zly5asrq4uu+OOOy7oa70Q1q1blz344INZU1NTFhHZyy+/XHT/hVrP48ePZ3V1ddn06dOzHTt2ZM3NzVl1dXW2cOHCPl+DvnS29Z03b152++23F53Phw8fLhpjfXt22223ZStXrszeeuutbOfOndkXvvCF7Kqrrso+/PDDwhjnb++dy/r29/P3so2R3/u938vmz59ftO93fud3sm984xuJZnTxWbp0aTZx4sQe7ztx4kQ2evTo7OGHHy7s+/Wvf53l8/ns6aefzrIsyz744INs0KBB2YsvvlgYc+DAgeyKK67Ivv/972dZlmVvv/12FhHZ66+/XhizdevWLCKyd955pw9e1cXht39YXsj1XLduXXbFFVdkBw4cKIx54YUXsrKysqyjo6NPXu+FdroY+aM/+qPTHmN9z93BgweziMg2bdqUZZnz93z77fXNsv5//l6WH9N0d3fH9u3bo6GhoWh/Q0NDbNmyJdGsLk579uyJ6urqqK2tjT/90z+Nd999NyIi9u7dG+3t7UVrWFZWFn/wB39QWMPt27fHsWPHisZUV1dHXV1dYczWrVsjn8/HjTfeWBgzZcqUyOfzl9X/xYVcz61bt0ZdXV3RH6667bbboqurK7Zv396nrzO1jRs3RmVlZVx33XXxl3/5l3Hw4MHCfdb33HV0dERExPDhwyPC+Xu+/fb6ntSfz9/LMkYOHToUH3300Sl/3G/UqFGn/FG/y9mNN94Yq1atin/7t3+Lb3/729He3h433XRTHD58uLBOZ1rD9vb2GDx4cHzyk58845jKyspTnruysvKy+r+4kOvZ3t5+yvN88pOfjMGDB/frNZ85c2Z85zvfiVdffTUeffTR+M///M/4/Oc/H11dXRFhfc9VlmXR2NgYN998c9TV1UWE8/d86ml9I/r/+Vvyr4PvT3K5XNHtLMtO2Xc5mzlzZuHfEyZMiKlTp8a1114b//RP/1S4cKo3a/jbY3oaf7n+X1yo9bwc1/zOO+8s/Luuri4mT54cY8eOjbVr18YXv/jF0x5nfYstXLgw3njjjfjhD394yn3O34/vdOvb38/fy/KdkZEjR8aAAQNOqbyDBw+eUoT8v2HDhsWECRNiz549hW/VnGkNR48eHd3d3fH++++fccz//u//nvJcP//5zy+r/4sLuZ6jR48+5Xnef//9OHbs2GW15lVVVTF27NjYs2dPRFjfc7Fo0aJYs2ZNbNiwIcaMGVPY7/w9P063vj3pb+fvZRkjgwcPjvr6+mhubi7a39zcHDfddFOiWV38urq6YteuXVFVVRW1tbUxevToojXs7u6OTZs2Fdawvr4+Bg0aVDSmra0t3nrrrcKYqVOnRkdHR/zHf/xHYcy///u/R0dHx2X1f3Eh13Pq1Knx1ltvRVtbW2HM+vXro6ysLOrr6/v0dV5MDh8+HPv374+qqqqIsL5nkmVZLFy4MFavXh2vvvpq1NbWFt3v/P14zra+Pel352+fXRp7kTv51d4VK1Zkb7/9dnbfffdlw4YNy/77v/879dQuGl/72teyjRs3Zu+++272+uuvZ3fccUdWXl5eWKOHH344y+fz2erVq7M333wz+7M/+7Mev8o3ZsyY7Ac/+EG2Y8eO7POf/3yPXzX77Gc/m23dujXbunVrNmHChH751d4jR45kLS0tWUtLSxYR2WOPPZa1tLQUvk5+odbz5Ff3ZsyYke3YsSP7wQ9+kI0ZM+aS/mpklp15fY8cOZJ97Wtfy7Zs2ZLt3bs327BhQzZ16tTs05/+tPU9B3/1V3+V5fP5bOPGjUVfLf3lL39ZGOP87b2zre/lcP5etjGSZVn2xBNPZGPHjs0GDx6cfe5znyv6GhVZ4fcEDBo0KKuurs6++MUvZj/5yU8K9584cSJbunRpNnr06KysrCy75ZZbsjfffLPoMX71q19lCxcuzIYPH54NHTo0u+OOO7LW1taiMYcPH86+9KUvZeXl5Vl5eXn2pS99KXv//fcvxEu8oDZs2JBFxCnbvHnzsiy7sOu5b9++7Atf+EI2dOjQbPjw4dnChQuzX//613358vvcmdb3l7/8ZdbQ0JB96lOfygYNGpRdddVV2bx5805ZO+vbs57WNSKylStXFsY4f3vvbOt7OZy/uSzLsr573wUA4Mwuy2tGAICLhxgBAJISIwBAUmIEAEhKjAAASYkRACApMQIAJCVGAICkxAgAkJQYAQCSEiMAQFJiBABI6v8AHt4JlG/+h0EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum amount value is  0.0\n",
      "Maximum amount value is  25691.16\n",
      "90% of the transactions have an amount less or equal than  203.0\n"
     ]
    }
   ],
   "source": [
    "# we provide our solution here\n",
    "plt.hist(big_raw_data.Amount.values, 6, histtype='bar', facecolor='g')\n",
    "plt.show()\n",
    "\n",
    "print(\"Minimum amount value is \", np.min(big_raw_data.Amount.values))\n",
    "print(\"Maximum amount value is \", np.max(big_raw_data.Amount.values))\n",
    "print(\"90% of the transactions have an amount less or equal than \", np.percentile(raw_data.Amount.values, 90))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2843150    4920]\n"
     ]
    }
   ],
   "source": [
    "print(sizes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"dataset_preprocessing\">\n",
    "    <h2>Dataset Preprocessing</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this subsection you will prepare the data for training. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X.shape= (2848070, 29) y.shape= (2848070,)\n"
     ]
    }
   ],
   "source": [
    "# data preprocessing such as scaling/normalization is typically useful for \n",
    "# linear models to accelerate the training convergence\n",
    "\n",
    "# standardize features by removing the mean and scaling to unit variance\n",
    "big_raw_data.iloc[:, 1:30] = StandardScaler().fit_transform(big_raw_data.iloc[:, 1:30])\n",
    "data_matrix = big_raw_data.values\n",
    "\n",
    "# X: feature matrix (for this analysis, we exclude the Time variable from the dataset)\n",
    "X = data_matrix[:, 1:30]\n",
    "\n",
    "# y: labels vector\n",
    "y = data_matrix[:, 30]\n",
    "\n",
    "# data normalization\n",
    "X = normalize(X, norm=\"l1\")\n",
    "\n",
    "# print the shape of the features matrix and the labels vector\n",
    "print('X.shape=', X.shape, 'y.shape=', y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"dataset_split\">\n",
    "    <h2>Dataset Train/Test Split</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the dataset is ready for building the classification models, you need to first divide the pre-processed dataset into a subset to be used for training the model (the train set) and a subset to be used for evaluating the quality of the model (the test set).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train.shape= (1993649, 29) Y_train.shape= (1993649,)\n",
      "X_test.shape= (854421, 29) Y_test.shape= (854421,)\n"
     ]
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)       \n",
    "print('X_train.shape=', X_train.shape, 'Y_train.shape=', y_train.shape)\n",
    "print('X_test.shape=', X_test.shape, 'Y_test.shape=', y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"dt_sklearn\">\n",
    "    <h2>Build a Decision Tree Classifier model with Scikit-Learn</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Scikit-Learn] Training time (s):  47.78989\n"
     ]
    }
   ],
   "source": [
    "# compute the sample weights to be used as input to the train routine so that \n",
    "# it takes into account the class imbalance present in this dataset\n",
    "w_train = compute_sample_weight('balanced', y_train)\n",
    "\n",
    "# import the Decision Tree Classifier Model from scikit-learn\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# for reproducible output across multiple function calls, set random_state to a given integer value\n",
    "sklearn_dt = DecisionTreeClassifier(max_depth=4, random_state=35)\n",
    "\n",
    "# train a Decision Tree Classifier using scikit-learn\n",
    "t0 = time.time()\n",
    "sklearn_dt.fit(X_train, y_train, sample_weight=w_train)\n",
    "sklearn_time = time.time()-t0\n",
    "print(\"[Scikit-Learn] Training time (s):  {0:.5f}\".format(sklearn_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"dt_sklearn_snapml\">\n",
    "    <h2>Evaluate the Scikit-Learn and Snap ML Decision Tree Classifier Models</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Decision Tree Classifier] Snap ML vs. Scikit-Learn speedup : 4.98x \n",
      "[Scikit-Learn] ROC-AUC score : 0.966\n",
      "[Snap ML] ROC-AUC score : 0.966\n"
     ]
    }
   ],
   "source": [
    "# Snap ML vs Scikit-Learn training speedup\n",
    "training_speedup = sklearn_time/snapml_time\n",
    "print('[Decision Tree Classifier] Snap ML vs. Scikit-Learn speedup : {0:.2f}x '.format(training_speedup))\n",
    "\n",
    "# run inference and compute the probabilities of the test samples \n",
    "# to belong to the class of fraudulent transactions\n",
    "sklearn_pred = sklearn_dt.predict_proba(X_test)[:,1]\n",
    "\n",
    "# evaluate the Compute Area Under the Receiver Operating Characteristic \n",
    "# Curve (ROC-AUC) score from the predictions\n",
    "sklearn_roc_auc = roc_auc_score(y_test, sklearn_pred)\n",
    "print('[Scikit-Learn] ROC-AUC score : {0:.3f}'.format(sklearn_roc_auc))\n",
    "\n",
    "# run inference and compute the probabilities of the test samples\n",
    "# to belong to the class of fraudulent transactions\n",
    "snapml_pred = snapml_dt.predict_proba(X_test)[:,1]\n",
    "\n",
    "# evaluate the Compute Area Under the Receiver Operating Characteristic\n",
    "# Curve (ROC-AUC) score from the prediction scores\n",
    "snapml_roc_auc = roc_auc_score(y_test, snapml_pred)   \n",
    "print('[Snap ML] ROC-AUC score : {0:.3f}'.format(snapml_roc_auc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown above both decision tree models provide the same score on the test dataset. However Snap ML runs the training routine 12x faster than Scikit-Learn. This is one of the advantages of using Snap ML: acceleration of training of classical machine learning models, such as linear and tree-based models. For more Snap ML examples, please visit [snapml-examples](https://ibm.biz/BdPfxP).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"svm_sklearn\">\n",
    "    <h2>Build a Support Vector Machine model with Scikit-Learn</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Scikit-Learn] Training time (s):  85.12\n"
     ]
    }
   ],
   "source": [
    "# import the linear Support Vector Machine (SVM) model from Scikit-Learn\n",
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "# instatiate a scikit-learn SVM model\n",
    "# to indicate the class imbalance at fit time, set class_weight='balanced'\n",
    "# for reproducible output across multiple function calls, set random_state to a given integer value\n",
    "sklearn_svm = LinearSVC(class_weight='balanced', random_state=31, loss=\"hinge\", fit_intercept=False)\n",
    "\n",
    "# train a linear Support Vector Machine model using Scikit-Learn\n",
    "t0 = time.time()\n",
    "sklearn_svm.fit(X_train, y_train)\n",
    "sklearn_time = time.time() - t0\n",
    "print(\"[Scikit-Learn] Training time (s):  {0:.2f}\".format(sklearn_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"svm_sklearn_snap\">\n",
    "    <h2>Evaluate the Scikit-Learn and Snap ML Support Vector Machine Models</h2>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Support Vector Machine] Snap ML vs. Scikit-Learn training speedup : 2.11x \n",
      "[Scikit-Learn] ROC-AUC score:   0.984\n",
      "[Snap ML] ROC-AUC score:   0.985\n"
     ]
    }
   ],
   "source": [
    "# compute the Snap ML vs Scikit-Learn training speedup\n",
    "training_speedup = sklearn_time/snapml_time\n",
    "print('[Support Vector Machine] Snap ML vs. Scikit-Learn training speedup : {0:.2f}x '.format(training_speedup))\n",
    "\n",
    "# run inference using the Scikit-Learn model\n",
    "# get the confidence scores for the test samples\n",
    "sklearn_pred = sklearn_svm.decision_function(X_test)\n",
    "\n",
    "# evaluate accuracy on test set\n",
    "acc_sklearn  = roc_auc_score(y_test, sklearn_pred)\n",
    "print(\"[Scikit-Learn] ROC-AUC score:   {0:.3f}\".format(acc_sklearn))\n",
    "\n",
    "# run inference using the Snap ML model\n",
    "# get the confidence scores for the test samples\n",
    "snapml_pred = snapml_svm.decision_function(X_test)\n",
    "\n",
    "# evaluate accuracy on test set\n",
    "acc_snapml  = roc_auc_score(y_test, snapml_pred)\n",
    "print(\"[Snap ML] ROC-AUC score:   {0:.3f}\".format(acc_snapml))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown above both SVM models provide the same score on the test dataset. However, as in the case of decision trees, Snap ML runs the training routine faster than Scikit-Learn. For more Snap ML examples, please visit [snapml-examples](https://ibm.biz/BdPfxP). Moreover, as shown above, not only is Snap ML seemlessly accelerating scikit-learn applications, but the library's Python API is also compatible with scikit-learn metrics and data preprocessors.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section evaluated the quality of the SVM models trained above using the hinge loss metric (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.hinge_loss.html). Run inference on the test set using both Scikit-Learn and Snap ML models. Computed the hinge loss metric for both sets of predictions. Print the hinge losses of Scikit-Learn and Snap ML.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Snap ML] Hinge loss:   0.228\n",
      "[Scikit-Learn] Hinge loss:   0.228\n"
     ]
    }
   ],
   "source": [
    "# get the confidence scores for the test samples\n",
    "sklearn_pred = sklearn_svm.decision_function(X_test)\n",
    "snapml_pred  = snapml_svm.decision_function(X_test)\n",
    "\n",
    "# import the hinge_loss metric from scikit-learn\n",
    "from sklearn.metrics import hinge_loss\n",
    "\n",
    "# evaluate the hinge loss from the predictions\n",
    "loss_snapml = hinge_loss(y_test, snapml_pred)\n",
    "print(\"[Snap ML] Hinge loss:   {0:.3f}\".format(loss_snapml))\n",
    "\n",
    "# evaluate the hinge loss metric from the predictions\n",
    "loss_sklearn = hinge_loss(y_test, sklearn_pred)\n",
    "print(\"[Scikit-Learn] Hinge loss:   {0:.3f}\".format(loss_snapml))\n",
    "\n",
    "# the two models should give the same Hinge loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Andreea Anghel\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Contributors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Joseph Santarcangelo\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|  Date (YYYY-MM-DD) |  Version | Changed By  |  Change Description |\n",
    "|---|---|---|---|\n",
    "| 2021-08-31  | 0.1  | AAN  |  Created Lab Content |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Copyright &copy; 2021 IBM Corporation. This notebook and its source code are released under the terms of the [MIT License](https://cognitiveclass.ai/mit-license/).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "anaconda-2022.05-py39",
   "language": "python",
   "name": "conda-env-anaconda-2022.05-py39-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}