Assignment_4.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tejb96/ml-pipelines/blob/main/Assignment_4.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "92778525",
      "metadata": {
        "id": "92778525"
      },
      "source": [
        "# Assignment 4: Pipelines and Hyperparameter Tuning (32 total marks)\n",
        "### Due: November 22 at 11:59pm\n",
        "\n",
        "### Name: Tejpreet Bal"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ce31b39a",
      "metadata": {
        "id": "ce31b39a"
      },
      "source": [
        "### In this assignment, you will be putting together everything you have learned so far. You will need to find your own dataset, do all the appropriate preprocessing, test different supervised learning models and evaluate the results. More details for each step can be found below.\n",
        "\n",
        "### You will also be asked to describe the process by which you came up with the code. More details can be found below. Please cite any websites or AI tools that you used to help you with this assignment."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cf275ca7",
      "metadata": {
        "id": "cf275ca7"
      },
      "source": [
        "## Import Libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2b67a661",
      "metadata": {
        "id": "2b67a661"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split, GridSearchCV\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.metrics import make_scorer, accuracy_score, f1_score"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8219f163",
      "metadata": {
        "id": "8219f163"
      },
      "source": [
        "## Step 1: Data Input (4 marks)\n",
        "\n",
        "Import the dataset you will be using. You can download the dataset onto your computer and read it in using pandas, or download it directly from the website. Answer the questions below about the dataset you selected.\n",
        "\n",
        "To find a dataset, you can use the resources listed in the notes. The dataset can be numerical, categorical, text-based or mixed. If you want help finding a particular dataset related to your interests, please email the instructor.\n",
        "\n",
        "**You cannot use a dataset that was used for a previous assignment or in class**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2af8bd32",
      "metadata": {
        "id": "2af8bd32",
        "outputId": "b9ea7da8-9378-40c6-fde4-f6537938ed18"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Rk</th>\n",
              "      <th>Player</th>\n",
              "      <th>Tm</th>\n",
              "      <th>Age</th>\n",
              "      <th>Pos</th>\n",
              "      <th>G</th>\n",
              "      <th>GS</th>\n",
              "      <th>Tgt</th>\n",
              "      <th>Rec</th>\n",
              "      <th>Ctch%</th>\n",
              "      <th>Yds</th>\n",
              "      <th>Y/R</th>\n",
              "      <th>TD</th>\n",
              "      <th>1D</th>\n",
              "      <th>Succ%</th>\n",
              "      <th>Lng</th>\n",
              "      <th>Y/Tgt</th>\n",
              "      <th>R/G</th>\n",
              "      <th>Y/G</th>\n",
              "      <th>Fmb</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>Justin Jefferson*+</td>\n",
              "      <td>MIN</td>\n",
              "      <td>23</td>\n",
              "      <td>WR</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>184</td>\n",
              "      <td>128</td>\n",
              "      <td>0.696</td>\n",
              "      <td>1809</td>\n",
              "      <td>14.1</td>\n",
              "      <td>8</td>\n",
              "      <td>80</td>\n",
              "      <td>57.6</td>\n",
              "      <td>64</td>\n",
              "      <td>9.8</td>\n",
              "      <td>7.5</td>\n",
              "      <td>106.4</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>Tyreek Hill*+</td>\n",
              "      <td>MIA</td>\n",
              "      <td>28</td>\n",
              "      <td>WR</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>170</td>\n",
              "      <td>119</td>\n",
              "      <td>0.700</td>\n",
              "      <td>1710</td>\n",
              "      <td>14.4</td>\n",
              "      <td>7</td>\n",
              "      <td>77</td>\n",
              "      <td>56.5</td>\n",
              "      <td>64</td>\n",
              "      <td>10.1</td>\n",
              "      <td>7.0</td>\n",
              "      <td>100.6</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>Travis Kelce*+</td>\n",
              "      <td>KAN</td>\n",
              "      <td>33</td>\n",
              "      <td>TE</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>152</td>\n",
              "      <td>110</td>\n",
              "      <td>0.724</td>\n",
              "      <td>1338</td>\n",
              "      <td>12.2</td>\n",
              "      <td>12</td>\n",
              "      <td>78</td>\n",
              "      <td>63.2</td>\n",
              "      <td>52</td>\n",
              "      <td>8.8</td>\n",
              "      <td>6.5</td>\n",
              "      <td>78.7</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>Stefon Diggs*</td>\n",
              "      <td>BUF</td>\n",
              "      <td>29</td>\n",
              "      <td>WR</td>\n",
              "      <td>16</td>\n",
              "      <td>16</td>\n",
              "      <td>154</td>\n",
              "      <td>108</td>\n",
              "      <td>0.701</td>\n",
              "      <td>1429</td>\n",
              "      <td>13.2</td>\n",
              "      <td>11</td>\n",
              "      <td>74</td>\n",
              "      <td>64.3</td>\n",
              "      <td>53</td>\n",
              "      <td>9.3</td>\n",
              "      <td>6.8</td>\n",
              "      <td>89.3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Austin Ekeler</td>\n",
              "      <td>LAC</td>\n",
              "      <td>27</td>\n",
              "      <td>RB</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>127</td>\n",
              "      <td>107</td>\n",
              "      <td>0.843</td>\n",
              "      <td>722</td>\n",
              "      <td>6.7</td>\n",
              "      <td>5</td>\n",
              "      <td>36</td>\n",
              "      <td>48.0</td>\n",
              "      <td>23</td>\n",
              "      <td>5.7</td>\n",
              "      <td>6.3</td>\n",
              "      <td>42.5</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Rk              Player   Tm  Age Pos   G  GS  Tgt  Rec  Ctch%   Yds   Y/R  \\\n",
              "0   1  Justin Jefferson*+  MIN   23  WR  17  17  184  128  0.696  1809  14.1   \n",
              "1   2       Tyreek Hill*+  MIA   28  WR  17  17  170  119  0.700  1710  14.4   \n",
              "2   3      Travis Kelce*+  KAN   33  TE  17  17  152  110  0.724  1338  12.2   \n",
              "3   4       Stefon Diggs*  BUF   29  WR  16  16  154  108  0.701  1429  13.2   \n",
              "4   5       Austin Ekeler  LAC   27  RB  17  17  127  107  0.843   722   6.7   \n",
              "\n",
              "   TD  1D  Succ%  Lng  Y/Tgt  R/G    Y/G  Fmb  \n",
              "0   8  80   57.6   64    9.8  7.5  106.4    0  \n",
              "1   7  77   56.5   64   10.1  7.0  100.6    1  \n",
              "2  12  78   63.2   52    8.8  6.5   78.7    1  \n",
              "3  11  74   64.3   53    9.3  6.8   89.3    1  \n",
              "4   5  36   48.0   23    5.7  6.3   42.5    5  "
            ]
          },
          "execution_count": 427,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Import dataset (1 mark)\n",
        "df=pd.read_excel('probowl2022.xlsx')\n",
        "df.head()\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "da80eb0a",
      "metadata": {
        "id": "da80eb0a",
        "outputId": "9b9fbf49-1ba6-452f-a3ab-b991e8dc359e"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Rk</th>\n",
              "      <th>Age</th>\n",
              "      <th>G</th>\n",
              "      <th>GS</th>\n",
              "      <th>Tgt</th>\n",
              "      <th>Rec</th>\n",
              "      <th>Ctch%</th>\n",
              "      <th>Yds</th>\n",
              "      <th>Y/R</th>\n",
              "      <th>TD</th>\n",
              "      <th>1D</th>\n",
              "      <th>Succ%</th>\n",
              "      <th>Lng</th>\n",
              "      <th>Y/Tgt</th>\n",
              "      <th>R/G</th>\n",
              "      <th>Y/G</th>\n",
              "      <th>Fmb</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>488.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.00000</td>\n",
              "      <td>500.000000</td>\n",
              "      <td>500.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>250.500000</td>\n",
              "      <td>25.920000</td>\n",
              "      <td>12.378000</td>\n",
              "      <td>5.650000</td>\n",
              "      <td>34.586000</td>\n",
              "      <td>23.210000</td>\n",
              "      <td>0.685114</td>\n",
              "      <td>254.076000</td>\n",
              "      <td>10.330533</td>\n",
              "      <td>1.500000</td>\n",
              "      <td>12.184000</td>\n",
              "      <td>47.814000</td>\n",
              "      <td>29.060000</td>\n",
              "      <td>6.728600</td>\n",
              "      <td>1.72560</td>\n",
              "      <td>18.663200</td>\n",
              "      <td>0.690000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>144.481833</td>\n",
              "      <td>2.926214</td>\n",
              "      <td>4.704826</td>\n",
              "      <td>5.810251</td>\n",
              "      <td>37.679093</td>\n",
              "      <td>25.362184</td>\n",
              "      <td>0.203655</td>\n",
              "      <td>312.712469</td>\n",
              "      <td>5.405415</td>\n",
              "      <td>2.207203</td>\n",
              "      <td>15.332329</td>\n",
              "      <td>21.610532</td>\n",
              "      <td>18.939848</td>\n",
              "      <td>3.581292</td>\n",
              "      <td>1.62995</td>\n",
              "      <td>20.145802</td>\n",
              "      <td>1.206985</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>21.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>-1.500000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>125.750000</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.590250</td>\n",
              "      <td>33.000000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>37.500000</td>\n",
              "      <td>15.000000</td>\n",
              "      <td>4.700000</td>\n",
              "      <td>0.50000</td>\n",
              "      <td>4.275000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>250.500000</td>\n",
              "      <td>25.000000</td>\n",
              "      <td>14.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>20.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>0.692000</td>\n",
              "      <td>129.500000</td>\n",
              "      <td>9.900000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>49.550000</td>\n",
              "      <td>26.000000</td>\n",
              "      <td>6.550000</td>\n",
              "      <td>1.10000</td>\n",
              "      <td>11.150000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>375.250000</td>\n",
              "      <td>28.000000</td>\n",
              "      <td>16.000000</td>\n",
              "      <td>11.000000</td>\n",
              "      <td>52.000000</td>\n",
              "      <td>35.000000</td>\n",
              "      <td>0.800000</td>\n",
              "      <td>378.750000</td>\n",
              "      <td>12.725000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>18.000000</td>\n",
              "      <td>57.100000</td>\n",
              "      <td>41.000000</td>\n",
              "      <td>8.400000</td>\n",
              "      <td>2.60000</td>\n",
              "      <td>27.650000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>500.000000</td>\n",
              "      <td>45.000000</td>\n",
              "      <td>17.000000</td>\n",
              "      <td>17.000000</td>\n",
              "      <td>184.000000</td>\n",
              "      <td>128.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1809.000000</td>\n",
              "      <td>42.000000</td>\n",
              "      <td>14.000000</td>\n",
              "      <td>80.000000</td>\n",
              "      <td>100.000000</td>\n",
              "      <td>98.000000</td>\n",
              "      <td>33.000000</td>\n",
              "      <td>8.30000</td>\n",
              "      <td>106.400000</td>\n",
              "      <td>8.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Rk         Age           G          GS         Tgt         Rec  \\\n",
              "count  500.000000  500.000000  500.000000  500.000000  500.000000  500.000000   \n",
              "mean   250.500000   25.920000   12.378000    5.650000   34.586000   23.210000   \n",
              "std    144.481833    2.926214    4.704826    5.810251   37.679093   25.362184   \n",
              "min      1.000000   21.000000    1.000000    0.000000    1.000000    0.000000   \n",
              "25%    125.750000   24.000000    9.000000    0.000000    6.000000    4.000000   \n",
              "50%    250.500000   25.000000   14.000000    3.000000   20.000000   13.000000   \n",
              "75%    375.250000   28.000000   16.000000   11.000000   52.000000   35.000000   \n",
              "max    500.000000   45.000000   17.000000   17.000000  184.000000  128.000000   \n",
              "\n",
              "            Ctch%          Yds         Y/R          TD          1D  \\\n",
              "count  500.000000   500.000000  488.000000  500.000000  500.000000   \n",
              "mean     0.685114   254.076000   10.330533    1.500000   12.184000   \n",
              "std      0.203655   312.712469    5.405415    2.207203   15.332329   \n",
              "min      0.000000   -10.000000   -6.000000    0.000000    0.000000   \n",
              "25%      0.590250    33.000000    7.000000    0.000000    1.000000   \n",
              "50%      0.692000   129.500000    9.900000    1.000000    6.000000   \n",
              "75%      0.800000   378.750000   12.725000    2.000000   18.000000   \n",
              "max      1.000000  1809.000000   42.000000   14.000000   80.000000   \n",
              "\n",
              "            Succ%         Lng       Y/Tgt        R/G         Y/G         Fmb  \n",
              "count  500.000000  500.000000  500.000000  500.00000  500.000000  500.000000  \n",
              "mean    47.814000   29.060000    6.728600    1.72560   18.663200    0.690000  \n",
              "std     21.610532   18.939848    3.581292    1.62995   20.145802    1.206985  \n",
              "min      0.000000   -6.000000   -6.000000    0.00000   -1.500000    0.000000  \n",
              "25%     37.500000   15.000000    4.700000    0.50000    4.275000    0.000000  \n",
              "50%     49.550000   26.000000    6.550000    1.10000   11.150000    0.000000  \n",
              "75%     57.100000   41.000000    8.400000    2.60000   27.650000    1.000000  \n",
              "max    100.000000   98.000000   33.000000    8.30000  106.400000    8.000000  "
            ]
          },
          "execution_count": 428,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df.describe()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "20316765",
      "metadata": {
        "id": "20316765"
      },
      "source": [
        "### Questions (3 marks)\n",
        "\n",
        "1. (1 mark) What is the source of your dataset?\n",
        "1. (1 mark) Why did you pick this particular dataset?\n",
        "1. (1 mark) Was there anything challenging about finding a dataset that you wanted to use?\n",
        "\n",
        "*ANSWER HERE*"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1b2b657f",
      "metadata": {
        "id": "1b2b657f"
      },
      "source": [
        "1. I am using the NFL pro bowl 2022 dataset from:https://www.pro-football-reference.com/years/2023/receiving.htm?sr&utm_source=direct&utm_medium=Share&utm_campaign=ShareTool#receiving\n",
        "2. I picked this dataset because I want to build a machine learning model that would be able to predict if a player is selected for the Pro Bowl based on the player stats.\n",
        "3. It was not challenging to find the data.  \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "42fea4cc",
      "metadata": {
        "id": "42fea4cc"
      },
      "source": [
        "## Step 2: Data Processing (5 marks)\n",
        "\n",
        "The next step is to process your data. Implement the following steps as needed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "afc244d4",
      "metadata": {
        "id": "afc244d4",
        "outputId": "cbff7854-158a-447b-beee-1a79babc3710"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Player    0\n",
              "Age       0\n",
              "Pos       0\n",
              "G         0\n",
              "GS        0\n",
              "Tgt       0\n",
              "Rec       0\n",
              "Ctch%     0\n",
              "Yds       0\n",
              "Y/R       0\n",
              "TD        0\n",
              "1D        0\n",
              "Succ%     0\n",
              "Lng       0\n",
              "Y/Tgt     0\n",
              "R/G       0\n",
              "Y/G       0\n",
              "Fmb       0\n",
              "dtype: int64"
            ]
          },
          "execution_count": 429,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Clean data (if needed)\n",
        "df.isnull().sum()\n",
        "\n",
        "# null_yr_rows = df[df['Y/R'].isnull()]\n",
        "# print(null_yr_rows)\n",
        "df['Y/R'].fillna(0, inplace=True)\n",
        "df['Pos'] = df['Pos'].replace('LB/OLB', 'LB')\n",
        "df['Pos'].fillna('LB', inplace=True)\n",
        "\n",
        "# this resulted in inconsistent results everytime I hit run because I am filling with different values each time for position\n",
        "# df.loc[df['Pos'].isnull(), 'Pos'] = random_positions = np.random.choice(df['Pos'].dropna().unique(), df['Pos'].isnull().sum())\n",
        "\n",
        "# df.dropna(subset='Y/R',inplace=True)\n",
        "# df.dropna(subset='Pos',inplace=True)\n",
        "# print(df['Pos'].unique())\n",
        "\n",
        "df.drop('Rk', axis=1, inplace=True)\n",
        "df.drop('Tm', axis=1, inplace=True)\n",
        "df.isnull().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "70a8c127",
      "metadata": {
        "id": "70a8c127",
        "outputId": "3e7024dc-441b-4a8f-c61f-2443dd5e2cb4"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>G</th>\n",
              "      <th>GS</th>\n",
              "      <th>Tgt</th>\n",
              "      <th>Rec</th>\n",
              "      <th>Ctch%</th>\n",
              "      <th>Yds</th>\n",
              "      <th>Y/R</th>\n",
              "      <th>TD</th>\n",
              "      <th>1D</th>\n",
              "      <th>...</th>\n",
              "      <th>R/G</th>\n",
              "      <th>Y/G</th>\n",
              "      <th>Fmb</th>\n",
              "      <th>Pos_FB</th>\n",
              "      <th>Pos_LB</th>\n",
              "      <th>Pos_QB</th>\n",
              "      <th>Pos_RB</th>\n",
              "      <th>Pos_T</th>\n",
              "      <th>Pos_TE</th>\n",
              "      <th>Pos_WR</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>184</td>\n",
              "      <td>128</td>\n",
              "      <td>0.696</td>\n",
              "      <td>1809</td>\n",
              "      <td>14.1</td>\n",
              "      <td>8</td>\n",
              "      <td>80</td>\n",
              "      <td>...</td>\n",
              "      <td>7.5</td>\n",
              "      <td>106.4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>28</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>170</td>\n",
              "      <td>119</td>\n",
              "      <td>0.700</td>\n",
              "      <td>1710</td>\n",
              "      <td>14.4</td>\n",
              "      <td>7</td>\n",
              "      <td>77</td>\n",
              "      <td>...</td>\n",
              "      <td>7.0</td>\n",
              "      <td>100.6</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>33</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>152</td>\n",
              "      <td>110</td>\n",
              "      <td>0.724</td>\n",
              "      <td>1338</td>\n",
              "      <td>12.2</td>\n",
              "      <td>12</td>\n",
              "      <td>78</td>\n",
              "      <td>...</td>\n",
              "      <td>6.5</td>\n",
              "      <td>78.7</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>29</td>\n",
              "      <td>16</td>\n",
              "      <td>16</td>\n",
              "      <td>154</td>\n",
              "      <td>108</td>\n",
              "      <td>0.701</td>\n",
              "      <td>1429</td>\n",
              "      <td>13.2</td>\n",
              "      <td>11</td>\n",
              "      <td>74</td>\n",
              "      <td>...</td>\n",
              "      <td>6.8</td>\n",
              "      <td>89.3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>27</td>\n",
              "      <td>17</td>\n",
              "      <td>17</td>\n",
              "      <td>127</td>\n",
              "      <td>107</td>\n",
              "      <td>0.843</td>\n",
              "      <td>722</td>\n",
              "      <td>6.7</td>\n",
              "      <td>5</td>\n",
              "      <td>36</td>\n",
              "      <td>...</td>\n",
              "      <td>6.3</td>\n",
              "      <td>42.5</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 23 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "   Age   G  GS  Tgt  Rec  Ctch%   Yds   Y/R  TD  1D  ...  R/G    Y/G  Fmb  \\\n",
              "0   23  17  17  184  128  0.696  1809  14.1   8  80  ...  7.5  106.4    0   \n",
              "1   28  17  17  170  119  0.700  1710  14.4   7  77  ...  7.0  100.6    1   \n",
              "2   33  17  17  152  110  0.724  1338  12.2  12  78  ...  6.5   78.7    1   \n",
              "3   29  16  16  154  108  0.701  1429  13.2  11  74  ...  6.8   89.3    1   \n",
              "4   27  17  17  127  107  0.843   722   6.7   5  36  ...  6.3   42.5    5   \n",
              "\n",
              "   Pos_FB  Pos_LB  Pos_QB  Pos_RB  Pos_T  Pos_TE  Pos_WR  \n",
              "0       0       0       0       0      0       0       1  \n",
              "1       0       0       0       0      0       0       1  \n",
              "2       0       0       0       0      0       1       0  \n",
              "3       0       0       0       0      0       0       1  \n",
              "4       0       0       0       1      0       0       0  \n",
              "\n",
              "[5 rows x 23 columns]"
            ]
          },
          "execution_count": 430,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Implement preprocessing steps. Remember to use ColumnTransformer if more than one preprocessing method is needed\n",
        "\n",
        "df = pd.get_dummies(df, columns=['Pos'], drop_first=True) # encodes player position column\n",
        "df['ProBowl'] = df['Player'].str.contains('*', regex=False).astype(int)\n",
        "df.drop('Player', axis=1, inplace=True)\n",
        "X = df.drop('ProBowl', axis=1)\n",
        "y = df['ProBowl']\n",
        "X.head()\n",
        "# y.isnull().sum()\n",
        "# y.sum()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b92c46b7",
      "metadata": {
        "id": "b92c46b7"
      },
      "source": [
        "### Questions (2 marks)\n",
        "\n",
        "1. (1 mark) Were there any missing/null values in your dataset? If yes, how did you replace them and why? If no, describe how you would've replaced them and why.\n",
        "2. (1 mark) What type of data do you have? What preprocessing methods would you have to apply based on your data types?\n",
        "\n",
        "*ANSWER HERE*"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "45ac5a0d",
      "metadata": {
        "id": "45ac5a0d"
      },
      "source": [
        "1. There were null values for Y/R which is a ratio of Yds over Rec. The null value was due to both Rec and Yds being 0. I filled the missing values with 0 using fillna instead of the mean since it made more sense in this case. There were missing values for the positions column which I filled with random positions.\n",
        "2. I have a mixed dataset containing numerical and categorical. I converted the categorical variable Pos to numerical using one-hot encoding. An asterisk at the end of the name is used to show that the player was selected for the Pro Bowl. I created a target column y which contain 1 for players with the * and 0 for without. The Rk (rank) and Tm (team) columns were dropped."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2a245d00",
      "metadata": {
        "id": "2a245d00"
      },
      "source": [
        "## Step 3: Implement Machine Learning Model (11 marks)\n",
        "\n",
        "In this section, you will implement three different supervised learning models (one linear and two non-linear) of your choice. You will use a pipeline to help you decide which model and hyperparameters work best. It is up to you to select what models to use and what hyperparameters to test. You can use the class examples for guidance. You must print out the best model parameters and results after the grid search."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5558a776",
      "metadata": {
        "id": "5558a776",
        "outputId": "9073ce13-f07f-440e-97da-c8364f2b70e0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Random Forest Results:\n",
            "Accuracy score: 0.9475\n",
            "F1 score: 0.46761904761904766\n",
            "\n",
            "Best Parameters for Random Forest based on F1: {'classifier__max_depth': 15, 'classifier__n_estimators': 50}\n",
            "\n",
            "Logistic Regression Results:\n",
            "Accuracy scores: 0.9324999999999999\n",
            "F1 scores: 0.4371184371184372\n",
            "\n",
            "Best Parameters for Logistic Regression based on F1: {'classifier__C': 0.01, 'classifier__penalty': 'l2', 'classifier__solver': 'liblinear'}\n",
            "\n",
            "SVM Results:\n",
            "Accuracy scores: 0.9574999999999999\n",
            "F1 scores: 0.5721212121212121\n",
            "\n",
            "Best Parameters for SVM based on F1: {'classifier__C': 10, 'classifier__kernel': 'rbf'}\n"
          ]
        }
      ],
      "source": [
        "# Implement pipeline and grid search here. Can add more code blocks if necessary\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Random Forest Pipeline\n",
        "rf_pipeline = Pipeline([\n",
        "    ('scaler', StandardScaler()),\n",
        "    ('classifier', RandomForestClassifier())\n",
        "])\n",
        "\n",
        "# Logistic Regression Pipeline\n",
        "lr_pipeline = Pipeline([\n",
        "    ('scaler', StandardScaler()),\n",
        "    ('classifier', LogisticRegression(max_iter=2500))\n",
        "])\n",
        "\n",
        "# SVM Pipeline\n",
        "svm_pipeline = Pipeline([\n",
        "    ('scaler', StandardScaler()),\n",
        "    ('classifier', SVC())\n",
        "])\n",
        "\n",
        "# param grids\n",
        "\n",
        "param_grid_rf = {\n",
        "    'classifier__n_estimators': [50, 100, 150, 200, 250],\n",
        "    'classifier__max_depth': [5, 10, 15, 20, 25]\n",
        "}\n",
        "\n",
        "param_grid_lr = {\n",
        "    'classifier__C': [0.01, 0.1, 1],\n",
        "    'classifier__penalty': ['l1', 'l2'],\n",
        "    'classifier__solver': ['liblinear', 'saga']\n",
        "}\n",
        "\n",
        "param_grid_svm = {\n",
        "    'classifier__C': [0.1, 1, 10, 100],\n",
        "    'classifier__kernel': ['linear', 'rbf']\n",
        "}\n",
        "\n",
        "# Define the scoring metrics you want to use\n",
        "scoring = {\n",
        "    'accuracy': make_scorer(accuracy_score),\n",
        "    'f1_score': make_scorer(f1_score)\n",
        "}\n",
        "\n",
        "grid_search_rf = GridSearchCV(rf_pipeline, param_grid_rf, cv=5, scoring=scoring, refit='f1_score')\n",
        "grid_search_lr = GridSearchCV(lr_pipeline, param_grid_lr, cv=5, scoring=scoring, refit='f1_score')\n",
        "grid_search_svm = GridSearchCV(svm_pipeline, param_grid_svm, cv=5, scoring=scoring, refit='f1_score')\n",
        "\n",
        "# Fit the models\n",
        "grid_search_rf.fit(X_train, y_train)\n",
        "grid_search_lr.fit(X_train, y_train)\n",
        "grid_search_svm.fit(X_train, y_train)\n",
        "\n",
        "# Get the best parameters based on F1\n",
        "best_params_rf = grid_search_rf.best_params_\n",
        "best_params_lr = grid_search_lr.best_params_\n",
        "best_params_svm = grid_search_svm.best_params_\n",
        "\n",
        "# Access the results for both scoring metrics\n",
        "results_rf = grid_search_rf.cv_results_\n",
        "results_lr = grid_search_lr.cv_results_\n",
        "results_svm = grid_search_svm.cv_results_\n",
        "\n",
        "# Print the results for accuracy and F1\n",
        "print(\"Random Forest Results:\")\n",
        "print(\"Accuracy score:\", results_rf['mean_test_accuracy'][results_rf['mean_test_f1_score'].argmax()])\n",
        "print(\"F1 score:\", max(results_rf['mean_test_f1_score']))\n",
        "print(\"\\nBest Parameters for Random Forest based on F1:\", best_params_rf)\n",
        "\n",
        "print(\"\\nLogistic Regression Results:\")\n",
        "print(\"Accuracy scores:\", results_lr['mean_test_accuracy'][results_lr['mean_test_f1_score'].argmax()])\n",
        "print(\"F1 scores:\", max(results_lr['mean_test_f1_score']))\n",
        "print(\"\\nBest Parameters for Logistic Regression based on F1:\", best_params_lr)\n",
        "\n",
        "print(\"\\nSVM Results:\")\n",
        "print(\"Accuracy scores:\", results_svm['mean_test_accuracy'][results_svm['mean_test_f1_score'].argmax()])\n",
        "print(\"F1 scores:\", max(results_svm['mean_test_f1_score']))\n",
        "print(\"\\nBest Parameters for SVM based on F1:\", best_params_svm)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3dbd7075",
      "metadata": {
        "id": "3dbd7075"
      },
      "source": [
        "### Questions (5 marks)\n",
        "\n",
        "1. (1 mark) Do you need regression or classification models for your dataset?\n",
        "1. (2 marks) Which models did you select for testing and why?\n",
        "1. (2 marks) Which model worked the best? Does this make sense based on the theory discussed in the course and the context of your dataset?\n",
        "\n",
        "*ANSWER HERE*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_o9p6e31y08d"
      },
      "source": [
        "1. I need classification models since the model will be predicting a categorical outcome, either the player is selected or not.\n",
        "2. I chose logistic regression for the linear model because logistic regression fits well to binary classification problems. Random forest was chosen because it builds multipe decision trees and combines their results, which can result in better accuracy and prevent overfitting. SVM was chosen since it performs relatively well if there are any outliers in the dataset.\n",
        "3. SVM model fit the dataset the best, with a F1 score of 0.57, Random Forest has F1 score of 0.45, and logistic regression has F1 score of 0.44. In class we discussed that SVM can fit really well to nonlinear and complex data and can separate data with a linear plane which resulted in the SVM fitting the best to this data."
      ],
      "id": "_o9p6e31y08d"
    },
    {
      "cell_type": "markdown",
      "id": "3f994e31",
      "metadata": {
        "id": "3f994e31"
      },
      "source": [
        "## Step 4: Validate Model (6 marks)\n",
        "\n",
        "Use the testing set to calculate the testing accuracy for the best model determined in Step 3."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "69e64c08",
      "metadata": {
        "id": "69e64c08",
        "outputId": "2100c7fc-ad86-47a2-9ebc-44d1cc28961a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "SVM Testing Accuracy: 0.97\n",
            "Test F1 Score: 0.5714285714285715\n"
          ]
        }
      ],
      "source": [
        "# Calculate testing accuracy (1 mark)\n",
        "y_pred_svm = grid_search_svm.predict(X_test)\n",
        "testing_accuracy_svm = accuracy_score(y_test, y_pred_svm)\n",
        "print(\"SVM Testing Accuracy:\", testing_accuracy_svm)\n",
        "\n",
        "f1=f1_score(y_test, y_pred_svm)\n",
        "print(\"Test F1 Score:\", f1)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1e4529ba",
      "metadata": {
        "id": "1e4529ba"
      },
      "source": [
        "\n",
        "### Questions (5 marks)\n",
        "\n",
        "1. (1 mark) Which accuracy metric did you choose?\n",
        "1. (1 mark) How do these results compare to those in part 3? Did this model generalize well?\n",
        "1. (3 marks) Based on your results and the context of your dataset, did the best model perform \"well enough\" to be used out in the real-world? Why or why not? Do you have any suggestions for how you could improve this analysis?\n",
        "\n",
        "*ANSWER HERE*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0hENLs0ky08e"
      },
      "source": [
        "1. I used F1 score and testing accuracy as the metrics. Since only 30 players are selected for pro bowl, F1 score is a good metric due to the imbalanced classes. I also did accuracy since there is no major downside to predicting false negatives.\n",
        "2. This model shows consistency with results from part 3. F1 score was 0.57 for part 3 and remained the same, therefore model did generalize well. The accuracy in part 3 was 0.96, which is also about the same.\n",
        "3. The model has an accuracy of 0.97, which is execellent. The F1 score is not that high which could mean that either precision or recall or both are low for this model. I dont think that this model is good enough currently to be used in the real world because of the low F1 score. I was able to improve the F1 score from 0.45 to 0.57 just by changing what I fill the empty values for the Pos column. I was filling it with random values and changed it to LB instead. To improve the model, i can revisit the feature pre-processing step again, also I can try different models as well."
      ],
      "id": "0hENLs0ky08e"
    },
    {
      "cell_type": "markdown",
      "id": "37b238f4",
      "metadata": {
        "id": "37b238f4"
      },
      "source": [
        "## Process Description (4 marks)\n",
        "Please describe the process you used to create your code. Cite any websites or generative AI tools used. You can use the following questions as guidance:\n",
        "1. Where did you source your code?\n",
        "1. In what order did you complete the steps?\n",
        "1. If you used generative AI, what prompts did you use? Did you need to modify the code at all? Why or why not?\n",
        "1. Did you have any challenges? If yes, what were they? If not, what helped you to be successful?"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93097bfe",
      "metadata": {
        "id": "93097bfe"
      },
      "source": [
        "*DESCRIBE YOUR PROCESS HERE*\n",
        "1. I used some code from lab 6, such as for the encoding and also the lecture examples.\n",
        "2. I attempted to complete he steps in order but ended up revisiting many of the steps multiple times.\n",
        "3. I used AI to understand my results better. Initially, I was filling the NaN values in the Pos column with other randomly selected values from the same column, I was getting different results for the F1 score ranging between 0.45 to 0.50. For part 4 it remained consistent at 0.57, which seemed a bit odd. I ended up just filling it with LB which did give me more consistent results.\n",
        "4. I used the labs and the lecture examples provided, which helped me to successfully complete this."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cd97b6ac",
      "metadata": {
        "id": "cd97b6ac"
      },
      "source": [
        "## Reflection (2 marks)\n",
        "Include a sentence or two about:\n",
        "- what you liked or disliked,\n",
        "- found interesting, confusing, challenging, motivating\n",
        "while working on this assignment.\n",
        "\n",
        "\n",
        "*ADD YOUR THOUGHTS HERE*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v4fn1hy_y08f"
      },
      "source": [
        "I liked that we got to chose our own data.\n",
        "\n",
        "Something I found interesting was that, accuracy is not always important. My model had a high accuracy, but if it simply predicts that no one will be selected for the Pro Bowl, it would still have a high accuracy; considering the imbalance of only 30 players being selected out of approximately 500 in the dataset. The F1 score is much more important to determine if the model is good enough, since it accounts for precision and recall."
      ],
      "id": "v4fn1hy_y08f"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.17"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}