Tree_Methods_Consulting_Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tree Methods Consulting Project "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project detail:\n",
    "\n",
    "A dog food company wants to predict why some batches of their dog food are spoiling much quicker than intended! Unfortunately this Dog Food company hasn't upgraded to the latest machinery, meaning that the amounts of the five preservative chemicals they are using can vary a lot, but which is the chemical that has the strongest effect? The dog food company first mixes up a batch of preservative that contains 4 different preservative chemicals (A,B,C,D) and then is completed with a \"filler\" chemical. The food scientists believe one of the A,B,C, or D preservatives is causing the problem, but need help to figure out which one!\n",
    "Use Machine Learning with RF to find out which parameter had the most predicitive power, thus finding out which chemical causes the early spoiling! So create a model and then find out how you can decide which chemical is the problem!\n",
    "\n",
    "* Pres_A : Percentage of preservative A in the mix\n",
    "* Pres_B : Percentage of preservative B in the mix\n",
    "* Pres_C : Percentage of preservative C in the mix\n",
    "* Pres_D : Percentage of preservative D in the mix\n",
    "* Spoiled: Label indicating whether or not the dog food batch was spoiled.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature importance:\n",
    "\n",
    "In this project, we use 3 different available tree classifiers in MLlib:\n",
    "\n",
    "A single decision tree (DT)\n",
    "\n",
    "A random forest (RF)\n",
    "\n",
    "A gradient boosted tree (GBT) \n",
    "\n",
    "These classification approaches can be trained on a dataset and provide the **feature importance**. Feature importance reveals which variables are important, therefore can help us in early detection or even improving our product/service.\n",
    "\n",
    "The **.featureImportances** function is implemented in DecisionTreeClassifier, RandomForestClassifier, and GBTClassifier.\n",
    "\n",
    "### How the featureImportances is calculated:\n",
    "\n",
    "According to the Pyspark MLlib Documentation:\n",
    "\n",
    "#### Decision Tree:\n",
    "\n",
    "This feature importance is calculated as follows:\n",
    "          - importance(feature j) = sum (over nodes which split on feature j) of the gain,\n",
    "            where gain is scaled by the number of instances passing through node\n",
    "          - Normalize importances for tree to sum to 1.\n",
    "\n",
    "**NB.** Feature importance for a single decision trees can have high variance due to correlated predictor variables. Consider using a RandomForestClassifier to determine feature importance instead.\n",
    "\n",
    "#### Random forest and GBT: \n",
    "\n",
    "        - Each feature's importance is the average of its importance across all trees in the ensemble\n",
    "        - The importance vector is normalized to sum to 1. \n",
    "        \n",
    "        \n",
    "### Hyperparameters:\n",
    "\n",
    "Here, two important hyperparameters are explained.\n",
    "\n",
    "**Maximum depth**: It simply limits the number of levels in the decision tree. It is the maximum number of chained decisions that the classifier will make to classify an example. It is useful to limit this to avoid overfitting the training data.\n",
    "\n",
    "**Maximum bins**: The decision tree algorithm is responsible for coming up with potential decision rules to try at each level, like the 'A' >= 50 or 'A' >= 10 decisions. Decisions are always of the same form: for numeric features, decisions are of the form feature >= value; and for categorical features, they are of the form feature in (value1, value2, …). So, the set of decision rules to try is really a set of values to plug in to the decision rule. These are referred to as “bins” in the Spark MLlib implementation. A larger number of bins requires more processing time but might lead to finding a more optimal decision rule.\n",
    "\n",
    "## Feature selection:\n",
    "\n",
    "Once the feature importances are extracted, the columns with feature importance of less than a threshold can be dropped. Other feature selection strategies like Forward, backward, or stepwise method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.sql import SparkSession\n",
    "spark = SparkSession.builder.appName('MLlibTreeProject').getOrCreate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = spark.read.csv('dog_food.csv', inferSchema=True, header=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- A: integer (nullable = true)\n",
      " |-- B: integer (nullable = true)\n",
      " |-- C: double (nullable = true)\n",
      " |-- D: integer (nullable = true)\n",
      " |-- Spoiled: double (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "data.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------+------------------+------------------+------------------+------------------+-------------------+\n",
      "|summary|                 A|                 B|                 C|                 D|            Spoiled|\n",
      "+-------+------------------+------------------+------------------+------------------+-------------------+\n",
      "|  count|               490|               490|               490|               490|                490|\n",
      "|   mean|  5.53469387755102| 5.504081632653061| 9.126530612244897| 5.579591836734694| 0.2857142857142857|\n",
      "| stddev|2.9515204234399057|2.8537966089662063|2.0555451971054275|2.8548369309982857|0.45221563164613465|\n",
      "|    min|                 1|                 1|               5.0|                 1|                0.0|\n",
      "|    max|                10|                10|              14.0|                10|                1.0|\n",
      "+-------+------------------+------------------+------------------+------------------+-------------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "data.describe().show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+---+---+---+---+-------+\n",
      "|  A|  B|  C|  D|Spoiled|\n",
      "+---+---+---+---+-------+\n",
      "|  0|  0|  0|  0|      0|\n",
      "+---+---+---+---+-------+\n",
      "\n",
      "+---+---+---+---+-------+\n",
      "|  A|  B|  C|  D|Spoiled|\n",
      "+---+---+---+---+-------+\n",
      "|  0|  0|  0|  0|      0|\n",
      "+---+---+---+---+-------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#Count null/nan values for all columns:\n",
    "from pyspark.sql.functions import isnan, isnull, when, count, col\n",
    "\n",
    "data.select([count(when(isnull(c), c)).alias(c) for c in data.columns]).show()\n",
    "data.select([count(when(isnan(c), c)).alias(c) for c in data.columns]).show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+---+---+----+---+-------+\n",
      "|  A|  B|   C|  D|Spoiled|\n",
      "+---+---+----+---+-------+\n",
      "|  4|  2|12.0|  3|    1.0|\n",
      "|  5|  6|12.0|  7|    1.0|\n",
      "|  6|  2|13.0|  6|    1.0|\n",
      "|  4|  2|12.0|  1|    1.0|\n",
      "|  4|  2|12.0|  3|    1.0|\n",
      "| 10|  3|13.0|  9|    1.0|\n",
      "|  8|  5|14.0|  5|    1.0|\n",
      "|  5|  8|12.0|  8|    1.0|\n",
      "|  6|  5|12.0|  9|    1.0|\n",
      "|  3|  3|12.0|  1|    1.0|\n",
      "|  9|  8|11.0|  3|    1.0|\n",
      "|  1| 10|12.0|  3|    1.0|\n",
      "|  1|  5|13.0| 10|    1.0|\n",
      "|  2| 10|12.0|  6|    1.0|\n",
      "|  1| 10|11.0|  4|    1.0|\n",
      "|  5|  3|12.0|  2|    1.0|\n",
      "|  4|  9|11.0|  8|    1.0|\n",
      "|  5|  1|11.0|  1|    1.0|\n",
      "|  4|  9|12.0| 10|    1.0|\n",
      "|  5|  8|10.0|  9|    1.0|\n",
      "+---+---+----+---+-------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "data.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "490"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------+-----+\n",
      "|Spoiled|count|\n",
      "+-------+-----+\n",
      "|    0.0|  350|\n",
      "|    1.0|  140|\n",
      "+-------+-----+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# count the values\n",
    "data.groupby('Spoiled').count().show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.feature import VectorAssembler\n",
    "featureList = ['A','B','C','D']\n",
    "assembler = VectorAssembler(inputCols = featureList,\n",
    "                            outputCol = 'features')\n",
    "df = assembler.transform(data).withColumnRenamed('Spoiled', 'label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.classification import DecisionTreeClassifier,\\\n",
    "                                RandomForestClassifier, GBTClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt_classifier = DecisionTreeClassifier(featuresCol='features',labelCol='label')\n",
    "rf_classifier = RandomForestClassifier(featuresCol='features',labelCol='label')\n",
    "gbt_classifier = GBTClassifier(featuresCol='features',labelCol='label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.evaluation import BinaryClassificationEvaluator\n",
    "from pyspark.ml.tuning import CrossValidator, ParamGridBuilder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt_paramGrid = ParamGridBuilder() \\\n",
    "    .addGrid(dt_classifier.maxDepth, [5, 10])\\\n",
    "    .addGrid(dt_classifier.maxBins, [20, 60])\\\n",
    "    .build()\n",
    "\n",
    "\n",
    "rf_paramGrid = ParamGridBuilder() \\\n",
    "    .addGrid(rf_classifier.maxDepth, [5, 10])\\\n",
    "    .addGrid(rf_classifier.numTrees, [10, 20])\\\n",
    "    .addGrid(rf_classifier.maxBins, [20, 60])\\\n",
    "    .build()\n",
    "\n",
    "\n",
    "gbt_paramGrid = ParamGridBuilder() \\\n",
    "    .addGrid(gbt_classifier.maxDepth, [5, 10])\\\n",
    "    .addGrid(gbt_classifier.maxBins, [20, 60])\\\n",
    "    .build()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "dt_crossval = CrossValidator(estimator=dt_classifier,\n",
    "                          estimatorParamMaps=dt_paramGrid,\n",
    "                          evaluator=BinaryClassificationEvaluator(),\n",
    "                          numFolds=3)\n",
    "\n",
    "rf_crossval = CrossValidator(estimator=rf_classifier,\n",
    "                          estimatorParamMaps=rf_paramGrid,\n",
    "                          evaluator=BinaryClassificationEvaluator(),\n",
    "                          numFolds=3)\n",
    "\n",
    "gbt_crossval = CrossValidator(estimator=gbt_classifier,\n",
    "                          estimatorParamMaps=gbt_paramGrid,\n",
    "                          evaluator=BinaryClassificationEvaluator(),\n",
    "                          numFolds=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SparseVector(4, {0: 0.0152, 1: 0.0137, 2: 0.9327, 3: 0.0384})"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt_model = dt_crossval.fit(df)\n",
    "dt_model.bestModel.featureImportances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SparseVector(4, {0: 0.0377, 1: 0.0287, 2: 0.9027, 3: 0.0309})"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_model = rf_crossval.fit(df)\n",
    "rf_model.bestModel.featureImportances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SparseVector(4, {0: 0.0296, 1: 0.0383, 2: 0.8286, 3: 0.1034})"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gbt_model = gbt_crossval.fit(df)\n",
    "gbt_model.bestModel.featureImportances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feature at index 2 (Chemical C) is the most important feature and causing the early spoilage! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BarContainer object of 4 artists>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWsAAAD4CAYAAAAqw8chAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAALp0lEQVR4nO3df6heBR3H8fe3u364crcf04pp3gxlmTPLW0T0y5ISR2koskGQYI3M6o8iGhQU9YeroCgSZEWkQk0zAmMp9MMIQ4s7nY4FitqiJmWKrWhWur798Zy1u9td99zd55x7vu39Aum5z07P/fTs9vbsPPfZjcxEkjRsT1vuAZKkhRlrSSrAWEtSAcZakgow1pJUwIquHnj16tU5NTXV1cNL0v+lHTt2PJqZJ8y9v7NYT01NMTMz09XDS9L/pYj47Xz3exlEkgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBnb0pZtfefUxt3t7Vwy/Zni3rl3uCJLXmmbUkFWCsJakAYy1JBRhrSSrAWEtSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAoy1JBWw4E+KiYgDwC7g6cBTwHXAlzPzXx1vkyQ12vxYrycy82yAiDgR+DawCvh0l8MkSYcs6jJIZj4CbAI+FBHRzSRJ0lyLvmadmQ8BE8CJc38tIjZFxExEzBzYv28c+yRJjPkFxszcmpnTmTk9sXJynA8tSce0Rcc6Ik4FDgCPjH+OJGk+i4p1RJwAXAN8LTOzm0mSpLnafDfIcRGxk0Pfunc98KVOV0mSDrNgrDNzoo8hkqQj8x2MklSAsZakAoy1JBVgrCWpAGMtSQUYa0kqwFhLUgHGWpIKMNaSVICxlqQCjLUkFWCsJakAYy1JBRhrSSrAWEtSAW1++MBRWbdmkpkt67t6eEk6pnhmLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAoy1JBVgrCWpAGMtSQV09g7GXXv3MbV5e1cPL0mDtKejd257Zi1JBRhrSSrAWEtSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAoy1JBVgrCWpAGMtSQW0jnVEvCgitkXEgxGxIyJ+GBGndzlOkjTS6sd6RUQA3weuzcwNzX2vBF4I3N/dPEkStP8ZjOcCT2bmNQfvyMx7upkkSZqr7WWQM4EdCx0UEZsiYiYiZg7s37e0ZZKk/xjrC4yZuTUzpzNzemLl5DgfWpKOaW1jvRs4p8shkqQjaxvrnwLPjIhNB++IiLMi4o3dzJIkzdYq1pmZwLuB85pv3dsNXAX8octxkqSRtt8NQmY+DFza4RZJ0hH4DkZJKsBYS1IBxlqSCjDWklSAsZakAoy1JBVgrCWpAGMtSQUYa0kqwFhLUgHGWpIKMNaSVICxlqQCjLUkFWCsJamA1n+f9WKtWzPJzJb1XT28JB1TPLOWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAjp7B+OuvfuY2rz9qP67e3znoyQdxjNrSSrAWEtSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAoy1JBVgrCWpAGMtSQUYa0kqoFWsI+JAROyMiHsi4q6IeH3XwyRJh7T9sV5PZObZABHxDuAq4M2drZIkHeZoLoOsAh4f9xBJ0pG1PbM+LiJ2As8CXgy8db6DImITsAlgYtUJYxkoSWp/Zv1EZp6dmWuB84HrIiLmHpSZWzNzOjOnJ1ZOjnWoJB3LFn0ZJDPvAFYDnjpLUk8WHeuIWAtMAI+Nf44kaT6LvWYNEMB7M/NAR5skSXO0inVmTnQ9RJJ0ZL6DUZIKMNaSVICxlqQCjLUkFWCsJakAYy1JBRhrSSrAWEtSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKqDtDx9YtHVrJpnZsr6rh5ekY4pn1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklRAZ+9g3LV3H1Obt7c6do/vdJSk/8kza0kqwFhLUgHGWpIKMNaSVICxlqQCjLUkFWCsJakAYy1JBRhrSSrAWEtSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkFGGtJKqB1rCPioojIiFjb5SBJ0n9bzJn1RuD25j8lST1qFeuIeA7wBuByYEOniyRJ/6XtmfWFwK2ZeT/wWEScM99BEbEpImYiYubA/n1jGylJx7q2sd4IbGtub+MIl0Iyc2tmTmfm9MTKyXHskyQBKxY6ICKeD7wVWBcRCUwAGREfz8zseqAkqd2Z9SXA9Zl5SmZOZebJwG+AN3Y7TZJ0UJtYbwS+P+e+7+F3hUhSbxa8DJKZ585z31e7mSNJmo/vYJSkAoy1JBVgrCWpAGMtSQUYa0kqwFhLUgHGWpIKMNaSVICxlqQCjLUkFWCsJakAYy1JBRhrSSrAWEtSAcZakgpY8O+zPlrr1kwys2V9Vw8vSccUz6wlqQBjLUkFGGtJKsBYS1IBxlqSCjDWklSAsZakAoy1JBVgrCWpgMjMbh444q/AfZ08+NKsBh5d7hFHMNRtQ90Fw9021F0w3G1D3QX9bjslM0+Ye2dnbzcH7svM6Q4f/6hExMwQd8Fwtw11Fwx321B3wXC3DXUXDGObl0EkqQBjLUkFdBnrrR0+9lIMdRcMd9tQd8Fwtw11Fwx321B3wQC2dfYCoyRpfLwMIkkFGGtJKmDJsY6I8yPivoh4ICI2z/Prz4yIG5pf/2VETC31c45p15si4q6IeCoiLulj0yK2fTQifh0R90bETyLilIHs+kBE7IqInRFxe0Sc0ceuNttmHXdxRGRE9PJtVi2es8si4k/Nc7YzIt7Xx64225pjLm2+1nZHxLeHsCsivjzr+bo/Iv7cx66W214SEbdFxN3N/z8v6GsbmXnU/wATwIPAqcAzgHuAM+Yc80Hgmub2BuCGpXzOMe6aAs4CrgMu6XrTIredC6xsbl8xoOds1azb7wJuHcpz1hx3PPBz4E5gegi7gMuAr/X19bXIbacBdwPPaz4+cQi75hz/YeCbA3rOtgJXNLfPAPb09Xu61DPr1wIPZOZDmflPYBtw4ZxjLgSubW7fBLwtImKJn3fJuzJzT2beC/yr4y1Hs+22zNzffHgncNJAdv1l1ofPBvp6dbrN1xnA54DPA38f2K7l0Gbb+4GrM/NxgMx8ZCC7ZtsIfKeHXdBuWwKrmtuTwMM9bVtyrNcAv5v18e+b++Y9JjOfAvYBL1ji5x3HruWy2G2XA7d0umik1a6IuDIiHgS+AHykh12ttkXEq4GTM3N7T5ta7Wpc3PyR+aaIOLmfaa22nQ6cHhG/iIg7I+L8gewCoLn891Lgpz3sgnbbPgO8JyJ+D/yQ0Zl/L3yBccAi4j3ANPDF5d5yUGZenZkvAz4BfGq59wBExNOALwEfW+4t8/gBMJWZZwE/4tCfModgBaNLIW9hdAb79Yh47rIuOtwG4KbMPLDcQ2bZCHwrM08CLgCub77+OrfUT7IXmH2mcFJz37zHRMQKRn90eGyJn3ccu5ZLq20RcR7wSeBdmfmPoeyaZRtwUaeLDllo2/HAmcDPImIP8Drg5h5eZFzwOcvMx2b9/n0DOKfjTa23MTpzvDkzn8zM3wD3M4r3cu86aAP9XQKBdtsuB24EyMw7gGcx+kueurfEC/IrgIcY/VHl4AX5V8w55koOf4Hxxh5eKFhw16xjv0W/LzC2ec5exeiFjtMGtuu0WbffCcwMZduc439GPy8wtnnOXjzr9ruBO4fynAHnA9c2t1czugTwguXe1Ry3FthD88a9AT1ntwCXNbdfzuiadS8bx/E/8AJG/0Z+EPhkc99nGZ0RwujfPN8FHgB+BZza0xO/0K7XMDqz+BujM/3dPX5RLLTtx8AfgZ3NPzcPZNdXgN3Nptv+VzD73jbn2F5i3fI5u6p5zu5pnrO1Q3nOgGB0+ejXwC5gwxB2NR9/BtjS13O1iOfsDOAXze/nTuDtfW3z7eaSVIAvMEpSAcZakgow1pJUgLGWpAKMtSQVYKwlqQBjLUkF/Bt7bXUe1Omc+wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.barh(featureList , gbt_model.bestModel.featureImportances)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}