clustering Toronto part 2.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "file = \"Toronto_Task1.csv\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Borough</th>\n",
       "      <th>Neighbourhood</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Postal Code</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>M1B</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Malvern, Rouge</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1C</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Highland Creek, Port Union, Rouge Hill</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1E</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Guildwood, Morningside, West Hill</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1G</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Woburn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1H</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Cedarbrae</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Borough                           Neighbourhood\n",
       "Postal Code                                                     \n",
       "M1B          Scarborough                          Malvern, Rouge\n",
       "M1C          Scarborough  Highland Creek, Port Union, Rouge Hill\n",
       "M1E          Scarborough       Guildwood, Morningside, West Hill\n",
       "M1G          Scarborough                                  Woburn\n",
       "M1H          Scarborough                               Cedarbrae"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "toronto_csv = pd.read_csv(file).set_index(\"Postcode\")#csv file created in the previous part \n",
    "T_df = toronto_csv.rename_axis(\"Postal Code\" , axis = 'index') \n",
    "T_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">renaming \"Postcode\" to \"Postal Code\" because its the used term in \"Geospatial_Coordinates\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">getting the latitude and longitude for each postal code, after that the dataframe will have two extra columns which are the longitude and latitude"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have built a dataframe of the postal code of each neighborhood along with the borough name and neighborhood name, in order to utilize the Foursquare location data, we need to get the latitude and the longitude coordinates of each neighborhood.\n",
    "\n",
    "In an older version of this course, we were leveraging the Google Maps Geocoding API to get the latitude and the longitude coordinates of each neighborhood. However, recently Google started charging for their API: http://geoawesomeness.com/developers-up-in-arms-over-google-maps-api-insane-price-hike/, so we will use the Geocoder Python package instead: https://geocoder.readthedocs.io/index.html.\n",
    "\n",
    "The problem with this Package is you have to be persistent sometimes in order to get the geographical coordinates of a given postal code. So you can make a call to get the latitude and longitude coordinates of a given postal code and the result would be None, and then make the call again and you would get the coordinates. So, in order to make sure that you get the coordinates for all of our neighborhoods, you can run a while loop for each postal code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = \"http://cocl.us/Geospatial_data\" # where we will be getting our coordinations  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'wget' is not recognized as an internal or external command,\n",
      "operable program or batch file.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Postal Code</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>M1B</th>\n",
       "      <td>43.806686</td>\n",
       "      <td>-79.194353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1C</th>\n",
       "      <td>43.784535</td>\n",
       "      <td>-79.160497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1E</th>\n",
       "      <td>43.763573</td>\n",
       "      <td>-79.188711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1G</th>\n",
       "      <td>43.770992</td>\n",
       "      <td>-79.216917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1H</th>\n",
       "      <td>43.773136</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1J</th>\n",
       "      <td>43.744734</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1K</th>\n",
       "      <td>43.727929</td>\n",
       "      <td>-79.262029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1L</th>\n",
       "      <td>43.711112</td>\n",
       "      <td>-79.284577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1M</th>\n",
       "      <td>43.716316</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1N</th>\n",
       "      <td>43.692657</td>\n",
       "      <td>-79.264848</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Latitude  Longitude\n",
       "Postal Code                      \n",
       "M1B          43.806686 -79.194353\n",
       "M1C          43.784535 -79.160497\n",
       "M1E          43.763573 -79.188711\n",
       "M1G          43.770992 -79.216917\n",
       "M1H          43.773136 -79.239476\n",
       "M1J          43.744734 -79.239476\n",
       "M1K          43.727929 -79.262029\n",
       "M1L          43.711112 -79.284577\n",
       "M1M          43.716316 -79.239476\n",
       "M1N          43.692657 -79.264848"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "!wget -q -O \"toronto_m.geospatial_data.csv\" url\n",
    "toronto_new_csv = pd.read_csv(url).set_index(\"Postal Code\")\n",
    "toronto_new_csv.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Toronto = T_df.join(toronto_new_csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Borough</th>\n",
       "      <th>Neighbourhood</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Postal Code</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>M1B</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Malvern, Rouge</td>\n",
       "      <td>43.806686</td>\n",
       "      <td>-79.194353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1C</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Highland Creek, Port Union, Rouge Hill</td>\n",
       "      <td>43.784535</td>\n",
       "      <td>-79.160497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1E</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Guildwood, Morningside, West Hill</td>\n",
       "      <td>43.763573</td>\n",
       "      <td>-79.188711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1G</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Woburn</td>\n",
       "      <td>43.770992</td>\n",
       "      <td>-79.216917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1H</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Cedarbrae</td>\n",
       "      <td>43.773136</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1J</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Scarborough Village</td>\n",
       "      <td>43.744734</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1K</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>East Birchmount Park, Ionview, Kennedy Park</td>\n",
       "      <td>43.727929</td>\n",
       "      <td>-79.262029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1L</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Clairlea, Golden Mile, Oakridge</td>\n",
       "      <td>43.711112</td>\n",
       "      <td>-79.284577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1M</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Cliffcrest, Cliffside, Scarborough Village West</td>\n",
       "      <td>43.716316</td>\n",
       "      <td>-79.239476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>M1N</th>\n",
       "      <td>Scarborough</td>\n",
       "      <td>Birch Cliff, Cliffside West</td>\n",
       "      <td>43.692657</td>\n",
       "      <td>-79.264848</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Borough                                    Neighbourhood  \\\n",
       "Postal Code                                                                 \n",
       "M1B          Scarborough                                   Malvern, Rouge   \n",
       "M1C          Scarborough           Highland Creek, Port Union, Rouge Hill   \n",
       "M1E          Scarborough                Guildwood, Morningside, West Hill   \n",
       "M1G          Scarborough                                           Woburn   \n",
       "M1H          Scarborough                                        Cedarbrae   \n",
       "M1J          Scarborough                              Scarborough Village   \n",
       "M1K          Scarborough      East Birchmount Park, Ionview, Kennedy Park   \n",
       "M1L          Scarborough                  Clairlea, Golden Mile, Oakridge   \n",
       "M1M          Scarborough  Cliffcrest, Cliffside, Scarborough Village West   \n",
       "M1N          Scarborough                      Birch Cliff, Cliffside West   \n",
       "\n",
       "              Latitude  Longitude  \n",
       "Postal Code                        \n",
       "M1B          43.806686 -79.194353  \n",
       "M1C          43.784535 -79.160497  \n",
       "M1E          43.763573 -79.188711  \n",
       "M1G          43.770992 -79.216917  \n",
       "M1H          43.773136 -79.239476  \n",
       "M1J          43.744734 -79.239476  \n",
       "M1K          43.727929 -79.262029  \n",
       "M1L          43.711112 -79.284577  \n",
       "M1M          43.716316 -79.239476  \n",
       "M1N          43.692657 -79.264848  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Toronto.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Toronto.to_csv(\"Toronto_task2.csv\",index =False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(103, 4)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Toronto.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}