Lesson 8 - More Python patterns and features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Additional features of Python & useful libraries\n",
    "\n",
    "This lecture is intended to teach you various things about Python that will help you understand more features of the language, to become more efficient coders, and to write code that is easier to read and understand.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Managing CSV files\n",
    "\n",
    "Python has a library ['csv'](https://docs.python.org/3/library/csv.html) that will make csv file management much easier than using a \"split\"\n",
    "\n",
    "\n",
    "### Reading CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Normal open and extract:\n",
      "Mark : 50\n",
      "Alejandro : 25\n",
      "Julia : 26\n",
      "Denise : 23\n",
      "Josef : 21\n",
      "\n",
      "\n",
      "Normal open and extract:\n",
      "<class 'csv.DictReader'>\n",
      "Name: Mark with age 50\n",
      "Name: Alejandro with age 25\n",
      "Name: Julia with age 26\n",
      "Name: Denise with age 23\n",
      "Name: Josef with age 21\n"
     ]
    }
   ],
   "source": [
    "import csv  # import the csv library\n",
    "#csv.list_dialects()\n",
    "\n",
    "print(\"Normal open and extract:\")\n",
    "with open('students.csv') as csvfile:\n",
    "     students = csv.reader(csvfile, delimiter=',', quotechar='\"')\n",
    "     for row in students:\n",
    "         print(' : '.join(row))  # note jow join works... join is a function of string objects\n",
    "\n",
    "print(\"\\n\\nNormal open and extract:\")\n",
    "with open('students.csv') as csvfile:\n",
    "    students = csv.DictReader(csvfile, fieldnames=(\"Name\", \"Age\"))\n",
    "    print(type(students))\n",
    "    for row in students:\n",
    "        #print(\"the row is an object of type \", type(row), \"with keys: \", row.keys())\n",
    "        print(\"Name: {} with age {}\".format(row[\"Name\"], row[\"Age\"]))\n",
    "        #print(\"Name: {} with age {}\".format(*row.values()))\n",
    "\n",
    "        #to convert to a normal Dictionary, do the following\n",
    "        #normaldict = dict(row)  # convert to normal dictionary\n",
    "        #print(list(normaldict.values()))\n",
    "        \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "\n",
    "### Writing CSV\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MEAL\tFOOD\tPRICE\n",
      "Desayuno\tHuevos rotos\t4.90\n",
      "Desayuno\tCafe con bolleria\t1.25\n",
      "Desayuno\tTosta con tomate\t1.90\n",
      "Cena\tPaella Valenciana\t12.90\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "import io\n",
    "\n",
    "with open('menu.csv', 'w', newline='') as csvfile:\n",
    "    menuwriter = csv.writer(csvfile, delimiter=\"\\t\", quotechar='\"')\n",
    "    menuwriter.writerow([\"MEAL\", \"FOOD\", \"PRICE\"])  # write a header!\n",
    "    \n",
    "    menuwriter.writerow([\"Desayuno\", \"Huevos rotos\", \"4.90\"])\n",
    "    menuwriter.writerow([\"Desayuno\", \"Cafe con bolleria\", \"1.25\"])\n",
    "    menuwriter.writerow([\"Desayuno\", \"Tosta con tomate\", \"1.90\"])\n",
    "    menuwriter.writerow([\"Cena\", \"Paella Valenciana\", \"12.90\"])\n",
    "csvfile.close\n",
    "\n",
    "csvfile = io.open(\"menu.csv\")\n",
    "print(csvfile.read())\n",
    "csvfile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "\n",
    "### Reading CSV with a header\n",
    "\n",
    "csv DictReader can automatically handle this situation (in this case, you don't set the 'fieldnames=[]' option)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OrderedDict([('MEAL', 'Desayuno'), ('FOOD', 'Huevos rotos'), ('PRICE', '4.90')])\n",
      "Huevos rotos\n",
      "OrderedDict([('MEAL', 'Desayuno'), ('FOOD', 'Cafe con bolleria'), ('PRICE', '1.25')])\n",
      "Cafe con bolleria\n",
      "OrderedDict([('MEAL', 'Desayuno'), ('FOOD', 'Tosta con tomate'), ('PRICE', '1.90')])\n",
      "Tosta con tomate\n",
      "OrderedDict([('MEAL', 'Cena'), ('FOOD', 'Paella Valenciana'), ('PRICE', '12.90')])\n",
      "Paella Valenciana\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "with open('menu.csv') as csvfile:\n",
    "    menu = csv.DictReader(csvfile, delimiter=\"\\t\", quotechar='\"')\n",
    "    for item in menu:\n",
    "        print(item)\n",
    "        print(item[\"FOOD\"])\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "# Efficient ways to work with CSV data\n",
    "\n",
    "Later in this course, and in subsequent courses, you will use complex libraries like 'panda' to deal with complex mathematical operations on complex data structures.  However, for very simple cases, you can often do things in just one line!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.237500000000001\n",
      "{'Desayuno', 'Cena'}\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "with open('menu.csv') as csvfile:\n",
    "    menu = list(csv.DictReader(csvfile, delimiter=\"\\t\"))  # NOTE!  using \"list\" here is memory-dangerous!\n",
    "\n",
    "    print(  sum(  float(item['PRICE']) for item in menu  ) / len(menu) ) # can ask for the length of the list\n",
    "    print(  set(  item['MEAL'] for item in menu))   # get distinct entries in that column (note datatype!)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27.8\n",
      "{'Desayuno', 'Cena'}\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "from decimal import *\n",
    "\n",
    "getcontext().prec = 3\n",
    "\n",
    "with open('menu.csv') as csvfile:\n",
    "    menu = list(csv.DictReader(csvfile, delimiter=\"\\t\"))  # NOTE!  using \"list\" here is memory-dangerous!\n",
    "\n",
    "    print(  sum(  Decimal(item['PRICE']) for item in menu  ) / len(menu) ) # Decimal sets number of digits...\n",
    "    print(  set(  item['MEAL'] for item in menu))   # get distinct entries in that column (note datatype!)\n",
    "\n",
    "# I will prove that we have not solved the problem!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27.7375\n",
      "{'Desayuno', 'Cena'}\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "with open('menu.csv') as csvfile:\n",
    "    menu = list(csv.DictReader(csvfile, delimiter=\"\\t\"))  # NOTE!  using \"list\" here is memory-dangerous!\n",
    "\n",
    "    print(  (sum(  (float(item['PRICE']) * 100) for item in menu  ) / len(menu)*100) / 10000 ) # Decimal sets number of digits...\n",
    "    print(  set(  item['MEAL'] for item in menu))   # get distinct entries in that column (note datatype!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "# More details about Functions\n",
    "\n",
    "We will look more closely about how to pass information into a function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = 1\n",
      "3\n",
      "x = 1\n",
      "6\n",
      "x = 1\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "def addition(x, y, z=None):\n",
    "    print(\"x = {}\".format(x))\n",
    "    if z:\n",
    "        result = x + y + z\n",
    "    else:\n",
    "        result = x + y\n",
    "        \n",
    "    return result\n",
    "\n",
    "\n",
    "print(addition(1,2))  # note that z is optional, because it has a default\n",
    "print(addition(1,2,3))\n",
    "print(addition(y=2,z=3,x=1))  # if you name them, they can be out-of-order\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "\n",
    "# Testing membership\n",
    "\n",
    "You will often want to ask if \"this\" appears somewhere in a list.  You can do this using the python keyword \"in\":\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      " Iterate over a list...\n",
      "it is not in the list\n",
      "\n",
      "\n",
      " Iterate over a Dictionary by key and value...\n",
      "the country IS in the list\n",
      "the currency IS in the list\n",
      "\n",
      "\n",
      " Iterate over all dictionary entries...\n",
      "Look at the datatype of currencies.items dict_items([('Spain', 'EUR'), ('America', 'USD'), ('Canada', 'CSD'), ('Japan', 'YEN')])\n",
      "Spain uses the currency EUR\n",
      "America uses the currency USD\n",
      "Canada uses the currency CSD\n",
      "Japan uses the currency YEN\n",
      "('Spain', 'EUR')\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n\\n Iterate over a list...\")\n",
    "\n",
    "mylist = [\"Spain\", \"Germany\", \"UK\", \"Netherlands\" ]\n",
    "x = \"Hungary\"\n",
    "\n",
    "if x in mylist:\n",
    "    print(\"It is in the list\")\n",
    "else:\n",
    "    print(\"it is not in the list\")\n",
    "\n",
    "    \n",
    "#################################################\n",
    "print(\"\\n\\n Iterate over a Dictionary by key and value...\")\n",
    "\n",
    "currencies = {\"Spain\": \"EUR\", \"America\": \"USD\", \"Canada\": \"CSD\", \"Japan\": \"YEN\"}\n",
    "country = \"Canada\"\n",
    "currency = \"YEN\"\n",
    "if country in currencies.keys():\n",
    "    print(\"the country IS in the list\")\n",
    "else:\n",
    "    print(\"the country is not in the list\")\n",
    "    \n",
    "if currency in currencies.values():\n",
    "    print(\"the currency IS in the list\")\n",
    "else:\n",
    "    print(\"the currency is not in the list\")\n",
    "    \n",
    "\n",
    "#########################################\n",
    "# Iterate over entities in a dictionary\n",
    "print(\"\\n\\n Iterate over all dictionary entries...\")\n",
    "print(\"Look at the datatype of currencies.items\", currencies.items())\n",
    "\n",
    "for (country, currency) in currencies.items():  # items will extract both key and value\n",
    "    print(\"{} uses the currency {}\".format(country, currency))\n",
    "    \n",
    "#print(currencies.items()[0])\n",
    "#print(list(currencies.items())[0])  # note that it is a TUPLE!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "\n",
    "# Managing your time\n",
    "\n",
    "there are specific functions and libraries in Python that manage dates and times (e.g. for correctly formatting date/time information before writing it to the database!)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Seconds/time:  1599731378.062345\n",
      "Now:  2020-09-10 05:49:38.065979\n",
      "Split:  2020 9 10 5 49 38\n",
      "Date:  2020-09-10\n"
     ]
    }
   ],
   "source": [
    "import time as time\n",
    "import datetime as date\n",
    "\n",
    "print(\"Seconds/time: \", time.time())  # number of seconds since Jan 1, 1970 \"the epoch\"\n",
    "now = date.datetime.fromtimestamp(time.time())\n",
    "print(\"Now: \", now)  # this is an ISO standard format (ISO 8601)\n",
    "print(\"Split: \", now.year, now.month, now.day, now.hour, now.minute, now.second)\n",
    "print(\"Date: \", now.date()) # this is an ISO standard format (ISO 8601)\n",
    "\n",
    "# ISO standard is required to write to mySQL date/time columns!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "\n",
    "# Fancy list generation\n",
    "\n",
    "You can put some functions into a List context, in order to generate a list output!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20]\n"
     ]
    }
   ],
   "source": [
    "my_list = [number for number in range(0,100) if number % 2 == 0]\n",
    "print(my_list[:11])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}