Section_2_Python_Concepts.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Section_2_Python_Concepts.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/rrajasek95/nlp-243-notebooks/blob/main/Section_2_Python_Concepts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0mnotTK6fGz"
      },
      "source": [
        "# NLP 243 Section 02 - Python Concepts\n",
        "\n",
        "In this notebook, we explore will be exploring intermediate concepts in Python that will become more relevant as we progress in the course. \n",
        "\n",
        "The outline of the section is as follows:\n",
        "* Python Classes\n",
        "* File I/O\n",
        "* Data Serialization\n",
        "* Object Serialization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0pTpwxzRXZHK"
      },
      "source": [
        "# New Section"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x88y_QX07PNl"
      },
      "source": [
        "## Python Classes\n",
        "\n",
        "Python supports Object Oriented Programming through the concept of classes. To give a refresher, classes are programming constructs that provide a template/blueprint to group information and behavior acting on that information into a single unit. \n",
        "\n",
        "This unit is what's referred to as an *object* or an *instance*. All values in Python are defined as objects. \n",
        "\n",
        "Examples of classes include the `int`, `str`, `float` classes. Examples of objects of those classes include `1`, `\"hello\"` and `1.0`. \n",
        "\n",
        "Thus, in this case, we can view classes as types for data. One useful aspect is that we can define more complex types and program units using Object Oriented Programming. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LDmlt9B3L5EU"
      },
      "source": [
        "### Case Study 1: Defining a min-heap using classes\n",
        "\n",
        "One powerful use case for Object Oriented Programming is to define custom data structures. Python has a [heapq](https://docs.python.org/3/library/heapq.html) module which uses functions to modify an array to use it as a priority queue.\n",
        "\n",
        "It defines the following methods:\n",
        "\n",
        "* `heapq.heappush(heap, item)` that pushes `item` into the array `heap`\n",
        "* `heapq.heappop(heap)` that pops and returns the smallest item from the heap.\n",
        "* `heapq.heappushpop(heap, item)` that pushes `item` and pops the smallest item off the heap. This is more efficient than calling `heappush` followed by `heappop`.\n",
        "\n",
        "The issue with this sort of implementation is that you have to annoyingly pass the same list as an argument over and over again to use it in your program. We could instead wrap this implementation into a class."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Q3cv6bu6alf"
      },
      "source": [
        "import heapq\n",
        "\n",
        "class Heap(object):\n",
        "    \"\"\"\n",
        "    Wrapper class for some of the heapq methods.\n",
        "    \"\"\"\n",
        "    def __init__(self): \n",
        "        # Constructor:\n",
        "        # Initializes the underlying heap array\n",
        "        self.heapArray = []\n",
        "\n",
        "    def push(self, item):\n",
        "        heapq.heappush(self.heapArray, item)\n",
        "    \n",
        "    def pop(self):\n",
        "        return heapq.heappop(self.heapArray)\n",
        "    \n",
        "    def pushpop(self, item):\n",
        "        return heapq.heappushpop(self.heapArray, item)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.heapArray)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cXaIDf4WM11p"
      },
      "source": [
        "This allows us to neatly use heaps as priority queues."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8_j3SsvELFJg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        },
        "outputId": "135437f5-7012-4439-b758-45122c4c6be6"
      },
      "source": [
        "hq = Heap()\n",
        "\n",
        "hq.push((2, 'Test'))\n",
        "hq.push((1, 'Hello'))\n",
        "hq.push((3, 'World'))\n",
        "\n",
        "while len(hq) > 0:\n",
        "    print(hq.pop())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(1, 'Hello')\n",
            "(2, 'Test')\n",
            "(3, 'World')\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yhlpEqdg3Plo"
      },
      "source": [
        "### Case Study 2: Extending heap for modifying behavior\n",
        "\n",
        "The heap class we defined behaves as a min-heap. However, this may not be useful for our purposes especially when we need to use them as a max-heap. Furhtermore, the `heapq` module does not provide a proper implementation of a max-heap. \n",
        "\n",
        "We will extend the heap class we supported to provide the max-heap behavior. One insight that will help us to implement it is how a min-heap is implemented. \n",
        "\n",
        "A min-heap places the smallest element at the root of the heap. It does so by comparing itself with some of the other elements such that it is less than them. Therefore, if we want max-heap behavior, we need to convert that \"less-than\" comparison to a \"greater-than\" comparison. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x5QIRBVB4Z97",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        },
        "outputId": "c5e47125-f061-4493-9a0d-ebf21b22c880"
      },
      "source": [
        "\"\"\"\n",
        "Solution derived from the following StackOverflow answer:\n",
        "https://stackoverflow.com/a/40455775\n",
        "\n",
        "This is merely an illustrative example to show how subclassing works. \n",
        "This may not be representative of best practice for implementing these classes.\n",
        "\"\"\"\n",
        "\n",
        "class MaxHeapItem(object):\n",
        "    # The MaxHeapItem inverts the comparison operator \n",
        "    # to convert a less-than comparison to a greater-than comparison\n",
        "    # this is a hack to force the min-heap to behave as a max-heap.\n",
        "    \n",
        "    def __init__(self, val):\n",
        "        self.val = val\n",
        "\n",
        "    def __lt__(self, other): # if x < y, place x before y; if x > y, we place x before y\n",
        "        # Note: normal implementation of less than would be\n",
        "        # self.val < other.val ; This inverts the comparison operator\n",
        "        return self.val > other.val\n",
        "\n",
        "    def __eq__(self, other):\n",
        "        return self.val == other.val\n",
        "\n",
        "    def __str__(self): \n",
        "        return str(self.val)\n",
        "    \n",
        "# Normal python implementation tries to place the smallest value at the top\n",
        "# What we want: is to place the largest value on top\n",
        "# compare(x, y): if x < y, it places x above y in the heap\n",
        "# What we want is compare(x, y): if x > y, we place x above y in the heap\n",
        "\n",
        "class MaxHeap(Heap): \n",
        "    # Placing Heap between parentheses indicates we are subclassing it\n",
        "    # i.e. we will inherit all the underlying methods of the heap\n",
        "\n",
        "    def push(self, item): \n",
        "        # We retain the underlying min heap behavior\n",
        "        # by calling super().push etc...\n",
        "        super().push(MaxHeapItem(item))\n",
        "    \n",
        "    def pop(self):\n",
        "        return super().pop().val\n",
        "    \n",
        "    def pushpop(self, item):\n",
        "        return super().pushpop(MaxHeapItem(item))\n",
        "\n",
        "hq2 = MaxHeap()\n",
        "\n",
        "hq2.push((2, 'Test'))\n",
        "hq2.push((1, 'Hello'))\n",
        "hq2.push((3, 'World'))\n",
        "\n",
        "while len(hq2) > 0:\n",
        "    print(hq2.pop())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(3, 'World')\n",
            "(2, 'Test')\n",
            "(1, 'Hello')\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HkvJyHhyLwtq"
      },
      "source": [
        "## Python File I/O"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hntt4UiuUaJb"
      },
      "source": [
        "Throughout this course, you will rely on reading and writing to files and storing data in various formats.\n",
        "\n",
        "This section will cover some specific ways to perform file I/O, as well as some custom formats such as JSON, CSV."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OWueJhNIW7Ir"
      },
      "source": [
        "#### Vanilla File I/O\n",
        "\n",
        "File I/O typically requires us to make an Operation System call (or syscall for short) to *open* a file for a certain action. \n",
        "\n",
        "There are three common actions (or modes) supported:\n",
        "* Read\n",
        "* Write (Starting from the beginning of the file)\n",
        "* Append (Write starting at the end of the file)\n",
        "\n",
        "You can open a file with one or more modes if necessary, but normally it's preferable to open a file with only one or the other mode to ensure you don't perform unintended actions such as writing to a file you only wanted to read.\n",
        "\n",
        "Once you're done reading or writing to a file, you will have to *close* the file. The reason is that the OS maintains information about open files in memory. Failing to close them can lead to that memory not being freed - causing a memory leak."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aK9UO_HFcF31"
      },
      "source": [
        "##### Manual File I/O\n",
        "\n",
        "A typical file I/O workflow that's commonly done is the following:\n",
        "* Open the file\n",
        "* Read/ Write/ Append to the file\n",
        "* Close the file\n",
        "\n",
        "The above is what we will be doing in this course.\n",
        "\n",
        "Other workflows include:\n",
        "* Open the file and keep writing to it (logger workflow)\n",
        "* Open the file and keep reading from it (streaming file workflow)\n",
        "\n",
        "The above cases occur with daemon processes or long-running programs, therefore, when those programs are terminated, we will need to close the files appropriately as a clean-up step."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VXc6D3JiZjpN",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b8f2cd33-724a-4a44-ece7-8b5ec52b6103"
      },
      "source": [
        "# Please refer to the documentation for further info:\n",
        "# https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files\n",
        "\n",
        "########################################################\n",
        "# Opening a file in python and writing some data to it #\n",
        "########################################################\n",
        "test_file = open('file.txt', 'w') # w indicates that only writing can be done\n",
        "\n",
        "# Using the flag 'w' allows us to write the data as a string\n",
        "test_file.write('Hello, World!')\n",
        "\n",
        "# Add more data\n",
        "test_file.write(' Additional Data')\n",
        "test_file.writelines(['\\nData on a new line\\n', 'Some more data'])\n",
        "test_file.close() # Close the file to free memory\n",
        "\n",
        "############################################\n",
        "# Now opening the same file and reading it #\n",
        "############################################\n",
        "same_file = open('file.txt', 'r') # r indicates that only reading can be done\n",
        "\n",
        "# Calling read without arguments will read the entire file\n",
        "content = same_file.readlines()\n",
        "\n",
        "same_file.close()\n",
        "\n",
        "print(content)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Hello, World! Additional Data\\n', 'Data on a new line\\n', 'Some more data']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pPDQhNd-b_L-"
      },
      "source": [
        "##### Context Managers\n",
        "\n",
        "In the above section, we talked about the first use case and the need for closing files. \n",
        "\n",
        "One particular concern that arises is what happens during failure.\n",
        "\n",
        "In our workflow:\n",
        "* Open the file\n",
        "* Read/ Write/ Append to the file\n",
        "* Close the file\n",
        "\n",
        "We have three failure cases:\n",
        "* We fail to open the file\n",
        "* We fail to read / write / append to the file\n",
        "* We fail to close the file\n",
        "\n",
        "This leads to the following handling cases to ensure that we have no open files:\n",
        "* File is not open, so we must do nothing\n",
        "* We failed to read/write, so we may have to close the file\n",
        "* We will have to try closing it, otherwise give up."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PboudLUeb9-a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        },
        "outputId": "dc3f1f35-8bbd-4d9f-934d-ecb35f7a85b0"
      },
      "source": [
        "print(\"Write failure case:\\n---\")\n",
        "try:\n",
        "    file2 = open('file2.txt', 'w')\n",
        "    # We are trying to write an integer when write expects a string\n",
        "    file2.write(3)\n",
        "    file2.close()\n",
        "except TypeError:\n",
        "    # Program throws an error\n",
        "    print(file2.closed) # file2 remains open, so we must close it\n",
        "    file2.close()\n",
        "\n",
        "print(\"\\nFile not found case:\\n---\")\n",
        "try:\n",
        "    file3 = open('file3.txt', 'r') # We are trying to read a nonexistent file\n",
        "    data = file3.read()\n",
        "    file3.close()\n",
        "except FileNotFoundError:\n",
        "    print(\"File not found\") # File is not found, so we should do nothing\n",
        "except IOError: # file is found but throws an error, so close in this case\n",
        "    file3.close()\n",
        "\n",
        "print(\"\\nAlternative # 1\\n---\")\n",
        "# Alternative way #1 to handle the previous case\n",
        "try:\n",
        "    file3 = None\n",
        "    file3 = open('file3.txt', 'r') # We are trying to read a nonexistent file\n",
        "    data = file3.read()\n",
        "    file3.close()\n",
        "except IOError:\n",
        "    print(\"Some IO error happened\")\n",
        "    if file3:\n",
        "        print(\"File exists, closing\") # this never gets executed\n",
        "        file3.close()\n",
        "\n",
        "print(\"\\nAlternative #2\\n---\")\n",
        "# Alternative way #2 to handle the previous case\n",
        "try:\n",
        "    file3 = None\n",
        "    file3 = open('file3.txt', 'r') # We are trying to read a nonexistent file\n",
        "    data = file3.read()\n",
        "except IOError:\n",
        "    print(\"Some IO error happened\")\n",
        "finally: # Finally block always gets executed, so close file here\n",
        "    if file3:\n",
        "        file3.close()\n",
        "\n",
        "try:\n",
        "    try:\n",
        "        file3 = open('file3.txt', 'r') # We are trying to read a nonexistent file\n",
        "        data = file3.read()\n",
        "        file3.close()\n",
        "    except FileNotFoundError:\n",
        "        print(\"File not found\") # File is not found, so we should do nothing\n",
        "    except IOError: # file is found but throws an error, so close in this case\n",
        "        file3.close()\n",
        "except IOError: # If we fail to close, try again to close\n",
        "    file3.close()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Write failure case:\n",
            "---\n",
            "False\n",
            "\n",
            "File not found case:\n",
            "---\n",
            "File not found\n",
            "\n",
            "Alternative # 1\n",
            "---\n",
            "Some IO error happened\n",
            "\n",
            "Alternative #2\n",
            "---\n",
            "Some IO error happened\n",
            "File not found\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JSW_5lfnniLS"
      },
      "source": [
        "In the above example, we had to handle so many cases of failure where we need to ensure that the file is closed correctly. Trying to recover from a I/O error such as this is very human-error prone, so Python provides constructs to handle this automatically. These are called context managers.\n",
        "\n",
        "Context management is invoked using the `with` keyword as follows:\n",
        "```\n",
        "with open('file.txt', 'r') as file:\n",
        "    # Do some action with file\n",
        "    read_data = file.read()\n",
        "```\n",
        "\n",
        "After the indented block, the file with automatically close itself since we have no access to the variable outside of the scope of the context manager. One advantage that context managers give is that they automatically clean up in the case of an exception. Thus, we don't have to write cascades of error management code to ensure that we clean up properly. However, we will still need to handle the exceptions according to our program specification and display the correct error messages, when needed."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lz0d-9mILaZy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 258
        },
        "outputId": "e9b9c58a-f832-426f-ed12-25de83d35828"
      },
      "source": [
        "# In this example, we will demonstrate writing a string to a file, followed\n",
        "# by reading it back\n",
        "\n",
        "STORY = \"\"\"\n",
        "A Hare was making fun of the Tortoise one day for being so slow.\n",
        "\n",
        "\"Do you ever get anywhere?\" he asked with a mocking laugh.\n",
        "\n",
        "\"Yes,\" replied the Tortoise, \"and I get there sooner than you think. I'll run you a race and prove it.\"\n",
        "\n",
        "The Hare was much amused at the idea of running a race with the Tortoise, but for the fun of the thing he agreed. So the Fox, who had consented to act as judge, marked the distance and started the runners off.\n",
        "\n",
        "The Hare was soon far out of sight, and to make the Tortoise feel very deeply how ridiculous it was for him to try a race with a Hare, he lay down beside the course to take a nap until the Tortoise should catch up.\n",
        "\n",
        "The Tortoise meanwhile kept going slowly but steadily, and, after a time, passed the place where the Hare was sleeping. But the Hare slept on very peacefully; and when at last he did wake up, the Tortoise was near the goal. The Hare now ran his swiftest, but he could not overtake the Tortoise in time.\n",
        "\"\"\"\n",
        "\n",
        "with open('hare_and_tortoise.txt', 'w') as hare_tortoise_file:\n",
        "    hare_tortoise_file.write(STORY)\n",
        "\n",
        "with open('hare_and_tortoise.txt', 'r') as hare_tortoise_file_read:\n",
        "    loaded_story = hare_tortoise_file_read.read()\n",
        "\n",
        "print(loaded_story)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "A Hare was making fun of the Tortoise one day for being so slow.\n",
            "\n",
            "\"Do you ever get anywhere?\" he asked with a mocking laugh.\n",
            "\n",
            "\"Yes,\" replied the Tortoise, \"and I get there sooner than you think. I'll run you a race and prove it.\"\n",
            "\n",
            "The Hare was much amused at the idea of running a race with the Tortoise, but for the fun of the thing he agreed. So the Fox, who had consented to act as judge, marked the distance and started the runners off.\n",
            "\n",
            "The Hare was soon far out of sight, and to make the Tortoise feel very deeply how ridiculous it was for him to try a race with a Hare, he lay down beside the course to take a nap until the Tortoise should catch up.\n",
            "\n",
            "The Tortoise meanwhile kept going slowly but steadily, and, after a time, passed the place where the Hare was sleeping. But the Hare slept on very peacefully; and when at last he did wake up, the Tortoise was near the goal. The Hare now ran his swiftest, but he could not overtake the Tortoise in time.\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bFUKW1Liv5Pm"
      },
      "source": [
        "## Data Serialization\n",
        "\n",
        "Very so often, we want to be able to store structured data into files or serialize different kinds of objects such as lists, dictionaries into file.\n",
        "\n",
        "For these use cases, it is useful to use the `json` and `csv` libraries of Python. We will not be covering the formats themselves in too much detail, but rather how to use them to read/write data structures."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QHPhXzWJxiuS"
      },
      "source": [
        "### JSON\n",
        "\n",
        "JSON refers to Javascript Object Notation which represents how data is represented in a structured way.\n",
        "\n",
        "JSON data comprises of a dictionary of key-value pairs:\n",
        "```\n",
        "{\n",
        "    \"key1\": value1,\n",
        "    \"key2\": value2,\n",
        "    ...\n",
        "    \"keyN\": valuen\n",
        "}\n",
        "```\n",
        "\n",
        "For the JSON format, the keys are double quoted strings and can be any string (including empty string!). \n",
        "\n",
        "Values can be of the following types:\n",
        "* Numbers (int or float)\n",
        "* Strings\n",
        "* Other JSON dictionaries\n",
        "* Lists (of any kind of value)\n",
        "\n",
        "For example, if we want to represent the information about our course, we can do it as follows\n",
        "\n",
        "```\n",
        "{\n",
        "    \"id\": 10,\n",
        "    \"course_code\": \"NLP243\",\n",
        "    \"term\": {\n",
        "        \"year\": 2020,\n",
        "        \"quarter\": \"Fall\"\n",
        "    },\n",
        "    \"name\": \"Machine Learning for NLP\",\n",
        "    \"instructor\": {\n",
        "        \"id\": 1,\n",
        "        \"name\": \"Dilek Hakkani-Tur\"\n",
        "    },\n",
        "    \"tas\": [\n",
        "        {\n",
        "            \"id\": 2,\n",
        "            \"name\": \"Rishi Rajasekaran\"\n",
        "        },\n",
        "        {\n",
        "            \"id\": 3,\n",
        "            \"name\": \"Zekun Zhao\"    \n",
        "        }\n",
        "    ]\n",
        "}\n",
        "```\n",
        "\n",
        "In the above object, we have \"id\" keys which are stored as integers, \"course_code\" and \"name\" stored as strings, \"instructor\"  and \"term\" as other dictionary objects, and \"tas\" as a list of other objects. \n",
        "\n",
        "In Python, we use the `json` library to store data as json. It supports creating JSON strings out of dictionaries or other basic data types and reading/writing them with files."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CHKeH1QmWB7u",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "outputId": "facaff8f-0c48-4fe2-a7df-b236494899e7"
      },
      "source": [
        "import json\n",
        "\n",
        "# This block demonstrates converting a python dictionary into a JSON string\n",
        "\n",
        "# using our course object, this is a python dictionary\n",
        "course = {\n",
        "    \"id\": 10,\n",
        "    \"course_code\": \"NLP243\",\n",
        "    \"term\": {\n",
        "        \"year\": 2020,\n",
        "        \"quarter\": \"Fall\"\n",
        "    },\n",
        "    \"name\": \"Machine Learning for NLP\",\n",
        "    \"instructor\": {\n",
        "        \"id\": 1,\n",
        "        \"name\": \"Dilek Hakkani-Tur\"\n",
        "    },\n",
        "    \"tas\": [\n",
        "        {\n",
        "            \"id\": 2,\n",
        "            \"name\": \"Rishi Rajasekaran\"\n",
        "        },\n",
        "        {\n",
        "            \"id\": 3,\n",
        "            \"name\": \"Zekun Zhao\"    \n",
        "        }\n",
        "    ]\n",
        "}\n",
        "print(\"Original course object:\")\n",
        "print(course)\n",
        "print(type(course))\n",
        "\n",
        "# Convert the dictionary to a JSON string\n",
        "course_json = json.dumps(course)\n",
        "print(\"\\nSeralized course into JSON:\")\n",
        "print(course_json)\n",
        "print(type(course_json))\n",
        "\n",
        "course_deserialized = json.loads(course_json)\n",
        "# Try and spot the difference on how python displays a dict vs the JSON string.\n",
        "print(\"\\nDeserialized course from JSON:\")\n",
        "print(course_deserialized) \n",
        "print(type(course_deserialized))\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original course object:\n",
            "{'id': 10, 'course_code': 'NLP243', 'term': {'year': 2020, 'quarter': 'Fall'}, 'name': 'Machine Learning for NLP', 'instructor': {'id': 1, 'name': 'Dilek Hakkani-Tur'}, 'tas': [{'id': 2, 'name': 'Rishi Rajasekaran'}, {'id': 3, 'name': 'Zekun Zhao'}]}\n",
            "<class 'dict'>\n",
            "\n",
            "Seralized course into JSON:\n",
            "{\"id\": 10, \"course_code\": \"NLP243\", \"term\": {\"year\": 2020, \"quarter\": \"Fall\"}, \"name\": \"Machine Learning for NLP\", \"instructor\": {\"id\": 1, \"name\": \"Dilek Hakkani-Tur\"}, \"tas\": [{\"id\": 2, \"name\": \"Rishi Rajasekaran\"}, {\"id\": 3, \"name\": \"Zekun Zhao\"}]}\n",
            "<class 'str'>\n",
            "\n",
            "Deserialized course from JSON:\n",
            "{'id': 10, 'course_code': 'NLP243', 'term': {'year': 2020, 'quarter': 'Fall'}, 'name': 'Machine Learning for NLP', 'instructor': {'id': 1, 'name': 'Dilek Hakkani-Tur'}, 'tas': [{'id': 2, 'name': 'Rishi Rajasekaran'}, {'id': 3, 'name': 'Zekun Zhao'}]}\n",
            "<class 'dict'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vy4_UKyR2DzJ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "outputId": "4a501a71-bbd0-4222-f60c-b63df1d3a2b3"
      },
      "source": [
        "# We also have convencience methods for reading/writing JSON into files\n",
        "import json\n",
        "\n",
        "course = {\n",
        "    \"id\": 10,\n",
        "    \"course_code\": \"NLP243\",\n",
        "    \"term\": {\n",
        "        \"year\": 2020,\n",
        "        \"quarter\": \"Fall\"\n",
        "    },\n",
        "    \"name\": \"Machine Learning for NLP\",\n",
        "    \"instructor\": {\n",
        "        \"id\": 1,\n",
        "        \"name\": \"Dilek Hakkani-Tur\"\n",
        "    },\n",
        "    \"tas\": [\n",
        "        {\n",
        "            \"id\": 2,\n",
        "            \"name\": \"Rishi Rajasekaran\"\n",
        "        },\n",
        "        {\n",
        "            \"id\": 3,\n",
        "            \"name\": \"Zekun Zhao\"    \n",
        "        }\n",
        "    ]\n",
        "}\n",
        "\n",
        "with open('nlp243.json', 'w') as nlp_json_file:\n",
        "    json.dump(course, nlp_json_file) # Dump into file\n",
        "\n",
        "with open('nlp243.json', 'r') as nlp_json_file:\n",
        "    deserialized_courses = json.load(nlp_json_file) # Load from file\n",
        "\n",
        "# dump and load operate on file objects\n",
        "# dump: wraps calling dumps and file.write\n",
        "# load: wraps calling file.read and loads\n",
        "\n",
        "print(\"Loaded course information:\")\n",
        "print(deserialized_courses)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loaded course information:\n",
            "{'id': 10, 'course_code': 'NLP243', 'term': {'year': 2020, 'quarter': 'Fall'}, 'name': 'Machine Learning for NLP', 'instructor': {'id': 1, 'name': 'Dilek Hakkani-Tur'}, 'tas': [{'id': 2, 'name': 'Rishi Rajasekaran'}, {'id': 3, 'name': 'Zekun Zhao'}]}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mcBOIbZJ3W5F"
      },
      "source": [
        "### CSV\n",
        "\n",
        "CSV stands for Comma Separated Values and is used to store tabular data. \n",
        "\n",
        "Here is an example of tabular data:\n",
        "\n",
        "| First Name | Last Name   | Role       |\n",
        "|------------|-------------|------------|\n",
        "| Dilek      | Hakkani-Tur | Instructor |\n",
        "| Rishi      | Rajasekaran | TA         |\n",
        "| Zekun      | Zhao        | TA         |\n",
        "\n",
        "This table can be stored easily in a text file as:\n",
        "```\n",
        "First Name, Last Name, Role\\n\n",
        "Dilek,Hakkani-Tur,Instructor\\n\n",
        "Rishi,Rajasekaran,TA\\n\n",
        "Zekun,Zhao,TA\\n\n",
        "```\n",
        "\n",
        "Here, the commas act as column delimiters and the `\\n` acts as a row delimiter.\n",
        "\n",
        "There will be many cases where we will be using CSVs to store data, including the homeworks where we will provide all data files as CSVs.\n",
        "\n",
        "We can commonly represent tabular data in Python in two ways:\n",
        "* A list of tuples/lists\n",
        "* A list of dicts\n",
        "\n",
        "These can then be serialized easily to a CSV file using the `csv` module."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XOyOorUV3J55",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        },
        "outputId": "8b4586fa-85fd-4a03-fd1c-069fb86fa511"
      },
      "source": [
        "import csv\n",
        "\n",
        "##############################################\n",
        "# Reading and writing lists of data directly #\n",
        "##############################################\n",
        "course_staff = [\n",
        "                ('Dilek', 'Hakkani-Tur', 'Instructor'),\n",
        "                ('Rishi', 'Rajasekaran', 'TA'),\n",
        "                ('Zekun', 'Zhao', 'TA')\n",
        "]\n",
        "\n",
        "with open('nlp243_staff.csv', 'w') as staff_csv:\n",
        "    fieldnames = ['First Name', 'Last Name', 'Role']\n",
        "\n",
        "    staff_writer = csv.writer(staff_csv, delimiter=',')\n",
        "    staff_writer.writerow(fieldnames)\n",
        "    staff_writer.writerows(course_staff)\n",
        "\n",
        "with open('nlp243_staff.csv', 'r') as staff_csv:\n",
        "\n",
        "    staff_reader = csv.reader(staff_csv)\n",
        "\n",
        "    for row in staff_reader:\n",
        "        print(row)\n",
        "\n",
        "###########################################\n",
        "# Reading and writing lists of dictionary #\n",
        "###########################################\n",
        "\n",
        "course_staff_dict_list = [\n",
        "    {'First Name': 'Dilek', 'Last Name': 'Hakkani-Tur', 'Role': 'Instructor'},\n",
        "    {'First Name': 'Rishi', 'Last Name': 'Rajasekaran', 'Role': 'TA'},\n",
        "    {'First Name': 'Zekun', 'Last Name': 'Zhao', 'Role': 'TA'}\n",
        "]\n",
        "\n",
        "with open('nlp243_staff2.csv', 'w') as staff_csv2:\n",
        "    writer = csv.DictWriter(staff_csv2, \n",
        "                            fieldnames = ['First Name', 'Last Name', 'Role'])\n",
        "    \n",
        "    writer.writerows(course_staff_dict_list)\n",
        "\n",
        "with open('nlp243_staff2.csv', 'r') as staff_csv2:\n",
        "    reader = csv.DictReader(staff_csv2, \n",
        "                            fieldnames = ['First Name', 'Last Name', 'Role'])\n",
        "    \n",
        "    for row in reader:\n",
        "        print(row)\n",
        "\n",
        "with open('nlp243_staff.csv', 'r') as staff_csv:\n",
        "    print(\"Raw CSV:\")\n",
        "    print(staff_csv.read())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['First Name', 'Last Name', 'Role']\n",
            "['Dilek', 'Hakkani-Tur', 'Instructor']\n",
            "['Rishi', 'Rajasekaran', 'TA']\n",
            "['Zekun', 'Zhao', 'TA']\n",
            "OrderedDict([('First Name', 'Dilek'), ('Last Name', 'Hakkani-Tur'), ('Role', 'Instructor')])\n",
            "OrderedDict([('First Name', 'Rishi'), ('Last Name', 'Rajasekaran'), ('Role', 'TA')])\n",
            "OrderedDict([('First Name', 'Zekun'), ('Last Name', 'Zhao'), ('Role', 'TA')])\n",
            "Raw CSV:\n",
            "First Name,Last Name,Role\n",
            "Dilek,Hakkani-Tur,Instructor\n",
            "Rishi,Rajasekaran,TA\n",
            "Zekun,Zhao,TA\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IQWTYQQoxine"
      },
      "source": [
        "### Miscellaneous Formats\n",
        "\n",
        "For the interest of brevity, we have covered only CSV and JSON. There are other data storage formats such as XML and YAML for which libraries exist. The principles we covered for JSON and CSV, however are very similar to the other formats (though not exactly) and will make it easy to pick up new libraries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dO0VMpFc36U_"
      },
      "source": [
        "## Storing Python Objects - Pickle\n",
        "\n",
        "The earlier data formats described are convenient to store data. However, there may be a necessity to also store Python objects directly, especially if those objects have specific function implementations.\n",
        "\n",
        "This will come into picture when you're storing your trained models, since we'd also want to also capture their behavior - for e.g. how their `predict` methods are called.\n",
        "\n",
        "Pickle is a Python standard that allows us to arbitrarily serialize any Python data - functions, object, classes, etc. It is a binary serialization format - i.e. it is not stored as readable text in a file. \n",
        "\n",
        "Therefore, to use pickle to store data, we will need to add a new flag to file open that indicates we're reading/writing byte-oriented data - `b`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uesvNEDC1M92",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 411
        },
        "outputId": "abe63d53-4ccd-4e64-8282-9dd21f13d24c"
      },
      "source": [
        "\n",
        "import pickle as pkl\n",
        "\n",
        "#####################################\n",
        "# Example 1 - Serializing functions #\n",
        "#####################################\n",
        "\n",
        "def f(x):\n",
        "    return x*x\n",
        "\n",
        "# Write our function to file in the form of bytes\n",
        "with open('dumped_code.pkl', 'wb') as function_pkl_file:\n",
        "    pkl.dump(f, function_pkl_file)\n",
        "\n",
        "# Read our function from the file\n",
        "with open('dumped_code.pkl', 'rb') as function_pkl_file:\n",
        "    g = pkl.load(function_pkl_file)\n",
        "\n",
        "print(g(3))\n",
        "\n",
        "############################################\n",
        "# Example 2 - Deserializing a TF-IDF model #\n",
        "############################################\n",
        "\n",
        "# Setup dependencies\n",
        "!pip install rank_bm25\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "import numpy as np\n",
        "import string\n",
        "from pprint import pprint\n",
        "\n",
        "# This is an example of a Pickle file that I created in the course of my summer\n",
        "# research work, to create an index of data for retrieving Reddit fun facts\n",
        "# for a model I was training\n",
        "\n",
        "# Link to Pickle file:\n",
        "# https://drive.google.com/file/d/1AdVRtjaCuxvXA7ErCdJgdwTioFoZBufn/view?usp=sharing\n",
        "\n",
        "# Either download the file or upload it to your Colab runtime to be able to use it\n",
        "knowledge_index_path = \"/content/drive/My Drive/NLP243/Sections/Section 02 - Files/knowledge_index.pkl\"\n",
        "\n",
        "\n",
        "with open(knowledge_index_path, 'rb') as knowledge_index_pickle:\n",
        "    knowledge_index = pkl.load(knowledge_index_pickle)\n",
        "\n",
        "# This class takes an index containing a TF-IDF vectorizer,\n",
        "# sentences of some fun facts and TF-IDF vectors for the fun facts.\n",
        "# \n",
        "# Using this information, it tries to retrieve the top-n closest fun facts\n",
        "# to the text that was provided.\n",
        "class TfIdfRankerRetriever(object):\n",
        "    \"\"\"\n",
        "    A module that performs retrieval from an index and also performs top-n ranking.\n",
        "    This forms a component of the heuristic knowledge selection policy for the KD-PD-NRG\n",
        "    \"\"\"\n",
        "\n",
        "    def _clean(self, s):\n",
        "        return ''.join([c for c in s.lower() if c not in string.punctuation])\n",
        "\n",
        "    def __init__(self, knowledge_index, new_index=False):\n",
        "        if new_index:\n",
        "            self.tfidf_vec = knowledge_index[\"vectorizer\"]\n",
        "            self.knowledge_sentences = knowledge_index[\"knowledge\"]\n",
        "            self.vectorized_sentences = knowledge_index[\"knowledge_vecs\"]\n",
        "        else:\n",
        "            self.tfidf_vec = knowledge_index[\"tfidf_vec\"]\n",
        "            self.knowledge_sentences = knowledge_index[\"knowledge_list\"]\n",
        "            self.vectorized_sentences = self.tfidf_vec.transform(self.knowledge_sentences)\n",
        "\n",
        "    def get_top_n(self, query, n=5):\n",
        "        # These two lines are derived from dynamic.py of the baseline code, with modifications\n",
        "        # to enable top-n selection\n",
        "        query_vector = self.tfidf_vec.transform([self._clean(query)])\n",
        "        similarity = np.squeeze(np.asarray(query_vector.dot(self.vectorized_sentences.transpose()).todense()))\n",
        "        top_n_indices = similarity.argsort()[-n:][::-1].tolist()\n",
        "        retrieve_rank_list = [(self.knowledge_sentences[i], similarity[i]) for i in top_n_indices]\n",
        "        return retrieve_rank_list\n",
        "\n",
        "ranker = TfIdfRankerRetriever(knowledge_index)\n",
        "\n",
        "pprint(ranker.get_top_n('I like fish'))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "9\n",
            "Requirement already satisfied: rank_bm25 in /usr/local/lib/python3.6/dist-packages (0.2.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from rank_bm25) (1.18.5)\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/base.py:318: UserWarning: Trying to unpickle estimator TfidfTransformer from version 0.23.1 when using version 0.22.2.post1. This might lead to breaking code or invalid results. Use at your own risk.\n",
            "  UserWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/base.py:318: UserWarning: Trying to unpickle estimator TfidfVectorizer from version 0.23.1 when using version 0.22.2.post1. This might lead to breaking code or invalid results. Use at your own risk.\n",
            "  UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[('there is a fish, called the Barreleye Fish, that has a transparent head',\n",
            "  0.3590835641510005),\n",
            " ('During the Brisbane G20 summit, when Putin shook the Canadian Prime '\n",
            "  'Minister\\'s hand, the latter said \"I guess I\\'ll shake your hand, but I '\n",
            "  'have only one thing to say to you. You need to get out of Ukraine.\"',\n",
            "  0.24303583493681213),\n",
            " ('a Seahorse is the only fish to have a neck', 0.24205519306373924),\n",
            " ('There is a deep-water fish called the Black Swallower (or great swallower) '\n",
            "  'that can swallow whole fish twice its length and up to ten times it mass.',\n",
            "  0.2252799662732104),\n",
            " ('freshwater fish only \"drink\" water through their skin via osmosis, while '\n",
            "  'saltwater fish also drink water through their mouths',\n",
            "  0.21354726243656907)]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Afa_iGB1vvn2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "ba0743c2-96f2-4189-8ab4-eef6a5eebcd8"
      },
      "source": [
        "### Appendix\n",
        "\n",
        "def add(a, b):\n",
        "    return a + b\n",
        "\n",
        "def addOneToFn(f):\n",
        "    return lambda a, b: f(a, b) + 1\n",
        "\n",
        "addWithOne = addOneToFn(add)\n",
        "print(addWithOne(1, 2))\n",
        "\n",
        "def multiply(a, b):\n",
        "    return a * b\n",
        "\n",
        "multiplyPlusOne = addOneToFn(multiply)\n",
        "\n",
        "print(multiplyPlusOne(2, 3))\n",
        "\n",
        "def naturalNumbers():\n",
        "    i = 1\n",
        "    while True:\n",
        "        yield i\n",
        "        i = i + 1\n",
        "\n",
        "for i, val in enumerate(naturalNumbers()):\n",
        "    if i < 100:\n",
        "        print(val)\n",
        "    else:\n",
        "        break"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4\n",
            "7\n",
            "1\n",
            "2\n",
            "3\n",
            "4\n",
            "5\n",
            "6\n",
            "7\n",
            "8\n",
            "9\n",
            "10\n",
            "11\n",
            "12\n",
            "13\n",
            "14\n",
            "15\n",
            "16\n",
            "17\n",
            "18\n",
            "19\n",
            "20\n",
            "21\n",
            "22\n",
            "23\n",
            "24\n",
            "25\n",
            "26\n",
            "27\n",
            "28\n",
            "29\n",
            "30\n",
            "31\n",
            "32\n",
            "33\n",
            "34\n",
            "35\n",
            "36\n",
            "37\n",
            "38\n",
            "39\n",
            "40\n",
            "41\n",
            "42\n",
            "43\n",
            "44\n",
            "45\n",
            "46\n",
            "47\n",
            "48\n",
            "49\n",
            "50\n",
            "51\n",
            "52\n",
            "53\n",
            "54\n",
            "55\n",
            "56\n",
            "57\n",
            "58\n",
            "59\n",
            "60\n",
            "61\n",
            "62\n",
            "63\n",
            "64\n",
            "65\n",
            "66\n",
            "67\n",
            "68\n",
            "69\n",
            "70\n",
            "71\n",
            "72\n",
            "73\n",
            "74\n",
            "75\n",
            "76\n",
            "77\n",
            "78\n",
            "79\n",
            "80\n",
            "81\n",
            "82\n",
            "83\n",
            "84\n",
            "85\n",
            "86\n",
            "87\n",
            "88\n",
            "89\n",
            "90\n",
            "91\n",
            "92\n",
            "93\n",
            "94\n",
            "95\n",
            "96\n",
            "97\n",
            "98\n",
            "99\n",
            "100\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q9Upgjwj-L3P"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}