Class 17 - Language/class_17_language.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "674ac2a7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# We'll start at 2:10ish PM \n",
    "\n",
    "Waiting for others to join the call :)\n",
    "\n",
    "https://www.youtube.com/watch?v=dzsuE5ugxf4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "843e46f7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/cdy.png' style='width:500px; float: left; margin: 0px 30px 15px 0px'></center>\n",
    "\n",
    "# What is NLP?\n",
    "## Class 17 - Data Science Curriculum \n",
    "\n",
    "<br>\n",
    "\n",
    "#### Women Building Change scholarship program 2023 🇧🇮\n",
    "April 21, 2023\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48500da2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ⏪ Recap last class\n",
    "\n",
    "- data science pipeline\n",
    "- how to read: .csv, .txt, .json\n",
    "- open source data\n",
    "- webscraping with beautiful soup\n",
    "- getting data from twitter"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82476939",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 🚀 Today's agenda\n",
    "\n",
    "- What is NLP?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f7409f0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Natural Language Processing (NLP)\n",
    "\n",
    "<br>\n",
    "<center><img src=\"img/clase01/nlp.png\" style='height:400px;'>\n",
    "    <small>Image credit: CleverTap</small></center>\n",
    "\n",
    "- It is the area of artificial intelligence that deals with human languages and derives valuable information from them\n",
    "\n",
    "- There are dozens of methods and strategies to solve a given problem"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1a91959",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 🤔 Why learn NLP?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca8998e9",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Text and Big Data\n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/unstructured.png' style='height:350px;'>\n",
    "</center>\n",
    "\n",
    "• There are more than 7,100 spoken languages in the world\n",
    "\n",
    "• From 80% to 90% of data generated and collected by organizations is unstructured → Most of it is in the form of text\n",
    "\n",
    "• Millions of data are being generated right now: WhatsApp, Twitter, YouTube, Reddit, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd7b2268",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world \n",
    "# Language modeling \n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/google.png' style='height:400px;'>\n",
    "</center>\n",
    "\n",
    "• Predict a sentence based on the history of the previous words"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c2a7ef5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Text classification \n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/spam.jpg' style='height:350px;'>\n",
    "</center>\n",
    "\n",
    "- Classify text in different categories based on its content\n",
    "\n",
    "- Most famous example: Spam or not"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c47d91a7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Information extraction\n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/whatsapp2.jpg' style='height:350px;'>\n",
    "</center>\n",
    "\n",
    "- As the name implies, this refers to extracting relevant information from the text\n",
    "\n",
    "- Examples: Calendar events and entities mentioned"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "969c824c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Information retrieval \n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/google_ret.png' style='height:350px;'>\n",
    "</center>\n",
    "\n",
    "- Find relevant documents in a large dataset based on a user's search\n",
    "\n",
    "- Related to text summarization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6336cc8a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Chatbots\n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/chatbot.png' style='height:400px;'>\n",
    "</center>\n",
    "\n",
    "• Systems that answer questions automatically"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eed57286",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Translation \n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/deepl.png' style='height:400px;'>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fd63f75",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n",
    "# Social media monitoring\n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/adele.png' style='height:300px;'>\n",
    "</center>\n",
    "\n",
    "- Companies want to understand their customers (targeted marketing campaigns)\n",
    "\n",
    "- They want to be able to take advantage of explicit and implicit feedback from their customers\n",
    "\n",
    "- Example: [Here](https://www.youtube.com/watch?v=sY9cjVK0mGU&ab_channel=MADSTARS%28%EB%B6%80%EC%82%B0%EA%B5%AD%EC%A0%9C%EA%B4%91%EA%B3%A0%EC%A0%9C%29)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30a608f5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### NLP in the real world\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54d7edb1",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### How do we talk to Siri? Does it speak English?\n",
    "\n",
    "<center><br>\n",
    "<img src='img/clase01/siri.jpg' style='height:350px;'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ebfe169",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Representation of data in numerical form\n",
    "\n",
    "**Example: Images**\n",
    "\n",
    "<center><img src='img/clase08/komp2.jpeg'>\n",
    "<small>Image credit: VideoNet</small></center>\n",
    "\n",
    "- An image is represented on a computer in the form of a matrix where each $m[i,j]$ represents pixel $i$,$j$ of the image\n",
    "\n",
    "- Similarly, a video is a collection of frames, where each frame is an image. Therefore, any video can be represented as a collection of matrices\n",
    "\n",
    "- (Un)fortunately, representing text numerically is not so simple"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f15b18ca",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 😫 🤯 Why is NLP so hard?\n",
    "\n",
    "\n",
    "### I saw the man on the hill with a telescope"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14151e55",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<img src='img/clase01/colina.jpeg' style='height:300px; float: left; margin: 30px 30px 30px 30px'> \n",
    "1. I saw the man. The man was on the hill. I was using a telescope.<br>\n",
    "2. I saw the man. I was on the hill. I was using a telescope.<br>\n",
    "3. I saw the man. The man was on the hill. The hill had a telescope.<br>\n",
    "4. I saw the man. I was on the hill. The hill had a telescope.<br>\n",
    "5. I saw the man. The man was on the hill. I saw him using a telescope.\n",
    "\n",
    "<br><br>\n",
    "<small>Image credit: All Things Linguistic</small>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d2f5bc8",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 😫 🤯 Why is NLP so hard?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b2908aa",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The city council denied the demonstrators the permit because **they** advocated violence. \n",
    "- The city council denied the demonstrators the permit because **they** feared violence. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc7fd7da",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/clase01/chicken.jpg'>\n",
    "<center>    \n",
    "    <big>Is the chicken ready to eat?</big>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca962bc7",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Ambiguity"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19416416",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/clase01/perro.jpg' style='height:350px;'>\n",
    "<center>    \n",
    "    <big>\"The dog bit the man\" vs \"The man bit the dog\"</big>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87778310",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Common knowledge"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20bc7d70",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/clase01/homesick.png' style='height:350px;'>\n",
    "<center>    \n",
    "    <big>How do you say \"Dépaysement\" in English?</big>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8ea8ab4",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Language diversity"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbfe1292",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/clase01/homer.gif' style='height:350px;'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ac0060d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Sarcasm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a96a93",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Break time!! See you in 5 minutes! \n",
    "@ 2: 49 PM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dabc5942",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 😫 🤯 So now that we have grasped why NLP is hard...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63a8759b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Representation of data in numerical form\n",
    "\n",
    "**Example: Images**\n",
    "\n",
    "<center><img src='img/clase08/komp2.jpeg'>\n",
    "<small>Image credit: VideoNet</small></center>\n",
    "\n",
    "- An image is represented on a computer in the form of a matrix where each $m[i,j]$ represents pixel $i$,$j$ of the image\n",
    "\n",
    "- Similarly, a video is a collection of frames, where each frame is an image. Therefore, any video can be represented as a collection of matrices\n",
    "\n",
    "- (Un)fortunately, representing text numerically is not so simple"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "480b0484",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 🤔 The issue at hand...\n",
    "\n",
    "<br>\n",
    "<center><img src='img/clase01/gigo.png' style='height:350px;'>\n",
    "<small>Image credit: R-Bloggers</small></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b633243",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    " - Let's take as an example the word **bow**. Can we just give it a number to represent it? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8872464",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 🙅‍♀ Then... how do we represent the meaning of a word? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7811fe37",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## How do we, humans, know what a word means?\n",
    "\n",
    "<br>\n",
    "<center><img src='img/clase08/reina.jpg' style='height:500px; float: center; margin: 0px 15px 15px 0px'></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dad1d05",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## We learn through experience\n",
    "\n",
    "<br>\n",
    "<center><img src='img/clase08/reyes2.png' style='height:600px; float: center; margin: 0px 15px 15px 0px'></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "203a4030",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Machines can also learn through experience\n",
    "\n",
    "<br>\n",
    "\n",
    "<div align='center'>\n",
    "    <img src='img/clase08/maquina1.png' style='height:500px; float: left; margin: 0px 15px 15px 0px'>\n",
    "    <img src='img/clase08/maquina2.png' style='height:500px; float: left; margin: 0px 15px 15px 0px'>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "061272f6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 🤔 🤔 🤔\n",
    "# What is meant by the meaning of a word?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "524149f8",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 🤔 🤔 🤔\n",
    "# What is tichiniky? \n",
    "(Don't look it up on Google)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97cafc56",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "\n",
    "- Joseph offered a glass of <font color='red'>tichiniky</font> to his girlfriend."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb6cb34b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- <font color='red'>Tichiniky</font> and steak make a great pairing for a nice meal."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50122440",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Charles stumbled, his face flushed from drinking too much <font color='red'>tichiniky</font>."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3eacb1be",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Last night I had bread, cheese, and this excellent <font color='red'>tichiniky</font> for dinner."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "332cb253",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 🤔 🤔 🤔\n",
    "I ask you again... \n",
    "# What is tichiniky? \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ca6abc9",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- An alcoholic drink\n",
    "<img src='img/clase08/amor_amistad.gif'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c76a464",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Distributional semantics \n",
    "\n",
    "- A bottle of <font color='red'>tichiniky</font> is on the table.\n",
    "- Not everyone likes <font color='red'>tichiniky</font>.\n",
    "- Don't drink <font color='red'>tichiniky</font> and drive.\n",
    "- We make <font color='red'>tichiniky</font> with grapes. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b12cfae5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Distributional semantics \n",
    "\n",
    "- A bottle of <font color='red'>________</font> is on the table.\n",
    "- Not everyone likes <font color='red'>________</font>.\n",
    "- Don't drink <font color='red'>________</font> and drive.\n",
    "- We make <font color='red'>________</font> with grapes. \n",
    "\n",
    "### 🤔 What other words would go well in those contexts?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa1fefa0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Distributional semantics \n",
    "\n",
    "- A bottle of ________ is on the table.\n",
    "- Not everyone likes ________.\n",
    "- Don't drink ________ and drive.\n",
    "- We make ________ with grapes. \n",
    "\n",
    "|                 | 1 | 2 | 3 | 4 | ... |\n",
    "|-----------------|---|---|---|---|-----|\n",
    "| tichiniky       |  |  |  |  |     |\n",
    "| strong          |  |  |  |  |     |\n",
    "| motor oil       |  |  |  |  |     |\n",
    "| tacos           |  |  |  |  |     |\n",
    "| wine            |  |  |  |  |     |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6c91cd3",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- ⚠️ Tichiniky & wine have the same **vector representation**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d90e61c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Common NLP terminology\n",
    "<center><img src='img/clase01/terms2.png' style='height:800px;'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909a484a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Once you have a numerical vector representation\n",
    "\n",
    "<center><img src='img/clase08/espacio.jpg' style='height:500px; float: center; margin: 0px 15px 15px 0px'></center>\n",
    "<br>\n",
    "\n",
    "<center><big><b>You can do math!</b></big></center>\n",
    "\n",
    "<br>\n",
    "\n",
    "- meaning($man$) - meaning($king$) + meaning($queen$) = meaning($woman$)\n",
    "\n",
    "Neural networks can make machines understand analogies like humans [Mikolov et al., 2013]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e624ea70",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import ast\n",
    "import pandas as pd\n",
    "import gensim.downloader as api\n",
    "from gensim.models import Word2Vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1f5ec193",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def analogy(model, worda, wordb, wordc):\n",
    "    '''\n",
    "    wordA is to wordB as wordC is to ...\n",
    "    '''\n",
    "    result = model.most_similar(negative=[worda], \n",
    "                                positive=[wordb, wordc])\n",
    "    return result[0][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e02e371a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "google_wv = api.load('word2vec-google-news-300')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0ccdb876",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Surgeon'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "analogy(google_wv, 'Man','Doctor','Woman')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c6a68b7e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('games', 0.7636998295783997),\n",
       " ('play', 0.6501179933547974),\n",
       " ('match', 0.648574948310852),\n",
       " ('matchup', 0.6120451092720032),\n",
       " ('agame', 0.5863147974014282),\n",
       " ('ballgame', 0.5731309056282043),\n",
       " ('thegame', 0.5718172788619995),\n",
       " ('opener', 0.5680000185966492),\n",
       " ('matches', 0.5580832958221436),\n",
       " ('tournament', 0.5496207475662231)]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "google_wv.most_similar(\"game\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7f248dc5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"data/rap.csv\")\n",
    "#df.pp = df.pp.apply(lambda row: [profanity.censor(w) for w in ast.literal_eval(row)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e2d36157",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ALink</th>\n",
       "      <th>SName</th>\n",
       "      <th>SLink</th>\n",
       "      <th>Lyric</th>\n",
       "      <th>language</th>\n",
       "      <th>pp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16332</th>\n",
       "      <td>/kool-moe-dee/</td>\n",
       "      <td>The Don</td>\n",
       "      <td>/kool-moe-dee/the-don.html</td>\n",
       "      <td>I'm The Don\\nArmed and dangerous\\nWith a mic I...</td>\n",
       "      <td>en</td>\n",
       "      <td>['armed', 'dangerous', 'mic', 'like', 'ordain'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6039</th>\n",
       "      <td>/linkin-park/</td>\n",
       "      <td>Figure.09 (demo 2002)</td>\n",
       "      <td>/linkin-park/figure-09-demo-2002.html</td>\n",
       "      <td>All these things I could not say,\\nMade me so ...</td>\n",
       "      <td>en</td>\n",
       "      <td>['things', 'could', 'say', 'made', 'much', 'af...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16932</th>\n",
       "      <td>/biz-markie/</td>\n",
       "      <td>I'm An Ugly Nigga (So What?)</td>\n",
       "      <td>/biz-markie/im-an-ugly-nigga-so-what.html</td>\n",
       "      <td>Hey hey hey, know what i heard?\\nBeauty's only...</td>\n",
       "      <td>en</td>\n",
       "      <td>['hey', 'hey', 'hey', 'know', 'heard', 'beauty...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1603</th>\n",
       "      <td>/chris-brown/</td>\n",
       "      <td>Love Music</td>\n",
       "      <td>/chris-brown/love-music.html</td>\n",
       "      <td>I got my keyboard\\nSymbols all ready\\nPull up ...</td>\n",
       "      <td>en</td>\n",
       "      <td>['got', 'keyboard', 'symbols', 'ready', 'pull'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16761</th>\n",
       "      <td>/chip/</td>\n",
       "      <td>Diamond Rings</td>\n",
       "      <td>/chip/diamond-rings.html</td>\n",
       "      <td>I rhyme rhyme, that's why I shine shine,\\nand ...</td>\n",
       "      <td>en</td>\n",
       "      <td>['rhyme', 'rhyme', 'shine', 'shine', 'roll', '...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6720</th>\n",
       "      <td>/eminem/</td>\n",
       "      <td>Stepdad (Intro)</td>\n",
       "      <td>/eminem/stepdad-intro.html</td>\n",
       "      <td>Get up\\nDad, what?\\nWhat'd I fuckin' tell you ...</td>\n",
       "      <td>en</td>\n",
       "      <td>['get', 'dad', 'fuckin', 'tell', 'turnin', 'li...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11921</th>\n",
       "      <td>/mystikal/</td>\n",
       "      <td>Y'all Ain't Ready Yet (Remix)</td>\n",
       "      <td>/mystikal/yall-aint-ready-yet-remix.html</td>\n",
       "      <td>(Precise)\\nMan, them niggas can't fuck with yo...</td>\n",
       "      <td>en</td>\n",
       "      <td>['precise', 'man', 'niggas', 'fuck', 'man', 'f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15608</th>\n",
       "      <td>/a-tribe-called-quest/</td>\n",
       "      <td>Whateva Will Be</td>\n",
       "      <td>/a-tribe-called-quest/whateva-will-be.html</td>\n",
       "      <td>Girl, this motherfucker's got rhythm\\n\\nSo am ...</td>\n",
       "      <td>en</td>\n",
       "      <td>['girl', 'motherfucker', 'got', 'rhythm', 'pos...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15897</th>\n",
       "      <td>/lil-yachty/</td>\n",
       "      <td>Fight Night Round 3 (Ft. Veeze, Babyface Ray)</td>\n",
       "      <td>/lil-yachty/fight-night-round-3-ft-veeze-babyf...</td>\n",
       "      <td>If I was down to my last dollar, give a fuck w...</td>\n",
       "      <td>en</td>\n",
       "      <td>['last', 'dollar', 'give', 'fuck', 'think', 'g...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10045</th>\n",
       "      <td>/bow-wow/</td>\n",
       "      <td>Get Up</td>\n",
       "      <td>/bow-wow/get-up.html</td>\n",
       "      <td>Lil' Bow Wow]+(J.D.)\\n(Let's go) Uh huh, uh ye...</td>\n",
       "      <td>en</td>\n",
       "      <td>['lil', 'bow', 'wow', 'j', 'let', 'go', 'uh', ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        ALink                                          SName  \\\n",
       "16332          /kool-moe-dee/                                        The Don   \n",
       "6039            /linkin-park/                          Figure.09 (demo 2002)   \n",
       "16932            /biz-markie/                   I'm An Ugly Nigga (So What?)   \n",
       "1603            /chris-brown/                                     Love Music   \n",
       "16761                  /chip/                                  Diamond Rings   \n",
       "6720                 /eminem/                                Stepdad (Intro)   \n",
       "11921              /mystikal/                  Y'all Ain't Ready Yet (Remix)   \n",
       "15608  /a-tribe-called-quest/                                Whateva Will Be   \n",
       "15897            /lil-yachty/  Fight Night Round 3 (Ft. Veeze, Babyface Ray)   \n",
       "10045               /bow-wow/                                         Get Up   \n",
       "\n",
       "                                                   SLink  \\\n",
       "16332                         /kool-moe-dee/the-don.html   \n",
       "6039               /linkin-park/figure-09-demo-2002.html   \n",
       "16932          /biz-markie/im-an-ugly-nigga-so-what.html   \n",
       "1603                        /chris-brown/love-music.html   \n",
       "16761                           /chip/diamond-rings.html   \n",
       "6720                          /eminem/stepdad-intro.html   \n",
       "11921           /mystikal/yall-aint-ready-yet-remix.html   \n",
       "15608         /a-tribe-called-quest/whateva-will-be.html   \n",
       "15897  /lil-yachty/fight-night-round-3-ft-veeze-babyf...   \n",
       "10045                               /bow-wow/get-up.html   \n",
       "\n",
       "                                                   Lyric language  \\\n",
       "16332  I'm The Don\\nArmed and dangerous\\nWith a mic I...       en   \n",
       "6039   All these things I could not say,\\nMade me so ...       en   \n",
       "16932  Hey hey hey, know what i heard?\\nBeauty's only...       en   \n",
       "1603   I got my keyboard\\nSymbols all ready\\nPull up ...       en   \n",
       "16761  I rhyme rhyme, that's why I shine shine,\\nand ...       en   \n",
       "6720   Get up\\nDad, what?\\nWhat'd I fuckin' tell you ...       en   \n",
       "11921  (Precise)\\nMan, them niggas can't fuck with yo...       en   \n",
       "15608  Girl, this motherfucker's got rhythm\\n\\nSo am ...       en   \n",
       "15897  If I was down to my last dollar, give a fuck w...       en   \n",
       "10045  Lil' Bow Wow]+(J.D.)\\n(Let's go) Uh huh, uh ye...       en   \n",
       "\n",
       "                                                      pp  \n",
       "16332  ['armed', 'dangerous', 'mic', 'like', 'ordain'...  \n",
       "6039   ['things', 'could', 'say', 'made', 'much', 'af...  \n",
       "16932  ['hey', 'hey', 'hey', 'know', 'heard', 'beauty...  \n",
       "1603   ['got', 'keyboard', 'symbols', 'ready', 'pull'...  \n",
       "16761  ['rhyme', 'rhyme', 'shine', 'shine', 'roll', '...  \n",
       "6720   ['get', 'dad', 'fuckin', 'tell', 'turnin', 'li...  \n",
       "11921  ['precise', 'man', 'niggas', 'fuck', 'man', 'f...  \n",
       "15608  ['girl', 'motherfucker', 'got', 'rhythm', 'pos...  \n",
       "15897  ['last', 'dollar', 'give', 'fuck', 'think', 'g...  \n",
       "10045  ['lil', 'bow', 'wow', 'j', 'let', 'go', 'uh', ...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9748e31c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "rap_model = Word2Vec(df['pp'].values,\n",
    "                     sg=1, # 1 skip-gram, 0 CBOW\n",
    "                     seed=1, vector_size=256, min_count=50, window=12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bda95459",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "\"Key 'game' not present\"",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m/var/folders/n7/q93jwpcs6jndz6qqvj4mhtcm0000gn/T/ipykernel_19036/1801291819.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mrap_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwv\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmost_similar\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"game\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/gensim/models/keyedvectors.py\u001b[0m in \u001b[0;36mmost_similar\u001b[0;34m(self, positive, negative, topn, clip_start, clip_end, restrict_vocab, indexer)\u001b[0m\n\u001b[1;32m    771\u001b[0m                 \u001b[0mmean\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweight\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    772\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 773\u001b[0;31m                 \u001b[0mmean\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweight\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_vector\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnorm\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    774\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhas_index_for\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    775\u001b[0m                     \u001b[0mall_keys\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/gensim/models/keyedvectors.py\u001b[0m in \u001b[0;36mget_vector\u001b[0;34m(self, key, norm)\u001b[0m\n\u001b[1;32m    436\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    437\u001b[0m         \"\"\"\n\u001b[0;32m--> 438\u001b[0;31m         \u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    439\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mnorm\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    440\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfill_norms\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/gensim/models/keyedvectors.py\u001b[0m in \u001b[0;36mget_index\u001b[0;34m(self, key, default)\u001b[0m\n\u001b[1;32m    410\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mdefault\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 412\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Key '{key}' not present\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    413\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    414\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mget_vector\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnorm\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyError\u001b[0m: \"Key 'game' not present\""
     ]
    }
   ],
   "source": [
    "rap_model.wv.most_similar(\"game\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b546fb86",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "google_wv.most_similar(\"game\", topn=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36c2e2f9",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Data science and ethics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d30d683",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src='img/libro1.jpg' style='height:600px; float: center; margin: 0px 15px 15px 0px'></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ac5c33c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The evolution of NLP\n",
    "\n",
    "<center><img src='img/clase11/evol.png' style='height:280px; float: center; margin: 0px 15px 15px 0px'></center>\n",
    "<center><small>Image credit: O'Reilly</small></center>\n",
    "\n",
    "- **Bag of Words / TFIDF**\n",
    "\n",
    "    - Simpler techniques to generate text embeddings\n",
    "<br>\n",
    "<br>\n",
    "- 2013: **Word2Vec**\n",
    "\n",
    "    - Technique that takes into account the semantic relationship between words\n",
    "    - One of the first models that use prediction techniques and neural networks\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85f7859e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Evolution of NLP\n",
    "\n",
    "<br>\n",
    "<center><img src='img/clase11/evolnnn.png' style='height:250px; float: center; margin: 0px 15px 15px 0px'></center>\n",
    "<center><small>Image credit: O'Reilly</small></center>\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "- 2018: **Transformers**\n",
    "\n",
    "    - Models that use attention to increase the speed of task-specific training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbb550e0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Data literacy\n",
    "\n",
    "<br>\n",
    "<center><img src='img/math.jpeg' style='height:500px'></center>\n",
    "\n",
    "- ChatGPT\n",
    "- [Video](https://www.youtube.com/watch?v=cQ54GDm1eL0&ab_channel=BuzzFeedVideo) Barack Obama\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54c2a4bb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ⏪ Today's recap\n",
    "\n",
    "- What is NLP?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3c47662",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src='img/bye.gif' style='height:250px;'></center> \n",
    "\n",
    "# Next class: Text pre-processing and Word Clouds\n",
    "# See you next Monday!"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}