Badge 4 - Visualisation and Data Cleaning v2.1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook:\n",
    "\n",
    "1. [Simple: **Counting Tokens**](#1)\n",
    "2. [Visualising **Frequency Distributions** (but first, **cleaning up the data**)](#2)\n",
    "3. [Advanced Visualisation: **Word Clouds**](#3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"1\"></a>\n",
    "# 1. Simple: Counting Tokens\n",
    "\n",
    "#### Questions & Objectives\n",
    "\n",
    "- How can I count tokens in text?\n",
    "\n",
    "#### Key Points\n",
    "\n",
    "- To count tokens, one can make use of NLTK’s `FreqDist` class from the `probability` package. The `N()` method can then be used to count how many tokens a text or corpus contains.\n",
    "- Counts for a specific token can be obtained using `fdist[\"token\"]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell now. It's the usual imports of text mining libraries.\n",
    "\n",
    "import nltk\n",
    "import numpy\n",
    "import string\n",
    "import matplotlib.pyplot as plt\n",
    "from nltk.tokenize import word_tokenize\n",
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's load the India corpus and lowercase it. This will take a minute to run.\n",
    "\n",
    "from nltk.corpus import PlaintextCorpusReader\n",
    "corpus_root = \"./data/Medical_History_of_British_India\"\n",
    "corpus_reader = PlaintextCorpusReader(corpus_root, '.*', encoding='latin1') \n",
    "corpus_tokens = corpus_reader.words() \n",
    "print(\"loaded tokens:\", len(corpus_tokens) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus_tokens = [word.lower()\n",
    "                 for word in corpus_tokens[0:5000000]] \n",
    "# again we just grab some of the data, to not run out of memory. \n",
    "# you can try to use it all by removing [0:5000000] but you might find your notebook freezes\n",
    "# if it does, go to Kernel > Restart and run your notebook from the top.\n",
    "print(\"finished lowercasing\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counting tokens in text\n",
    "\n",
    "You can also do other useful things like count the number of tokens in a text, determine the count and percentage of particular tokens in a corpus, and plot the count distributions as a graph. To do this we have to import the `FreqDist` class from the NLTK `probability` package. When calling this class, a list of tokens from a text or corpus needs to be specified as a parameter in brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This will take a minute, too.\n",
    "\n",
    "from nltk.probability import FreqDist\n",
    "frequencies = FreqDist(corpus_tokens)\n",
    "print(frequencies)\n",
    "# you will see a short summary of the frequencies object printed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can print the counts of most common tokens with `freq_dist_object.most_common( how_many )`,\n",
    "\n",
    "e.g., `fdist.most_common(100)` for the most common 100 words.\n",
    "\n",
    "The results will be arranged starting from the top most frequent tokens with the frequency counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(frequencies.most_common(100))\n",
    "\n",
    "# Fun fact: notice that this is a list of tuples [('word1', 233), ('word2', 2324), ...]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instead of printing, you can just return the value from the cell,\n",
    "# which will be easier to read but very long:\n",
    "frequencies.most_common(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `frequency_distribution.N()` we can count the total number of tokens in a corpus:\n",
    "\n",
    "e.g., `fdist.N()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(frequencies.N())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `frequencies[ your_word ]` you can count the number of times a particular token appears in a corpus,\n",
    "\n",
    "e.g., `frequencies['hospital']` returns `28280` which means that the word 'hospital' appears 28,280 times in our corpus of reports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(frequencies['hospital'])\n",
    "print(frequencies['he'])\n",
    "print(frequencies['she'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `frequencies.freq( your_word )` you can also determine the relative frequency of a token in a corpus, meaning what % of the corpus a term is,\n",
    "\n",
    "e.g., `frequencies.freq('hospital')` returns `0.000997673635341749` which means that 0.1% of all words are 'hospital'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(frequencies.freq('hospital'))\n",
    "print(frequencies.freq('he'))\n",
    "print(frequencies.freq('she')) \n",
    "print(frequencies.freq('purple')) \n",
    "\n",
    "# Notice, sometimes this fraction is so small that it goes into 'scientific notation' (e-05)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Note on counting tokens that match a Regular Expression\n",
    "\n",
    "If you have a list of tokens created using Regular Expression matching, as in the previous section, and you’d like to count them, then you can also simply count the length of the list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This will take a minute.\n",
    "import re\n",
    "womaen_strings = [word \n",
    "                  for word in corpus_tokens \n",
    "                  if re.search('^wom[ae]n$', word)]\n",
    "print(len(womaen_strings))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Frequency counts of tokens are useful to compare different corpora in terms of the occurrences of different words or expressions, for example: in order to see if a word appears more rarely in one corpus versus another.\n",
    "\n",
    "Counts of tokens, documents and an entire corpus can also be used to compute simple pairwise document similarity of two documents (later, have a look at Jana Vembunarayanan’s blogpost for a hands-on example of how to do that: https://janav.wordpress.com/2013/10/27/tf-idf-and-cosine-similarity/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"2\"></a>\n",
    "# 2. Visualising Frequency Distributions\n",
    "# (but first, cleaning up the data)\n",
    "\n",
    "\n",
    "#### Questions & Objectives\n",
    "\n",
    "- How can I draw a frequency distribution of the most frequent words in a collection?\n",
    "- How can I visualise this data as a word cloud?\n",
    "\n",
    "#### Key Points\n",
    "\n",
    "- A frequency distribution can be created using the `plot()` method.\n",
    "- In this episode you will also learn how to clean data by removing stop words and other types of tokens from the text.\n",
    "- A word cloud can be used to visualise tokens and their frequency in text in a different way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualising Frequency Distributions of Tokens in Text\n",
    "\n",
    "### Graph of the frequency of the words as they are:\n",
    "\n",
    "The `plot()` method can be called to draw the frequency distribution as a graph for the most common tokens in the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "frequencies.plot(30,title='Frequency distribution for the 30 most common tokens in our text collection')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the distribution contains a lot of non-content words like “the”, “of”, “and”, etc. (we call these stop words) and punctuation. This is not very useful. Let's have a small peek on what these words are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "frequencies.most_common(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yes, definitely not useful. Many of these words look like noise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🐛Minitask:  Identify 3-4 categories of not-very-helpful tokens in the above set\n",
    "\n",
    "Look at the above set of most popular tokens. Some of them look important and meaningful ('government', 'disease', etc.) but many of them are not very useful.\n",
    "\n",
    "- Identify some families of not helpful tokens and write names for these families below.\n",
    "\n",
    "Do not spend too much time on this (2 minutes maximum.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "<details><summary style='color:blue'>CLICK HERE TO SEE THE THE ANSWER. BUT REALLY TRY TO DO IT YOURSELF FIRST!</summary>\n",
    "\n",
    "    There's no code involved here. You might have come up with: numbers, punctuation marks, typical but not very meaningful words. Keep reading for more information.\n",
    "</details>\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Removing tokens that are just noise\n",
    "\n",
    "We can remove these before drawing the graph. We need to import `stopwords` from the `corpus` package to do this. The list of stop words is combined with a list of punctuation and a list of single digits using + signs into a new list of tokens to be ignored."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some of Python's built-in 'cheat sheets' of punctuation and other 'meaningless characters', and some provided by the NLTK library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nltk.download('stopwords')\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "# Let's have a look what words are usually discarded:\n",
    "print(string.punctuation)\n",
    "print(string.digits)\n",
    "print(stopwords.words('english'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: we turn the strings of punctuation and digits into a list, \n",
    "# so they can be added to the other list of stop words.\n",
    "print(list(string.punctuation))\n",
    "print(list(string.digits))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove stop words, punctuation marks and digits. \n",
    "# Note: The set( ... ) syntax removes duplicates from the list.\n",
    "\n",
    "remove_these = set(stopwords.words('english') + list(string.punctuation) + list(string.digits))\n",
    "\n",
    "filtered_text = [token \n",
    "                 for token in corpus_tokens \n",
    "                 if not token in remove_these]\n",
    "\n",
    "# Note: the above 3-line version could be a one-liner (see below). \n",
    "# It's up to you, which format you prefer. The format above, or this:\n",
    "# filtered_text = [word for word in corpus_tokens if not word in remove_these]\n",
    "\n",
    "frequencies_filtered = FreqDist(filtered_text)\n",
    "print(frequencies_filtered.most_common(30))\n",
    "frequencies_filtered.plot(30,title='Frequency distribution (excluding stop words, digits, and punctuation)')\n",
    "\n",
    "# This graph should be a bit better..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manually adding more tokens to be ignored\n",
    "\n",
    "The above graph is already much better, but sometimes we want to manually add some tokens to be ignored. It is easy, we just need to add more elements to the `remove_these` list.\n",
    "\n",
    "We looked at the top 100 tokens and found these to be not particularly useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print([token \n",
    "       for (token,count) in frequencies_filtered.most_common(100)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These elements do not seem meaningful for our analysis:\n",
    "\n",
    "- Rogue punctuation: '...', '..', etc.\n",
    "- A lot of multi-digit numbers: '12', '000'\n",
    "- Individual letters: 'a', 'j', etc.\n",
    "- Some other not very meaningful words: 'also', 'would'\n",
    "\n",
    "Let's create more lists of things we want to remove:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's create a range of numbers from 0 to 100 and turn them into strings, so they are like '45' not like 45.\n",
    "numbers_1_to_100 = [str(integer) \n",
    "                    for integer in range(101)]\n",
    "print(numbers_1_to_100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here are the weird puctuation marks:\n",
    "extra_punctuation_to_remove = ['.', '..','...','....','.....','......', ').', '.,']\n",
    "print(extra_punctuation_to_remove)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "individual_letters = list(string.ascii_lowercase)\n",
    "print(individual_letters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🐛Minitask: Identify more words that are potentially noise\n",
    "\n",
    "In a minute we will remove tokens that are noise. Based on your previous minitask and the current most popular tokens:\n",
    "\n",
    "- Run the cell below\n",
    "- Identify 10 more words that are most likely noise\n",
    "- Add them to the list `some_more_tokens_to_remove`\n",
    "- Run the cell again and continue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell to see current 100 most popular tokens, as part of the task above.\n",
    "# Then, add more items to some_more_tokens_to_remove\n",
    "\n",
    "print( [token \n",
    "        for (token,count) in frequencies_filtered.most_common(100)])\n",
    "\n",
    "some_more_tokens_to_remove = [ 'rs', 'per', 'would', '000']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "<details><summary style='color:blue'>CLICK HERE TO SEE THE THE ANSWER. BUT REALLY TRY TO DO IT YOURSELF FIRST!</summary>\n",
    "\n",
    "    ### BEGIN SOLUTION\n",
    "    some_more_tokens_to_remove = [ 'rs', 'per', 'would','one','two','first'\n",
    "                                 '000',  '00',  'co', 'ditto', '1st', 'ii', \n",
    "                                 'total', 'number', 'year', 'years']\n",
    "    ### END SOLUTION\n",
    "</details>\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Note about cleaning up data carefully\n",
    "Sometimes you might find that certain tokens do not communicate content, like 'would', and we are showing you how to remove them here.\n",
    "\n",
    "This comes with a warning, though: you have to be very careful performing steps like this, because it has the potential to completely bias your data. Careful cleaning of messy datasets is very important.\n",
    "\n",
    "Also: While it makes sense to remove stop words for this type of frequency analysis, it essential to keep them in the data for other text analysis tasks. Retaining the original text is crucial, for example, when deriving part-of-speech tags for a text or for recognising names in a text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can re-do the visualisation again, this time using an expanded and customised list of items to ignore.\n",
    "This should be a more meaningful graph!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's combine it all together and generate our new graph.\n",
    "\n",
    "remove_these = set(stopwords.words('english') + list(string.punctuation) + list(string.digits) \n",
    "        + numbers_1_to_100 + extra_punctuation_to_remove + individual_letters+some_more_tokens_to_remove)\n",
    "\n",
    "filtered_text = [token \n",
    "                 for token in corpus_tokens \n",
    "                 if not token in remove_these]\n",
    "    \n",
    "frequencies_filtered = FreqDist(filtered_text)\n",
    "print(frequencies_filtered.most_common(30))\n",
    "frequencies_filtered.plot(30,title='Frequency distribution (excluding stop words, numbers, and punctuation)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"3\"></a>\n",
    "# 3. Advanced Visualisation: Word Clouds\n",
    "\n",
    "## Basic word cloud:\n",
    "\n",
    "We can also present the filtered tokens as a word cloud. A word cloud is a modern type of graph where words' sizes communicate their frequency.  It's often used to have a quick an overview of the corpus. Additionally, word clouds can be customised as we'll see below.\n",
    "\n",
    "We will use the `WordCloud( ).generate_from_frequencies()` method.  The input to this method is a frequency dictionary of all tokens and their counts in the text.\n",
    "\n",
    "You will also see another way to create a simplified frequency count of words. That's because a word cloud requires words to be in a dictionary format:\n",
    "\n",
    " `{'total': 94009, 'year': 82829, 'number': 63358, 'cases': 39876 .... }`\n",
    " \n",
    "We will use another Python package, `Counter`, to create such a dictionary using the `filtered_text` variable as input. Note that `Counter` is much less powerful than `FreqDist`, but you might see it in other people's code, so we want you to be familiar with it.\n",
    "\n",
    "Once we have the data in the correct format, we generate the word cloud using the frequency dictionary and plot the figure to a specified size. We can show the plot using `plt.show()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "simple_frequencies_dict = Counter(filtered_text)\n",
    "\n",
    "# Let's have a peek into this dictionary. How many times does the word 'hospital' appear in filtered_text?\n",
    "print(simple_frequencies_dict['hospital'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Note on installing extra software in Jupyter Notebooks with !pip\n",
    "\n",
    "To create a word cloud, we'll first show you one of the most powerful features of of Jupyter Notebooks: you can download and install almost any software from the internet into your Notebook, including the virtualized server your Notebooks run on in Noteable. Because it is 'sandboxed' it is primarily safe. To install things we use `!pip`, the Python package installer command, which we'll use for installing `wordcloud`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this now.  It will install wordcloud on your machine.\n",
    "!pip install wordcloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from wordcloud import WordCloud\n",
    "\n",
    "cloud = WordCloud(width=800, height=400, max_font_size=160,colormap=\"hsv\").generate_from_frequencies(simple_frequencies_dict)\n",
    "plt.figure(figsize=(16,12))\n",
    "\n",
    "# You can play with the width, height, max_font_size, and figsize parameters to change the size, layout, \n",
    "# and clarity of your word cloud.  Try commenting out the previous word cloud code and running the code below.\n",
    "# What differences to you observe?\n",
    "# cloud = WordCloud(width=400, height=200, max_font_size=80,colormap=\"hsv\").generate_from_frequencies(simple_frequencies_dict)\n",
    "# plt.figure(figsize=(8,6))\n",
    "\n",
    "plt.imshow(cloud, interpolation='bilinear')\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "\n",
    "# You should see a colourful diagram below. It is generated uniquely for you at this moment in time. \n",
    "# Generate it again and you will see that it changes slightly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🖇💬Buddy discussion: What text data would a word cloud be really good and really bad for?\n",
    "\n",
    "#### Ask your buddy now if they reached the **BUDDY TASK**. Once you both did, complete this task:\n",
    "\n",
    "Try to identify strengths and weaknesses of word clouds.\n",
    "\n",
    "Don't spend too much time on this (2 minutes maximum) but take note of your favourite idea."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fancy-shaped word cloud\n",
    "\n",
    "And now a shaped word cloud for a bit of fun! This will present your word cloud in the shape of a given image.\n",
    "\n",
    "You need a shape file, which we provide for you in the form of the medical symbol (it looks like two snakes wrapped around a staff with wings)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mask image needs to have a transparent background so that only the black shape is used as a mask for the word cloud - parts that are black will be filled with your words, parts that are transparent will be left empty. You can use your own images later!\n",
    "\n",
    "It would be fun if we can customise:\n",
    "\n",
    "- The shape in which the words arrange themselves (e.g., circle, shape of the UK, question mark)\n",
    "- Colours to use for the words\n",
    "\n",
    "The **geeky bits** about how we use images and colours **(feel free to skip reading them)**:\n",
    "\n",
    "- To display the shaped word cloud you need to import the Image package from `PIL` as well as `numpy`. The image first needs to be opened and converted into a numpy array which we call `med_mask`. \n",
    "- A customised colour map (cmap) is created to present the words in black font. \n",
    "- Colours use #RRGGBB format where two hexadecimal characters (0123456789ABCDEF) describe amount of Red, Green and Blue we want. E.g., `#FF0000` means full red, no green, no blue. `#000000` means no red, nor green, nor blue, which is black. `#FFFFFF` is white, `#111111` and `#222222` are shades of grey, getting darker as amount of colour (represented in hexadecimal characters) increases.\n",
    "- The word cloud is created with a white background, with the mask and the colour map set as parameters, and generated from the dictionary containing the number of occurrences for each word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "import numpy as np\n",
    "from matplotlib.colors import LinearSegmentedColormap\n",
    "\n",
    "# Masking image\n",
    "medical_icon_mask_image = np.array(Image.open(\"./images/medical.png\"))\n",
    "\n",
    "# Custom Colormap\n",
    "colors = [\"#000000\", \"#111111\", \"#101010\", \"#121212\", \"#212121\", \"#222222\"]\n",
    "cmap = LinearSegmentedColormap.from_list(\"mycmap\", colors)\n",
    "\n",
    "# Image details: background, shape-mask, colours \n",
    "wordcloud = WordCloud(width=800, height=1600, \n",
    "                      background_color=\"white\", mask=medical_icon_mask_image, colormap=cmap)\n",
    "\n",
    "wordcloud.generate_from_frequencies(simple_frequencies_dict)\n",
    "plt.figure(figsize=(16,12))\n",
    "plt.imshow(wordcloud, interpolation=\"bilinear\")\n",
    "plt.axis(\"off\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🦋 Extra task (optional): if you have finished everything else already:\n",
    "\n",
    "- Can you create a wordcloud of the presidential speeches corpus?\n",
    "- Then, can you use your own masking image?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}