PaRSEC Animations Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0e587b2a",
   "metadata": {},
   "source": [
    "# ParSEC Animation utilities demo\n",
    "Daniel Mishler"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aed239c",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c6f24561",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "080e7869",
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-5-26118aa37c42>, line 5)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-5-26118aa37c42>\"\u001b[0;36m, line \u001b[0;32m5\u001b[0m\n\u001b[0;31m    import \"../parsec_animation_utils.py\" as panimate\u001b[0m\n\u001b[0m           ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython import display\n",
    "\n",
    "import parsec_animation_utils as panimate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77164729",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.ioff() # a requirement for funcAnimation in parsec_animation_utils"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3b749b1",
   "metadata": {},
   "source": [
    "## One more thing\n",
    "\n",
    "parsec_animation_utils uses another library that depends on FFmpeg, you might need to download it on your system first:\n",
    "\n",
    "https://ffmpeg.org/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c23f644",
   "metadata": {},
   "source": [
    "## Demos\n",
    "- the following demos use a run of the DGEMM improved rectangle method (whose effort will be obvious by looking at the graphs)\n",
    "    - N=M=K=4800\n",
    "    - tilesize=200\n",
    "    - after each 6x6 rectangle of tile gemm tasks are inserted, the master thread will go and execute tasks with the worker threads until there are `eacb` tasks left.\n",
    "        - This `eacb` parameter is short for 'execute and come back', the ParSEC function used to implement this functionality\n",
    "- This run of DGEMM on Methane, a 36-core machine with Intel Xeon Gold 6140 Processors at 2.4 GHz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f75f4296",
   "metadata": {},
   "outputs": [],
   "source": [
    "trace_eacb072 = pd.HDFStore(\"eacb072_profile.prof.h5\") # Data from the Methane run"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23011078",
   "metadata": {},
   "source": [
    "### eacb = 072"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42e69172",
   "metadata": {},
   "source": [
    "### Animate progress of work on Matrix C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c17bb3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_eacb072,\n",
    "                     \"gemm\",\n",
    "                     order_func = panimate.rectangle_order_generator_improved,\n",
    "                     which_animate = \"progress\",\n",
    "                     title=\"enforced rectangle eacb 072\",\n",
    "                     num_frames = 30,\n",
    "                     fps = 10,\n",
    "                     N = 4800,\n",
    "                     M = 4800,\n",
    "                     K = 4800,\n",
    "                     tilesize = 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d8ae6aa",
   "metadata": {},
   "source": [
    "### Enforce a time interval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ce93822",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_eacb072,\n",
    "                     \"gemm\",\n",
    "                     order_func = panimate.rectangle_order_generator_improved,\n",
    "                     which_animate = \"progress\",\n",
    "                     title=\"enforced rectangle eacb 072\",\n",
    "                     enforce_interval = 0.001,\n",
    "                     fps = 10,\n",
    "                     N = 4800,\n",
    "                     M = 4800,\n",
    "                     K = 4800,\n",
    "                     tilesize = 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b361a24",
   "metadata": {},
   "source": [
    "## Other possible plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5e9bf1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_eacb072,\n",
    "                     \"gemm\",\n",
    "                     order_func = panimate.rectangle_order_generator_improved,\n",
    "                     which_animate = \"tasks\",\n",
    "                     title=\"enforced rectangle eacb 072\",\n",
    "                     num_frames = 50,\n",
    "                     fps = 10,\n",
    "                     N = 4800,\n",
    "                     M = 4800,\n",
    "                     K = 4800,\n",
    "                     tilesize = 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eaa0cc18",
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"tasks_per_frame_(enforced rectangle eacb 072).png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cd56ebf",
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"tasks_times_execution_order_(enforced rectangle eacb 072).png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23eb265b",
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"tasks_times_sorted_(enforced rectangle eacb 072).png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84330348",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_eacb072,\n",
    "                     \"gemm\",\n",
    "                     order_func = panimate.rectangle_order_generator_improved,\n",
    "                     which_animate = \"abcprogress\",\n",
    "                     title=\"enforced rectangle eacb 072\",\n",
    "                     num_frames = 50,\n",
    "                     fps = 10,\n",
    "                     N = 4800,\n",
    "                     M = 4800,\n",
    "                     K = 4800,\n",
    "                     tilesize = 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1335cb76",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_eacb072,\n",
    "                     \"gemm\",\n",
    "                     order_func = panimate.rectangle_order_generator_improved,\n",
    "                     which_animate = \"abctasks\",\n",
    "                     title=\"enforced rectangle eacb 072\",\n",
    "                     num_frames = 50,\n",
    "                     fps = 10,\n",
    "                     N = 4800,\n",
    "                     M = 4800,\n",
    "                     K = 4800,\n",
    "                     tilesize = 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d78b3184",
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"tasks_per_frame_(enforced rectangle eacb 072).png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d90b6df6",
   "metadata": {},
   "source": [
    "## POTRF\n",
    "You can also run traces on POTRF - more is coming in the future.\n",
    "The below data was also run on Methane.\n",
    "\n",
    "- potrf with N=5400, tilesize=180"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c194f8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "trace_potrf     = pd.HDFStore(\"2022-06-15_mid_dpotrf.prof.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c51ecde",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_potrf,\n",
    "                     \"potrf\",\n",
    "                     which_animate = \"progress\",\n",
    "                     title = \"potrf trace testing\",\n",
    "                     num_frames = 60,\n",
    "                     fill = \"absolute\",\n",
    "                     fps = 10,\n",
    "                     N = 5400,\n",
    "                     M = 5400,\n",
    "                     K = 5400,\n",
    "                     tilesize = 180)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ae89e26",
   "metadata": {},
   "outputs": [],
   "source": [
    "panimate.animate_trace(trace_potrf,\n",
    "                     \"potrf\",\n",
    "                     which_animate = \"progress\",\n",
    "                     title = \"potrf trace testing\",\n",
    "                     num_frames = 60,\n",
    "                     fill = \"relative\",\n",
    "                     fps = 10,\n",
    "                     N = 5400,\n",
    "                     M = 5400,\n",
    "                     K = 5400,\n",
    "                     tilesize = 180)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1dd1dbf6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}