updating deploy notebooks for hardware requirement (#30)

NVIDIA-AI-Blueprints · Dec 12, 2024 · f85251e · f85251e
1 parent f7128e7
commit f85251e
Showing 1 changed file with 31 additions and 28 deletions.
diff --git a/deploy/1_Deploy_CVE.ipynb b/deploy/1_Deploy_CVE.ipynb
@@ -13,10 +13,12 @@
    "id": "8a0f2acb-52d9-49dd-9676-2d58715786f5",
    "metadata": {},
    "source": [
-    "> \n",
-    "This notebook will deploy the NVIDIA AI Blueprint for vulnerability analysis using NVIDIA NIM microservices and [NVIDIA Morpheus](https://github.com/nv-morpheus/Morpheus) which demonstrates accelerated analysis on common vulnerabilities and exposures (CVEs) at an enterprise scale, reducing mitigation from days and hours to just seconds. You will install the necessary prerequisites, spin up NVIDIA Morpheus SDK and NVIDIA NIM™ microservices on 4xA100 GPU nodes. For improved paralleled performance, we recommend 8x or more H100s for inference. Depending on the application, the Morpheus pipeline operation affords more flexibility on GPU needs.\n",
+    "This notebook will deploy the NVIDIA AI Blueprint for vulnerability analysis using NVIDIA NIM microservices and [NVIDIA Morpheus](https://github.com/nv-morpheus/Morpheus) which demonstrates accelerated analysis on common vulnerabilities and exposures (CVEs) at an enterprise scale, reducing mitigation from days and hours to just seconds. You will install the necessary prerequisites, spin up a NVIDIA Morpheus pipeline and connect to NVIDIA hosted NIM APIs. Depending on the application, the Morpheus pipeline operation affords more flexibility on GPU needs.\n",
     "\n",
-    "Once deployed, you will have a fully functional pipeline which determines whether a software package includes exploitable and vulnerable components, be able to run the blueprint on an example input including viewing the results from an output file with all the gathered and generated information for a final review by the security analyst.\n"
+    "Once deployed, you will have a fully functional pipeline which determines whether a software package includes exploitable and vulnerable components, be able to run the blueprint on an example input including viewing the results from an output file with all the gathered and generated information for a final review by the security analyst.\n",
+    "\n",
+    "\n",
+    "**Note**: this notebook is designed to run as a [brev.dev launchable](https://console.brev.dev/launchable/deploy/?launchableID=env-2olGFXbhH0qEtU47MJ4tpxCAZIn) on 1XL40S GPU."
    ]
   },
   {
@@ -142,7 +144,7 @@
    "id": "5d26d7bf-0044-45c4-bace-18b63c41dd04",
    "metadata": {},
    "source": [
-    "### Get an API Key\n",
+    "### Obtain API Keys\n",
     "To run the pipeline, you need to obtain API keys for the following APIs. These will be needed in a later step to Set up the environment file.\n",
     "\n",
     "\n"
@@ -339,9 +341,9 @@
    "id": "3d6a8a62-fc0c-4c26-87d4-f2b3b1aaba95",
    "metadata": {},
    "source": [
-    "Export **API Key**\n",
+    "Authenticate Docker with **NGC**\n",
     "\n",
-    "To download the appropriate models and resources, you will need to set NGC_API_KEY environment variable. This will allow you to pass the API key value to a docker run command."
+    "In order to pull images required by the workflow from NGC, you must first authenticate Docker with NGC. You can use same the NVIDIA API Key obtained in the [Obtain API Keys](#obtain-api-keys) section (saved as `NVIDIA_API_KEY` in the `.env` file)."
    ]
   },
   {
@@ -354,8 +356,7 @@
    "outputs": [],
    "source": [
     "%%bash\n",
-    "export NGC_API_KEY=\"$NVIDIA_API_KEY\"\n",
-    "echo \"${NGC_API_KEY}\" | docker login nvcr.io -u '$oauthtoken' --password-stdin"
+    "echo \"${NVIDIA_API_KEY}\" | docker login nvcr.io -u '$oauthtoken' --password-stdin"
    ]
   },
   {
@@ -476,7 +477,7 @@
    "source": [
     "## OPTIONAL SECTION : Pull <name> **Self-hosted NIMs**\n",
     "\n",
-    "***NOTE***: If you want to self-host this entire blueprint, you will need an 8xH100 instance at minimum. Multiple nodes are highly recommended!\n"
+    "**Note**: Only run this section if you are using notebook to deploy self-hosted NIMs, e.g. for production purposes. To self-host this entire blueprint, you will need a 4XA100 configuration at minimum. (This configuration can be launched using [brev.dev](https://www.brev.dev/)). Refer to the [Hardware Requirements](https://github.com/NVIDIA-AI-Blueprints/vulnerability-analysis?tab=readme-ov-file#hardware-requirements) section for more information."
    ]
   },
   {
@@ -728,7 +729,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! curl -v -X POST http://0.0.0.0:26466/scan -d @data/input_messages/morpheus:24.03-runtime.json"
+    "! curl -v -X POST http://0.0.0.0:26466/scan -d @data/input_messages/morpheus:23.11-runtime.json"
    ]
   },
   {
@@ -835,6 +836,26 @@
     "4. Explore the vulnerability analysis outputs using `meta/llama-3.1-70b-instruct`!"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f0181f3a",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-block alert-info\">\n",
+    "    <b>Note:</b>  This explore section is optional and independent of the deployment. You will need to restart the kernel after installing the openai module.\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "44ec8875",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install openai"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -1046,24 +1067,6 @@
     "Error requesting [1/10]: (Retry 0.1 sec) https://services.nvd.nist.gov/rest/json/cves/2.0: 503, message='Service Unavailable', url=URL('https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2023-50447')\n",
     "```"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f88c1421-e582-4f09-9fee-522eb498a404",
-   "metadata": {},
-   "source": [
-    "<div class=\"alert alert-block alert-info\">\n",
-    "    <b>Note:</b> By default, GPU IDs 0-3 are for LLM, 4 for the embedding model, and 5 for the reranking model.\n",
-    "</div>\n",
-    "    \n",
-    ">To change the GPUs used for NIM deployment, set the following environment variables:\n",
-    "\n",
-    ">>**LLM_MS_GPU_ID**: Update this to specify the LLM GPU IDs (e.g., 0,1,2,3).\n",
-    "\n",
-    ">>**EMBEDDING_MS_GPU_ID**: Change this to set the embedding GPU ID.\n",
-    ">>\n",
-    ">>**RANKING_MS_GPU_ID**: Modify this to adjust the reranking LLM GPU ID."
-   ]
   }
  ],
  "metadata": {