SimpleIndexDemo-ChatGPT.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "7ae43f8b",
      "metadata": {
        "id": "7ae43f8b"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jerryjliu/llama_index/blob/main/docs/examples/customization/llms/SimpleIndexDemo-ChatGPT.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9c48213d-6e6a-4c10-838a-2a7c710c3a05",
      "metadata": {
        "id": "9c48213d-6e6a-4c10-838a-2a7c710c3a05"
      },
      "source": [
        "# ChatGPT"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cf6108b0",
      "metadata": {
        "id": "cf6108b0"
      },
      "source": [
        "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "590874bc",
      "metadata": {
        "id": "590874bc"
      },
      "outputs": [],
      "source": [
        "%pip install llama-index-llms-openai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "14790912",
      "metadata": {
        "id": "14790912"
      },
      "outputs": [],
      "source": [
        "!pip install llama-index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "690a6918-7c75-4f95-9ccc-d2c4a1fe00d7",
      "metadata": {
        "id": "690a6918-7c75-4f95-9ccc-d2c4a1fe00d7"
      },
      "outputs": [],
      "source": [
        "import logging\n",
        "import sys\n",
        "\n",
        "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
        "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
        "\n",
        "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
        "from llama_index.core import Settings\n",
        "from llama_index.llms.openai import OpenAI\n",
        "from IPython.display import Markdown, display"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9bc1c58d",
      "metadata": {
        "id": "9bc1c58d"
      },
      "source": [
        "#### Download Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "18c8d851",
      "metadata": {
        "id": "18c8d851"
      },
      "outputs": [],
      "source": [
        "!mkdir -p 'data/paul_graham/'\n",
        "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "50d3b817-b70e-4667-be4f-d3a0fe4bd119",
      "metadata": {
        "id": "50d3b817-b70e-4667-be4f-d3a0fe4bd119"
      },
      "source": [
        "#### Load documents, build the VectorStoreIndex"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "03d1691e-544b-454f-825b-5ee12f7faa8a",
      "metadata": {
        "id": "03d1691e-544b-454f-825b-5ee12f7faa8a"
      },
      "outputs": [],
      "source": [
        "# load documents\n",
        "documents = SimpleDirectoryReader(\"./data/paul_graham\").load_data()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6cc980e8-f4e1-4fad-93f8-ab1bbaa874f3",
      "metadata": {
        "id": "6cc980e8-f4e1-4fad-93f8-ab1bbaa874f3"
      },
      "outputs": [],
      "source": [
        "# set global settings config\n",
        "llm = OpenAI(temperature=0, model=\"gpt-3.5-turbo\")\n",
        "Settings.llm = llm\n",
        "Settings.chunk_size = 512"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ad144ee7-96da-4dd6-be00-fd6cf0c78e58",
      "metadata": {
        "id": "ad144ee7-96da-4dd6-be00-fd6cf0c78e58"
      },
      "outputs": [],
      "source": [
        "index = VectorStoreIndex.from_documents(documents)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b6caf93b-6345-4c65-a346-a95b0f1746c4",
      "metadata": {
        "id": "b6caf93b-6345-4c65-a346-a95b0f1746c4"
      },
      "source": [
        "#### Query Index"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "83e2905e-3789-4793-82b9-0ac488246824",
      "metadata": {
        "id": "83e2905e-3789-4793-82b9-0ac488246824"
      },
      "source": [
        "By default, with the help of langchain's PromptSelector abstraction, we use\n",
        "a modified refine prompt tailored for ChatGPT-use if the ChatGPT model is used."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "85466fdf-93f3-4cb1-a5f9-0056a8245a6f",
      "metadata": {
        "id": "85466fdf-93f3-4cb1-a5f9-0056a8245a6f",
        "outputId": "6ccacb11-7819-4802-bad4-044110ff1ce6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
            "> [retrieve] Total LLM token usage: 0 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 8 tokens\n",
            "> [retrieve] Total embedding token usage: 8 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 0 tokens\n",
            "> [get_response] Total LLM token usage: 0 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
            "> [get_response] Total embedding token usage: 0 tokens\n"
          ]
        }
      ],
      "source": [
        "query_engine = index.as_query_engine(\n",
        "    similarity_top_k=3,\n",
        "    streaming=True,\n",
        ")\n",
        "response = query_engine.query(\n",
        "    \"What did the author do growing up?\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b0262da8",
      "metadata": {
        "id": "b0262da8",
        "outputId": "c266c4dc-2e31-4dd0-9cb3-58510eb4ca64"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Before college, the author worked on writing short stories and programming on an IBM 1401 using an early version of Fortran. They also worked on programming with microcomputers and eventually created a new dialect of Lisp called Arc. They later realized the potential of publishing essays on the web and began writing and publishing them. The author also worked on spam filters, painting, and cooking for groups."
          ]
        }
      ],
      "source": [
        "response.print_response_stream()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ec88df57",
      "metadata": {
        "id": "ec88df57",
        "outputId": "3259e60c-d7b4-4be1-ec3f-7dee4bec02e1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
            "> [retrieve] Total LLM token usage: 0 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 12 tokens\n",
            "> [retrieve] Total embedding token usage: 12 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 0 tokens\n",
            "> [get_response] Total LLM token usage: 0 tokens\n",
            "INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
            "> [get_response] Total embedding token usage: 0 tokens\n"
          ]
        }
      ],
      "source": [
        "query_engine = index.as_query_engine(\n",
        "    similarity_top_k=5,\n",
        "    streaming=True,\n",
        ")\n",
        "response = query_engine.query(\n",
        "    \"What did the author do during his time at RISD?\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "67e8e675-1b03-423a-b53e-23ab278ba03b",
      "metadata": {
        "id": "67e8e675-1b03-423a-b53e-23ab278ba03b",
        "outputId": "b3e48b17-d261-4c28-d3be-9e022703c331"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The author attended RISD and took classes in fundamental subjects like drawing, color, and design. They also learned a lot in the color class they took, but otherwise, they were basically teaching themselves to paint. The author dropped out of RISD in 1993."
          ]
        }
      ],
      "source": [
        "response.print_response_stream()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "88ca1808-d112-4c28-b110-b65dcc9b7207",
      "metadata": {
        "id": "88ca1808-d112-4c28-b110-b65dcc9b7207"
      },
      "source": [
        "**Refine Prompt**: Here is the chat refine prompt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2f0c270d-9de5-40bf-88fc-83a360523db0",
      "metadata": {
        "id": "2f0c270d-9de5-40bf-88fc-83a360523db0"
      },
      "outputs": [],
      "source": [
        "from llama_index.core.prompts.chat_prompts import CHAT_REFINE_PROMPT"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4db38651-9790-4a61-ac3d-689ce6dfa369",
      "metadata": {
        "id": "4db38651-9790-4a61-ac3d-689ce6dfa369"
      },
      "outputs": [],
      "source": [
        "dict(CHAT_REFINE_PROMPT.prompt)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6cb664e8-f53f-4d6c-a086-1f2784cc1dc8",
      "metadata": {
        "id": "6cb664e8-f53f-4d6c-a086-1f2784cc1dc8"
      },
      "source": [
        "#### Query Index (Using the standard Refine Prompt)\n",
        "\n",
        "If we use the \"standard\" refine prompt (where the prompt is one text template instead of multiple messages), we find that the results over ChatGPT are worse."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "29c416f8-d5ab-47d6-8b16-f615bfa58219",
      "metadata": {
        "id": "29c416f8-d5ab-47d6-8b16-f615bfa58219"
      },
      "outputs": [],
      "source": [
        "from llama_index.core.prompts.default_prompts import DEFAULT_REFINE_PROMPT"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3df1acc4-735a-48ac-9fb4-73d9d7eabc02",
      "metadata": {
        "id": "3df1acc4-735a-48ac-9fb4-73d9d7eabc02"
      },
      "outputs": [],
      "source": [
        "query_engine = index.as_query_engine(\n",
        "    refine_template=DEFAULT_REFINE_PROMPT,\n",
        "    similarity_top_k=5,\n",
        "    streaming=True,\n",
        ")\n",
        "response = query_engine.query(\n",
        "    \"What did the author do during his time at RISD?\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b8938077-6527-4008-8d0c-af7a8178ff10",
      "metadata": {
        "id": "b8938077-6527-4008-8d0c-af7a8178ff10"
      },
      "outputs": [],
      "source": [
        "response.print_response_stream()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "llama_index_v2",
      "language": "python",
      "name": "llama_index_v2"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}