Skip to content

Commit

Permalink
issue #4: llama index deployment design
Browse files Browse the repository at this point in the history
  • Loading branch information
k-allagbe committed Mar 8, 2024
1 parent 79a0fff commit 413098d
Show file tree
Hide file tree
Showing 5 changed files with 101 additions and 0 deletions.
24 changes: 24 additions & 0 deletions docs/deployment_design/components.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
@startuml
left to right direction
interface REST

component "LlamaIndex API" as api {
package app as "flask app"
package LlamaIndex as "llamaIndex_db" {
package llama_index
}
}
app --> LlamaIndex: uses
REST -- api

folder Container as "Index folder" {
file doc as "default__vector_store.json"
file index as "docstore.json"
file vector as "graph_store.json"
file graph as "image__vector_store.json"
file image as "index_store.json"
}

llama_index --> Container : reads

@enduml
Binary file added docs/deployment_design/components.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 16 additions & 0 deletions docs/deployment_design/deployment.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
@startuml
left to right direction
interface REST

cloud Azure {
node docker as "Docker container" {
component "LlamaIndex API" as api
}
database volume as "Docker Volume" {
folder Container as "Index folder"
}
}

REST -- api
api --> Container : reads
@enduml
Binary file added docs/deployment_design/deployment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
61 changes: 61 additions & 0 deletions docs/deployment_design/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Llama index deployment design

## Component diagram

![Alt text](components.png)

## Deployment diagram

![Alt text](deployment.png)

## LlamaIndex API startup sequence diagram

```mermaid
sequenceDiagram
participant creator as App Creator
create participant sc as :StorageContext
creator->>sc: create with index_folder
create participant index as :BaseIndex
creator->>index: create with storage_context
create participant retriever as :VectorIndexRetriever
creator->>retriever: create with index
create participant rqe as :RetrieverQueryEngine
creator->>rqe: create with retriever
create participant conf as :Config
creator->>conf: create with query_engine
create participant app as :Flask
creator->>app: create with config
```

**Resources:**

- [Loading data in LlamaIndex using a remote
filesystem](https://docs.llamaindex.ai/en/stable/module_guides/storing/save_load.html#using-a-remote-backend)

## LLamaIndex API search sequence diagram

```mermaid
sequenceDiagram
actor user
participant app as app:Flask
participant config as config:Config
participant rqe as :RetrieverQueryEngine
user->>+app: POST /search query_string
app->>config: get query_engine
app->>+rqe: query with query_string
rqe-->>-app: response
app->app: generate results from response.nodes
app-->>-user: results
```

**Resources:**

- [Creating a flask API with
LlamaIndex](https://docs.llamaindex.ai/en/stable/understanding/putting_it_all_together/apps/fullstack_app_guide.html#flask-backend)

**Note:**

The retriever query potentially takes some time (more than 5 seconds in some instances). This could be problematic as the user expects near instant results. We might have to optimize this process.

0 comments on commit 413098d

Please sign in to comment.