Skip to content

Commit

Permalink
issue #4: llama index deployment design
Browse files Browse the repository at this point in the history
  • Loading branch information
k-allagbe committed Mar 8, 2024
1 parent 79a0fff commit f9b08b8
Show file tree
Hide file tree
Showing 5 changed files with 102 additions and 0 deletions.
26 changes: 26 additions & 0 deletions docs/deployment_design/components.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
@startuml
left to right direction
interface REST

component "LlamaIndex API" as api {
package app as "flask app"
package LlamaIndex as "llamaIndex_db" {
package llama_index
package azure_storage_blob
}
}
azure_storage_blob <- llama_index: uses
app --> LlamaIndex: uses
REST -- api

folder Container as "Index folder" {
file doc as "default__vector_store.json"
file index as "docstore.json"
file vector as "graph_store.json"
file graph as "image__vector_store.json"
file image as "index_store.json"
}

azure_storage_blob --> Container : reads

@enduml
Binary file added docs/deployment_design/components.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 16 additions & 0 deletions docs/deployment_design/deployment.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
@startuml
left to right direction
interface REST

cloud Azure {
node docker as "Docker container" {
component "LlamaIndex API" as api
}
database volume as "Docker Volume" {
folder Container as "Index folder"
}
}

REST -- api
api --> Container : reads
@enduml
Binary file added docs/deployment_design/deployment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
60 changes: 60 additions & 0 deletions docs/deployment_design/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Llama index deployment design

## Component diagram

![Alt text](components.png)

## Deployment diagram

![Alt text](deployment.png)

## LlamaIndex API startup sequence diagram

```mermaid
sequenceDiagram
participant creator as App Creator
create participant cc as container_client:AzureContainerClient
creator->>cc: instantiate
create participant sc as storage_context:StorageContext
creator->>sc: create with container_client
create participant rqe as retriever:RetrieverQueryEngine
creator->>rqe: create with storage_context
create participant conf as config:Config
creator->>conf: create with retriever
create participant app as app:Flask
creator->>app: create with config
```

**Resources:**

- [Loading data in LlamaIndex using a remote
filesystem](https://docs.llamaindex.ai/en/stable/module_guides/storing/save_load.html#using-a-remote-backend)

## LLamaIndex API search sequence diagram

```mermaid
sequenceDiagram
actor user
participant app as app:Flask
participant config as config:Config
participant rqe as retriever:RetrieverQueryEngine
user->>+app: POST /search query_string
app->>config: get retriever
app->>+rqe: query with query_string
rqe-->>-app: response
app->app: generate results from response.nodes
app-->>-user: results
```

**Resources:**

- [Creating a flask API with
LlamaIndex](https://docs.llamaindex.ai/en/stable/understanding/putting_it_all_together/apps/fullstack_app_guide.html#flask-backend)

**Note:**

The retriever query potentially takes some time (more than 5 seconds in some instances). This could be problematic as the user expects near instant results. We might have to optimize this process.

0 comments on commit f9b08b8

Please sign in to comment.