-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
issue #4: llama index deployment design
- Loading branch information
Showing
5 changed files
with
101 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
@startuml | ||
left to right direction | ||
interface REST | ||
|
||
component "LlamaIndex API" as api { | ||
package app as "flask app" | ||
package LlamaIndex as "llamaIndex_db" { | ||
package llama_index | ||
} | ||
} | ||
app --> LlamaIndex: uses | ||
REST -- api | ||
|
||
folder Container as "Index folder" { | ||
file doc as "default__vector_store.json" | ||
file index as "docstore.json" | ||
file vector as "graph_store.json" | ||
file graph as "image__vector_store.json" | ||
file image as "index_store.json" | ||
} | ||
|
||
llama_index --> Container : reads | ||
|
||
@enduml |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
@startuml | ||
left to right direction | ||
interface REST | ||
|
||
cloud Azure { | ||
node docker as "Docker container" { | ||
component "LlamaIndex API" as api | ||
} | ||
database volume as "Docker Volume" { | ||
folder Container as "Index folder" | ||
} | ||
} | ||
|
||
REST -- api | ||
api --> Container : reads | ||
@enduml |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# Llama index deployment design | ||
|
||
## Component diagram | ||
|
||
 | ||
|
||
## Deployment diagram | ||
|
||
 | ||
|
||
## LlamaIndex API startup sequence diagram | ||
|
||
```mermaid | ||
sequenceDiagram | ||
participant creator as App Creator | ||
create participant sc as :StorageContext | ||
creator->>sc: create with index_folder | ||
create participant index as :BaseIndex | ||
creator->>index: create with storage_context | ||
create participant retriever as :VectorIndexRetriever | ||
creator->>retriever: create with index | ||
create participant rqe as :RetrieverQueryEngine | ||
creator->>rqe: create with retriever | ||
create participant conf as :Config | ||
creator->>conf: create with query_engine | ||
create participant app as :Flask | ||
creator->>app: create with config | ||
``` | ||
|
||
**Resources:** | ||
|
||
- [Loading data in LlamaIndex using a remote | ||
filesystem](https://docs.llamaindex.ai/en/stable/module_guides/storing/save_load.html#using-a-remote-backend) | ||
|
||
## LLamaIndex API search sequence diagram | ||
|
||
```mermaid | ||
sequenceDiagram | ||
actor user | ||
participant app as app:Flask | ||
participant config as config:Config | ||
participant rqe as :RetrieverQueryEngine | ||
user->>+app: POST /search query_string | ||
app->>config: get query_engine | ||
app->>+rqe: query with query_string | ||
rqe-->>-app: response | ||
app->app: generate results from response.nodes | ||
app-->>-user: results | ||
``` | ||
|
||
**Resources:** | ||
|
||
- [Creating a flask API with | ||
LlamaIndex](https://docs.llamaindex.ai/en/stable/understanding/putting_it_all_together/apps/fullstack_app_guide.html#flask-backend) | ||
|
||
**Note:** | ||
|
||
The retriever query potentially takes some time (more than 5 seconds in some instances). This could be problematic as the user expects near instant results. We might have to optimize this process. |