Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added document: Accelerate document indexing and retrieval #4600

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

writinwaters
Copy link
Contributor

What problem does this PR solve?

Type of change

  • Documentation Update


Please note that several of your settings may *significantly* increase the time required for document parsing and retrieval. If you often find that document parsing and question answering are time-consuming, here is a checklist to consider:

1. Use GPU to reduce embedding time.
Copy link
Contributor

@JinHai-CN JinHai-CN Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two chapter:

  1. Accelerate document indexing
  • Use GPU for embedding model
  • Disable RAPTOR
  • GraphRAG will cost lots of time
  1. Accelerate question answer, but not retrieval
  • Disable multiple-turn dialog
  • Disable reranker
  • Disable keyword extraction. (will be enable in this version, check with Kevin)
  • Check the 'light' after each answer. Give and detailed description on each phase of the answer
    • Total: including chunk retrieval and answer generation.
    • Check LLM: check if LLM is valid
    • Create retriever: create chunk retriever
    • Binding embedding: binding embedding model for answer
    • Bing LLM: binding LLM for tuning question of multiple turn dialog
    • Tune question: time cost of tuning question for multiple turn dialog
    • Bind reranker: binding reranker model for chunking retrieval
    • Generate keyword: keyword extraction from question by LLM.
    • Retrieval: data chunk retrieval cost
    • Generate answer: answer generation by LLM time cost

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants