What is the difference between OCR_PROVIDER and VISION_LLM_PROVIDER?

[rosaLux161]

What is the difference between OCR_PROVIDER and VISION_LLM_PROVIDER?

      OCR_PROVIDER: "llm" # Default OCR provider
      VISION_LLM_PROVIDER: "ollama" # openai or ollama

Are these two technics to OCR the document? What do I have to set to use the OCR generated by Paperless-ngx with Tesseract?

[gardar]

You have the option to use either:

• A specialized AI OCR service (currently only google document ai)
• A LLM provider (ollama or openai) which are services that can do OCR pretty well, although they are not specially designed as OCR services.

To use the specialized AI OCR service you set the OCR_PROVIDER to google_docai

To use a LLM provider you have to set both OCR_PROVIDER to llm and also set VISION_LLM_PROVIDER to either ollama or openai, then you also might want to look into customizing the prompt to get the ocr results you want.

The OCR data from paperless-gpt is saved in the 'Content' section of the document in paperless-ngx. However, unlike Tesseract OCR in paperless-ngx, it is not yet embedded in the PDF. This functionality is still a work in progress: #212.

[rosaLux161]

But how to use exactly this by paperless-ngx generated OCR. I don't want to replace something, I just want to use paperless-gpt with the already exisiting OCR data and generate titles, tags and corresponds with this data instead of sending every document to Google AI or a Vision LLM if there is already existing OCR data (even if produced results by Google AI or Vision LLM may be better).

[gardar]

Ah in that case you skip the OCR in paperless-gpt and use LLM_PROVIDER not VISION_LLM_PROVIDER

[rosaLux161]

And also don't set OCR_PROVIDER to anything?

But that should not be possible according to the readme:

OCR_PROVIDER 	OCR provider to use (llm or google_docai). 	No 	llm
VISION_LLM_PROVIDER 	AI backend for LLM OCR (openai or ollama). Required if OCR_PROVIDER is llm. 	Cond. 	
VISION_LLM_MODEL 	Model name for LLM OCR (e.g. minicpm-v). Required if OCR_PROVIDER is llm. 	Cond.

If I don't set OCR_PROVIDER should the default be 'llm'. And if OCR_PROVIDER is llm, the fields VISION_LLM_PROVIDER and VISION_LLM_MODEL are required.

[gardar]

Well it doesn't really matter what you set the OCR settings to. If you don't press the OCR button in the paperless-gpt webui then no OCR is done.

[rosaLux161]

You mean Vision LLM oder Document AI are only used on this subpage /experimental-ocr? Then I didn't had a question at all. I thought it is somehow used in the process of generating tags and stuff when clicking on 'Generate'.