How do I grayscale my documentfile?

[DJay921]

I wanted to improve the inference time and accuracy so I wanted to experiment with Grayscaling the image, I understand Grayscale transform is currently not available, right? I was trying to pass the grayscale param to my DocumentFile.from_pdf(...grayscale=True) which is accepted by pypdfium2. But that does not seem to work and throws an this error TypeError: Cannot handle this data type: (1, 1, 1),. What will be the correct/efficient way to do grayscale? can someone please share a minimum example for this? Thanks.

[felixdittrich92]

Hi @DJay921 👋 ,

You can do the following:

doc = DocumentFile.from_pdf("my-pdf-path.pdf")
# Convert to grayscale but keep 3 channels which is required
doc = [cv2.merge([cv2.cvtColor(page, cv2.COLOR_BGR2GRAY)] * 3) for page in doc]

But this shouldn't change anything on the inference latency additional all the models are mainly trained on RGB images and only while training some samples are augmented by grayscaling.

Could you share an short snippet how you use OnnxTR ? Maybe there are some tricks you missed :)

Best,
Felix

[DJay921]

Oh I see, thanks for that information @felixdittrich92. I was using pretty much the default params, but this seems to be taking some time to run the inference, I tried scaling down the image to 1.4. What should be an appropriate scale and image size for running inference though?

# doc loading
from onnxtr.io import DocumentFile
from onnxtr.models import ocr_predictor, from_hub, EngineConfig
from pathlib import Path
import numpy as np
# from onnxtr.transforms import Normalize

pdf_path = r"pathf"

doc = DocumentFile.from_pdf(str(Path(pdf_path)), scale=1.4)

# Load your model 
from onnxtr.models import detection, recognition

# Path setup for models

detection_model_path = 'onnx_models/onnxtr-db-mobilenet-v3-large.onnx'
recognition_model_path = 'onnx_models/onnxtr-crnn-mobilenet-v3-large.onnx'

det_model = detection.db_mobilenet_v3_large(detection_model_path)
rec_model = recognition.crnn_mobilenet_v3_large(recognition_model_path)

predictor = ocr_predictor(det_arch=det_model,
                           reco_arch=rec_model, 
                           assume_straight_pages=False,
                            det_engine_cfg=EngineConfig(),  # detection model engine configuration (default: internal predefined configuration)
    reco_engine_cfg=EngineConfig(),  # recognition model engine configuration (default: internal predefined configuration)
    clf_engine_cfg=EngineConfig(),  # classification (orientation) model engine configuration (default: internal predefined configuration)                      
                          )

export = result.export()

[felixdittrich92]

So under the hood every image is resized to 1024x1024 by keeping the aspect ratio and symmetric padding (If not changed by the users args).

Especially with assume_straight_pages=False you have following options:

disable_crop_orientation=True: If your images doesn't contain vertical rotated text elements - huge speed-up
disable_page_orientation=True: If your images are maximum by -90 - 90 degrees rotated - small speed-up

Additional there is also the option for straighten_pages=True

It does really depend on the documents you want to process which combination works best for you :)