Skip to content
#

internvideo

Here is 1 public repository matching this topic...

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

  • Updated Mar 10, 2025
  • C++

Improve this page

Add a description, image, and links to the internvideo topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the internvideo topic, visit your repo's landing page and select "manage topics."

Learn more