General Guide

Endpoints can be reached via HTTP Requests. Each endpoint has an associated query object that it expects as body, and writes back something as a Response.

When marking stream = true in a query, the server will keep writing to the HTTP Response's body as soon as some info is available. Otherwise all at once on end.

Streamed responses follow this format: $"data: {JSON.Serialize(outputObj)}\n\n". Check the javascript examples for a way to parse JUST the response.

Chat Endpoint "/chat/completion":POST

• Code: ChatEndpoint.cs
• Expects ChatQuery as the HTTP Request's body.
• Outputs LocalResponse.

Completion Endpoint "/text-completion/completion":POST

• Code: CompletionEndpoint.cs
• Expects CompletionQuery as the HTTP Request's body.
• Outputs LocalResponse.

Batch Endpoint "/batch":POST

• Code: BatchEndpoint.cs
• Expects QueryBatch as the HTTP Request's body.
• Outputs BatchResponse with the mapped ids and LocalResponses.

Note: Query Batches can contain both Chat & Completion Queries. In this case, the Completion queries will obtain an ID first.

(Beta) Classification Endpoint "/classify":POST

• Code: ClassificationEndpoint.cs
• Expects ClassificationQuery as the HTTP Request's body.
• Outputs string with the index of the option that was chosen by the LLM.

Note: The classification endpoint probably needs user customization before usage.