fix: return HTTP 404 when route not found #396
Conversation
Signed-off-by: David Xia <david@davidxia.com>
| }, | ||
| } | ||
|
|
||
| if err := stream.Send(resp); err != nil { |
There was a problem hiding this comment.
cannot use resp (variable of type *"github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3".ImmediateResponse) as *"github.com/envoyproxy/go-control-plane/envoy/service/ext_proc/v3".ProcessingResponse value in argument to stream.Send
There was a problem hiding this comment.
In the model_processor.go you have an example of how to return an immediate response that you can use in the chat completion procesor.
| @@ -84,6 +84,25 @@ func (c *chatCompletionProcessor) ProcessRequestBody(ctx context.Context, rawBod | |||
| c.requestHeaders[c.config.modelNameHeaderKey] = model | |||
| b, err := c.config.router.Calculate(c.requestHeaders) | |||
| if err != nil { | |||
| // if err.Error() == "no matching rule found" { | |||
There was a problem hiding this comment.
Instead of doing this and looking the error tree like you do, let's create a proper ErrRouteNotFound in the filterapi/x package. This way all implementations of Calculate can return that error consistently.
Then you can modify the default Calculate method to return that error, and here can cleanly just use errors.Is and return the immediate response directly.
| @@ -168,6 +170,25 @@ func (s *Server) Process(stream extprocv3.ExternalProcessor_ProcessServer) error | |||
|
|
|||
| resp, err := s.processMsg(ctx, p, req) | |||
| if err != nil { | |||
| s.logger.Info("AAAAA") | |||
There was a problem hiding this comment.
The fact that routes are calculated is an implementation detail of the chat completion processor, because it's based on the request body for requests to that endpoint. Therefore, I would not have this logic here as it couples the server to semantics of a particular processor.
Let's keep this unchanged and just have the chat completion processor return the immediate response.
| }, | ||
| } | ||
|
|
||
| if err := stream.Send(resp); err != nil { |
There was a problem hiding this comment.
In the model_processor.go you have an example of how to return an immediate response that you can use in the chat completion procesor.
|
new one #399 |
No description provided.