validation errors for users

[davidism]

I've been starting to think about a framework I want to build, that handles structuring and validating both HTML form data and JSON data, as well as building HTML form inputs and OpenAPI schema. I've been looking at various validation and API libraries for inspiration.

As part of that, I was looking into cattrs, and was curious how you were handling validation errors both in cattrs and in uapi. It looks like (but maybe I've missed it) that uapi returns a generic "400 bad request" (or similar) error message on any errors. It looks like cattrs can return detailed error messages, but the messages it produces are still somewhat "developer facing", with messages like "Structuring list[int] @ index 0" including type annotations, indicies, etc.

A previous API library I wrote, magql, produces error messages in a nested structure matching the nested input it received. The messages themselves are "user friendly" messages such as "This field is required", "Enter a whole number", etc. An API client can use that structure to show the messages next to each specific form input that caused them.

I'm interested to hear your thoughts about validation UI and whether you have plans for that in uapi, as well as about generating HTML forms or other UIs.

[Tinche]

Howdy!

I have to admit since I've changed jobs a year ago I haven't been working on uapi a lot, since I don't really get a chance to use it. You're right in that uapi doesn't make use of cattrs validation yet to return errors. I haven't figured out a response format for it yet.

That said, let's go over cattrs validation real quick. cattrs has two modes - detailed_validation=True or False (the default is true). No detailed validation is much faster but it short-circuits on the first error, and there's no exception wrapping so there are no error paths. This mode is more suitable for APIs where you control both ends and you just want speed.

When detailed validation is turned on, cattrs will produce ExceptionGroups (there are a couple of different ones - ClassValidationError and IterableValidationError) and attach notes to exceptions contained within. This data structure is essentially a tree of errors, with leaves the actual errors.

The error message you mention ("Structuring list[int] @ index 0") is from rendering an IterableValidationErorr note. This is what gets printed out to the console when you log out a traceback with something that supports ExceptionGroups. There's also a helper method, transform_error. This traverses the exception tree and produces a list of error messages with paths (the paths are somewhat modeled on msgspec paths). It can be customized somewhat, or you can implement your own. I was planning on using this to expose error messages. If you wrote your own you could get output very similar to what you're describing for magql.

Given all this, there's an optimization opportunity here I've also been considering. The APIs I've implemented so far at my jobs deal with successful requests the vast majority of the time - 99.9%. It feels a waste to use a detailed validation converter on these and give up efficiency in such a large majority of cases. My idea was using two converters, one without detailed validation, and one with, and rerunning the deserialization operation with the second converter in the case the first converter fails. That means we do double the work on errors, but if errors are a vanishingly small percentage of total operations it's worth it.

I don't have plans on rendering HTML responses for errors. I'm more interested in api use cases and don't really have time for this. I don' imagine it'd be particularly challenging in a technical sense (the UI part is probably a different story, but I suck at this ;)

Let me know if you have any other questions or comments. I love discussing cattrs design decisions ;)

[salotz]

Great thread!

I've been thinking about similar things. I'm developing an API that makes heavy use of cattrs. In many ways I'm (unfortunately) reimplementing parts of uapi (with a few different design decisions).

Eventually would love to share work here and build a common thing. I've already discussed elsewhere thoughts on OpenAPI generation #63

While I hadn't considered it too much, I was also planning on using the detailed validation exception trees to do the message generation. Although one could imagine more detailed introspection based on the actual attrs class definitions and metadata attached to those fields.