[ENH] - add support for Cohere assistants

[smokestacklightnin]

This addresses Issue #227

As always, comments and change requests are welcome and encouraged

[smokestacklightnin]

Quick status update

The code seems to receive the JSON stream http responses without error, but is not parsing them properly. No error is reported in the terminal console when running the UI, but the UI response says that something went wrong. I believe this has to do with the type of streamedchatresponse. I'll do more work to try to fix this.

Also, mypy complains when I run it after adapting AsyncIteratorReader from the google assistant code. I tried running mypy on that file too, and got similar complaints. Does anyone have any input? I haven't pushed anything up yet because pre-commit won't pass if mypy complains

[pmeier]

@smokestacklightnin I took the liberty and had a look to unblock you. I've pushed bfc868b that implements the streaming. While Cohere also uses a streaming HTTP request, in contrast to Google they don't stream JSON, but rather JSONL. Meaning, each line is valid JSON. Thus, we don't need an extra library to resolve the stream, but rather just ingest the stream line by line and convert to JSON ourselves.

[smokestacklightnin]

@smokestacklightnin I took the liberty and had a look to unblock you. I've pushed bfc868b that implements the streaming. While Cohere also uses a streaming HTTP request, in contrast to Google they don't stream JSON, but rather JSONL. Meaning, each line is valid JSON. Thus, we don't need an extra library to resolve the stream, but rather just ingest the stream line by line and convert to JSON ourselves.

@pmeier Thanks for the help! I'll continue working on this

[smokestacklightnin]

I added a preamble_override, which seems to be the last substantial change that is needed. I will then add links in the code and the FAQ documentation.

I tested it, and it seems to be working nominally correctly

[pmeier]

Only a few minor comments left. Otherwise this is good to go. Thanks @smokestacklightnin!

[smokestacklightnin]

Only a few minor comments left. Otherwise this is good to go. Thanks @smokestacklightnin!

Everything is done. Glad to contribute!