Added support for custom API host #105
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
apparently PR #92 addresses the bug that made it seem like there was no way to change the API host, closing this PR in favor of the bug fix. |
|
Just came here to say +1 for ollama support. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have very limited expirience with Go but I really want to use yai with local LLM's so I tried to implement support for custom API host and seemingly succeeded.
It is possible because Ollama(open source program that allows to run LLM's locally) recently added Openai API compatibility and now all programs written for OpenAI's API that allow to edit API host can run using local LLM's.
Because of my limited expirience with go and lacking familiarity with this project's codebase I am not sure if I implemented everything correctly strongly advise to review this PR carefully and add anything that I forgot to(if I did) before merging.
I feel like my expirience is not enough to implement custom system prompts without actually learning the language. But people who run local LLM's will appreciate if you do implement custom system prompts because some system prompts that are beneficial for ChatGPT are confusing for smaller and dumber 7b models. If custom system prompts are implemented people will be able to fine tune their prompts for the model that they use.