Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add support for <think></think> tag. #1115

Closed
braveltd opened this issue Jan 22, 2025 · 8 comments
Closed

feature: add support for <think></think> tag. #1115

braveltd opened this issue Jan 22, 2025 · 8 comments
Labels
enhancement New feature or request

Comments

@braveltd
Copy link

Feature request

I tried DeepSeek-R1 today, and noticed. It generated before each generation

Image

i think avante should support it, and make it kinda darker (much less visible) or smth.

or maybe i just dont know how to use this model yet!))

Motivation

Ugly ahhh tags at the start of each generation....

Other

No response

@braveltd braveltd added the enhancement New feature or request label Jan 22, 2025
@jtabke
Copy link

jtabke commented Jan 23, 2025

I was able to strip out the think preamble like this:

parse_stream_data = function(data, handler_opts)
	local json_data = vim.fn.json_decode(data)

	if json_data then
		-- Check for final message with "done: true"
		if json_data.done then
			handler_opts.on_complete(nil) -- Signal completion
			return
		end

		if json_data.message and json_data.message.content then
			local content = json_data.message.content

			-- Track and accumulate content after <think> tag
			if not handler_opts.in_think_block and content:match("<think>") then
				handler_opts.in_think_block = true
				return
			end

			if handler_opts.in_think_block and content:match("</think>") then
				handler_opts.in_think_block = false
				return
			end

			-- Only pass content when not in think block
			if not handler_opts.in_think_block and content ~= "" then
				handler_opts.on_chunk(content)
			end
		end
	end
end,

If your using parse_response with API that follows SSE you might need to alter this slightly but should give you a simple hack to address this.

@braveltd
Copy link
Author

can we make it gray + maybe closeable?

not everytime you wanna see what is AI thinking before answering, and feel's like a good idea to make it grayish color...

@weizheheng
Copy link

@braveltd Do you happen to know why it's not rendering the <think> tag on my end?

Image

@braveltd
Copy link
Author

@weizheheng you're using deepseek-reasoner, not DeepSeek-R1.

@weizheheng
Copy link

@weizheheng you're using deepseek-reasoner, not DeepSeek-R1.

Ah, let me try it out thank you.

Cause on their API doc, it mentioned that deepseek-reasoner points to the new model DeepSeek-R1.

@braveltd
Copy link
Author

@weizheheng i use hyperbolic as my provider. try to use hyperbolic (xyz) or kluster (ai). they give free credits and good for testing and using. (not sponsored ofc)


sorry if that is forbidden to tell that here. im new bro 👑

@braveltd
Copy link
Author

Image

Image

it's fine now, with kluster ai.

Should i close this issue?

@braveltd
Copy link
Author

Image

yes. i should from 2ead26f!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants