Skip to content

Commit

Permalink
更新 grpo_config.py
Browse files Browse the repository at this point in the history
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
  • Loading branch information
ji-huazhong and qgallouedec authored Feb 19, 2025
1 parent 47f1e60 commit 6492607
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions trl/trainer/grpo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,8 @@ class GRPOConfig(TrainingArguments):
`vllm_gpu_memory_utilization`, leading to a reduced KV cache size. If not set, vLLM will use the model
context size, which might be much larger than the KV cache, leading to inefficiencies.
vllm_enable_prefix_caching (`bool`, *optional*, defaults to `True`):
Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the GPU used support
this feature, because enabling prefix cache on GPUs older than Ampere architecture (like the V100) may
cause errors, see: https://github.com/huggingface/trl/issues/2798.
Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the model and the hardware support
this feature.
vllm_guided_decoding_regex (`str` or `None`, *optional*, defaults to `None`):
Regex for vLLM guided decoding. If `None` (default), guided decoding is disabled.
Expand Down

0 comments on commit 6492607

Please sign in to comment.