Skip to content

Commit

Permalink
apply review suggestion
Browse files Browse the repository at this point in the history
  • Loading branch information
ji-huazhong committed Feb 19, 2025
1 parent 47f1e60 commit 416908b
Showing 1 changed file with 4 additions and 6 deletions.
10 changes: 4 additions & 6 deletions trl/trainer/grpo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,8 @@ class GRPOConfig(TrainingArguments):
`vllm_gpu_memory_utilization`, leading to a reduced KV cache size. If not set, vLLM will use the model
context size, which might be much larger than the KV cache, leading to inefficiencies.
vllm_enable_prefix_caching (`bool`, *optional*, defaults to `True`):
Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the GPU used support
this feature, because enabling prefix cache on GPUs older than Ampere architecture (like the V100) may
cause errors, see: https://github.com/huggingface/trl/issues/2798.
Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the model and the hardware
support this feature.
vllm_guided_decoding_regex (`str` or `None`, *optional*, defaults to `None`):
Regex for vLLM guided decoding. If `None` (default), guided decoding is disabled.
Expand Down Expand Up @@ -211,9 +210,8 @@ class GRPOConfig(TrainingArguments):
vllm_enable_prefix_caching: Optional[bool] = field(
default=True,
metadata={
"help": "Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the GPU used "
"support this feature, because enabling prefix cache on GPUs older than Ampere architecture (like the V100) "
"may cause errors, see: https://github.com/huggingface/trl/issues/2798."
"help": "Whether to enable prefix caching in vLLM. If set to `True` (default), ensure that the model and "
"the hardware support this feature."
},
)
vllm_guided_decoding_regex: Optional[str] = field(
Expand Down

0 comments on commit 416908b

Please sign in to comment.