Skip to content

Issues: huggingface/trl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

159 Open 1,228 Closed
159 Open 1,228 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

GRPO VLLM does not work with Lora 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#2698 opened Jan 30, 2025 by gagan3012
5 tasks done
I cannot launch PPOTrainning script with accelerate launch ⚡accelerate Related to accelerate ⚡ PEFT Related to PEFT 🏋 PPO Related to PPO
#2696 opened Jan 30, 2025 by daehuikim
5 tasks done
OOM 8xH100 using latest GRPO code with vLLM 🐛 bug Something isn't working 🚀 deepspeed Related to deepspeed 🏋 GRPO Related to GRPO
#2688 opened Jan 30, 2025 by abacaj
5 tasks done
6
empty Cache after logps_per_token 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#2686 opened Jan 29, 2025 by shirinyamani
3
rewards_funcs set to eval mode 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information 🏋 Reward Related to Reward modelling
#2685 opened Jan 29, 2025 by shirinyamani
1
Support iterative GRPO ✨ enhancement New feature or request 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#2684 opened Jan 29, 2025 by howardzhou
@shirinyamani
8
About the Implementation of GRPO 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information
#2681 opened Jan 29, 2025 by macheng6
1
Ability to provide a static completion for GRPO ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#2680 opened Jan 29, 2025 by Palmik
1
logging issue: generation_config in rloo_trainer.py's generate_completions() is not reflective of actual model generations 🐛 bug Something isn't working 🏋 RLOO Related to RLOO
#2678 opened Jan 28, 2025 by swkarlekar
5 tasks done
TypeError: type list doesn't define __round__ method - why I am getting this error 🐛 bug Something isn't working ⏳ needs more info Additional information or clarification is required to proceed 🏋 Reward Related to Reward modelling
#2674 opened Jan 28, 2025 by Tarak200
1
"None of the inputs have requires_grad=True" with online DPO and GRPO 🐛 bug Something isn't working 🏋 GRPO Related to GRPO 🏋 Online DPO Related to Online DPO
#2671 opened Jan 28, 2025 by benjamin-marie
5 tasks done
3
whiten_rewards parameter in RLOO config is not used. 🐛 bug Something isn't working 🏋 RLOO Related to RLOO
#2665 opened Jan 26, 2025 by velezbeltran
5 tasks done
Issues with DPOTrainer and Qwen2-VL processor 🐛 bug Something isn't working 🏋 DPO Related to DPO
#2660 opened Jan 25, 2025 by baichuanzhou
1
Resuming from checkpoint doesn't seem to work 🐛 bug Something isn't working ⏳ needs more info Additional information or clarification is required to proceed 🏋 PPO Related to PPO 🏋 RLOO Related to RLOO
#2657 opened Jan 25, 2025 by Superskyyy
5 tasks done
2
Judge API feedback: Structured inputs ✨ enhancement New feature or request
#2655 opened Jan 25, 2025 by DreamGenX
Update Documentation about SFT Config 📚 documentation Improvements or additions to documentation 👶 good first issue Good for newcomers 🏋 SFT Related to SFT
#2649 opened Jan 24, 2025 by ParagEkbote
5 tasks done
2
How to stop SFTTrainer from auto tokenizing my messages ? ❓ question Seeking clarification or more information 🏋 SFT Related to SFT
#2642 opened Jan 24, 2025 by MohamedAliRashad
3
../aten/src/ATen/native/cuda/Indexing.cu:1284: indexSelectLargeIndex: block: [193,0,0], thread: [66,0,0] Assertion srcIndex < srcSelectDimSize failed. 🐛 bug Something isn't working 🙋 help from community wanted Open invitation for community members to contribute 🏋 PPO Related to PPO
#2641 opened Jan 23, 2025 by JohnConnor123
5 tasks done
1
RLOO
#2634 opened Jan 23, 2025 by qgallouedec
Reward
#2633 opened Jan 23, 2025 by qgallouedec
@qgallouedec
1
PPO
#2629 opened Jan 23, 2025 by qgallouedec
ORPO
#2628 opened Jan 23, 2025 by qgallouedec
GKD
#2624 opened Jan 23, 2025 by qgallouedec
@kashif
1
[Tracking issue] Wrong loss scaling when accumulating gradient 🐛 bug Something isn't working 🏋 DPO Related to DPO 🏋 DPPO Related to DDPO 🏋 GKD Related to GKD 🏋 GRPO Related to GRPO 🏋 Iterative SFT Related to Iterative SFT 🏋 KTO Related to KTO 🏋 Online DPO Related to Online DPO 🏋 ORPO Related to ORPO 🏋 PPO Related to PPO 🏋 PRM Related to PRM 🏋 Reward Related to Reward modelling 🏋 RLOO Related to RLOO 🏋 SFT Related to SFT 🏋 XPO Related to XPO
#2617 opened Jan 23, 2025 by qgallouedec
13 of 18 tasks
PPOTrainer.__init__() missing 2 required positional arguments: 'reward_model' and 'train_dataset' 🐛 bug Something isn't working 🏋 PPO Related to PPO
#2614 opened Jan 23, 2025 by ansonctyu
5 tasks done
2
ProTip! What’s not been updated in a month: updated:<2024-12-30.