You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
多卡序列并行 这两个好像乘起来需要等于总的进程数,
assert args.ulysses_size * args.ring_size == world_size, f"The number of ulysses_size and ring_size should be equal to the world size."
然后ulysses_size 需要是num_heads 的银子
assert cfg.num_heads % args.ulysses_size == 0, f"num_heads must be divisible by ulysses_size."
但是具体这两个参数是什么意义 该怎么调有没有谁能解释一下。
The text was updated successfully, but these errors were encountered:
多卡序列并行 这两个好像乘起来需要等于总的进程数,
assert args.ulysses_size * args.ring_size == world_size, f"The number of ulysses_size and ring_size should be equal to the world size."
然后ulysses_size 需要是num_heads 的银子
assert cfg.num_heads % args.ulysses_size == 0, f"
num_heads
must be divisible byulysses_size
."但是具体这两个参数是什么意义 该怎么调有没有谁能解释一下。
The text was updated successfully, but these errors were encountered: