[Support] Request for supporting Sd3.5 Medium

[pprp]

We’ve encountered an issue where our code works correctly with SD3, but fails to function as expected with SD3.5. After running the same code on SD3.5, we’re seeing discrepancies that don’t appear on SD3. Could you please assist us in identifying the bug or compatibility issue?

Specifically, we got the following output from SD3.5

[feifeibear]

Would like to give a merge request for SD3.5. We are willing to help you debug this problem.

[Zctoylm0927]

Hi, I met the same problem here.

After running the same code on SD3.5

I guess we only change the model path to SD3.5, and the model type remains "SD3" using sd3_example.py. Because the difference between SD3.5 and SD3 is that the former has dual_attention, which changes in norm1 and attn2.
If I set the $GPU to single A800, it could generate the normal picture. But when I run it on 2 A800 and set pp=2, it generates a strange image like above.
Also, I tried it with CFG parallel, which was unable to generate images and exited midway.
Could you please assist us in identifying the bug or compatibility issue?

[feifeibear]

We will find someone to support SD 3.5 medium in xDiT. Maybe directly apply SD3 code on SD3.5 is not right.

[LWang1016]

@feifeibear Any update for this issue?