How to Add DiTFastAttn in our custom model? do i need to write a pipeline or will it work if i can just swap out some attention layers/ processors

[asahni04]

How to Add DiTFastAttn in our custom model? do i need to write a pipeline or will it work if i can just swap out some attention layers/ processors

[feifeibear]

DiTFastAttn is a relatively independent feature, which essentially replaces the attention module with DiTFastAttn. You can refer to the following MR for details.

We haven't tested this feature for a while, so I welcome you to submit a useful MR to help improve this functionality.

#297

[asahni04]

hi @feifeibear will it require changing the sampling code or only swapping attention processors would be enough? the model i have is nn.Module how do i go about using xfuser DiTFastAttn with it? if you can give me an idea i can come up with a MR to improve functionality

[feifeibear]

We have currently integrated xFuserFastAttention in diffusers, and you can replace the original attention with this class in your nn.Module, which offers great flexibility.

[asahni04]

@feifeibear that's amazing can you please share the PR or reference code?