Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Explicitly check for None when using prefill
attn_mask
(#983)
When you attempt to implicitly null-check the attention_mask, you hit a torch error: ```bash RuntimeError: Cannot call numel() on tensor with symbolic sizes/strides ``` Simply adding an explicit check for null fixes it, and the `--use-attention-mask` export path for prefill
- Loading branch information