Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch.aten.view.dtype from f8E4M3FNUZ to si8 with dynamic dimension fails to lower #20002

Closed
sogartar opened this issue Feb 14, 2025 · 0 comments · Fixed by #20005
Closed

torch.aten.view.dtype from f8E4M3FNUZ to si8 with dynamic dimension fails to lower #20002

sogartar opened this issue Feb 14, 2025 · 0 comments · Fixed by #20005
Assignees
Labels
bug 🐞 Something isn't working compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm)

Comments

@sogartar
Copy link
Contributor

sogartar commented Feb 14, 2025

What happened?

    %0 = torch.aten.view.dtype %arg0, %int1
        : !torch.vtensor<[?,32,8,128],f8E4M3FNUZ>, !torch.int
        -> !torch.vtensor<[?,32,8,128],si8>

results in error

small-reproducer.mlir:5:10: error: 'flow.tensor.bitcast' op value set has 1 dynamic dimensions but only 0 dimension values are attached
    %0 = torch.aten.view.dtype %arg0, %int1
         ^
small-reproducer.mlir:5:10: note: see current operation: %1 = "flow.tensor.bitcast"(%0) <{operandSegmentSizes = array<i32: 1, 0, 0>}> : (tensor<?x32x8x128xf8E4M3FNUZ>) -> tensor<?x32x8x128xi8>

Steps to reproduce your issue

Compile the below IR with

iree-compile small-reproducer.mlir \
  -o=small-reproducer.vmfb \
  --iree-hip-target=gfx942 \
  --iree-hal-target-device=hip
func.func @main(
  %arg0: !torch.vtensor<[?,32,8,128],f8E4M3FNUZ>
) -> !torch.vtensor<[?,32,8,128],si8> {
    %int1 = torch.constant.int 1
    %0 = torch.aten.view.dtype %arg0, %int1
        : !torch.vtensor<[?,32,8,128],f8E4M3FNUZ>, !torch.int
        -> !torch.vtensor<[?,32,8,128],si8>
  return %0 : !torch.vtensor<[?,32,8,128],si8>
}

What component(s) does this issue relate to?

MLIR, Compiler

Version information

8fab35c

Additional context

This was encountered when trying to compile Lllam f8 and the bitcasting was required to appease Torch for not supporting torch.Tensor.index_copy_ for torch.float8_e4m3fnuz.
More context at nod-ai/shark-ai#896 (comment)

@sogartar sogartar added bug 🐞 Something isn't working compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) labels Feb 14, 2025
@IanWood1 IanWood1 self-assigned this Feb 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants