Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Int8DynamicActivationIntxWeightConfig out of experimental #1968

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

metascroy
Copy link
Contributor

@metascroy metascroy commented Mar 26, 2025

This PR moves Int8DynamicActivationIntxWeightConfig and its quantizer into torchao.quantization.quant_api (out of experimental). Int8DynamicActivationIntxWeightConfig is refactored to closely mirror Int8DynamicActivationInt4WeightConfig when weight_dtype=torch.int4, layout=QDQLayout().

Quantization in Int8DynamicActivationIntxWeightConfig is done with QDQLayout, and then packing is done separately with make_packed_linear_int8_dynamic_activation_intx_weight_tensor. This is to separate the quantization algorithm from the storage. Both the packed and QDQLayout quantize with the same algorithm, and this is made explicit.

Copy link

pytorch-bot bot commented Mar 26, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1968

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 4a421db with merge base df532f0 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 26, 2025
@metascroy metascroy changed the title Create utility to construct QuantizedLinear from plain data Move Int8DynamicActivationIntxWeightConfig out of experimental Apr 1, 2025
@metascroy metascroy requested a review from jerryzh168 April 1, 2025 03:43
@metascroy metascroy force-pushed the add-torchao-module-constructors branch 2 times, most recently from c80e35e to fb1eca3 Compare April 6, 2025 00:02
@metascroy metascroy force-pushed the add-torchao-module-constructors branch from f63ca78 to 90bc3aa Compare April 6, 2025 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants