Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated Resnet50 model for Blackhole, with Batch = 32 #17985

Merged
merged 1 commit into from
Mar 2, 2025
Merged

Updated Resnet50 model for Blackhole, with Batch = 32 #17985

merged 1 commit into from
Mar 2, 2025

Conversation

mywoodstock
Copy link
Contributor

@mywoodstock mywoodstock commented Feb 19, 2025 •
edited
Loading

Ticket

#17393
#18341

Problem description

This PR enables larger batch sizes 20 and 32 for Resnet50 on Blackhole.

What's changed

Some updates to the model itself to allow batch 32.
More updates to the fold op to allow non-rectangular grids.

Checklist

@mywoodstock mywoodstock force-pushed the asarje/rn50-bh-largebatch-20250218 branch 2 times, most recently from 72712db to 90fb639 Compare February 26, 2025 23:37
@mywoodstock mywoodstock changed the title [DO NOT METGE] Asarje/rn50 bh largebatch 20250218 Updated Resnet50 model for Blackhole, with Batch = 32 Feb 26, 2025
@mywoodstock mywoodstock marked this pull request as ready for review February 26, 2025 23:45
bbradelTT
bbradelTT reviewed Feb 27, 2025
View reviewed changes
...p/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp Outdated
@@ -202,7 +202,7 @@ void MAIN {
#ifdef ARCH_BLACKHOLE
// FIXME: This is a temporary workaround to avoid hangs on blackhole.
// https://github.com/tenstorrent/tt-metal/issues/16439
for (uint32_t i = 0; i < 10; i++) {
for (uint32_t i = 0; i < 100; i++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to add an order of magnitude to the delay?

Someone said this was a 30% slowdown that is now a 300% slowdown.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, this snuck in --> didn't mean to push this in

models/demos/ttnn_resnet/tt/ttnn_functional_resnet50.py Outdated
@@ -914,6 +966,19 @@ def run(self, input_tensor, device, ops_parallel_config, conv_op_cache={}) -> tt
),
}
)
# ## 128
# core_range_set = ttnn.CoreRangeSet(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove commented out code here and in other parts of the PR.

models/demos/ttnn_resnet/tt/ttnn_functional_resnet50.py
and layer_module
and (layer_module == "layer1_module2" or layer_module == "layer1_module3")
):
conv_kwargs_2["conv_config"].act_block_h_override = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point it may be better to have a function go through all three cases, and then just do something like

if f(batch_size, layer_module):
    conv_kwargs_2["conv_config"].act_block_h_override = 0

@mywoodstock mywoodstock force-pushed the asarje/rn50-bh-largebatch-20250218 branch from af8258a to cad363d Compare February 27, 2025 17:19
@mywoodstock mywoodstock mentioned this pull request Feb 27, 2025
Closed
@mywoodstock mywoodstock force-pushed the asarje/rn50-bh-largebatch-20250218 branch from 0648d6a to 752a333 Compare February 27, 2025 17:35
sankarmanoj-tt
sankarmanoj-tt reviewed Feb 28, 2025
View reviewed changes
...v/conv2d/device/kernels/reader_conv_activations_2d_mcast_padded_with_halo_3x3_weights_v2.cpp
@@ -273,4 +273,5 @@ void kernel_main() {
} // act_w_num_outer
cb_pop_front(tilized_in0_cb_id, act_block_num_tiles);
}
noc_async_write_barrier();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?
Should the width sharded reader kernel also be updated similarly?

Copy link
Contributor

@s-jovic s-jovic Feb 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just be careful here, from my experience width sharded reader kernel hung when I added the barrier. #18341

Copy link
Contributor Author

@mywoodstock mywoodstock Feb 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need these since watcher was catching outstanding writes for these kernels. And adding it to this these kernels included do not hang.

@mywoodstock mywoodstock force-pushed the asarje/rn50-bh-largebatch-20250218 branch 3 times, most recently from 6975cc9 to 20bf23b Compare March 1, 2025 02:56
@mywoodstock mywoodstock force-pushed the asarje/rn50-bh-largebatch-20250218 branch from 20bf23b to bc243cc Compare March 1, 2025 23:10
@mywoodstock mywoodstock merged commit 8150c70 into main Mar 2, 2025
16 checks passed
@mywoodstock mywoodstock deleted the asarje/rn50-bh-largebatch-20250218 branch March 2, 2025 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@s-jovic s-jovic s-jovic left review comments

@ntarafdar ntarafdar ntarafdar approved these changes

@yugaoTT yugaoTT yugaoTT approved these changes

@bbradelTT bbradelTT bbradelTT approved these changes

@sankarmanoj-tt sankarmanoj-tt sankarmanoj-tt approved these changes

@jaykru-tt jaykru-tt jaykru-tt approved these changes

@tt-aho tt-aho Awaiting requested review from tt-aho tt-aho is a code owner

@ayerofieiev-tt ayerofieiev-tt Awaiting requested review from ayerofieiev-tt ayerofieiev-tt is a code owner

@dmakoviichuk-tt dmakoviichuk-tt Awaiting requested review from dmakoviichuk-tt dmakoviichuk-tt is a code owner

@rfurko-tt rfurko-tt Awaiting requested review from rfurko-tt rfurko-tt is a code owner

@cfjchu cfjchu Awaiting requested review from cfjchu cfjchu is a code owner

@TT-BrianLiu TT-BrianLiu Awaiting requested review from TT-BrianLiu TT-BrianLiu is a code owner

@razorback3 razorback3 Awaiting requested review from razorback3 razorback3 is a code owner

@dongjin-na dongjin-na Awaiting requested review from dongjin-na dongjin-na is a code owner

@omilyutin-tt omilyutin-tt Awaiting requested review from omilyutin-tt omilyutin-tt is a code owner

@vsureshTT vsureshTT Awaiting requested review from vsureshTT vsureshTT is a code owner

@edwinleeTT edwinleeTT Awaiting requested review from edwinleeTT edwinleeTT is a code owner

@sjameelTT sjameelTT Awaiting requested review from sjameelTT sjameelTT is a code owner

@yugi957 yugi957 Awaiting requested review from yugi957 yugi957 is a code owner

@jvegaTT jvegaTT Awaiting requested review from jvegaTT jvegaTT is a code owner

@llongTT llongTT Awaiting requested review from llongTT llongTT is a code owner

@nardoTT nardoTT Awaiting requested review from nardoTT nardoTT is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@mywoodstock @ntarafdar @yugaoTT @s-jovic @bbradelTT @sankarmanoj-tt @jaykru-tt