Skip to content

Commit

Permalink
[skip ci] Update llms.md (#18464)
Browse files Browse the repository at this point in the history
### Ticket
N/A

### Problem description
Order of sections was incorrect.

### What's changed
Section order corrected.

### Checklist
- [ ] [All post
commit](https://github.com/tenstorrent/tt-metal/actions/workflows/all-post-commit-workflows.yaml)
CI passes
- [ ] [Blackhole Post
commit](https://github.com/tenstorrent/tt-metal/actions/workflows/blackhole-post-commit.yaml)
CI passes (if applicable)
- [ ] [Model
regression](https://github.com/tenstorrent/tt-metal/actions/workflows/perf-models.yaml)
CI passes (if applicable)
- [ ] [Device performance
regression](https://github.com/tenstorrent/tt-metal/actions/workflows/perf-device-models.yaml)
CI passes (if applicable)
- [ ] **(For models and ops writers)** Full [new models
tests](https://github.com/tenstorrent/tt-metal/actions/workflows/full-new-models-suite.yaml)
CI passes (if applicable)
- [ ] New/Existing tests provide coverage for changes
  • Loading branch information
bbeggsTT authored Mar 4, 2025
1 parent e03382a commit bf54914
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tech_reports/LLMs/llms.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ ttnn_gamma_rm = ttnn.as_tensor(

The distributed implementation is designed for cases where activations are **sharded along the embedding dimension** across multiple devices. It ensures the correct computation of mean and variance across shards by leveraging cross-device communication. Both interleaved and width-sharded inputs are supported.

##### 2.3.2.2.1 Steps to Perform Distributed Normalization on TT-Devices
##### 2.3.1.2.1 Steps to Perform Distributed Normalization on TT-Devices

1. **Compute Local Statistics** - Each device computes the required statistics (e.g., \(E[x]\), \(E[x^2]\)) locally on its shard of the input tensor.
- For **RMSNorm**, only \(E[x^2]\) is required.
Expand Down

0 comments on commit bf54914

Please sign in to comment.