Could someone elighten me on how to compute layers in parallel? I was under the ...

delusional on April 30, 2025 | parent | context | favorite | on: Can LLMs do randomness?

Could someone elighten me on how to compute layers in parallel? I was under the impression that the linearity of the layer computation was why we were mostly bandwidth constrained. If you can compute the layers In parallel then why do we need high bandwidth?

dist-epoch on April 30, 2025 [–]

https://developer.nvidia.com/blog/mastering-llm-techniques-i...