3 ms·
Wire typically means internet connection, and it’s hardly the bottleneck
by Kerbonut 9d ago
Wire typically means internet connection, and it’s hardly the bottleneck
- deleted 9d ago[deleted]
- 317070 9d agoin the case of large language models, the wire is the communication of your parameters between your layers of memory that is often the bottleneck. To do a forward pass, you need to use all parameters once, and so the communication between the compute and the storage is the bottleneck, and that bottleneck is also a bunch of wires.
- mitxela 9d agoThe other bottleneck is the amount of fast storage, which compression also improves.