2 ms·
Depends on the details, as always. If this hypothetical future is one where mixtures of experts is predominant, where each expert fits on a node, then the node
by ben_w 2y ago
Depends on the details, as always.
If this hypothetical future is one where mixtures of experts is predominant, where each expert fits on a node, then the nodes only need the bandwidth to accept inputs and give responses — they won't need the much higher bandwidth required to spread a single model over the planet.