2 ms·
Significantly higher latency than you have within a single datacenter. Think "my GPU working with your GPU".
by staunton 4y ago
Significantly higher latency than you have within a single datacenter. Think "my GPU working with your GPU".
- alexvoda 4y agoThere are already LLMs hosted across the internet (Folding@Home style) instead of in a single data center. Just because the swarm infrastructure hosting an LLM has higher latency across certain paths does not make it a swarm of LLMs.
- staunton 4y ago> There are already LLMs hosted across the internet (Folding@Home style) Interesting, I haven't heard of that. Can you name examples?
- alexvoda 4y agoI read about Petals (1) some time ago here on HN. There are surely others too, but I don't remember the names. 1. https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals