4 ms·
Now I want to see someone figure out a way to performantly distribute a trillion parameter model on a million smartphones looking at a web page.
by BulgarianIdiot 4y ago
Now I want to see someone figure out a way to performantly distribute a trillion parameter model on a million smartphones looking at a web page.
- sebzim4500 4y agoThe bandwidth would end up costing way more than the compute would.
- BulgarianIdiot 4y agoDepends how you do it. LLM are huge, but their input and output is miniscule bits of text. If you find a way to put "narrow paths" in the hidden layers, basically to subdivide a model into smaller interconnected models, then the bandwidth will be similarly massively reduced. This is not without precedent, look up how your brain hemispheres and the regions within are connected.