3 ms·
is everyone just waiting for teh DGX Spark? Are they really going to ban local inference?
by ionwake 1y ago
is everyone just waiting for teh DGX Spark? Are they really going to ban local inference?
- neuroelectron 1y agoWhat do you mean ban? The bandwidth between macs isn't enough to do inference effectively.
- zozbot234 1y agoI'm pretty sure you can network Macs together via the latest Thunderbolt standards and get pretty decent performance overall. Sure, it will be a bottleneck to some extent but it's still useful for many purposes.
- neuroelectron 1y agoYes you can do that and shard a very large model across the devices but it's way too slow so you will get no performance gains beyond being able to run a much larger model at all.
- ionwake 1y agothats a performance gain
- neuroelectron 1y agoit's paying more for less performance.
- ionwake 1y agoAre you a teenager or something
- neuroelectron 1y agoJust look at the people who tried it on youtube
- jsjohnst 1y ago> The bandwidth between macs isn't enough to do inference effectively. While it’s certainly no where near the memory bandwidth, 80Gbps is on par with most high end, but still affordable, machine to machine connections. Then add on the fact you can have hundreds of gigabytes of shared ram on each machine.