13 ms·
That slot is called USB-C. I can fully imagine inference ASICs coming in powerbank form factor that you'd just plug and play.
by roncesvalles 8mo ago
That slot is called USB-C. I can fully imagine inference ASICs coming in powerbank form factor that you'd just plug and play.
- XorNot 8mo agoPretty sure it'd just be a thumbdrive. Are the Taalas chips particularly large in surface area?
- dmurray 8mo agoThe only product they've announced at the moment [0] is a PCI-e card. It's more like a small power bank than a big thumb drive. But sure, the next generation could be much smaller. It doesn't require battery cells, (much) heat management, or ruggedization, all of which put hard limits on how much you can miniaturise power banks. [0] https://taalas.com/the-path-to-ubiquitous-ai/ https://taalas.com/the-path-to-ubiquitous-ai/
- ChrisMarshallNY 8mo agoI’m old enough to remember your typical computer filling warehouse-sized buildings. Nowadays, your average cellphone has more computing power than those behemoths. I have a micro SD card with 256GB capacity, and I think they are up to 2TB. On a device the size of a fingernail.
- slfnflctd 8mo agoThat is all definitely amazing, but data storage is a fundamentally different process with far fewer constraints than continuous computation.
- ChrisMarshallNY 8mo agoIt all uses the same miniaturization techniques, though.
- yonatan8070 8mo agoI wouldn't call that size a small power bank. That chip is in the same ballpark as gaming GPUs, and based on the VRMs in the picture it probably draws about as much power. But as you said, the next generations are very likely to shrink (especially with them saying they want to do top of the line models in 2 generations), and with architecture improvements it could probably get much smaller.
- layla5alive 8mo agoTop of the line models will need more weights and more transistors, so the shrinking factors will be competing with growing factors, I'd expect them to keep maxing out the ASIC sizes to whatever is economically feasible.
- yonatan8070 8mo agoNaturally they'll always have a big expensive SKU, but the existence of a Threadripper doesn't automatically obsolete the Ryzen 3
- thesz 8mo ago800 mm2, about 90mm per side, if imagined as a square. Also, 250 W of power consumption. The form factor should be anything but thumbdrive.
- layla5alive 8mo agoYes, bigger than a 5090's GB202 ASIC! :)
- zupa-hu 8mo agoThis would be a hell of a hot power bank. It uses about as much power as my oven. So probably more like inside a huge cooling device outside the house. Or integrated into the heating system of the house. (Still compelling!)
- fennecbutt 8mo ago*the whole server uses 2.2kw or whatever, not a single board. I think that was for 8 boards or something.
- zupa-hu 8mo agoOh does it? Thanks for the clarification then. Their home page said 2.5kW so I assumed that's what it is. To be fair, 2.5kW does sound too much for a single 3x3cm chip, it would probably melt.
- fennecbutt 8mo agoMore powwwwaaa! Yeah, though I suppose once we get properly 3d silicon I would not be surprised at power rating for that, 3cm^3 would be something to behold.
- ekianjo 8mo agoNot if you need 200w power to run inference.
- stavros 8mo agoUSB-C can do up to 240W. These days I power all my devices with a USB hub, even my Lipo charger.
- grayhatter 8mo agoHave you seen a device that can supply 240w and act as a data host? Or is the 240w only from dedicated chargers?
- stavros 8mo agoI haven't seen one, but I also don't tend to use it for anything other than a power supply, so I wouldn't know. Since the standard supports it, though, it's just a matter of the market needing a device like that.
- amelius 8mo ago> USB-C With these speeds you can run it over USB2, though maybe power is limiting.
- bagful 8mo agoLike the chip-software in Gibson’s sprawl, from the micro-soft to the ROM cowboy to the Aleph, the endgame of computertool distribution is via single-use chunks of quasi-biological computronium
- avisser 8mo agoMichael Bay just read "computronium" and spawned an 8 movie franchise in his head.