5 ms·
Tenstorrent engineers talk open-sourced bare-metal stack
- raphlinus 3y agoThe Metalium repo is here: https://github.com/tenstorrent-metal/tt-metal/ https://github.com/tenstorrent-metal/tt-metal/ There's also an "unboxing video" from Dr Ian Cuttress and Jasmina Vasiljevic from a few months ago: https://www.youtube.com/watch?v=WJpJkvNw9Ts https://www.youtube.com/watch?v=WJpJkvNw9Ts . It's a bit out of date because a lot of the stuff that was promised as "coming soon" has actually happened. I personally think making this open source is huge and am cautiously optimistic that the hardware has potential for general purpose parallel computing, so I've ordered a card.
- deleted 3y ago[deleted]
- shubham_saboo 3y agoVery recently we have also opensourced BUDA, top-down software stack for running ML models on Tenstorrent Hardware: https://github.com/tenstorrent/tt-buda https://github.com/tenstorrent/tt-buda Metalium being the bottom-up software stack giving open access to Tenstorrent Hardware.
- sashank_1509 3y agoSo right now it’s only for model inference is it and not for training
- wslh 3y agoGood question. I have been compiling an analysis of the hardware market with the idea of analyzing a cloud market for LLMs. Feel free to comment: https://docs.google.com/document/d/1nehciIpaKj3-sEta5Jx2q9in5P041ppSUFGVuImvlbw/edit https://docs.google.com/document/d/1nehciIpaKj3-sEta5Jx2q9in... I am mainly interested in the training side because it is the more difficult one.
- deleted 3y ago[deleted]
- adifrancescoTT 3y agoCorrect, working on enabling training on next iterations
- deleted 3y ago[deleted]
- shubham_saboo 3y agoYes, Grayskull is Tenstorrent's entry-level devkit for inference only. Future generation of chips to feature training.
- zozbot234 3y agoVery interesting, this should broaden the case for Tenstorrent well beyond the usual kind of "AI" and into traditional HPC workloads, which will` tend to involve quite a bit of custom code and heterogenous compute where every "core" is not working on the same thing.
- cmrdporcupine 3y agoYeah, I'm not an ML guy, but I was looking at the matmult.h header here, as well as the stuff in eltwise_unary and thinking there'd definitely be some more general purpose uses for this stuff. I wonder what the memory transfer throughput/latency is from host to card and back https://github.com/tenstorrent-metal/tt-metal/blob/main/tt_metal/include/compute_kernel_api/matmul.h https://github.com/tenstorrent-metal/tt-metal/blob/main/tt_m...
- dist1ll 3y ago> “We want to open [Metalium] to the point that everything is 100% transparent,” she said. “We will click ‘open’ on our Github repo, and the world will be able to see everything that we’re doing internally. That's would be massive. Will that also apply to firmware?
- ojn 3y agoFirmware can sometimes get a bit complicated due to third party IP used and whether that can be opensourced, but within those constraints this is the ambition. I.e things like memory controller setup and link training is often constrained by the IP vendors on whether you can share it in source form. Good news in general though -- there isn't a lot of firmware on these cards in the path of the workloads, most of it is around system management and thermals/power. Terminology-wise some of pieces in the code base are called firmware, but are loaded at runtime and built as part of the usual build process. It's more of a linked-with-the-workload libraries than resident FW. (There's some of both though).
- everyone 3y agoBased on the name I assumed this was gonna be about a torrent site I hadnt heard of.
- wmf 3y agoTranscutaneous electrical nerve stimulation torrents.
- userbinator 3y agoMy thoughts went towards a bare-metal torrent server, suited for seedboxes.
- brcmthrowaway 3y agoDidnt Tenstorrent pivot out of ML?
- adifrancescoTT 3y agoCurious why you think that?
- cloudhan 3y agoThe memory is too small to be useful nowadays.