7 ms·
This is purely a developer kit, letting people get used to the hardware configuration and the software stacks before the big hardware comes later in future gene
by andy99 3y ago
This is purely a developer kit, letting people get used to the hardware configuration and the software stacks before the big hardware comes later in future generations.
I'd like to know what different precisions/ quantizations if any are supported. For LLMs 8GB is fine for playing around if weights are quantized. And as the article mentions it's more than enough for lots of computer vision models.
- camel-cdr 3y agoGrayskull supports FP8, FP16, BFP16, VFP2, BFP4, BFP8 [0], and some sort of 19-bit floating point in the SFPU. [1] I don't know how and with what performance those are supported. FP4 has 221 TFLOPS on the Grayskull e75, and 332 TFLOPS on the Grayskull e150. [0] While wormhole has 82 INT8 TOPS per card. [2] I haven't found any other numbers. Edit: Some more info about data formats: https://docs.tenstorrent.com/tenstorrent/v/tt-buda/dataformats https://docs.tenstorrent.com/tenstorrent/v/tt-buda/dataforma... [0] https://docs.tenstorrent.com/tenstorrent/add-in-boards-and-cooling-kits/grayskull-tm-e75-e150/specifications https://docs.tenstorrent.com/tenstorrent/add-in-boards-and-c... [1] https://tenstorrent-metal.github.io/tt-metal/latest/tt_metal/apis/kernel_apis/sfpu/llk.html https://tenstorrent-metal.github.io/tt-metal/latest/tt_metal... [2] https://tenstorrent.com/systems/galaxy/ https://tenstorrent.com/systems/galaxy/