25 ms·
Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast an
by MasterScrat 3y ago
Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work.
I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!
- memossy 3y agoThe v5es and v5ps are pretty amazing at running SD, giving code for SD3 now to optimise it on those. v5es are particularly interesting given the millions that will land and the large pod sizes, particularly well constructed for million token context windows.
- renewiltord 3y agoWhich TPUs do you use? Cloud-hosted or your own hardware? Interesting insight.
- 1024core 3y ago"TPUs" are a Google-only product, available* only on GCP. * Notwithstanding the Choral boards
- memossy 3y agoWe use v4s, v5es & v5ps. Mostly v5ps, very stable int8 training (versus the horror that is fp8 stability)
- MasterScrat 3y agoWe started on V3s, now fully moved to V4s with some V5Es, investigating a full move towards V5E & V5P
- doctorpangloss 3y agoBut you and I can't buy a TPU. You and I can buy an H100.
- isoprophlex 3y agoSpeak for yourself! I can't even afford 1/10th of an H100.
- elorant 3y agoHe's probably speaking about availability, not affordability.
- leansensei 3y agoIt was an attempt at humor, obviously.