4 ms·
This is somewhat off topic, but does anyone know why they haven't tried putting SSD in a GPU like chip for inference? My understanding is that SSDs have a diffe
by foota 3y ago
This is somewhat off topic, but does anyone know why they haven't tried putting SSD in a GPU like chip for inference? My understanding is that SSDs have a different access pattern for reading that adds latency vs RAM, but I don't otherwise know why it couldn't serve as a high bandwidth source for reads (ignoring write latencies and durability concerns).
- anoother 3y agoIt has been tried once -- Radeon Pro SSG
- foota 3y agoMaybe. I'm not sure how it was implemented and I didn't see much in depth discussion about it, but it seems to me like that's more just a normal SSD sitting on the card, I was thinking something closer to an SSD with a vastly different (and likely more expensive) interface allowing for much higher bandwidth than an SSD typically offers, since there's not much advantage to having the SSD if you can't read it into cache quickly.