3 ms·
Memory bound token generation is a limitation of transformer decoders. In the past, hardware has motivation algorithm innovations. I’m curious how long it wil
by gdiamos 3y ago
Memory bound token generation is a limitation of transformer decoders.
In the past, hardware has motivation algorithm innovations.
I’m curious how long it will take until we see more hardware friendly models.
- marmaduke 3y agoThe Rwkv family of models qualifies, since it computes like a recurrent network at inference time.