7 ms·
"Attention" is a feature that makes this whole thing work in the first place, it's not a flaw, although all current models are non-ideal at it in practice. Coul
by embedding-shape 1mo ago
"Attention" is a feature that makes this whole thing work in the first place, it's not a flaw, although all current models are non-ideal at it in practice. Could be better for sure :)
- thewhitetulip 1mo agoRight but when the context window fills up, whatever miniscule guardrails are there magically disappear - even if this is by design, this is bad. Especially when it was marketed as magic