4 ms·
> The Transformer architecture revolutionized sequence modeling with its introduction of attention, a mechanism by which models look back at earlier inputs to p
by dmix 11mo ago
> The Transformer architecture revolutionized sequence modeling with its introduction of attention, a mechanism by which models look back at earlier inputs to prioritize relevant input data
I've always wanted to read how something like Cursor manages memory. It seems to have developed a long history of all of prompts and understands both the codebase and what I'm building slightly more over time, causing less errors.
- russdill 11mo agoThat's not what they are talking about here. This is just a description of what goes on with a transformer and the context window
- dmix 11mo agoAh so 'long-term memory' in this case is just really large context windows with a long series of user inputs. That makes sense.
- deleted 11mo ago[deleted]