4 ms·
What do you think context is, if not 'attention'?
by lwhi 9mo ago
What do you think context is, if not 'attention'?
- XenophileJKO 9mo agoContext is the information you give the model, attention is what parts it focuses on. And this is finite in capacity and emergent from the architecture.
- lwhi 9mo agoSo attention is based on a smaller subset of context?
- chickensong 9mo agoYou can create a context that includes info and instructions, but the agent may not pay attention to everything in the context, even if context usage is low.
- 3371 9mo agoIMO "Attention" is an abstraction over the result of prompt engineering, the chain reaction of input converging the output (both "thinking" and response).