3 ms·
IMO this is a context window issue. Humans are pretty good are memorizing super broad context without great accuracy. Sometimes our "recall" function doesn't ev
by zobzu 1y ago
IMO this is a context window issue. Humans are pretty good are memorizing super broad context without great accuracy. Sometimes our "recall" function doesn't even work right ("How do you say 'blah' in German again?"), so the more you specialize (say, 10k hours / mastery), the better you are at recalling a specific set of "skills", but perhaps not other skills.
On the other hand, LLMs have a programatic context with consistent storage and the ability to have perfect recall, they just don't always generate the expected output in practice as the cost to go through ALL context is prohibitive in terms of power and time.
Skills.. or really just context insertion is simply a way to prioritize their output generation manually. LLM "thinking mode" is the same, for what it's worth - it really is just reprioritizing context - so not "starting from scratch" per se.
When you start thinking about it that way, it makes sense - and it helps using these tools more effectively too.
- dwaltrip 1y agoThere are ways to compensate for lack of “continual learning”, but recognizing that underlying missing piece is important.
- ryancnelson 1y agoI commented here already about deli-gator ( https://github.com/ryancnelson/deli-gator https://github.com/ryancnelson/deli-gator ) , but your summary nailed what I didn’t mention here before: Context. I’d been re-teaching Claude to craft Rest-api calls with curl every morning for months before i realized that skills would let me delegate that to cheaper models, re-using cached-token-queries, and save my context window for my actual problem-space CONTEXT.
- dingnuts 1y ago>I’d been re-teaching Claude to craft Rest-api calls with curl every morning for months what the fuck, there is absolutely no way this was cheaper or more productive than just learning to use curl and writing curl calls yourself. Curl isn't even hard! And if you learn to use it, you get WAY better at working with HTTP! You're kneecapping yourself to expend more effort than it would take to just write the calls, helping to train a bot to do the job you should be doing
- jmtulloss 1y agoMy interpretation of the parent comment was that they were loading specific curl calls into context so that Claude could properly exercise the endpoints after making changes.
- F7F7F7 1y agoHe’s likely talking about Claude’s hook system that Anthropic created to provide better control over context.
- ryancnelson 1y agoi know how to use curl. (I was a contributor before git existed) … watching Claude iterate to re-learn whether to try application/x-form-urle ncoded or GET /?foo wastes SO MUCH time and fills your context with “how to curl” that you re-send over again until your context compacts. You are bad at reading comprehension. My comment meant I can tell Claude “update jira with that test outcome in a comment” and, Claude can eventually figure that out with just a Key and curl, but that’s way too low level. What I linked to literally explains that, with code and a blog post.
- mbesto 1y ago> IMO this is a context window issue. Not really. It's a consequential issue. No matter how big or small the context window is, LLMs simply do not have the concept of goals and consequences. Thus, it's difficult for them to acquire dynamic and evolving "skills" like humans do.
- adastra22 1y agoWorth noting, even though it isn’t critical to your argument, that LLMs do not have perfect recall. I got to great lengths to keep agentic tools from relying on memory, because they often get it subtly wrong.