4 ms·

I'm generally a believer in trading compute for insight. So this makes sense. I'l also curious how adding 100x of compute into a longer prompt compares with u

• josephg 3y ago
  > For example, is it better to have 100x longer prompts, or 100x bigger models? Ultimately we need both. There's no point having a superintelligent code assistant which doesn't have enough working memory to understand what your program is doing. And there's no point having 100x longer prompts if the system isn't smart enough to contribute code changes. I think we can have both, but we'll need to do more work on our language models first. I mean, humans have extremely limited working memory, but we can work on arbitrarily large programs. We do it by paging context in and out of our minds. As such, I don't need to think about the entire google chrome codebase to make a change to one small part of it. I'm interested in the approach of the LongMem paper (from Microsoft Research). As I understand it, their approach does something like humans where the system learns to page parts of the input in and out of working memory as needed. (I haven't read the paper in detail yet). https://arxiv.org/abs//2306.07174 https://arxiv.org/abs//2306.07174