4 ms·

What makes you think we're lacking fundamental plumbing? I am personally still quite convinced it will simply take a bit of time and development to plug in the

What makes you think we're lacking fundamental plumbing? I am personally still quite convinced it will simply take a bit of time and development to plug in the current AI into sufficient self-testing structures (basically the next versions of AutoGPT) and let it run long enough to map out an entire domain with accurate measurements. Seems more like just a matter of time and a bit of funding - but mostly inevitable and very straightforward. It doesn't even really take GPT4 quality levels to do - that's just a bonus. https://twitter.com/fablesimulation/status/1681352904152850437 https://twitter.com/fablesimulation/status/16813529041528504... These guys seem to be on the leading edge there - making self-referential character-driven narratives where agents talk to each other and build the collective world understanding, all through a South Park Westworld lol

sheepscreek 3y ago
I completely agree with you. I failed to articulate it well. Like the current top HN post suggests (https://eugeneyan.com/writing/llm-patterns/ https://eugeneyan.com/writing/llm-patterns/), we’re still discovering patterns that work well with LLMs. That said, anecdotally - they already excel at being logic engines. Capable of filling in the gaps between instructions. Using their worldly knowledge or “common sense” to do so. But ever so often, they’ll miss an important bit. And I have to be quite involved to catch that. Kinda defeats the purpose. Here, I think we can benefit from supervisor LLMs. A second layer, whose sole job is to ensure the output quality. A QA bot - essentially.
- dogcomplex 3y ago
  Yeah with their own QA from a variety of different personas/perspectives/concerns/contexts I reckon you'll get a very decent accuracy - or at least self-assessment of inaccuracy. All these can ever do is propagate the data they know so far to the context/prompt you desire, but I don't see obvious limits there. And GPT4 is already a superb conversation partner as smart as nearly any person - so it's really like piecing experts together. If we run into any fundamental limitations from piecing these all together, it's gonna be the same limit that any group of humans trying to make a coherent/consistent organization of knowledge encounters, I think. Coincidentally, that appears to be how GPT4 was made - apparently it's actually about 8 personas with designated roles running GPT3.5 trained together ("Panel of experts"? there's an AI name for the technique). Makes you wonder how far that one trick scales. (P.S. great link. Gah - another long read on the todo list)

sheepscreek 3y ago

I completely agree with you. I failed to articulate it well. Like the current top HN post suggests (https://eugeneyan.com/writing/llm-patterns/ https://eugeneyan.com/writing/llm-patterns/), we’re still discovering patterns that work well with LLMs. That said, anecdotally - they already excel at being logic engines. Capable of filling in the gaps between instructions. Using their worldly knowledge or “common sense” to do so. But ever so often, they’ll miss an important bit. And I have to be quite involved to catch that. Kinda defeats the purpose. Here, I think we can benefit from supervisor LLMs. A second layer, whose sole job is to ensure the output quality. A QA bot - essentially.

dogcomplex 3y ago
Yeah with their own QA from a variety of different personas/perspectives/concerns/contexts I reckon you'll get a very decent accuracy - or at least self-assessment of inaccuracy. All these can ever do is propagate the data they know so far to the context/prompt you desire, but I don't see obvious limits there. And GPT4 is already a superb conversation partner as smart as nearly any person - so it's really like piecing experts together. If we run into any fundamental limitations from piecing these all together, it's gonna be the same limit that any group of humans trying to make a coherent/consistent organization of knowledge encounters, I think. Coincidentally, that appears to be how GPT4 was made - apparently it's actually about 8 personas with designated roles running GPT3.5 trained together ("Panel of experts"? there's an AI name for the technique). Makes you wonder how far that one trick scales. (P.S. great link. Gah - another long read on the todo list)