4 ms·
I did similar experiments with GPT4 and Soulver[1], though I "tuned" it by teaching it Soulver interactively before continuing to prompt. It can then be used in
by summarity 4y ago
I did similar experiments with GPT4 and Soulver[1], though I "tuned" it by teaching it Soulver interactively before continuing to prompt. It can then be used in the same way. My primary goal was to add basic calculation capabilities to GPT, that are: a) guaranteed to halt (all Soulver sheets are functions) and b) readable by both GPT and the user (a Python program may be too dev-oriented for normal people).
It worked quite well. To well almost: I started a meta-conversation where I asked another GPT4 instance to come up with conversations SoulverGPT could have with a user where the addition of solving is beneficial. This worked, and eventually even found a bug in Soulver - essentially fuzzing the language.
[1] https://github.com/soulverteam/SoulverCore https://github.com/soulverteam/SoulverCore
- UltimateEdge 4y agoHuh, I wonder if LLMs are good at fuzzing in general.
- summarity 4y agoAlso did some early experiments but didn't go very far. If someone wants to investigate: use structural code analysis tools (parser-parser-generators) like Comby[1] to factor existing code, letting GPT remix it instead of generating test cases all by itself. [1] https://comby.dev/blog/2022/04/11/comby-decomposer-compiler-fuzzing https://comby.dev/blog/2022/04/11/comby-decomposer-compiler-...