3 ms·

Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. Given that you're talking about ChatGPT, I assume you

Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. Given that you're talking about ChatGPT, I assume you aren't accessing GPT-3.5 or GPT-4 directly through the API but using the app or the interface provided at chat.openai.com. The magic that makes the kinds of interactions you're describing possible amounts to a bit of clever prompting sprinkled on top of some rather impressive frontend design and engineering. Correctly prompted, even Mistral-7B can write and run code in response to questions, and it's a model that can run on laptops from half a decade ago, with two or three orders of magnitude fewer parameters that GPT-4.

MuffinFlavored 3y ago
> Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. By default, the ChatGPT "model" knows to not try to do math and instead write code to do the math then run it. I get that it's set up infrastructure wise to be able to run it, but why is Claude's main chat UI not trying to instead respond "hey, do this calculation on your own since I can't" or something of this nature instead of responding to math incorrectly
- cosmojg 3y ago
  Because ChatGPT ships with a system prompt which instructs the underlying model to do exactly that. A similar web application could be developed for Claude, and it would perform similarly with the right prompt, as it's quite good at tool use. For example, I'm able to get Claude-3-Opus to write Python and call a Python interpreter of its own accord when questioned about time series data in some of my data analysis workflows, though I haven't glued together a pretty GUI for it yet (e.g., plots are simply saved to disk). While I haven't run into any problems around calculations yet, I'm sure it wouldn't be too hard to further refine the system prompt and ensure that all calculations are performed or checked using Python.

MuffinFlavored 3y ago

> Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. By default, the ChatGPT "model" knows to not try to do math and instead write code to do the math then run it. I get that it's set up infrastructure wise to be able to run it, but why is Claude's main chat UI not trying to instead respond "hey, do this calculation on your own since I can't" or something of this nature instead of responding to math incorrectly

cosmojg 3y ago
Because ChatGPT ships with a system prompt which instructs the underlying model to do exactly that. A similar web application could be developed for Claude, and it would perform similarly with the right prompt, as it's quite good at tool use. For example, I'm able to get Claude-3-Opus to write Python and call a Python interpreter of its own accord when questioned about time series data in some of my data analysis workflows, though I haven't glued together a pretty GUI for it yet (e.g., plots are simply saved to disk). While I haven't run into any problems around calculations yet, I'm sure it wouldn't be too hard to further refine the system prompt and ensure that all calculations are performed or checked using Python.