3 ms·
Cool project, but it reminds me of one of the golden rules of AI “if you can solve it with a simple rules based system, you should” Having to tell chatGPT to m
by Gigachad 4y ago
Cool project, but it reminds me of one of the golden rules of AI “if you can solve it with a simple rules based system, you should”
Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring.
- mintplant 4y agoI see your point, but I think that's kinda neat, personally. Fixing a bug by asking nicely for the bug not to happen. The pinnacle of declarativeness.
- MereInterest 4y agoIt makes me think of INTERCAL, that you need to prepend commands with "please" some of the time, but not so often that it chastises you for being overly polite.
- ncr100 4y agoWild times to be alive and a software engineer. Wonderful example @mintplant.
- lelandfe 4y agoIt also did not work perfectly for me, running GP's prompt. ChatGPT left off the shared "Egg Rolls" entry entirely :(
- MikeTheGreat 4y agoCan I ask why it's effective to tell it to get it's math correct? Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why would telling it to check it's math actually make it's predictions more accurate? 2) Why would asking it to check it's math change the results it produces? Shouldn't it just do the same thing a second time and produce answers that are just as (in)accurate as it did the first time?
- mintplant 4y agoMy intuition is: the AI is trying to predict a likely response to the input query given what it's seen in the dataset it's been trained on. That dataset will include instances where math has been done incorrectly, so the AI producing a response with incorrect math isn't inherently "wrong" from its perspective. However, queries containing reminders to be careful with the math are more likely to be followed by responses with correct numbers.
- hansworst 4y agoSo it’s pretending to be an average human, and the average human gets it wrong a lot? I wonder if you can improve coding results by telling it to act as an expert programmer?
- sharemywin 4y agoI told it to sound 100% more human when writing a blog post and it moves the score with the AI detectors. Then I told it to sound even more human and it does even better.
- the_gipsy 4y agoIt's not "pretending". As the comment said, the dataset has something like "paths" and that prompt narrows it down to those that (hopefully) are related to "getting the math right".
- kaetemi 4y agoI noticed as well that by default the output is effectively average. The right context bias does help with getting better quality output.
- sharemywin 4y agoYou can also have it check it's self after the fact. which is the advantage of this over GPT3.
- 4y ago
- ryanjshaw 4y agoA calculator is a "simple rules based system" but I still do mental arithmetic.
- visarga 4y ago> Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring. We don't compute in our heads, we use calculators. chatGPT could have a calculator too. Why kill a fly with a sledge hammer?
- nigamanth 4y agoExactly, ChatGPT has a calculator to compute the math, the calculator with regard to one query may be correct, but to a query specifying that the calculator must be right the calculator to that query will most likely be correct.
- visarga 4y agoIt's just a regex. When the model outputs "[calculate]12+34=[/calculate]" you append the result "46" and continue with the model. In some papers they even use named symbols. So you could have "[code]width=10; height=20[/code]. What is the area? ... The area is [code]width * height[/code].