5 ms·

* We're seeing much less of "it's making mistakes" these days.* Perhaps less than before, but still making very fundamental errors. Anything involving number I

by yibg 2y ago

* We're seeing much less of "it's making mistakes" these days.* Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious. Pretty frequently I'd get different answers for the same question (to a human). e.g. ChatGPT will give an effective tax rate of n for some income amount. Then when asked to break down the calculation will come up with an effective tax rate of m instead. When asked how much tax is owed on that income will come up with a different number such that the effective rate is not n or m. Until this is addressed to a sufficient degree, it seems difficult to apply to anything that involves numbers and can't be quickly verified by a human.

Animats 2y ago
> Perhaps less than before, but still making very fundamental errors. Yes. Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built. Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning. If only they knew when they didn't know something. There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today. [4] https://hungleai.substack.com/p/uncertainty-confidence-and-hallucination https://hungleai.substack.com/p/uncertainty-confidence-and-h...
- bennettnate5 2y ago
  I remember watching IBM's Watson soundly beat Ken Jennings on Jeapordy. One of the things that sticks out most to me about the memory is that Watson had a confidence rating score for each answer it gave (and there were a few questions for which it had very low confidence). I didn't realize it at the time, but that was actually pretty impressive given the overconfidence issues LLMs have nowadays. Of course, citing sources would always trump any kind of confidence rating in my mind; sources provide provenance, while confidence rating can be fudged just like a bogus answer can.
  - _heimdall 2y ago
    Providing sources gets interesting to because (a) LLMs have been known to hallucinate nonexistent sources and (b) there isn't a perfect way of knowing whether the source itself was generated by an LLM and contains hallucinations.
LASR 2y ago
Yes. Numbers / math is pretty much instant hallucination. But. Try this approach instead: have it generate python code, with print statements before every bit of math it performs. It will write pretty good code, which you then execute to generate the actual answer. Simpler example: paste in a paragraph of text, ask it to count the number of words. The answer will be incorrect most of the time. Instead, ask it to out each word in the text in a numbered list and then output the word count. It will be correct almost always. My anecdotal learning from this: LLMs are pretty human-like in their mental abilities. I wouldn't be able to simply look at some text and give you an accurate word count. I would point my finger / cursor to every word and count up. The solutions above are basically giving LLMs some additional techniques or tools, very similar to how a human may use a calculator, or count words. In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values. The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs. Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet. Takes a couple of minutes, but pretty bulletproof and also generalizable to other complex math calculations.
- ddfs123 2y ago
  >I wouldn't be able to simply look at some text and give you an accurate word count. I actually think you would, lots of time I've been surprised by how accurate people can eye-ball stuffs.
- AdieuToLogic 2y ago
  > Yes. Numbers / math is pretty much instant hallucination. Programming is codified applied mathematics and involves numbers in all but the most trivial programs. > LLMs are pretty human-like in their mental abilities. LLM's are algorithms. Algorithms do not have "mental abilities", people do. Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable. > In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values. > The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs. > Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet. How is this not a classic definition of overfitting[0]? Or is the generated code intentionally specific to, and only applicable for, a single spreadsheet? 0 - https://en.wikipedia.org/wiki/Overfitting https://en.wikipedia.org/wiki/Overfitting
_sys49152 2y ago
i tell chatgpt i pay 5 dollars for every excellent response, and i make it keep track of how much i spend per chat session (of course in addition to my normal course of work) it does 2 things. 1 tells me how deep i am in the conversation and 2 when the computation falls apart, i can assume other things in its response will be trash as well. and sometimes number 3 how well the software is working. ranges from 20$ to 35$ on average but a couple days they go deep to 45$ (4,7,9 responses in chat session) today i learned i can knock the computation loose in a session somewhere around the 4th reply or $20 by injecting a random number in my course work and it was latching onto the number instead of computing $
Propelloni 2y ago
LLMs don't do math in that sense. They build a string of tokens out of a billion pre-weighted ones that gets a favorable probably distribution when taking your prompt into account. Change your prompt, get a different printout. There is no semantic understanding (in the sense of does what is printed make sense) and it therefore cannot plausibility check its response. A LLM will just print gibberish if it gets the best probability distribution of tokens. I'm sure that's something that will be addressed over time, but we are not there yet. I'm not keen on marketing words like "superintelligence" but boiling it down that's what in my mind the OP said. These systems are limited in ways that we do not yet fully appreciate. They are not silver bullets for all or maybe even many problems. We need to figure out where they can be deployed for greater benefit.
madeofpalk 2y ago
> Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious Totally. They are large language models, not math models. I think the problem is that 'some people' overhype them as universal tools to solve any problem, and to answer any question. But really, LLMs excel in generating pretty regular text.
- yibg 2y ago
  Problem is also they DO work in SOME cases. e.g. ask ChatGPT to breakdown the calculation to determine tax owed on $xxx of income, and the result is correct in most cases. So there is this perception of intelligence but also fails spectacularly in very simple cases.
  - madeofpalk 2y ago
    In the same way that you can drive a tractor to the shops and sometimes it's a fine mode of transport.

by yibg 2y ago

Animats 2y ago

> Perhaps less than before, but still making very fundamental errors. Yes. Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built. Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning. If only they knew when they didn't know something. There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today. [4] https://hungleai.substack.com/p/uncertainty-confidence-and-hallucination https://hungleai.substack.com/p/uncertainty-confidence-and-h...

bennettnate5 2y ago
I remember watching IBM's Watson soundly beat Ken Jennings on Jeapordy. One of the things that sticks out most to me about the memory is that Watson had a confidence rating score for each answer it gave (and there were a few questions for which it had very low confidence). I didn't realize it at the time, but that was actually pretty impressive given the overconfidence issues LLMs have nowadays. Of course, citing sources would always trump any kind of confidence rating in my mind; sources provide provenance, while confidence rating can be fudged just like a bogus answer can.
- _heimdall 2y ago
  Providing sources gets interesting to because (a) LLMs have been known to hallucinate nonexistent sources and (b) there isn't a perfect way of knowing whether the source itself was generated by an LLM and contains hallucinations.

LASR 2y ago

Yes. Numbers / math is pretty much instant hallucination. But. Try this approach instead: have it generate python code, with print statements before every bit of math it performs. It will write pretty good code, which you then execute to generate the actual answer. Simpler example: paste in a paragraph of text, ask it to count the number of words. The answer will be incorrect most of the time. Instead, ask it to out each word in the text in a numbered list and then output the word count. It will be correct almost always. My anecdotal learning from this: LLMs are pretty human-like in their mental abilities. I wouldn't be able to simply look at some text and give you an accurate word count. I would point my finger / cursor to every word and count up. The solutions above are basically giving LLMs some additional techniques or tools, very similar to how a human may use a calculator, or count words. In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values. The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs. Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet. Takes a couple of minutes, but pretty bulletproof and also generalizable to other complex math calculations.

ddfs123 2y ago
>I wouldn't be able to simply look at some text and give you an accurate word count. I actually think you would, lots of time I've been surprised by how accurate people can eye-ball stuffs.
AdieuToLogic 2y ago
> Yes. Numbers / math is pretty much instant hallucination. Programming is codified applied mathematics and involves numbers in all but the most trivial programs. > LLMs are pretty human-like in their mental abilities. LLM's are algorithms. Algorithms do not have "mental abilities", people do. Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable. > In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values. > The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs. > Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet. How is this not a classic definition of overfitting[0]? Or is the generated code intentionally specific to, and only applicable for, a single spreadsheet? 0 - https://en.wikipedia.org/wiki/Overfitting https://en.wikipedia.org/wiki/Overfitting

_sys49152 2y ago

i tell chatgpt i pay 5 dollars for every excellent response, and i make it keep track of how much i spend per chat session (of course in addition to my normal course of work) it does 2 things. 1 tells me how deep i am in the conversation and 2 when the computation falls apart, i can assume other things in its response will be trash as well. and sometimes number 3 how well the software is working. ranges from 20$ to 35$ on average but a couple days they go deep to 45$ (4,7,9 responses in chat session) today i learned i can knock the computation loose in a session somewhere around the 4th reply or $20 by injecting a random number in my course work and it was latching onto the number instead of computing $

Propelloni 2y ago

LLMs don't do math in that sense. They build a string of tokens out of a billion pre-weighted ones that gets a favorable probably distribution when taking your prompt into account. Change your prompt, get a different printout. There is no semantic understanding (in the sense of does what is printed make sense) and it therefore cannot plausibility check its response. A LLM will just print gibberish if it gets the best probability distribution of tokens. I'm sure that's something that will be addressed over time, but we are not there yet. I'm not keen on marketing words like "superintelligence" but boiling it down that's what in my mind the OP said. These systems are limited in ways that we do not yet fully appreciate. They are not silver bullets for all or maybe even many problems. We need to figure out where they can be deployed for greater benefit.

madeofpalk 2y ago

> Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious Totally. They are large language models, not math models. I think the problem is that 'some people' overhype them as universal tools to solve any problem, and to answer any question. But really, LLMs excel in generating pretty regular text.

yibg 2y ago
Problem is also they DO work in SOME cases. e.g. ask ChatGPT to breakdown the calculation to determine tax owed on $xxx of income, and the result is correct in most cases. So there is this perception of intelligence but also fails spectacularly in very simple cases.
- madeofpalk 2y ago
  In the same way that you can drive a tractor to the shops and sometimes it's a fine mode of transport.