3 ms·

Funny enough just today I asked ChatGPT a tax question, which it very confidently gave me the wrong answer for. For those curious, the question was: is interes

by yibg 3y ago

Funny enough just today I asked ChatGPT a tax question, which it very confidently gave me the wrong answer for. For those curious, the question was: is interest earned in Canadian RRSP accounts considered taxable income in California for state tax.

fragmede 3y ago
Yikes. I'm definitely asking my CPA that question and not an LLM.
- NewJazz 3y ago
  First result on DuckDuckGo, for now. Interesting bit of tax law trivia. https://www.kahntaxlaw.com/california-reporting-requirements-canadian-rrsps/ https://www.kahntaxlaw.com/california-reporting-requirements...
Closi 3y ago
Treat an LLM as a confident smart person who isn’t an expert in anything, and doesn’t have access to resources to check their answer (unless you give it access). If you assume the above, it’s unsurprising that it doesn’t get your question right. What’s the correct answer to your question? If you say “research the question before answering” I bet it could probably solve it.
- linsomniac 3y ago
  >Treat an LLM as a confident smart person who isn’t an expert in anything When you say it that way, I'm reminded that IRL I find that sort of person to be just the worst. Largely because I've mostly encountered them in mailing lists and users group meetings, confidently leading other people down a blind alley. When it's just a 1-on-1, with me asking ChatGPT, I guess I'm more willing to accept it -- because the blast radius is limited.
- daveguy 3y ago
  LLMs don't have any clue what the concept "research the question before answering" means. Maybe if you asked it a series of leading questions to get some of the "research" in the context window first. Then you might as well just search. Otherwise it'll just confidently sound like it researched without any bearing on the answer. LLMs are sloppy, and we shouldn't have to do "sociology" experiments to find out whether or not an LLM might give the right answer slightly more often if we trick it into adding more context to the answer.
  - Workaccount2 3y ago
    While something like GPT4 cannot "do more research before answering a question", it is well established that telling an LLM it's an expert in something and asking it to spell out the steps in it's decision making process does make it more accurate in it's responses.
    - daveguy 3y ago
      That is allowing the LLM to place minor steps into its own context window where a more difficult result is more likely to follow as the next series of words. That is using our intelligence of how answers are determined to manipulate the output. It doesn't say anything about the "intelligence" of an LLM. Telling it that it's an expert does not have it somehow change the rote algorithm or have it "pretend" anything. It's just not that complicated.
    - Closi 3y ago
      GPT-4 absolutely can do more research. Adding this to a prompt will encourage GPT-4 to do web searches in the background to augment its answer.
  - Closi 3y ago
    > LLMs don't have any clue what the concept "research the question before answering" means. GPT-4 in ChatGPT will absolutely do research if you ask it (in the practical sense that if you say “research first” like in my original post, it will search the web and interpret/synthesise the results, and then use the findings to inform its answer).
- danShumway 3y ago
  > Treat an LLM as a confident smart person who isn’t an expert in anything, and doesn’t have access to resources to check their answer (unless you give it access). Sure, but I don't want that person helping me with my taxes. Like, the idea that I get an answer and then I need to do research to figure out if it's correct... The whole point -- literally the entire point -- is that I don't want to do the research. If I wanted to do the research, I wouldn't use the LLM. I would just do the research. And now I'm using the LLM and I have to do the research anyway? If we're talking about brainstorming or getting topic overviews or helping you start research, sure, I could see that. An LLM could be useful there. But asking direct questions and getting back plausible-sounding incorrect answers? This is a domain where an LLM just shouldn't be used at all. I've brought up this analogy in the past, but it's like people proposing an LLM as a calculator, and when it gets answers wrong they say, "well, humans get math problems wrong too." Yes, they do. Why do you think we started using calculators instead of humans? There is a reason why I don't go on Reddit to ask for tax advice. It's cool that people can make a computer that simulates that same use-case, but that is not a use-case that was useful to simulate. If I have to read the tax forms anyway, then I might as well just read the tax forms.
  - Closi 3y ago
    The LLM can do the research itself, just current LLM's often don't by default (and require a bit of prompting). LLM's also use calculators (and again, if you prompt ChatGPT it will use a calculator to generate the result). These things are solvable, we are still in the 'Commodore-PET' phase of LLM's development, i.e. impressive for today but lots more to do to make it more useful for most people.
    - danShumway 3y ago
      The thing is, a confident smart person who isn’t an expert in anything, who DOES have access to resources to check their answers is still not someone I want doing my taxes. The only way that an LLM doing my taxes for me is useful is if I trust it to the same level that I trust an expert; and that includes being able to very reliably know when that person is actually confident and when they're not. I haven't seen strong evidence that this is possible with the current LLM approach. So if we're thinking of LLMs as a confident smart person who sometimes spouts bullcrap -- that's still just not a good fit for this kind of product category. It's a good fit for other tasks, just... not for these. It's definitely not a good fit for putting in front of problems that are already pretty-well solved. Calculators replaced human calculation, it's a solved problem, you put the numbers into the calculator. Even natural-language input into the calculator is a reasonably solved problem, it's not a task that really requires a full LLM of the scale of GPT-4. So is there value in introducing a human-like source of error into the middle of that process again? I feel like the answer is 'no'. > These things are solvable, we are still in the 'Commodore-PET' phase of LLM's development Sure, if we get to the point where an LLM can give such good tax information that there literally is no need to read the tax documents to confirm any of it, then great! But that's not even remotely where we are, and products like this straight-up shouldn't be launched until we're at that point. This is sort of like selling me an scuba tank and saying, "it might not dispense oxygen so make sure you hold your breath the entire time", and when I say, "well what's the point of the tank then" the reply I get back is, "well eventually we're going to get it to reliably dispense oxygen!" Okay, but the products are all launching now, I'm being asked to buy into this now, not eventually. The future doesn't change the fact that the current products that I'm supposed to get excited about are literally useless at a conceptual level -- that they do not even make sense given their caveats. I'm not complaining about AI chat bots or funny toys or even toy phone assistants, I'm complaining that companies like Intuit are literally launching tax assistants that give incorrect information and their response is, "well, double check what the bot says." Well that is not a tax assistant then, that's just work. The service that the chatbot is offering me is that I'll have to work more. Fundamentally, there appears to be a misunderstanding from these companies about what a product is.
ripberge 3y ago
What do you believe was incorrect? I asked Chat GPT 3.5 and 4 this question verbatim, seems they both gave the correct answer: it is taxable income. 4 was very thorough. 3.5 not so much.
- yibg 3y ago
  This is the answer I got: “Interest earned in Canadian Registered Retirement Savings Plan (RRSP) accounts is generally not taxable income in California for state tax purposes. California does not typically tax income earned in foreign retirement accounts like RRSPs. However, it's always a good idea to consult with a tax professional or accountant for personalized advice based on your specific situation.”
- yoyohello13 3y ago
  This is exactly the problem. The answer is not deterministic.
dustypotato 3y ago
Don't know why, but I found Gemini more accurate for this kind of stuff. It yaps a lot, so you have to focus on brevity going forward.
ogogmad 3y ago
Oh my God, here we go again. WHICH VERSION of chatgpt did you use?

by yibg 3y ago

fragmede 3y ago

Yikes. I'm definitely asking my CPA that question and not an LLM.

NewJazz 3y ago
First result on DuckDuckGo, for now. Interesting bit of tax law trivia. https://www.kahntaxlaw.com/california-reporting-requirements-canadian-rrsps/ https://www.kahntaxlaw.com/california-reporting-requirements...

Closi 3y ago

Treat an LLM as a confident smart person who isn’t an expert in anything, and doesn’t have access to resources to check their answer (unless you give it access). If you assume the above, it’s unsurprising that it doesn’t get your question right. What’s the correct answer to your question? If you say “research the question before answering” I bet it could probably solve it.

linsomniac 3y ago
>Treat an LLM as a confident smart person who isn’t an expert in anything When you say it that way, I'm reminded that IRL I find that sort of person to be just the worst. Largely because I've mostly encountered them in mailing lists and users group meetings, confidently leading other people down a blind alley. When it's just a 1-on-1, with me asking ChatGPT, I guess I'm more willing to accept it -- because the blast radius is limited.
daveguy 3y ago
LLMs don't have any clue what the concept "research the question before answering" means. Maybe if you asked it a series of leading questions to get some of the "research" in the context window first. Then you might as well just search. Otherwise it'll just confidently sound like it researched without any bearing on the answer. LLMs are sloppy, and we shouldn't have to do "sociology" experiments to find out whether or not an LLM might give the right answer slightly more often if we trick it into adding more context to the answer.
- Workaccount2 3y ago
  While something like GPT4 cannot "do more research before answering a question", it is well established that telling an LLM it's an expert in something and asking it to spell out the steps in it's decision making process does make it more accurate in it's responses.
  - daveguy 3y ago
    That is allowing the LLM to place minor steps into its own context window where a more difficult result is more likely to follow as the next series of words. That is using our intelligence of how answers are determined to manipulate the output. It doesn't say anything about the "intelligence" of an LLM. Telling it that it's an expert does not have it somehow change the rote algorithm or have it "pretend" anything. It's just not that complicated.
  - Closi 3y ago
    GPT-4 absolutely can do more research. Adding this to a prompt will encourage GPT-4 to do web searches in the background to augment its answer.
- Closi 3y ago
  > LLMs don't have any clue what the concept "research the question before answering" means. GPT-4 in ChatGPT will absolutely do research if you ask it (in the practical sense that if you say “research first” like in my original post, it will search the web and interpret/synthesise the results, and then use the findings to inform its answer).
danShumway 3y ago
> Treat an LLM as a confident smart person who isn’t an expert in anything, and doesn’t have access to resources to check their answer (unless you give it access). Sure, but I don't want that person helping me with my taxes. Like, the idea that I get an answer and then I need to do research to figure out if it's correct... The whole point -- literally the entire point -- is that I don't want to do the research. If I wanted to do the research, I wouldn't use the LLM. I would just do the research. And now I'm using the LLM and I have to do the research anyway? If we're talking about brainstorming or getting topic overviews or helping you start research, sure, I could see that. An LLM could be useful there. But asking direct questions and getting back plausible-sounding incorrect answers? This is a domain where an LLM just shouldn't be used at all. I've brought up this analogy in the past, but it's like people proposing an LLM as a calculator, and when it gets answers wrong they say, "well, humans get math problems wrong too." Yes, they do. Why do you think we started using calculators instead of humans? There is a reason why I don't go on Reddit to ask for tax advice. It's cool that people can make a computer that simulates that same use-case, but that is not a use-case that was useful to simulate. If I have to read the tax forms anyway, then I might as well just read the tax forms.
- Closi 3y ago
  The LLM can do the research itself, just current LLM's often don't by default (and require a bit of prompting). LLM's also use calculators (and again, if you prompt ChatGPT it will use a calculator to generate the result). These things are solvable, we are still in the 'Commodore-PET' phase of LLM's development, i.e. impressive for today but lots more to do to make it more useful for most people.
  - danShumway 3y ago
    The thing is, a confident smart person who isn’t an expert in anything, who DOES have access to resources to check their answers is still not someone I want doing my taxes. The only way that an LLM doing my taxes for me is useful is if I trust it to the same level that I trust an expert; and that includes being able to very reliably know when that person is actually confident and when they're not. I haven't seen strong evidence that this is possible with the current LLM approach. So if we're thinking of LLMs as a confident smart person who sometimes spouts bullcrap -- that's still just not a good fit for this kind of product category. It's a good fit for other tasks, just... not for these. It's definitely not a good fit for putting in front of problems that are already pretty-well solved. Calculators replaced human calculation, it's a solved problem, you put the numbers into the calculator. Even natural-language input into the calculator is a reasonably solved problem, it's not a task that really requires a full LLM of the scale of GPT-4. So is there value in introducing a human-like source of error into the middle of that process again? I feel like the answer is 'no'. > These things are solvable, we are still in the 'Commodore-PET' phase of LLM's development Sure, if we get to the point where an LLM can give such good tax information that there literally is no need to read the tax documents to confirm any of it, then great! But that's not even remotely where we are, and products like this straight-up shouldn't be launched until we're at that point. This is sort of like selling me an scuba tank and saying, "it might not dispense oxygen so make sure you hold your breath the entire time", and when I say, "well what's the point of the tank then" the reply I get back is, "well eventually we're going to get it to reliably dispense oxygen!" Okay, but the products are all launching now, I'm being asked to buy into this now, not eventually. The future doesn't change the fact that the current products that I'm supposed to get excited about are literally useless at a conceptual level -- that they do not even make sense given their caveats. I'm not complaining about AI chat bots or funny toys or even toy phone assistants, I'm complaining that companies like Intuit are literally launching tax assistants that give incorrect information and their response is, "well, double check what the bot says." Well that is not a tax assistant then, that's just work. The service that the chatbot is offering me is that I'll have to work more. Fundamentally, there appears to be a misunderstanding from these companies about what a product is.

ripberge 3y ago

What do you believe was incorrect? I asked Chat GPT 3.5 and 4 this question verbatim, seems they both gave the correct answer: it is taxable income. 4 was very thorough. 3.5 not so much.

yibg 3y ago
This is the answer I got: “Interest earned in Canadian Registered Retirement Savings Plan (RRSP) accounts is generally not taxable income in California for state tax purposes. California does not typically tax income earned in foreign retirement accounts like RRSPs. However, it's always a good idea to consult with a tax professional or accountant for personalized advice based on your specific situation.”
yoyohello13 3y ago
This is exactly the problem. The answer is not deterministic.

dustypotato 3y ago

Don't know why, but I found Gemini more accurate for this kind of stuff. It yaps a lot, so you have to focus on brevity going forward.

ogogmad 3y ago

Oh my God, here we go again. WHICH VERSION of chatgpt did you use?