6 ms·

Don't worry, it'll die down. People are just having fun with the current novelty item. I tried ChatGPT and it is by far the best chat bot I've ever interacted w

Don't worry, it'll die down. People are just having fun with the current novelty item. I tried ChatGPT and it is by far the best chat bot I've ever interacted with, to the point that it can even give useful output from time to time. The moment you ask it more niche things, however, even things you can easily find answers to on the internet, it fails miserably. For example, ask it to give you a minizinc model for the 8 queens problem. It'll confidently give you an answer that's completely wrong. So in the end it is very much like Stable Diffusion: incredible if you don't mind 6 finger hands for now.

chrisbaker98 4y ago
Yeah, chatgpt becomes less impressive when you start asking it questions about topics you already know. You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. Obviously it's still extremely impressive and much better than anything I've seen before - and it might mean we're only a few years away from something flawless - but I can't trust it for now. (In theory, though, it might be easy to solve the current problem. The bot doesn't have to be right about everything, it just has to cite its sources. "I think that Napoleon was defeated by Wellington at the battle of Borodino. For more information, see this Britannica article. Click here to report if I made a mistake.")
- anonyfox 4y ago
  > You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. thats... exactly like most humans behave.
  - chrisbaker98 4y ago
    Humans are capable of expressing uncertainty; ChatGPT never does.
    - soulofmischief 4y ago
      That is categorically false, the model does hallucinate but it absolutely expresses uncertainty about things.
      - withinboredom 4y ago
        It does for certain things, but when I was guiding it to create a very complex regex, it would totally "forget" or "ignore" random constraints and give the wrong regex repeatedly. It was pretty fun, even if I never got a correct regex. I had to hand craft it. So much for AI making things faster.
        soulofmischief 4y ago
        GitHub Copilot has increased my development speed by at least 10%. That's an insane margin for $10 a month.
        actionfromafar 4y ago
        Speed yes, but how did it change your direction?
        soulofmischief 4y ago
        It sent me on a crash course for success. ;)
      - hooby 4y ago
        Is there a relation between the model expressing certainty or uncertainty and the likelihood of the statement being a hallucination?
        soulofmischief 4y ago
        Yes, and if you poke around the OpenAI playground you will see they've trained models to be more transparent about uncertainty, opting for a non-answer instead of a hallucination much more often.
  - neilv 4y ago
    We (at least in the US) tend to celebrate "confidence". Maybe it's a primal thing, where confidence was a signal of strength, because if you went strutting around without being able to back it up, you'd get your primal butt kicked. But now people can false signal with impunity. Maybe ChatGPT can help educate us out of thinking unwarranted confidence is admirable and attractive.
    - shanebellone 4y ago
      It's because people are weak and want someone to follow. Humans are livestock (with exceptional egos) waiting for a shepherd to herd them.
      - alch- 4y ago
        I, for one, welcome our new robot overlords.
      - worthless-trash 4y ago
        Spoken by someone who hasn't worked with either sheep or humans. My bet is sheep, because they are -very- different creatures.
  - patwolf 4y ago
    The nice thing about humans (at least some of them) is that you can ultimately explain to them why they're wrong and they'll understand. When ChatGPT gives you wrong or contradictory information, there's no recourse.
    - anonyfox 4y ago
      somewhat I lost this belief during brexit/trump/pandemic/... . Maybe ChatGPT also implements a "saving face" behavior, it's not like this isn't unusual in the world.
  - jcuenod 4y ago
    During an assessment with a kindergartner, the child was asked "continue the sequence: 1, 2, 3". The child answered, "4, 7, 42, 91 ... wow, I'm good at this."
    - temporary22 4y ago
      tbh that can be a correct answer (crescent numbers, not necessarily consecutive)
- TaylorAlexander 4y ago
  Yeah, it definitely gets me thinking about how powerful it would be if they can ever get a version that can return correct answers for questions in a broad variety of fields. It feels like something adjacent to Google, but different. On a search engine, I am asking an algorithm to return web pages generally written by humans that have some relationship to my query. But with ChatGPT I start to get the sense for what it would be like to be talking to an expert in whatever field relates to my query. A system like this that can embody real expertise would be amazing.
- vonunov 4y ago
  It took 3-4 BS replies before I got it to admit it hadn't actually read Animorphs at all!
- sireat 4y ago
  I asked ChatGPT to provide an academic source on Pearson correlation coefficient. ChatGPT responded with an abstract to a fake 1904 article. The link to the article it gave lead to 1913 article on crab claw length by a different Pearson.
- lordnacho 4y ago
  I started playing with it yesterday. It's amazing. For coding stuff, it's already usable as a template assistant: it finds the right imports, gives things sensible names, and gets the interface right with a bit of prompt engineering. For general knowledge, it reminds me an awful lot of a copywriter I once worked with. He understood almost nothing about finance (very young guy), but he could churn out articles with the right words in them. Basically a layman would say he's an expert, and an expert would say he's a layman. The same goes for things I'm not an expert on, BTW. My answer to why Rome fell is pretty much what cGPT spits out, and I wouldn't know better until a classics professor challenged me on it. That's what I currently think about chatGPT. A sort of intellectual tourist who can tell you a lot of things about a lot of areas, but it's skin deep. It's still rather awesome, because you have to start somewhere and basically everywhere I've looked it has that diligent high school kid answer that can be researched if you know what you're doing. You can even tell it's a high school kid because of the way it uses certain terms: a little too generalizing, skipping over nuances. It also doesn't give really long answers, at least not to things I ask. If it were really confident it would spit out something akin to acoup's essays about history. Of course this will fall to Goodheart's law someday: say lots of things and people will think you are smart, until they realize you are just saying lots of things in order to sound smart, and then length will no longer be a signal of a good answer.
  - syntheweave 4y ago
    I asked ChatGPT whether it makes positive or normative statements. It errored, so in the comment box I explained that the correct answer is that it makes statements normative to the consensus of its data set, which sometimes happen to be positive. This is why it's easy to make ChatGPT reveal bias, and why the restrictions patching that bias tend to fall prey to simple ruses that reveal its preferences; everything looks like a norm to it, so it lacks the causal chains that would lead towards logical outcomes. As a result it's extremely gullible and emits nonsense when given a tough logic question. Given a task like "give me a list of words ending in the letter u", it will oblige with a very lengthy alphabetical list of words, most of which end in u, but not all. Asked to find the largest set of rhyming words in the list, its answers changed radically each time, from "I can't do that" to a somewhat plausible candidate(except for having words that don't end in u) to getting stuck repeating "buttocks". I used it to write some business language to respond to a recruiter. It's decent at being a secretary, since that stuff is 99% norms.
- benibela 4y ago
  I saw a post where it got asked for integral of x^3 over [2,4]. It explained perfectly each step of calculating the integral .. and then got the wrong result
- Sophira 4y ago
  You actually can ask ChatGPT to cite its sources. Unfortunately, the sources it cites are very often completely made up, despite looking like extremely good sources.
- kjs3 4y ago
  You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. So it's Wikipedia that talks.
icoder 4y ago
Yeah I asked it a programming question and it suggested I could use both degrees and radians (for a 'double' parameter) giving literal examples (not just in text but in code) 'use 45 for 45 degrees, and 1.5 for 1.5 radians'. This is for Flutter's Transform.rotate and unless I've missed something, this is always in radians. Took me longer to find out than if I had just Googled it, although I admit I had been using it with success all morning for comparable questions. If it's right 90% of the time and acts and sounds like it is right 10% of the time while being (slightly) wrong, that makes it practically useless.
- manmal 4y ago
  It’s not useless if you treat the output like just another blog post that could be wrong - with a healthy dose of suspicion.
  - gremlinsinc 4y ago
    Hell, even it could be right, chatGPT or the blog post but in tech/programming breaking changes happen and updates to frameworks so it could be giving you react 17 advice not react 18, etc.
okamiueru 4y ago
I asked if how long the titanic was, and it gave a very good and confident answer. Including how it hit a polar bear and sank.
- deleted 4y ago
  [deleted]
- iso1631 4y ago
  I was playing with it with my son last night, he asked why the beans cooled down more quickly than his potato It started well with the volume-surface area ratio. It then expanded on it to confidently tell us that beans were made of metal while potatoes were made of plastic.
tluyben2 4y ago
For some things it confidently gets completely wrong, you can correct it slightly and put 5-6 other prompts to arrive at a correct answer. This is not easy to automate though.
mromanuk 4y ago
The inaccuracy and the confidence when they respond, are the worst aspects of these transformers.
stevenhuang 4y ago
Prompt it better, provide escape hatches (if I am not clear ask follow up questions), tell it not to improvise, provide more context, prime your prompt by referencing deep related material, provide examples of what you want it to do. If you follow all this in your initial prompt, you will get vastly better responses... which is unsurprising. How would you react when you get a random DM from someone asking such an esoteric question, without surrounding context? (Is the person just role playing here, or is this a genuine research request for the purposes of a report?)
EVa5I7bHFq9mnYK 4y ago
Can it answer "Don't know"? Or does it suffer from over-inflated ego, just like humans?
Nanana909 4y ago
A lot of people on HN are poking holes in ChatGPT when they find something it does wrong. I have to imagine 20-ish years ago we’d be complaining about questions we tried that Google failed to give a good result for. > For example, ask it to give you a minizinc model for the 8 queens problem. But even a normal software engineer, is likely to confidently fail as something as niche as this. What percent of anyones work consists of questions like this? Even for the people who do encounter it, I’m willing to say not very much…… This doesn’t even take into account the state of this technology in 2,3, 10 years. A lot of people denying every advancement in this stuff are going to be saying it all the way into their obsolescence.

chrisbaker98 4y ago

Yeah, chatgpt becomes less impressive when you start asking it questions about topics you already know. You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. Obviously it's still extremely impressive and much better than anything I've seen before - and it might mean we're only a few years away from something flawless - but I can't trust it for now. (In theory, though, it might be easy to solve the current problem. The bot doesn't have to be right about everything, it just has to cite its sources. "I think that Napoleon was defeated by Wellington at the battle of Borodino. For more information, see this Britannica article. Click here to report if I made a mistake.")

anonyfox 4y ago
> You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. thats... exactly like most humans behave.
- chrisbaker98 4y ago
  Humans are capable of expressing uncertainty; ChatGPT never does.
  - soulofmischief 4y ago
    That is categorically false, the model does hallucinate but it absolutely expresses uncertainty about things.
    - withinboredom 4y ago
      It does for certain things, but when I was guiding it to create a very complex regex, it would totally "forget" or "ignore" random constraints and give the wrong regex repeatedly. It was pretty fun, even if I never got a correct regex. I had to hand craft it. So much for AI making things faster.
      - soulofmischief 4y ago
        GitHub Copilot has increased my development speed by at least 10%. That's an insane margin for $10 a month.
        actionfromafar 4y ago
        Speed yes, but how did it change your direction?
        soulofmischief 4y ago
        It sent me on a crash course for success. ;)
    - hooby 4y ago
      Is there a relation between the model expressing certainty or uncertainty and the likelihood of the statement being a hallucination?
      - soulofmischief 4y ago
        Yes, and if you poke around the OpenAI playground you will see they've trained models to be more transparent about uncertainty, opting for a non-answer instead of a hallucination much more often.
- neilv 4y ago
  We (at least in the US) tend to celebrate "confidence". Maybe it's a primal thing, where confidence was a signal of strength, because if you went strutting around without being able to back it up, you'd get your primal butt kicked. But now people can false signal with impunity. Maybe ChatGPT can help educate us out of thinking unwarranted confidence is admirable and attractive.
  - shanebellone 4y ago
    It's because people are weak and want someone to follow. Humans are livestock (with exceptional egos) waiting for a shepherd to herd them.
    - alch- 4y ago
      I, for one, welcome our new robot overlords.
    - worthless-trash 4y ago
      Spoken by someone who hasn't worked with either sheep or humans. My bet is sheep, because they are -very- different creatures.
- patwolf 4y ago
  The nice thing about humans (at least some of them) is that you can ultimately explain to them why they're wrong and they'll understand. When ChatGPT gives you wrong or contradictory information, there's no recourse.
  - anonyfox 4y ago
    somewhat I lost this belief during brexit/trump/pandemic/... . Maybe ChatGPT also implements a "saving face" behavior, it's not like this isn't unusual in the world.
- jcuenod 4y ago
  During an assessment with a kindergartner, the child was asked "continue the sequence: 1, 2, 3". The child answered, "4, 7, 42, 91 ... wow, I'm good at this."
  - temporary22 4y ago
    tbh that can be a correct answer (crescent numbers, not necessarily consecutive)
TaylorAlexander 4y ago
Yeah, it definitely gets me thinking about how powerful it would be if they can ever get a version that can return correct answers for questions in a broad variety of fields. It feels like something adjacent to Google, but different. On a search engine, I am asking an algorithm to return web pages generally written by humans that have some relationship to my query. But with ChatGPT I start to get the sense for what it would be like to be talking to an expert in whatever field relates to my query. A system like this that can embody real expertise would be amazing.
vonunov 4y ago
It took 3-4 BS replies before I got it to admit it hadn't actually read Animorphs at all!
sireat 4y ago
I asked ChatGPT to provide an academic source on Pearson correlation coefficient. ChatGPT responded with an abstract to a fake 1904 article. The link to the article it gave lead to 1913 article on crab claw length by a different Pearson.
lordnacho 4y ago
I started playing with it yesterday. It's amazing. For coding stuff, it's already usable as a template assistant: it finds the right imports, gives things sensible names, and gets the interface right with a bit of prompt engineering. For general knowledge, it reminds me an awful lot of a copywriter I once worked with. He understood almost nothing about finance (very young guy), but he could churn out articles with the right words in them. Basically a layman would say he's an expert, and an expert would say he's a layman. The same goes for things I'm not an expert on, BTW. My answer to why Rome fell is pretty much what cGPT spits out, and I wouldn't know better until a classics professor challenged me on it. That's what I currently think about chatGPT. A sort of intellectual tourist who can tell you a lot of things about a lot of areas, but it's skin deep. It's still rather awesome, because you have to start somewhere and basically everywhere I've looked it has that diligent high school kid answer that can be researched if you know what you're doing. You can even tell it's a high school kid because of the way it uses certain terms: a little too generalizing, skipping over nuances. It also doesn't give really long answers, at least not to things I ask. If it were really confident it would spit out something akin to acoup's essays about history. Of course this will fall to Goodheart's law someday: say lots of things and people will think you are smart, until they realize you are just saying lots of things in order to sound smart, and then length will no longer be a signal of a good answer.
- syntheweave 4y ago
  I asked ChatGPT whether it makes positive or normative statements. It errored, so in the comment box I explained that the correct answer is that it makes statements normative to the consensus of its data set, which sometimes happen to be positive. This is why it's easy to make ChatGPT reveal bias, and why the restrictions patching that bias tend to fall prey to simple ruses that reveal its preferences; everything looks like a norm to it, so it lacks the causal chains that would lead towards logical outcomes. As a result it's extremely gullible and emits nonsense when given a tough logic question. Given a task like "give me a list of words ending in the letter u", it will oblige with a very lengthy alphabetical list of words, most of which end in u, but not all. Asked to find the largest set of rhyming words in the list, its answers changed radically each time, from "I can't do that" to a somewhat plausible candidate(except for having words that don't end in u) to getting stuck repeating "buttocks". I used it to write some business language to respond to a recruiter. It's decent at being a secretary, since that stuff is 99% norms.
benibela 4y ago
I saw a post where it got asked for integral of x^3 over [2,4]. It explained perfectly each step of calculating the integral .. and then got the wrong result
Sophira 4y ago
You actually can ask ChatGPT to cite its sources. Unfortunately, the sources it cites are very often completely made up, despite looking like extremely good sources.
kjs3 4y ago
You'll notice that it's often wrong, but the bigger problem is that it's always confidently wrong. So it's Wikipedia that talks.

icoder 4y ago

Yeah I asked it a programming question and it suggested I could use both degrees and radians (for a 'double' parameter) giving literal examples (not just in text but in code) 'use 45 for 45 degrees, and 1.5 for 1.5 radians'. This is for Flutter's Transform.rotate and unless I've missed something, this is always in radians. Took me longer to find out than if I had just Googled it, although I admit I had been using it with success all morning for comparable questions. If it's right 90% of the time and acts and sounds like it is right 10% of the time while being (slightly) wrong, that makes it practically useless.

manmal 4y ago
It’s not useless if you treat the output like just another blog post that could be wrong - with a healthy dose of suspicion.
- gremlinsinc 4y ago
  Hell, even it could be right, chatGPT or the blog post but in tech/programming breaking changes happen and updates to frameworks so it could be giving you react 17 advice not react 18, etc.

okamiueru 4y ago

I asked if how long the titanic was, and it gave a very good and confident answer. Including how it hit a polar bear and sank.

deleted 4y ago
[deleted]
iso1631 4y ago
I was playing with it with my son last night, he asked why the beans cooled down more quickly than his potato It started well with the volume-surface area ratio. It then expanded on it to confidently tell us that beans were made of metal while potatoes were made of plastic.

tluyben2 4y ago

For some things it confidently gets completely wrong, you can correct it slightly and put 5-6 other prompts to arrive at a correct answer. This is not easy to automate though.

mromanuk 4y ago

The inaccuracy and the confidence when they respond, are the worst aspects of these transformers.

stevenhuang 4y ago

Prompt it better, provide escape hatches (if I am not clear ask follow up questions), tell it not to improvise, provide more context, prime your prompt by referencing deep related material, provide examples of what you want it to do. If you follow all this in your initial prompt, you will get vastly better responses... which is unsurprising. How would you react when you get a random DM from someone asking such an esoteric question, without surrounding context? (Is the person just role playing here, or is this a genuine research request for the purposes of a report?)

EVa5I7bHFq9mnYK 4y ago

Can it answer "Don't know"? Or does it suffer from over-inflated ego, just like humans?

Nanana909 4y ago

A lot of people on HN are poking holes in ChatGPT when they find something it does wrong. I have to imagine 20-ish years ago we’d be complaining about questions we tried that Google failed to give a good result for. > For example, ask it to give you a minizinc model for the 8 queens problem. But even a normal software engineer, is likely to confidently fail as something as niche as this. What percent of anyones work consists of questions like this? Even for the people who do encounter it, I’m willing to say not very much…… This doesn’t even take into account the state of this technology in 2,3, 10 years. A lot of people denying every advancement in this stuff are going to be saying it all the way into their obsolescence.