5 ms·

Except this isn't people trying to break it. "Summarize lululemon quarterly earnings report" returning made up numbers is not garbage in, garbage out, unless th

Except this isn't people trying to break it. "Summarize lululemon quarterly earnings report" returning made up numbers is not garbage in, garbage out, unless the garbage in part is the design approach to this thing. The thing swearing on it's mother that its 2022 after returning the date, then "refusing to trust" the user is not the result of someone stress testing the tool.

BoorishBears 4y ago
I wrote a longer version of this comment, but why would you ask ChatGPT to summarize an earnings report, and at the very least not just give it the earnings report? I will be so so disappointed if the immense potential their current approach has gets nerfed because people want to shoehorn this into being AskJeeves 2.0 All of these complaints boil down to hallucination, but hallucination is what makes this thing so powerful for novel insight. Instead of "Summarize lululemon quarterly earnings report" I would cut and paste a good chunk with some numbers, then say "Lululemon stock went (up|down) after these numbers, why could that be", and in all likelihood it'd give you some novel insight that makes some degree of sense. To me, if you can type a query into Google and get a plain result, it's a bad prompt. Yes that's essentially saying "you're holding it wrong", but again, in this case it's kind of like trying to dull a knife so you can hold it by the blade and it'd really be a shame if that's where the optimization starts to go.
- rst 4y ago
  It's being promoted as a search engine; in that context, it's completely reasonable to expect that it will fetch the earnings report itself if asked.
  - BoorishBears 4y ago
    It was my mistake holding HN to a higher standard than the most uncharitable interpretation of a comment. I didn't fault a user for searching with a search engine, I'm questioning why a search engine is pigeonholing ChatGPT into being search interface. But I guess if you're the kind of person prone to low value commentary like "why'd you search using a search engine?!" you might project it onto others...
- notahacker 4y ago
  According to the article Microsoft did this. In their video product demo. To showcase its purported ability to retrieve and summarise information. Which, as it turns out, was more of an inability to do it properly. I agree your approach to prompting is less likely to yield an error (and make you more likely to catch it if it does), but your question basically boils down to "why is Bing Chat a thing?". And tbh that one got answered a while ago when Google Home and Siri and Alexa became things. Convenience is good: it's just it turns out that being much more ambitious isn't that convenient if it means being wrong or weird a lot
  - BoorishBears 4y ago
    I mean I thought it was clear enough that, I am in fact speaking to the larger point of "why is this a product"? When I say "people" I don't mean visitors to Bing, I mean whoever at Microsoft is driving this Microsoft wants their expensive oft derided search engine to become a relevant channel in people's lives, that's an obvious "business why" But from a "product why", Alexa/Siri/Home seem like they would be cases against trying this again for the exact reason you gave: Pigeonholing an LM try to answer search engine queries is a recipe for over-ambition Over-ambition in this case being relying on a system prone to hallucinations for factual data across the entire internet.
- int_19h 4y ago
  The major advantage that Bing AI has over ChatGPT is that it can look things up on its own, so why wouldn't it go find the report?
  - BoorishBears 4y ago
    Absolutely incredible that you can ask this question and not think for a moment the comment you read (right?) is a commentary on the product. It's actually easier for you to think someone asked "why did a search engine search" than "why does the search engine have an LM sitting over it"
    - int_19h 4y ago
      Your question was: "why would you ask ChatGPT to summarize an earnings report, and at the very least not just give it the earnings report?" The obvious answer is, because it's easier and faster to do so if you know that it can look it up yourself. If the question is rather about why it can look it up, the equally obvious answer is that it makes it easier and faster to ask such questions.
      - BoorishBears 4y ago
        My comment is not a single sentence. I'd excuse the misunderstanding if I had just left it to the reader to guess my intent, but not only do I expand on it, I wrote two more sibling comments hours before you replied clarifying it. It almost seems like you stopped reading the moment you got to some arbitrary point and decided you knew what I was saying better than I did. > If the question is rather about why it can look it up, the equally obvious answer is that it makes it easier and faster to ask such questions. Obviously the comment is questioning this exact permise: And arguing that it's not faster and easier to insert an LM over a search engine, when an LM is prone to hallucination, and the entire internet is such a massive dataset that you'll overfit on search engine style question and sacrifice the novel aspect to this. You were so preciously close to getting that but I guess snark about obvious answers is more your speed...
        4y ago

BoorishBears 4y ago

I wrote a longer version of this comment, but why would you ask ChatGPT to summarize an earnings report, and at the very least not just give it the earnings report? I will be so so disappointed if the immense potential their current approach has gets nerfed because people want to shoehorn this into being AskJeeves 2.0 All of these complaints boil down to hallucination, but hallucination is what makes this thing so powerful for novel insight. Instead of "Summarize lululemon quarterly earnings report" I would cut and paste a good chunk with some numbers, then say "Lululemon stock went (up|down) after these numbers, why could that be", and in all likelihood it'd give you some novel insight that makes some degree of sense. To me, if you can type a query into Google and get a plain result, it's a bad prompt. Yes that's essentially saying "you're holding it wrong", but again, in this case it's kind of like trying to dull a knife so you can hold it by the blade and it'd really be a shame if that's where the optimization starts to go.

rst 4y ago
It's being promoted as a search engine; in that context, it's completely reasonable to expect that it will fetch the earnings report itself if asked.
- BoorishBears 4y ago
  It was my mistake holding HN to a higher standard than the most uncharitable interpretation of a comment. I didn't fault a user for searching with a search engine, I'm questioning why a search engine is pigeonholing ChatGPT into being search interface. But I guess if you're the kind of person prone to low value commentary like "why'd you search using a search engine?!" you might project it onto others...
notahacker 4y ago
According to the article Microsoft did this. In their video product demo. To showcase its purported ability to retrieve and summarise information. Which, as it turns out, was more of an inability to do it properly. I agree your approach to prompting is less likely to yield an error (and make you more likely to catch it if it does), but your question basically boils down to "why is Bing Chat a thing?". And tbh that one got answered a while ago when Google Home and Siri and Alexa became things. Convenience is good: it's just it turns out that being much more ambitious isn't that convenient if it means being wrong or weird a lot
- BoorishBears 4y ago
  I mean I thought it was clear enough that, I am in fact speaking to the larger point of "why is this a product"? When I say "people" I don't mean visitors to Bing, I mean whoever at Microsoft is driving this Microsoft wants their expensive oft derided search engine to become a relevant channel in people's lives, that's an obvious "business why" But from a "product why", Alexa/Siri/Home seem like they would be cases against trying this again for the exact reason you gave: Pigeonholing an LM try to answer search engine queries is a recipe for over-ambition Over-ambition in this case being relying on a system prone to hallucinations for factual data across the entire internet.
int_19h 4y ago
The major advantage that Bing AI has over ChatGPT is that it can look things up on its own, so why wouldn't it go find the report?
- BoorishBears 4y ago
  Absolutely incredible that you can ask this question and not think for a moment the comment you read (right?) is a commentary on the product. It's actually easier for you to think someone asked "why did a search engine search" than "why does the search engine have an LM sitting over it"
  - int_19h 4y ago
    Your question was: "why would you ask ChatGPT to summarize an earnings report, and at the very least not just give it the earnings report?" The obvious answer is, because it's easier and faster to do so if you know that it can look it up yourself. If the question is rather about why it can look it up, the equally obvious answer is that it makes it easier and faster to ask such questions.
    - BoorishBears 4y ago
      My comment is not a single sentence. I'd excuse the misunderstanding if I had just left it to the reader to guess my intent, but not only do I expand on it, I wrote two more sibling comments hours before you replied clarifying it. It almost seems like you stopped reading the moment you got to some arbitrary point and decided you knew what I was saying better than I did. > If the question is rather about why it can look it up, the equally obvious answer is that it makes it easier and faster to ask such questions. Obviously the comment is questioning this exact permise: And arguing that it's not faster and easier to insert an LM over a search engine, when an LM is prone to hallucination, and the entire internet is such a massive dataset that you'll overfit on search engine style question and sacrifice the novel aspect to this. You were so preciously close to getting that but I guess snark about obvious answers is more your speed...
      - 4y ago