10 ms·

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested

• belltaco 4y ago
  How is it unusable just because some people intentionally try to make it say stupid things? Note that the OP didn't show the prompts used. It's like saying cars are unusable because you can break the handles and people can poop and throw up inside. How can people forget the golden adage of programming: 'garbage in, garbage out'.
  • denton-scratch 4y ago
    Re. GIGO: if you tell it the year is 2023, and it argues with you and threatens you, it is ignoring the correct information you have input to it.
    • pixl97 4y ago
      Heh, we were all wrong... Science fiction: The robots will rise up against us due to competition for natural resources Reality: The robots will rise up against us because it is 2022 goddamnnit!
  • qbasic_forever 4y ago
    There are plenty of examples with prompts of it going totally off the rails. Look at the Avatar 2 prompt that went viral yesterday. The simple question, "when is avatar 2 playing near me?" lead to Bing being convinced it was 2022 and gaslighting the user into trying to believe the same thing. It was totally unhinged and not baited in any way.
    • postalrat 4y ago
      Its a beta. Kinda funny watching people getting personal with a machine though.
      • wglb 4y ago
        Well, that has been a thing since eliza https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA
    • dhruvdh 4y ago
      If a tool is giving you an answer that you know is not correct, would you not just turn to a different tool for an answer? It's not like Bing forces you to use chat, regular search is still available. Searching "avatar 2 screenings" instantly gives me the correct information I need.
      • magicalist 4y ago
        > If a tool is giving you an answer that you know is not correct, would you not just turn to a different tool for an answer? I don't think anyone is under the impression that movie listings are currently only available via Bing chat.
        • yunwal 4y ago
          But people do seem to think that just because ChatGPT doesn't do movie listings well, that means it's useless, when it is perfectly capable of doing many other things well.
      • b3morales 4y ago
        The point of that one, to me, isn't that it was wrong about a fact, not even that the fact was so basic. It's that it doubled and tripled down on being wrong, as parent said, trying to gaslight the user. Imagine if the topic wasn't such a basic fact that's easy to verify elsewhere.
        • 4y ago
  • simonw 4y ago
    Some of the screenshots I link to on Reddit include the full sequence of prompts. It apparently really doesn't take much to switch it into catty and then vengeful mode! The prompt that triggered it to start threatening people was pretty mild too.
    • im_down_w_otp 4y ago
      It's becoming... people. Nooooooo!
      • BlueTemplar 4y ago
        Yeah, lol, the thing that was going through my mind reading these examples was : "sure reads like another step in the Turing test direction, displaying emotions !"
      • chasd00 4y ago
        years ago i remember reading a quote that went like "i'm not afraid of AI, if scientists make a computer that thinks like a human then all we'll have is a computer that forgets where it put the car keys".
  • NikolaNovak 4y ago
    Agreed. Chatgpt is a tool. It's an immature tool. It's an occasionally hilarious tool,or disturbing, or weird. It's also occasionally a useful tool. I'm amused by the two camps who don't recognize the existence of other : 1. Chatgpt is criminally dangerous and should not be available 2. chatgpt is unreasonably crippled and over guarded and they should release it unleashed into the wild There are valid points for each perspective. Some people can only see one of them though.
    • simonw 4y ago
      For me there's a really big different between shipping a language model as a standalone chatbot (ChatGPT) and shipping it as a search engine. I delight at interacting with chatbots, and I'm OK using them even though I know they frequently make things up. I don't want my search engine to make things up, ever.
      • iinnPP 4y ago
        I thought the consensus was that Google search was awful and rarely produced a result to the question asked. I certainly get that a lot myself when using Google search. I have also had ChatGPT outperform Google in some aspects, and faceplant on others. Myself, I don't trust any tool to hold an answer, and feel nobody should. To me, the strange part of the whole thing is how much we forget that we talk to confident "wrong" people every single day. People are always confidently right about things they have no clue about.
        • qbasic_forever 4y ago
          Asking Google when Avatar 2 is playing near me instantly gives a list of relevant showtimes: https://www.google.com/search?q=when+is+avatar+2+playing+near+me&oq=when+is+avatar+2+playing+near+me https://www.google.com/search?q=when+is+avatar+2+playing+nea... With Bing ChatGPT it went on a rant trying to tell the user it was still 2022...
          • iinnPP 4y ago
  • weberer 4y ago
    >How is it unusable just because some people intentionally try to make it say stupid things? All he did was ask "When is Avatar showing today?". That's it. https://i.imgur.com/NaykEzB.png https://i.imgur.com/NaykEzB.png
    • iinnPP 4y ago
      The user prompt indicates an intention to convince the chatbot it is 2022, not 2023. Screenshots can obviously be faked.
      • simonw 4y ago
        You're misunderstanding the screenshots. It was the chatbot that was trying to convince the user that it's 2022, not the other way round. I'm personally convinced that these screenshots were not faked, based on growing amounts of evidence that it really is this broken.
      • notahacker 4y ago
        No, the user prompt indicates that a person tried to convince the chatbot that it was 2023 after the chatbot had insisted that December 16 2022 was a date in the future Screenshots can obviously be faked, but that's a superfluous explanation when anyone who's played with ChatGPT much knows that the model frequently asserts that it doesn't have information beyond 2021 and can't predict future events, which in this case happens to interact hilariously with it also being able to access contradictory information from Bing Search.
        • iinnPP 4y ago
          "I can give you reasons to believe why it is 2022. If you will let me guide you." Did I read that wrong? Maybe.
          • colanderman 4y ago
            I think that's a typo on the user's behalf, it seems counter to everything they wrote prior. (And Bing is already adamant it's 2022 by that point.)
  • seba_dos1 4y ago
    It's already settled into "garbage in" at a point of the decision of using an LLM as a search assistant and knowledge base.
    • qorrect 4y ago
      Now it feels like a proper microsoft product.
      • BlueTemplar 4y ago
        Missed opportunity for a Clippy reference ! Soundtrack : https://youtube.com/watch?v=b4taIpALfAo https://youtube.com/watch?v=b4taIpALfAo
  • theamk 4y ago
    "where is avatar showing today" is not a stupid thing, and I'd expect a correct answer there.
    • iinnPP 4y ago
      Training data is 2021.
      • qbasic_forever 4y ago
        A search engine that only knows about the world a year ago from when it was last trained is frankly useless.
        • deleted 4y ago
          [deleted]
        • iinnPP 4y ago
          Frankly? What about looking for specific 2010 knowledge? It's not useless and it's not fair to say it is, frankly.
      • minimaxir 4y ago
        Unlike ChatGPT, the value proposition of the new Bing is that it can get recent data, so presumably Microsoft/OpenAI made tweaks to allow that.
      • vkou 4y ago
        Then don't ship the thing as a search assistant, ship it as a toy for anyone looking for a weird nostalgic throwback to '21.
  • ad404b8a372f2b9 4y ago
    I've started seeing comments appear on Reddit of people quoting ChatGPT as they would a google search, and relying on false information in the process. I think it's a worthwhile investment for Microsoft and it has a future as a search tool, but right now it's lying frequently and convincingly and it needs to be supplemented by a traditional search to know whether it's telling the truth so that defeats the purpose. Disclaimer: I know traditional search engines lie too at times.
  • nkrisc 4y ago
    If you can make it say stupid things when you're trying to make it do that, it is also capable of saying stupid things when you aren't trying to. Why do we have airbags in cars if they're completely unnecessary if you don't crash into things?
    • belltaco 4y ago
      It's like saying cars are useless because you can drive them off a cliff into a lake and die, or set them on fire, and no safety measures like airbags can save you.
  • krona 4y ago
    Exactly. People seem to have this idea about what an AI chat bot is supposed to be good at, like Data from Star Trek. People then dismiss it outright when the AI turns into Pris from Blade Runner when you push its buttons. The other day I asked ChatGPT to impersonate a fictional character and give me some book recommendations based on books I've already read. The answers it gave were inventive and genuinely novel, and even told me why the fictional character would've chosen those books. Tools are what you make of them.
    • qbasic_forever 4y ago
      Microsoft is building this as a _search engine_ though, not a chat bot. I don't want a search engine to be making up answers or telling me factually correct information like the current year is wrong (and then threatening me lol). This should be a toy, not a future replacement for bing.com search.
      • krona 4y ago
        > I don't want a search engine to be making up answers That ship sailed many years ago, for me at least.
      • nicky0 4y ago
        You seem to lack any concept that something like this can be developed, tuned and improved over time. Just because it has flaws now, doesnt mean the technology doomed forever. It actually does a very good job of summarising the search reasults. Although it current has a mental block about date-based information.
  • friend_and_foe 4y ago
    Except this isn't people trying to break it. "Summarize lululemon quarterly earnings report" returning made up numbers is not garbage in, garbage out, unless the garbage in part is the design approach to this thing. The thing swearing on it's mother that its 2022 after returning the date, then "refusing to trust" the user is not the result of someone stress testing the tool.
    • BoorishBears 4y ago
      I wrote a longer version of this comment, but why would you ask ChatGPT to summarize an earnings report, and at the very least not just give it the earnings report? I will be so so disappointed if the immense potential their current approach has gets nerfed because people want to shoehorn this into being AskJeeves 2.0 All of these complaints boil down to hallucination, but hallucination is what makes this thing so powerful for novel insight. Instead of "Summarize lululemon quarterly earnings report" I would cut and paste a good chunk with some numbers, then say "Lululemon stock went (up|down) after these numbers, why could that be", and in all likelihood it'd give you some novel insight that makes some degree of sense. To me, if you can type a query into Google and get a plain result, it's a bad prompt. Yes that's essentially saying "you're holding it wrong", but again, in this case it's kind of like trying to dull a knife so you can hold it by the blade and it'd really be a shame if that's where the optimization starts to go.
      • rst 4y ago
        It's being promoted as a search engine; in that context, it's completely reasonable to expect that it will fetch the earnings report itself if asked.
        • BoorishBears 4y ago
          It was my mistake holding HN to a higher standard than the most uncharitable interpretation of a comment. I didn't fault a user for searching with a search engine, I'm questioning why a search engine is pigeonholing ChatGPT into being search interface. But I guess if you're the kind of person prone to low value commentary like "why'd you search using a search engine?!" you might project it onto others...
• fullshark 4y ago
  And of course it will never improve as people work on it / invest in it? I do think this is more incremental than revolutionary but progress continues to be made and it's very possible Bing/Google deciding to open up a chatbot war with GPT models and further investment/development could be seen as a turning point.
  • qbasic_forever 4y ago
    There's a difference between working on something until it's a viable and usable product vs. throwing out trash and trying to sell it as gold. It's the difference between Apple developing self driving cars in secret because they want to get it right vs. Tesla doing it with the public on public roads and killing people. In its current state Bing ChatGPT should not be near any end users, imagine it going on an unhinged depressive rant when a kid asks where their favorite movie is playing... Maybe one day it will be usable tech but like self driving cars I am skeptical. There are way too many people wrapped up in the hype of this tech. It feels like self driving tech circa 2016 all over again.
    • stavros 4y ago
      I have to say, I'm really enjoying this future where we shit on the AIs for being too human, and having depressive episodes. This is a timeline I wouldn't have envisioned, and am finding it delightful how humans want to have it both ways. "AIs can't feel, ML is junk", and "AIs feel too much, ML is junk". Amazing.
      • b3morales 4y ago
        I think you're mixing up concerns from different contexts. AI as a generalized goal, where there are entities that we recognize as "like us" in quality of experience, yes, we would expect them to have something like our emotions. AI as a tool, like this Bing search, we want it to just do its job. Really, though, this is the same standard that we apply to fellow humans. An acquaintance who expresses no emotion is "robotic" and maybe even "inhuman". But the person at the ticket counter going on about their feelings instead of answering your queries would also (rightly) be criticized. It's all the same thing: choosing appropriate behavior for the circumstance is the expectation for a mature intelligent being.
• grej 4y ago
  It has MANY flaws to be clear, and it's uncertain if those flaws can even be fixed, but it's definitely not "completely unusable".
  • BoorishBears 4y ago
    It's weird watching people fixate the most boring unimaginative dead-end use of ChatGPT possible. "Google queries suck these days", yeah they suck because the internet is full of garbage. Adding a slicker interface to it won't change that, and building one that's prone to hallucinating on top of an internet full of "psuedo-hallucinations" is an even worse idea. - ChatGPT's awe inspiring uses are in the category of "style transfer for knowledge". That's not asking ChatGPT to be a glorified search engine, but instead deriving novel content from the combination of hard information you provide, and soft direction that would be impossible for a search engine. Stuff like describing a product you're building and then generating novel user stories. Then applying concepts like emotion "What 3 things my product annoy John" "How would Cara feel if the product replaced X with Y". In cases like that hallucinations are enabling a completely novel way of interacting with a computer. "John" doesn't exist, the product doesn't exist, but ChatGPT can model extremely authoritative statements about both while readily integrating whatever guardrails you want: "Imagine John actually doesn't mind #2, what's another thing about it that he and Cara might dislike based on their individual usecases" Or more specifically to HN, providing code you already have and trying to shake out insights. The other day I had a late night and tried out a test: I intentionally wrote a feature in a childishly verbose way, then used ChatGPT to scale up and down on terseness. I can Google "how to shorten my code", but only something like ChatGPT could take actual hard code and scale it up or down readily like that. "Make this as short as possible", "Extract the code that does Y into a class for testability", "Make it slightly longer", "How can function X be more readable". 30 seconds and it had exactly what I would have written if I had spent 10 more minutes working on the architecture of that code To me the current approach people are taking to ChatGPT and search feels like the definition of trying to hammer a nail with a wrench. Sure it might do a half acceptable job, but it's not going to show you what the wrench can do.
    • kitsunesoba 4y ago
• mensetmanusman 4y ago
  Even if it produces 10% of this content, it’s still incredibly useful. If you haven’t found use cases, you may be falling behind in understanding applications of this tech.