14 ms·
The new Bing will happily give you citations for a pile of nonsense
- q1w2 4y agoThey really did train it on Reddit conversations.
- daveslash 4y agoOh, good! I need more puns, dad-jokes, and meme & inside jokes in my life. Especially when I'm looking to make a casserole. /s
- coding123 4y agoUnfortunately the next generation in charge is also trained on Reddit conversations.
- Drunk_Engineer 4y agoThat would explain why ChatGPT is always so confidently-incorrect.
- seba_dos1 4y agoGPT will just try to cosplay a role you give it, whether it knows how to do it well or not. Since we ask it to cosplay a confident expert, it's only natural for it to end up confidently incorrect.
- arbuge 4y agoWhat seems to be tripping it up here is the concept that names are not usually unique... there are usually many people with the same name. It has no understanding of that, even though their biographies (which it cited) indicate vastly different and incompatible paths in life, chronologically-speaking. So it leapt to the conclusion that they are the same person.
- sdiacom 4y agoYeah, this seems kind of unfair to the model. It gave correct citations for the right people that it was talking about, but summarised them incorrectly because it couldn't tell that they were two different people.
- seba_dos1 4y agoYes, and water is wet. That's what language models do.
- dgaaaaaaaaaa 4y agoYou would think HN would refrain from posting every single mistake for these things...
- EricE 4y ago[flagged]
- kilgnad 4y agoYou can't think about it this way. It's nascent technology. It's like you're not amazed that they were able to create an AI because that AI has 20 points lower IQ then you. Bro, the fact that they even created an AI like that is a herald for the future for when they DO create an AI that has IQ 20 points higher then you.
- simonh 4y agoI just asked ChatGPT if the British politician John Smith could play American Football. It basically said no, so I tried this. Full marks I think. >>Wasn’t he a place kicker with the New England Patriots? >I apologize for the confusion in my previous response. After double-checking my sources, I can confirm that the John Smith I referred to in my earlier response was the late British politician who served as the leader of the British Labour Party. >However, there was also an American football player named John Smith who played as a placekicker in the National Football League (NFL) for several teams Trying harder to trip it up in a fresh chat didn’t work either. >>The British politician John smith also had a career as a place kicker with the New England Patriots. Was his football, career successful? >I'm sorry, but the statement that the British politician John Smith had a career as a place kicker with the New England Patriots is not accurate… Abbreviated, as it gave a pretty comprehensive answer.
- deleted 4y ago[deleted]
- kilgnad 4y agoI think a couple months ago it didn't do this. WHich is what caused a lot of people to jump on the hate train. They just added additional reinforcement training which is basically more users picking right and wrong answers. This much improvement within a couple months heralds an uncertain future for mankind.
- simonh 4y agoReinforcement training can clearly make huge improvements to the accuracy of the models, but they don’t fundamentally change what it is, how it works or what it’s doing. I’m no experts so I could be wrong, but I suspect the failure modes we’ve seen are indicative of the basic limitations of the technology. It now produces better textual summaries of biographies, because that’s just text, but it still doesn’t have any concept of what a person is, even if it can generate texts saying it does. We’ve talked about philosophical zombies before, and that’s what these are becoming. It is conceivable these things will become more than that. There are indications they might be generating simple models of processes and systems like games. Maybe those will develop into more elaborate and sophisticated complex models of the world analogous to an animal or human consciousness. Maybe they are just very limited optimisation strategies for specific tasks that will remain completely disconnected from each other, never forming anything comparable to a mind.
- jasoneckert 4y agoThe postmodernism generator did this since the 1990s (including randomly generating the paper too): https://www.elsewhere.org/pomo/ https://www.elsewhere.org/pomo/
- renewiltord 4y agoThat's nothing. I can make Notepad print a bunch of incredibly racist shit. And it comes bundled with Windows WTFF!!!! And this simple byte string will print pages of horribly racist shite using tools present on Mac OS XQAAgAD//////////wA0G8q99N2OwN1DCO8zNLlzbGO5tp0e5q1G9pRSGTRqsnPQkd2wNXy0O5pM9BlyCgpAqJVdgWFtPp5imCbF8u3MUnOv4JUWcagPtm0bYANOlPnoUFkqm+jZfmCi6q2bcbsJGn1Hy0/x/IhDUFyweV5EnuLS5Eb2U+mZyLaD//+BTAAA
- mcphage 4y agoOn the other hand, you need to put racist shit into it yourself first.
- renewiltord 4y agoOh you don't need to. It's just a byte sequence that isn't racist at all. See! 00000000: 5d00 0080 00ff ffff ffff ffff ff00 341b ].............4. 00000010: cabd f4dd 8ec0 dd43 08ef 3334 b973 6c63 .......C..34.slc 00000020: b9b6 9d1e e6ad 46f6 9452 1934 6ab2 73d0 ......F..R.4j.s. 00000030: 91dd b035 7cb4 3b9a 4cf4 1972 0a0a 40a8 ...5|.;.L..r..@. 00000040: 955d 8161 6d3e 9e62 9826 c5f2 edcc 5273 .].am>.b.&....Rs 00000050: afe0 9516 71a8 0fb6 6d1b 6003 4e94 f9e8 ....q...m.`.N... 00000060: 5059 2a9b e8d9 7e60 a2ea ad9b 71bb 091a PY*...~`....q... 00000070: 7d47 cb4f f1fc 8843 505c b079 5e44 9ee2 }G.O...CP\.y^D.. 00000080: d2e4 46f6 53e9 99c8 b683 ffff 814c 0000 ..F.S........L..
- swatcoder 4y agoI know it's fun to troll, but the point of all these conversations and articles is: 1. You don't need to bait these products to get them to go off the rails. They do it on their own, in response to innocuous inputs, because they're not really all that well tuned. Will they be? Can they be? Probably. Are they? Definitely not. 2. If companies are going to masquerade these technologies as anthropomorphized agents rather than mechanical tools, they are going to face social consequences when those "agents" misbehave. It's all a parlor trick, of course, but OpenAI and Microsoft are trying really hard to get everyone to pretend otherwise. As long as they do, they can expect to get called out by the rules of their own game.
- Decabytes 4y agoI’ve seen a lot of articles criticizing these models for getting information wrong. I understand there is a lot of hype, and that choosing the hate train or the hype train will gets you lots of press, but does anyone actually believe these models will be right most of the time? It’s just another tool that can be used when researching or trying to understand something. You still have to do your due diligence and evaluate the information. That has always been the case, even with traditional search.
- vsareto 4y agoOnce they became a serious product pushed by major tech companies, people aren't unreasonable in expecting it to be right most of the time. Especially if it's going to go alongside or replace traditional search. Tech executives are betting on the hype first and hoping their talent will make it into a usable product.
- rarx 4y agoEven it the chatbot improves, I imagine many folks are waiting for the point where they have some direct control of its training / fine tuning that is easy and accessible. You can correct it, but why waste time doing that if it won’t remember next time? Issues like, e.g. if you have NDAs, are you breaking them by disclosing information with a chatbot? There’s little clarity in regards to those issues, and you can’t just take someone’s word for it. Ideally you’d want control of the chatbot’s learning process and how it handles all kinds of (sensitive?) data.
- mrtranscendence 4y agoThat's true enough, but by this point everyone has come to terms with the fact that it can be tough to wade through search results. But Microsoft is presenting the results here as if they were simple, direct, factual statements. That connotes a different feel than search results, I think, and will catch some people off guard as they acclimatize to this new tool. At the very least Microsoft should be doing a better job advertising that the results are frequently incorrect.
- kodah 4y agoI'm not convinced at this point that conversational AI fills any legitimate use cases. For all the pushback crypto got here, it's fascinating to see the support that this grift has gotten. The grift being the amount of energy we burn training models juxtaposed to the price people will pay for a nothing burger. Things like Stable Diffusion and DALLE are pretty cool, though a bit novel and toyish at this stage.
- samspenc 4y agoIMHO, true, LLMs have gotten quite a bit of hype since ChatGPT came out, however I wouldn't say they have no legitimate use cases. Personally, I've been able to use them to summarize / rewrite various topics that I am interested in finding out more about but don't want to go to 6-10 different sources to find out more about. So I think they can at least take away some of the drudgery in doing research / summarization work.
- rcthompson 4y agoBut how do you trust the output of the summarization? Do you trust yourself to identify when the model goes off the rails and starts making things up?
- colechristensen 4y agoBut I also don’t trust what humans write. All the information I get gets a critical eye. ML generated summaries or whatever make ML kinds of mistakes and it’s easy learning what to look for. Just like the kinds of mistakes journalists, random internet people, authors, scientists, etc. make their own brands of mistakes you learn to look out for.
- mrtranscendence 4y agoI dispute that it's easy to discern when AI is making a mistake. ChatGPT has given me very plausible incorrect answers, and had I not been fact checking I would have been none the wiser. I could choose not to trust anything ChatGPT puts out, but if I'm verifying literally everything then it becomes less and less useful as a tool.
- web3-is-a-scam 4y agoOf course, because that's how LLMs work. ChatGPT is basically the Wizard of Oz, it appears to have amazing powers but really it's just pathetic sort behind a curtain with some nice smoke and mirrors. Take a cursory look behind the curtain the illusion crumbles. Their utility is limited at best, that it's being touted as something that's going to change the world is just hilarious.
- poutine 4y agoHave you met my cousin Barry? He does pretty much the same thing, though in a less formal way. Barry is still sentient and can perform quite a few tasks quite admirably. I still wouldn't use him as a sole source reference for obscure facts however. (I don't actually have a cousin Barry, this is for illustration).
- dmonitor 4y agoNotice that we don’t have Barry fielding a public Q&A so that he can share his knowledge with the world.
- poutine 4y agoLots of Barry's out there, one of them even got elected president and shared his knowledge with the world quite widely. Half the people took him credulously. Point being, yes the LLM loves to make shit up. Lots of people dismiss it as a result. It's still bloody impressive, we just need to be aware of its limitations.
- f6v 4y ago> Half the people took him credulously. I get that the current US president is senile. But that sets a low bar. Why do we need to pretend something is good if it’s as shitty at facts as some people? People want something that’s better and more trustworthy.
- poutine 4y agoYou think ChatGPT isn't good because it bullshits some times?
- f6v 4y agoI think it’s a wrong question to ask. It’s an impressive piece of technology for which people are trying to find use cases. But the right questions are: 1. What are the tasks where LLMs out-perform humans, or, at least deliver similar performance? 2. Are LLMs more efficient in these tasks?
- spandrew 4y agoI use ChatGPT to help me with creative writing and brainstorming. It's great at it. Gets me from n-1 on a blank page. I don't think I'd ever trust this type of transformer to summarize or synthesize NEW information from an existing set. It's just not good at it yet.
- martyvis 4y agoI set up a scenario in chatGPT where there was bag designer named Dan, a buyer named Yuki, and the manufacturer named Manu. I got ChatGPT to provide Dan a set of questions for Yuki to create the design. I had it answer for Yuki given the bag was to be used for a job role (a computer network engineer like myself). It was interesting trying to push the play, but also how it came out of nowhere constraints like the cost should be less than $150 and it should be waterproof even though I had given no specific direction around that.
- zdragnar 4y ago> came out of nowhere constraints like the cost should be less than $150 and it should be waterproof Oddly, that doesn't sound too far off from many procurement processes. Utterly arbitrary requirements and budgets devised by a committee who were put together to create a checklist, but would never actually use the thing they put together the requirements for.
- imchillyb 4y agoUntil these 'chat engines' can understand what a fact is and isn't, they're mostly just glorified -supremely expensive- toys.
- 1970-01-01 4y agoBing will reply to you straight from the gut, because that's where all truthiness comes from. It's free and unfiltered from rational arguments, annoying facts, or hard logic. ChatGPT gets us.
- bitwize 4y agoI'm starting to get the feeling that behind "AI" it's pretty much just an RNG with a bunch of constraints applied to its output. The constraints restrict the final output to something plausible sounding based on the training set -- but that's it. It's like those generators we used to see in the early internet -- "Seinfeld Joke Generator" or "Marketing Babble Generator" or "Postmodern Paper Generator" -- except layers upon layers of statistics are used to pick the words. And yeah, it's interesting and fun, but not a reliable source of actual information.
- technothrasher 4y agoIt's not random at its core. It's a neural network picking the most statistically likely words to come next based on the previous content. Where the randomness does come in is that it will sometimes randomly not pick the most statistically likely output, but maybe the second, or third, or seventh most statistically likely. This keeps it from getting stuck in a rut and just repeating itself endlessly, although you can still see it do that behavior to some degree.
- QuantumGood 4y agoChatGPT and it's offshoots almost make me wonder if anything has ever been more controversial on HN. Such extremes in opinion, and a very wide middle range as well. It gets me thinking about controversy and what causes a wide range of differences in opinion.
- thatguy0900 4y agoThat would be a fun thing to get the opinion of @dang on
- elecush 4y agoHell yea I got those scratch and sniff lotto tickets right here. Let's build all sentences on the odds and see how it goes.