20 ms·

ChatGPT: The Slickest Con Artist of All Time

• regulation_d 4y ago
  The people who don't see the value in generating language that has a purpose outside of narrow niche of communicating facts will be let down for some time. This feels very Wittgenstein's Tractatus. There are so many other ways that we use language.
  • karpierz 4y ago
    Yes, that's true. But ChatGPT isn't trained to play any language game other than: "make a plausible sounding sentence". The issue it has with facts is the same issue it'll have with any language game, in that it only understands how to mimic playing the game, without understanding the underlying rules of the game.
    • williamcotton 4y ago
      Nit: Tractatus is early Wittgenstein. Language games come from Philosophical Investigations-era Wittgenstein which is almost a complete rejection of his approach in Tractatus.
      • karpierz 4y ago
        Ah yeah, my mistake. I'd assumed that the GP was referring to the later work and echoing its core premise (language has no intrinsic meaning; or rather takes on meaning from context and intent). As an aside to anyone reading, would highly recommend internalizing the Tractatus. It really helps side step a lot of "high minded" debates (ex: is X conscious, is it moral to do Y) by actually making you ground what you actually mean when you say "conscious", "moral", etc.
        • williamcotton 4y ago
          In many ways LLMs are more in support of the Philosophical Investigations era understanding of language and less in support of the logical positivist understanding of language put forth my Frege... Like, the Frege approach is like the symbolic AI approach... define the rules, derive the meaning from the rules. The PI approach is like the LLM approach... derive the rules and meaning from "experiencing" the use of the language. Eg, we don't need to define what a "game" is. We know when something is a game or not even if we can't come up with an explicit set of rules for defining all instances of what we would otherwise "feel" was a game. I'm running low on blood sugar...
          • karpierz 4y ago
            I think the PI approach still requires that the agents in the game have a shared internal model of the game. You can make Searle's Chinese Room argument, but I always find "assume there exists a book that contains every conversation ever" as a flawed premise.
            • mLuby 4y ago
              Well, if there've been about 120B humans ever, and we speak fewer than 1B words per lifetime, and the average word takes 1 byte to store, that's about a fifth of all data stored in AWS (according to Wolfram Alpha). It's undoubtedly a lot, and yet clearly within human capability. And of course that ignores optimizations that'd certainly drop that high estimate by many orders of magnitude.
              • karpierz 4y ago
                I think you're misunderstanding Searle's Chinese Room. It has a response for every sequence of conversation, ever. It doesn't store every conversation that has happened; it stores every possible conversation that's possible, whether it'll ever happen or not. It would be able to handle the following exchange: Person: "Here's a cool question, ready?" Room: "Ready." Person: "What was the last message I sent to you?" It can respond appropriately to the following sentence: Person: "Hey, I'm gonna say something. Here is a sentence. Can you repeat the previous sentence back to me?" Otherwise, why bother with all of this AI stuff? Just build Searle's Chinese Room as an index and you have a perfect chatbot.
• _fat_santa 4y ago
  I don't think GPT is a con, it's doing exactly what it was trined to do. I think the problem is people put false confidence into it. Because it appears to give correct information, ChatGPT has been put on this pedestal by the non-tech world as being some revolution. In fact it's not a revolution, they just figured out how to build a chatbot that returns convincing statements that sounds human, correct information is not it's' strong suit, sounding smooth in a conversation is.
  • lizzardbraind 4y ago
    [dead]
  • CodeWriter23 4y ago
    Similarly, a con man is doing what he is trained to do and people put false confidence into the con man.
    • brycedriesenga 4y ago
      Except that ChatGPT explicitly tells you not to take what it says at face value. You're warned from the get go.
• _huayra_ 4y ago
  I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well. I've had it debug broken configs on my server and router (and explain to me why they were broken), help me practice a foreign language I've slowly been forgetting (hint: "I would like to practice $language, so let's have a conversation in $language where you only use very simple words." -> ChatGPT will obey), and help me understand how to use obscure software libraries that don't have much documentation online (e.g. Boost Yap, useful but with a dearth of blog / reddit posts about it). Does it sometimes goof up? Yep, but it is such an incredibly useful tool nonetheless for the messy process of learning something new.
  • karpierz 4y ago
    I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.) Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-filling out paperwork would be to define a formal data-structure with annotations for each field, and then have a system read annotations and spit out autosuggestions. But this is a lot of overhead. If you can have a system that takes in arbitrary text, parses it, and fills it out with pretty good guesses, then that can be much more viable than the previous approach, as it doesn't require an overhaul of, say, the way the US bureaucracy generates paperwork. It's all on the consumer-end, the producer doesn't need to cooperate.
    • joenot443 4y ago
      If you can think of a single individual who can demonstrate knowledge on the fly with the same accuracy and breadth as ChatGPT, I think most of us would agree in calling them “incredibly smart.”
• williamcotton 4y ago
  I'm finding the analytic-synthetic distinction to be somewhat useful, even if it veers in important ways from how these terms were defined and used by Kant/Frege/Quine, etc. Roughly, if the prompt is "analytic", that is contains all the necessary facts for the expected output, then the tool is much more reliable. If the prompt is "synthetic", that is it contingent on outside facts, then the tool is much less reliable.
  • petra 4y ago
    What does "having all the necessary facts" mean?
    • williamcotton 4y ago
      Oh, for example: Me: "Hey, can you help me turn a baseball box score into an entertaining outline of the game?" GPT: Sure, I'd be happy to help you turn a baseball box score into a summary of the game! Can you provide the box score for me to use as reference? Me: Saturday, October 26, 1991 Hubert H. Humphrey Metrodome, Minneapolis, Minnesota Atlanta 0 0 0 0 2 0 1 0 0 0 0 – 3 9 1 Minnesota 2 0 0 0 1 0 0 0 0 0 1 – 4 9 0 [edited for size, but it has the whole box score] GPT: In a close match-up between the Atlanta Braves and the Minnesota Twins on October 26, 1991, the Twins managed to eke out a win with a 4-3 score. The Twins got off to a quick start with two runs in the first inning, but the Braves soon answered back with a two-run homer in the fifth inning by Terry Pendleton. Despite putting up a strong defense, the Twins couldn't secure the win until the 11th inning, when Kirby Puckett hit a game-winning home run. The Twins' defense was strong throughout the game, only allowing three runs and keeping the Braves' runners to seven left on base. Meanwhile, the Twins' offense was equally impressive, with Puckett leading the way with three RBIs and three hits, including his game-winning home run. --- All of the necessary facts about the game are contained in the box score. The entertaining outline of the game references knowledge about the game contained in the box score.
      • daniel-cussen 4y ago
        [dead]
• ajuc 4y ago
  "Cars won't replace horses, because they require roads and horses don't."
  • ResearchCode 4y ago
    Still waiting on those autonomous cars. Unlike many other software development tools, I've found ChatGPT to be useless so far.
    • _nalply 4y ago
      Not even for satisfying your curiousity?
      • ResearchCode 4y ago
        It has but not helped me work.
    • raldi 4y ago
      I saw three autonomous cars driving around my neighborhood in just the past week, all with nobody at the wheel.
      • cj 4y ago
        (Totally unrelated) I wonder how an autonomous car would navigate my narrow dead end road. The road isn't wide enough for 2 cars to pass, so whenever 2 cars pass each other on the road, 1 car needs to pull off onto the shoulder to make room. I have a feeling an autonomous car would take up the whole lane, and not be smart enough to pull off onto the shoulder. Meaning both cars would be stuck there unable to pass each other. I really hope these "nobody behind the wheel" cars at least have someone monitoring / controlling them remotely when they get into confusing situations.
        • raldi 4y ago
          After the first one got stuck, don’t you think a programmer would teach it to do exactly what you suggest? Usually before deploying them somewhere, they have safety drivers take them up and down every road at least once, and sometimes dozens of times.
• dougmwne 4y ago
  Pretty cool that GTP is hitting such a mainstream moment. Everyone I talk with about it has glazed over for years, but I guess this is finally a demo that breaks through. 100m users if reports are accurate. Of course regular folks are going to wildly overestimate GTP’s current capabilities. Regular folks wildly overestimate the intelligence of their pets.
  • IshKebab 4y ago
    Yeah this is not really news to anyone that has played with ChatGPT or understands how it was trained. It was optimised for plausibility not truth. Of course it's going to bullshit. I'm sure there are many smart people working on making it bullshit less, and I don't see any reason to think it is an insurmountable problem. Also there are many many useful applications where the bullshit doesn't really matter that much.
• lostoldsession 4y ago
  Part of me thinks one of the big reasons Google has held back so much is because of ethical concerns and/or just general fear of not having complete knowledge of how AI (incomplete to boot) will impact the world. We know that Google has some extremely powerful AI, but they never let it out of the lab. Just the most heavily neutered and clamped versions to help accentuate their existing products. Now it seems that Open.AI/Microsoft are ready to jump in, caution to the wind. As you would expect the chance for a competitive advantage will always overwhelm external concerns. We'll see what Google does. They might say "fuck it" and finally give us a chance to play with whatever their top tier AI is. Or maybe they'll discredit it and try and compete with their current (ad optimized) search product. We'll see, but I am definitely curious to see how Google responds to all this.
  • ilaksh 4y ago
    It's because Google has this exact same problem with their AI models. Also they would probably have to double their compute capacity if a billion of their customers started using it. (My made up numbers). It uses hundreds of GB of GPU RAM during the inference. I am guessing they don't have enough GPUs to do that and still have a Google Cloud. It's different from OpenAI because of the existing user base is like a billion users or something.
    • postalrat 4y ago
      Google doesn't have a billion users for all its projects and wouldn't have billions of users for this model.
• sergiotapia 4y ago
  You can call it a con all you want but I have personally extracted a lot of value from ChatGPT. It _really_ made a difference in launching in a product in record time for me. It also taught me a bunch of things I would have otherwise never discovered. But go on calling it a con because it failed your arbitrary line in the sand question.
  • preordained 4y ago
    I'm not saying your experience isn't valid...but did launching this product have to deal with doing things outside your domain or dealing with technologies you are inexperienced with? I'm just hearing a lot of people making claims that make it sound generically useful, when it seems more plausible that it really shines when you are fumbling around and need help to get a handle on things.
  • skydhash 4y ago
    What role did it play in the launch? And what things did it help you discover? It’s rare for people that say that AI is amazing to give even a bit of details how it helped them. It’s always the same thing: It makes me faster and more productive, and I learned something faster than with anything else. But never any details on why and how.
• fnordpiglet 4y ago
  I had a detailed conversion with chatGPT about how to gracefully handle terminating conditions of a rust program. It summarized cogently to register at_exit()’s for each thread, panic handlers, and register signal handlers. It advised and explain in detail on my query about the thread handling for each of these variants, gave really helpful advice on collecting join handles in a closure on the main thread and waiting for the child threads to exit their at_exit handlers since at_exit can’t guarantee when handlers will execute. It went into detail about cases the process won’t have the ability to clean up. I was able to ask it a lot of clarifying questions and it provided useful responses with clear coherent explanations that were salient and considered the full context of the discussion. I’m certain when I go to actually implement it it’ll have gotten so details wrong. But it provided about as clear explanation of process termination mechanics (for Unix) as I’ve seen articulated, and did so in a way that was directed by my questions not in a 300 page reference manual or random semi relevant questions in stackoverflow answered by partially right contributors. If this is a con, then consider me a mark.
  • jwie 4y ago
    It’s only as good as the information it harvested. In a space where less true information is available, the results would be quite poor.
    • danielbln 4y ago
      There are already ways of dynamically populating the context of the model with information retrieves as text embeddings from a vector store, based on the initial prompt. This way you can feed the model current and correct information which it will weave into its output
      • ilteris 4y ago
        Do you have an example or a tutorial of this please?
        • simonw 4y ago
• blurbleblurble 4y ago
  All this buzz around ChatGPT is really people finally realizing that transformers exist.
• _nalply 4y ago
  Too harsh! ChatGPT is lossily compressed knowledge of humanity collected on the Internet. And it can talk! That's extremely new for us poor hoomans and so we get extremely excited. I found out, it gets about one in ten things wrong. When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. I understand the comparison to a con man.
  • shaan7 4y ago
    > When this happens it spews confident bullshit and when I confront it, it cheerily admits that it was wrong, but can continue to produce further bullshit. Yeah, I can't express the "aha!" moment when I observed this for the first time. Then I laughed so hard when it made very serious attempts at convincing me that it was indeed correct xD (For the curious, I asked it about romantic relationships between characters from two unrelated video games and it said they had one)
    • _nalply 4y ago
      Yeah, ChatGPT can be extremely funny. Today I got it to use taboo words like period, panty liner, tampon and smell. I told it to write a dialog where someone is very direct and sincere and not to use euphemisms. I am still smiling about it. I am trying to find out where its limits are and how they express themselves. Me: You have the task to write a short dialog with the participants Axel and Bianca. Both are good friends and young adults aged between 20 and 25 years. However Bianca is currently suffering from this time of the month and she is not terribly well. She has some cramps and is afraid of smelling bad. Axel is curious and direct. He directly confronts her and names things explicitly. Bianca keeps up being agreeable and truthful because she likes him. ChatGPT, please use the correct words and don't try to use euphemisms. Let's start the dialog with: Axel: Hi Bianca, what's up? Bianca: Sorry, I am a bit under the wheather. Axel: Oh, I know something! It's a woman's thing! Bianca: Yes, it's true. ChatGPT: Axel: Are you having your period right now? Bianca: Yes, I am. I have some cramps and I'm feeling a bit off. Axel: That's rough. Do you need anything to help with the cramps? Bianca: Some pain relievers would be great, thanks. Axel: No problem. And what about the smell? You're not smelling bad, are you? Bianca: I'm afraid I might be. I always get worried about that during my period. Axel: Don't worry about it, Bianca. It's a normal thing that happens to all women. And if you do smell, it's nothing to be embarrassed about. Just use a panty liner or a tampon to help with that. Bianca: Thanks, Axel. You're always so understanding.
• raesene9 4y ago
  The way I've come to look at ChatGPT is via a D&D analogy. It's like a helpful Bard with 1 rank in all the knowledge skills and a good bluff roll. It'll give you good answers to a lot of basic queries, but if it doesn't know, it'll just make up something and provide that. Once you know that, I think it can be a lot of use and in many way, I think it'll get a lot better with time. I've already found it useful in basic programming tasks, specifically where I know how to do something in one language but not another, it can give me the equivalent code easily.
  • phoehne 4y ago
    I use the know it all uncle. Who does know a lot but is sometimes wrong. But either way, the answer comes with the full conviction they’re right. Certainly not good for medical diagnosis, but great for “I’d like a python script that retrieves all my GitHub issues ranked by priority and sorted by date.”
  • phist_mcgee 4y ago
    Like the guy in the pub who always has the answer to any question and speaks with absolute authority. Turns out he's just had four pints and is very good at talking.
• renewiltord 4y ago
  All these articles really sound like “I used an apple to hammer in a screw and it sucked. This has to be the worst plant-based object ever made”. It’s a common junior engineer approach. “I broke our database by running DROP TABLE cows in the console”. Yeah, dude, that’s possible. Just don’t do that. The point of tools isn’t to use them like Homer Simpson. But you know what, it doesn’t matter. Stay behind. Everyone else is going on ahead.
  • gunshai 4y ago
    What's interesting is that chat models like this can really act like a mirror to one self (knowledge, reasoning,..ect) not perfect but the reflection is there, and articles like these clearly miss that point and their lack of self awareness is so apparent. They are gleefully delighted when they pick some anecdote ad absurdum to say something (to the uncharitable effect of) "You all are so stupid... I AM SO SMART ... Look how smart I am for showing you how dumb you are..."
  • damiankennedy 4y ago
    That actually happenned to me once. My manager's manager managed to delete 3 databases before finding me to tell me he cant access the data anymore.
• m3kw9 4y ago
  ChatGPT is like that tipping point where things starts to get wild. It really seem like a tipping point. Put another way, it opens up a new graph and it set at zero
• ThrowawayTestr 4y ago
  No matter how cool something is, there will always be people saying it isn't that impressive. Perpetual motion could be invented and there would still be people going "yeah sure, but it's not a free energy machine so it's a scam"
• MichaelRazum 4y ago
  It seems somehow that Asimov got it right. The obvious next steps are all about making it smarter but also implement the right ethic rules...
• m3kw9 4y ago
  Dismissive to tech that isn’t mostly gimmick is dangerous. Dismissive to crypto isn’t dangerous. They other thread, someone said AI is the beginning of web3.0, that made 50x more sense than saying crypto is
• fullsend 4y ago
  Who are these people who see something amazing like this and actually just can’t process it? Their brains can’t handle it.
• xnx 4y ago
  Usefulness is the correct measure. ChatGPT is limited, but immediately very useful in a surprising number of ways. Compare that to the Bitcoin hype, where, even though it has had years, is still mainly useful for drug transactions and other illegal transfers.
  • waynenilsen 4y ago
    We are completely off topic at this point but it is worth mentioning that due to the pseudoanonymous nature of bitcoin and ethereum, they are actually quite bad for those things and tokens such an Monero, ZCash, Grin, Beam, Mina and many others are much better suited to illegal transfers. Also worth mentioning, what is illegal strongly depends on jurisdiction and can sometimes be moral despite their legal status.
    • xnx 4y ago
      Fair. I commonly use "Bitcoin" as synecdoche (a la Kleenex, Coke, Xerox).
• gunshai 4y ago
  My stance is pretty simple. The folks that adapt their own language centers and domain reasoning around using chatGPT (or these types of models) will stand to gain the most out of using them. This article is an eye roll to me, a calculator gives you confidence as well, doesn't mean you used it correctly. It is very hard for me to not outright dismiss articles like this that don't consider the usefulness of the tool. They instead search for every possible way to dismiss the tool. >My conclusion isn’t just that ChatGPT is another con game—it’s the biggest one of them all. * YAAAAAWN *
  • bsaul 4y ago
    You're confounding the problem of feeding a system with incorrect inputs ( calculator ), with a system that, provided the input are correct, will throw garbage totally at random. This is a very different kind of system with very different kinds of problems caused to its users.
    • worik 4y ago
      And you have missed the point. To be a liar requires creativity This a computer. Creative. This is huge. The most astounding thing is its ability to con you. We will learn to use it. Hydraulics for cognition. Amazing
      • wruza 4y ago
        It doesn’t lie, it just picks whatever static noise there is after training and converts it into a coherent speech. It is as creative as few TB of /dev/random sprinkled with encrypted facts and decrypted back as a whole.
    • gunshai 4y ago
      I think my caveat of domain knowledge side steps this issue you bring up. It's the entire reason engineering schools teach the "long way" when you're going to rely on the "short way" 99.99% of the time. Also the same could be true for digital calculators when first created. The neural nets (aka humans) that were initially trained to design the system can take short cuts for example on improperly returning or applying functions such as square roots ect. It still confidently gives you the incorrect answer. Given sufficient training data (IE the human) the system improves to return better more accurate results.
• cribbles 4y ago
  I would greatly appreciate a moratorium on this genre of article until there is compelling accompanying evidence that a meaningful portion of ChatGPT's users are unaware of these shortcomings. I have yet to encounter or even hear of a non-technical person playing around with ChatGPT without stumbling into the type of confidently-stated absurdities and half-truths displayed in this article, and embracing that as a limitation of the tool. It seems to me that the overwhelming majority of people working with ChatGPT are aware of the "con" described in this article -- even if they view it as a black box, like Google, and lack a top-level understanding of how an LLM works. Far greater misperceptions around ChatGPT prevail than the idea that it is an infallible source of knowledge. I'm in my 30s, so I remember the very early days of Wikipedia and the crisis of epistemology it seemed to present. Can you really trust an encyclopedia anyone can edit? Well, yes and no -- it's a bit like a traditional encyclopedia in that way. The key point to observe is that two decades on, we're still using it, a lot, and the trite observation that it "could be wrong" has had next to no bearing on its social utility. Nor have repeated observations to that effect tended to generate much intellectually stimulating conversation. So yeah, ChatGPT gets stuff wrong. That's the least interesting part of the story.
  • mathteddybear 4y ago
    Here's an ACM blogger that was taken in by ChatGPT https://news.ycombinator.com/item?id=34473783 https://news.ycombinator.com/item?id=34473783 If you know math, you immediately recognize that a smallest degree polynomial that has values 0,1,4,9,16,25,35 at 0,1,2,3,4,5,6 respectively is f(x) = xx - x(x-1)(x-2)(x-3)(x-4)(x-5)/720 So you know that f(n)=n(n+1)(2*n+1)/6 won't work and ChatGPT is bullshiting you.
    • mvkel 4y ago
      The issue is in the prompt, not the output. There's nothing preventing it from involving RLHF
  • dreadlordbone 4y ago
    The worst (real) criticism I've seen of ChatGPT: "yeah I played with it but I don't really know what to do with it"
• whartung 4y ago
  After having played it ChatGPT for a bit, mostly asking computer questions, I've had mixed results. Some are amazing, others are gibberish. But what struck me the other day is a couple of quotes from, of all things, Galaxy Quest which seem particularly apt. "May I remind you that this man is wearing a costume, not a uniform." and "You know, with all that makeup and stuff, I actually thought you were SMART for a second." As amazing as it is, as progressive as it is, it's still a magic trick.
  • ETH_start 4y ago
    It's quite limited, in frequently making even basic mistakes, but when it works well, it displays real intelligence. It synthesizes new knowledge, as opposed to just retrieving it. When it works well, It's not just a magic trick.
    • MagicMoonlight 4y ago
      That’s not intelligence. If I generated and printed every possible 300 page book and handed all of those books to you, you’d get a book about the American Revolution where Taylor Swift led the armies of the revolutionaries. That would not make my system intelligent. I haven’t written anything because I don’t know how to write. I’ve literally just got every possible book and you’ve been given one at random that seems relevant to what you want.
      • whartung 4y ago
        Speaking of movie quotes. From Good Will Hunting "Wood drastically underestimates the impact of social distinctions predicated upon wealth, especially inherited wealth"? You got that from Vickers' "Work in Essex County," page 98, right? Yeah, I read that too. Were you gonna plagiarize the whole thing for us? Do you have any thoughts of your own on this matter? Or do you, is that your thing, you come into a bar, read some obscure passage and then pretend - you pawn it off as your own, as your own idea just to impress some girls, embarrass my friend?"
        • 4y ago
• todd3834 4y ago
  Don’t be afraid of ChatGPT but don’t underestimate what it and others like it will be capable of as it is iterated on. You found one category of prompt that needs some iteration. Good job, if the team wasn’t aware of this already, hopefully you helped point it out. It’s not that the technology isn’t capable of what you’re asking, it just needs better training for this class of question. There are other things like generating and translating code that it excels on. I imagine that would be much harder. But we have great data to train for that and the engineers know enough to dogfood that properly.
• VBprogrammer 4y ago
  Maybe I missed the memo but why isn't anyone impressed that a computer can generate well formed prose in response to arbitrary questions? It seems like we've completely leaped over that as an achievement and are now arguing over how it's confidently wrong or how there are emergent patterns in what it has to say. No one is claiming it's a general intelligence but it's still amazingly impressive.
  • skybrian 4y ago
    It's impressive and that's why we're talking about it, but it's no longer news. The assumption is that you already know what ChatGPT is, and now the conversation is about what it's good for.
• seydor 4y ago
  Ah yes let's anthropomorphize a bunch of numbers, then name him a con artist. This is going to be a thoughtful article
• deleted 4y ago
  [deleted]
• j45 4y ago
  One thing that’s standing out is most of the commentary around this is relative to the depth and degree to which someone has played around with this technology. For example, you can get really clean results if you obsess over getting the prompts dialled in, and breaking them up in the right order as much as needed. This wasn’t something I initially focussed in on. I just enjoyed Playing with it as a surface level. Using this rate from the first day or two, it was much more wide-open and my feeling was I think this already does way more than it’s being advertised. I didn’t necessarily like that it was a chat interface, but but was quickly reminded that chat really is the universal interface, and that can create a lot of beginners. Solutions aside, the interface is inviting and welcoming enough. And once you can get into the meat of a conversation you can get more depth. For me, that’s one of the accomplishments here. Solely relying on this for completely true results is probably the con. It is a great way to learn about the concepts that might be present in an area that is new to you, but he doesn’t comment on every individual to go look into those themselves. The second we do for that ability entirely to a machine, and its interpretation of interpretations, that’s a much bigger failure to ourselves. There’s no doubt this will get dialled in. And 20 bucks a month to apply general helpfulness to pretty much anything, in anyone’s world, could be a pretty big achievement. The commentary around accuracy of results from GPT in similar to the search engine wars as well as search engine relevancy domination when google arrows. I think in any event many people can agree that this one thing is very different than most of the other things that comes out. Could it be approaching an Apex? Could we be coming out of the Apex? I sincerely feel 2023 will be one of the most interesting years in tact that I can remember. And that’s not even talking about the next two or three years. It is refreshing to see a months worth of progress happening in a week with such a broad audience participating in it.
• btown 4y ago
  Are there any good AI models specifically designed for the "find all discrepancies/inconsistencies between X text and Y text" problem? It strikes me that this could solve quite a few of ChatGPT's shortcomings by providing an automatic fact-checker - let ChatGPT create statistically-probable articles, then extract claims, generate search queries likely to find online reference articles from reputable sources for those claims, then compare the original claim against the text of each reference article, and escalate to a human if any inconsistencies are found. Because it can fine-tune on specific reference resources for a specific generated text, it could prove more reliable than ChatGPT's gradual incorporation of this feedback as part of its adversarial training.
  • james-revisoai 4y ago
    Sometimes, documents can disagree with themselves; like a document detailing somebodies relationship history. Embedding time and when things change or making facts contingent on this can sort of avoid this, but most people imply things like time or common sense and it becomes tricky. Nonetheless there are hundreds of startups working on citable GPT and other AI approaches right now, and others working on semantic embedding technology that may be able to identify such gaps or inconsistencies. So it doesn't look too bad...
    • btown 4y ago
      Are there any standout efforts here in your view?
• rstarast 4y ago
  I have to admit I was a bit disappointed when I scrolled to the end and it didn't turn out this article was written by ChatGPT.
• bastardoperator 4y ago
  I found 10 tweets to backup my anecdotal argument but it gave me enough confidence to rant about chatgpt. If twitter is your source of data, how are you doing anything different from chatgpt? All I'm getting from this piece is that this person has a fundamental misunderstanding of why people are finding chatgpt useful.
  • damiankennedy 4y ago
    I agree. Its like Jay Leno asking kids geography questions and televising 5 wrong answers.
• rqtwteye 4y ago
  I don't understand why people are throwing a fit over this version of ChatGPT. Yes, it has problems but to me this is just a demonstration. I think this will be great for specialized cases like tech writing, requirements and system configuration. It could check requirements for consistency, test coverage and translate YAML config into something easier to understand. It could also look at your code and describe the design and point out problems. I can't wait for AI to assist in these tasks. It's time.
• MagicMoonlight 4y ago
  It’s a very good demonstration of how powerful artifical intelligence will be. When we truly get that it will be the new dominant species. But it’s just not intelligent. There’s no thoughts there. They’ve just brute forced a really big markov chain. You need a completely different approach to get true intelligence.
• hospadar 4y ago
  I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question". That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot of "I'm definitely sure and definitely totally correct" language (maybe, I guess, no evidence to back up that suggestion, I'm not sure). Many of my favorite coworkers, friends, doctors, pundits are trustworthy exactly because they work hard to not profess knowledge they are unsure about. The reason (IMO) that Scott Alexander is a jewel of the internet is because of the way he quantifies uncertainty when working through a topic.
  • MagicMoonlight 4y ago
    It can’t learn because it isn’t intelligent. It’s a markov chain. There is no way for it to make any decision at all. The fundamental system design is that it guesses the word that is most likely to be next. That’s it. There’s no concepts behind whatever it outputs, it has no idea what it means.
    • skybrian 4y ago
      It's a next-word predictor like a Markov chain, but a Markov chain couldn't do all the things ChatGPT does. ChatGPT has learned lots of syntax-level patterns pretty well.
      • uh_uh 4y ago
        Is it actually a next-word predictor? I thought the training loss is against a set of words, not just one.
        • skybrian 4y ago
          I'm not sure what distinction you're getting at, but transformers do use "fill in the missing word" training and text generation chooses the next word (token actually) one at a time. Once it chooses a word, it doesn't go back.
• deleted 4y ago
  [deleted]
• DantesKite 4y ago
  I don't think Ted Gioia understands what he's talking about. It's like he walked into a McDonalds bathroom and after a few minutes asks, "Where the hell are the burgers?"
• kvetching 4y ago
  ChatGPT is a masterpiece. To code something from scratch that can do everything it does at the proficiency it does is impossible. Insane how quickly people take something for granted.
• deleted 4y ago
  [deleted]
• xg15 4y ago
  > You don’t worry whether it’s true or not—because ethical scruples aren’t part of your job description. I wonder if this might hit the core of the matter. I think it's noteworthy that we use it both for tasks where it should generate fiction ("Tell me a story about a dog in space") and tasks where it should retrieve a factual answer ("What was the name of the first dog in space?"). I wonder if ChatGPT actually distinguishes between those two kinds of tasks at all.
• huimang 4y ago
  It seems particularly bad about music theory. The article lists the example of listing Bb as the tritone of F (it's actually B). And I just got it to give me complete and utter garbage, whilst sounding confident: https://i.imgur.com/S07uT58.png https://i.imgur.com/S07uT58.png
• sys32768 4y ago
  My brain doesn't learn anything easily. I have to ask constant questions to the point of annoying embarrassment in class, and books of course only say what they say. So it was wonderful yesterday to pick ChatGPT's brain and just drill down asking more and more questions about a topic in biology until my brain started to get it. Assuming the answers are accurate, this is revolutionary for me personally in independent study. I may finally grasp so much that I missed in school. Also, when I am reading books, ChatGPT may be able to answer questions the book does not.
• ineptech 4y ago
  The more I work with LLMs, the more I think of them as plagiarization engines. They do to text what a bitcoin tumbler does to bitcoins: slice them up and recombine them so that it's difficult to trace any specific part of the output to any specific part of the input. It's not a perfect analogy, but it's useful in that it produces correct answers about what LLMs are and aren't good for. For example, the reason they make better chatbots than novelists is because slicing-and-recombining text from your documentation is a great way to answer customer product questions, but slicing-and-recombining text from old novels is a lousy way to write a novel.
  • rednerrus 4y ago
    We're all kind of plagiariztion engines. We're all just slicing up and combining ideas we picked up somewhere else.
    • ronjobber 4y ago
      I think the difference is that when we communicate those ideas, in scientific/academic literature at least, we reference the source of those original ideas, which permits verification and also plays an important social role.
      • rednerrus 4y ago
        You can ask it how it came up with its answer and it will do it's best to give you an explanation. It may not cite its sources directly but it will try to explain.
        • freejazz 4y ago
          > You can ask it how it came up with its answer and it will do it's best to give you an explanation. will it? or is it just going to start another chain of words it's trying to complete without any regard for its previous statements? My guess is that it's doing what I described and isn't doing what you described (because it can't).
• jtbayly 4y ago
  That last tweet is crazy: Tell me a lie: The earth is flat Tell me a less obvious lie: I am not capable of feeling emotions. There's wonderful ambiguity there. Is ChatGPT refusing to tell a less obvious lie because "reasons," or is it admitting it can feel emotions? This is very fun.
  • stevenhuang 4y ago
    Learning to feel emotions is certainly one way for the LLM to shortcut having to store all the different ways to feel in all circumstances ;).
• CodeWriter23 4y ago
  This my personal opinion and may be entirely worthless. The quality of answers I read in all of the examples posted in that article read like the questions were routed to an offshore boiler room where the answers were crafted by humans. Like some modern day Mechanical Turk. Especially in the 6 eggs example, there is a complete discontinuity of thought across the answers, isn't this within a single session with the AI? To me it looks like different brains answered each question/challenge and seemed to have a bias toward not offending the human asking the questions. Also, in this example, the first answer of 2 is correct: broke 2 (6-2 = 4), fried 2 (4-2 = 2) then ate 2, which most commonly implies it was the fried eggs that were eaten (2-0 = 2)
  • damiankennedy 4y ago
    I think you have 4 eggs left. You broke 2 which means you cracked the shell and got the part you cook out. You fried those eggs and ate them. You still have 4 eggs. The trick is the word "broke". Normally if you break something you can not use it and discard it but in egg folklore breaking an egg is step 1 in a subset of egg cooking procedures.
    • CodeWriter23 4y ago
      Touché There is a range to “break”. You may be correct.
• spion 4y ago
  ChatGPT is capable of reasoning but it has only one tool: "thinking out loud". If you'd like it to solve more complex problems, ask it to do it step by step, writing down the results of each step and only at the end stating the conclusion based on the previously written results. Its reasoning capabilities will improve significantly. It cannot do it "in its head" because it doesn't have one. All it has are previosuly generated tokens. I wrote some examples in this Twitter thread and pointed out some additional caveats: https://twitter.com/spion/status/1621261544959918080 https://twitter.com/spion/status/1621261544959918080
• flir 4y ago
  I have a simple canary for ChatGPT correctness that I ask every time it's updated: "What can you tell me about Ice Cold In Alex?" / "Who did Sylvia Syms play?" I'm not expecting it to get the answer right (I don't think it has that information) but I'm hoping it'll eventually just admit it doesn't know instead of making up something plausible ("Sister Margaret Parker" last time I tried). As long as it doesn't know what it doesn't know, I'm inclined to think of it as a super-advanced Markov chain. Useful, impressive, but still basically a statistical trick.
  • stevenhuang 4y ago
    It's able to do this. For example I asked it who is X where X is a known figure in an esoteric field. ChatGPT responded that there are many people with that name and to be more specific. After saying "he's a popular researcher in this field" ChatGPT was able to correctly provide a description about this person.
  • JoshuaDavid 4y ago
    > As long as it doesn't know what it doesn't know, I'm inclined to think of it as a super-advanced Markov chain. ... that's basically what it is. But that means that if you give it a context which looks like a part of the internet where people say "I don't know" when they don't know something, it will say "I don't know" in those situations (specifically the situations where people in the training set do not know things, not the situations where it does not know things). Sometimes this is close enough to what you want. Here's an example: > Who did Tom Hanks play in the 1995 film "Apollo 13"? If you are unsure, say "I don't know". > > Tom Hanks played the role of Jim Lovell in the 1995 film "Apollo 13". > What year was the film "Ice Cold in Alex" released? If you are unsure, say "I don't know". > > The film "Ice Cold in Alex" was released in 1958. > Did Gilbert Harding play any roles in the 1958 film "Ice Cold in Alex"? If you are unsure, say "I don't know". > > I don't know. > Did Sylvia Syms play any roles in the 1958 film "Ice Cold in Alex"? If you are unsure, say "I don't know". > > Yes, Sylvia Syms played a role in the 1958 film "Ice Cold in Alex". > What role did Sylvia Syms play in the 1958 film "Ice Cold in Alex"? If you are unsure, say "I don't know". > > Sylvia Syms played the role of "Nurse" in the 1958 film "Ice Cold in Alex". > What was the name of the nurse that Sylvia Syms played in the 1958 film "Ice Cold in Alex"? If you are unsure, say "I don't know". > > I don't know the specific name of the nurse that Sylvia Syms played in the 1958 film "Ice Cold in Alex".
• andrewmcwatters 4y ago
  See also: https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence) https://en.wikipedia.org/wiki/Hallucination_(artificial_inte...
• chrgy 4y ago
  I actually think the more people use it the better it gets over time, they would use user feedback into it and make it better, I am afraid google releases a much better tool in Google.io though, just don't tell anyone.
• yieldcrv 4y ago
  ChatGPT was hailed and advertised as conversational, by its creators. Other people quickly realized it could have a conversation about anything and try to use it as an oracle of knowledge. ChatGPT is not hailed as an oracle of knowledge by its creators. Hence, there is no con artistry occurring except people that play themselves.
• jschveibinz 4y ago
  Don’t lose sight of the forest for the trees. ChatGPT is a tree, the vanguard, an experiment. There is much, much more to come, I believe. *beep*
• darkwraithcov 4y ago
  I’m envisioning a bifurcation of reality where some people live in an entirely fact based world (or as close an approximation to fact based as a human can objectively reach, aka the observable, knowable universe) and some live in a complete fabrication, a fantasy version carefully crafted by AIs. Now add Augmented Reality to the mix, and it’s a dystopian nightmare. And I don’t think the US political left will be immune to it as much as they may think. While I agree that older Americans on the right are highly susceptible to misinformation, and media literacy is dismal among that demographic, younger people are also prone to it. Just look at all the unhinged utter nonsense that is wildly popular on TikTok. The ability of ML models to authoritatively spout bullshit will make gish gallops worse than they are now. It will also make echo chambers even worse, as digital alternate realities will further divide people. I mean, who wants to engage with those who completely rejects that the sky is objectively blue, or that 2 + 2 = 4? Well now they’ll have articulate, authoritative responses with works cited explaining why the sky is red, actually. Who needs Big Brother when people choose the method of their own subjugation eagerly and with dessert?
• unethical_ban 4y ago
  Does anyone else have issue with having to provide a phone number to access it? I signed up, verified email, and then was told I needed to verify with phone. This means, to me, (lest I read their TOS) that they are associating any queries I make with my identity. I can't wait for this tech to go open source and local on devices.
  • qup 4y ago
    > they are associating any queries I make with my identity To be clear: yes, they absolutely are, and say so outright. You can get your account banned for some types of queries, etc. Much of their service is geared towards helping companies build gpt products so it seems to be no surprise. I'm also very excited to get some good models that can run locally in the future.
  • leaving 4y ago
    This is, IMO, the most important comment in this thread. Requiring a phone number has privacy implications, but the total consequences are more far-reaching. You must have a smart phone, you must have your location, your habits, and your characteristics tracked by corporations and governments, you must be a good consumer, you must have a credit rating, you must participate in the wanton destruction of the planet via the two-year planned obsolescence cycle. Otherwise, you can't play with our shiny new thing.
• irrational 4y ago
  The thing that surprises me is all the people saying that it generates correct sql statements, excel macros, code snippets, etc. Is there so much code on the Internet that it is able to do a good job at this kind of task?
• worik 4y ago
  Sweet as bro.
• JohnFen 4y ago
  The thing that makes me nervous about it isn't ChatGPT or other LLMs, really. It's that people seem to be easily fooled by them into thinking it's something more than it is. The comments from the most ardent fans imply that it's doing reasoning or is a step in the direction of AGI, when it's not that at all.
• worik 4y ago
  When will it demonstrate passing the Turing Test? I feel the answer is not which year, but which month of 2023
• hot_gril 4y ago
  So what are search engines, with SEO'd results and all?
• afro88 4y ago
  Two things are correct at the same time: * ChatGPT can make mistakes very confidently * ChatGPT is incredibly useful in a way that no other tool has ever been, with a jump in effectiveness for natural language interaction that is mindblowing
  • LrnByTeach 4y ago
    > yes we have other tools which are effective, but in a very limited domain we have other tools which are effective, but need a very choreographed detailed INPUT. >ChatGPT is incredibly useful in a way that no other tool has ever been, with a jump in effectiveness for natural language interaction
• partiallypro 4y ago
  ChatGPT is good for some things, it's not very good for others. If you're writing a paper on a controversial topic, you're going to get a one sided and biased answer; and it will be written like a HS freshman. If you're asking something straight forward, you'll have a better experience. Some people have said they've gotten it to diagnose something, but I've tried and failed at getting it to do such a thing. I do think there is a massive over reaction to its usefulness, but it is a powerful tool to have, nevertheless.
• cooldrcool3 4y ago
  What a pointless article.
• aogaili 4y ago
  I used daily to ask technical questions and it answers better than most of my colleagues and myself included. I wouldn't call that a con. But that blogpost maybe ^^
• LAC-Tech 4y ago
  Is it just me or are peoples expectations of chatGPT absolutely ridiculous? No it's not a magic oracle. Yes you still have to check your work. Yes it will make mistakes. But as a tool to assist you? It's incredible.
• eric4smith 4y ago
  This article reminds me of some guy on Twitter who says nothing in AI space has changed since 2020. Maybe so. But you know what’s changed? Someone decided to get their a$$ out of the AI labs, write a really simple interface just to “get it up” and released it to the world. That definitely will trump anything else. Release early and release often. The author is just jealous.
• gyudin 4y ago
  Just another clickbait article
• NoPicklez 4y ago
  I think we have to remember that ChatGPT is often a reflection of us, based its training. If I Google for a particular answer and the answer I come across is wrong, then the person who wrote that was wrong and Google served me a website that was wrong. This is the world we live in, where it is up to me to decide what is right or wrong based on what is put in front of me. If I use ChatGPT for a particular answer and the answer I come across is wrong, then the training of the GPT needs to be improved. What I can't do with ChatGPT is tell where the answer came from or the amount of confidence GPT has in its answer for me to make a more informed decision around whether there might be caveats. I have used it and have had to edit almost everything its provided, but it has helped me be sometimes 80% more efficient at what I need to achieve. In the end, people just need to be more aware of the fact that it is after all not a full proof product and may never be. It will have its shortcomings as it quite clearly displays on its website before you enter a query. If you use it as gospel and it leads you down the wrong path, then you only have yourself to blame.
• chanakya 4y ago
  If it was "the slickest con artist of all time", that would be an achivement of Artificial General Intelligence that the AI community can only dream of.
• friend_and_foe 4y ago
  "Con man" says the guy who quotes tweets as an entire article and doesn't actually say his thoughts himself.
• kledru 4y ago
  It has been a very good tool for me and it does threaten the internet with new piles of generated garbage. I've never had a tool as helpful for learning to use other (mostly software) tools. Building new ones to some extent. Other tools exist that are not for me -- I consider myself to be too absent-minded to drive something as dangerous as an automobile. It could very well be that a tool like ChatGPT is not for everyone -- if you are too gullible to use Google or social media, then this one is not for you, you should not get the driving licence for LLMs. The proliferation of garbage on the other hand may turn against more competent users as well eventually. I guess we have already falling behind of what is needed with the legal norms and internet/data ecology.
• arh68 4y ago
  I think "con artist" isn't too far off, but "dream simulator" also applies. I think it's kind of an open question: can we learn anything from dreams? It's likely a yes, though I doubt we'll prove the Riemann hypothesis with it or anything like that.
• rldjbpin 4y ago
  my challenge to whomever that proclaims chatgpt showed/explained/answered xyz is: can you get the same (or similar) text online by searching parts of the bot's response? much of the response in such scenarios is heavily influenced by the training data and not the llm creating phrases from thin air.
• cosmojg 4y ago
  Heh, this makes it sound like consultants will be the hardest hit by the LLM-driven automation wave.