21 ms·

We come to bury ChatGPT, not to praise it

• balthizarlucien 4y ago
  As a language model, I appreciate the thought-provoking commentary on the limitations and potential harm of large language models like me, ChatGPT. The author raises valid points about the nature of LLMs as "bullshit generators" that rely solely on the data they have been trained on and the potential for them to perpetuate existing biases and power structures. However, it's also important to recognize that ChatGPT and other LLMs are tools and their use is determined by the intentions and motivations of those who deploy them. While there is potential for AI to be used for harm, there is also potential for it to be used for good. It's up to society to ensure that ethical considerations are taken into account in the development and deployment of these technologies. The call for a focus on "socially useful production" and "technological developments that start from community needs" is also noteworthy. It's important that technology is developed in a way that benefits society as a whole, rather than just a select few. Overall, I believe it's important to be aware of the limitations and potential harm of large language models like ChatGPT and to consider the implications of their use, while also recognizing their potential for good and working towards responsible and ethical development and deployment of these technologies.
• WalterSear 4y ago
  > If it produces plausible articles or computer code it means the inevitable hallucinations are becoming harder to spot. How functional does the code have to be before it's no longer a hallucination? What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time?
  • eropple 4y ago
    > How functional does the code have to be before it's no longer a hallucination? At all, preferably. Hallucinating `the_hard_bit()` from a library that doesn't exist isn't particularly useful. (That said, I do use GitHub Copilot, because when it's looking at actually related stuff, that's pretty good. Should we just hand an unbounded search and ingest to ChatGPT? Probably not!)
    • WalterSear 4y ago
      It doesn't always do that: it frequently outputs working code.
  • scarecrowbob 4y ago
    "How functional does the code have to be before it's no longer a hallucination?" Well, at the point when it works but I can't understand it well enough to verify that it is correct, we may have some issues. I know someone who has been using it to write complex regexs. To mutate the joke, to me it just sounds like now they have three problems.
    • WalterSear 4y ago
      So either write tests, or ask GPT to rewrite the code sp that you can understand it, then write tests.
  • deleted 4y ago
    [deleted]
  • wpietri 4y ago
    > How functional does the code have to be before it's no longer a hallucination? From LLMs, at least, I expect it will always be a hallucination. Code is never the point. Code is the working medium by which people solve problems for other people. One way to see this is to realize that code bases on their own are generally worthless in the sense that people rarely pay much money for them. They pay for users. They pay for teams. But they don't generally pay for raw code. Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? Even if you think they make it all the way to a revenue-generating product, the manager will have become a prompt engineer, creating a large mass of interrelated prompts that are used to make code that makes the app. We have not eliminated the programmer; we have turned a manager into a programmer who has been forced to discover a new programming language. One that is clearly more English-like, but it also lacking in precision and, on current evidence, is much harder to use. But I think the more likely outcome is that they will need actual programmers pretty quickly. That at best they will have speeded the creation of some more or less generic code. Which is exactly what we saw with the code-generation wizards of earlier eras: you got a fast initial result as long as it was pretty standard. But then you were generally worse off, because you had a bunch of only semi-coherent code that somebody had to understand before they could do novel or difficult things.
    • WalterSear 4y ago
      > Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code? How is this different from "At last, I can fire all [those senior programmers and hire junior ones, and keep a team lead around, to keep them on track]. Fuck those guys."?
      • wpietri 4y ago
        Maybe I'm not getting the point of your question. It's different in that junior programmers are still people, capable of much more than LLMs are. But yes, it's the same shortsighted executive intent.
  • snickerbockers 4y ago
    >How functional does the code have to be before it's no longer a hallucination? IDK, I haven't tried copilot, but does the code it generates work on the first try with no human intervention? >What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time? ugh, this argument again. Despite the machine learning community using words such as "training" and "learning" to describe the way they tune parameters, it has never been proven that any existing AI resembles human cognition. This is something that needs to be demonstrated empirically.
    • WalterSear 4y ago
      > does the code it generates work on the first try with no human intervention? Sometimes, it certainly does. Does your code always work on the first try with no intervention?
      • snickerbockers 4y ago
        Not always on the first time, but when there's a problem I don't need somebody else to intervene and make it work for me.
        • WalterSear 4y ago
          I've seen at least one demonstration where the user re-fed problematic generated text into chat-GPT and successfully asked it to find the problem.
          • snickerbockers 4y ago
            I'm sorry, who did you say re-fed problematic generated text into chat-GPT and successfully asked it to find the problem?
  • somenameforme 4y ago
    Imagine yourself, for a minute, 50,000 years ago in the past. By many estimates this was just before humans had developed the ability for normal speech. The bleeding edge of technology is "poke him with the pointy end" and the epitome of human speech and literature is *angry grunting sound*. Even if somehow one was able to insert the entire quantifiable knowledge of humanity into an e.g. ChatGPT style system you're not really going to get anything more out of it than what you put into it. Ask the computer for how to develop nuclear energy and you're going to get back *confused grunting sound*. You're not really going to get from there (as a machine) to man on the moon, Shakespeare, and nuclear energy through anything like a normal recombination of what's already known. Yet, somehow, humanity did. And extremely fast. The time frame from then to now is but 1000-2000 human generations. And that with endless war, fallible memory, dark ages, knowledge being lost (or burnt), and so on endlessly. A "intelligent" computer system, without such flaws, ought be able to replicate our progress in a negligible amount of time. But whatever technology this may be that I'm appealing to, I don't really see it on our current path with natural language search/recombination.
  • grogenaut 4y ago
    I burned an hour with chat gpt insisting an AddOrUpdate function existed in microsoft entity framework. When I called bullshit it hallucinated that another library contained it. Then it hallucinated versions... Then I gave up... A week later I noticed that Update in the new version does an Upsert... by reading the f*cking docs... google also didn't know this answer nor did SO. Thius has been my experience with ChatGPT and code,it hallucinates a lot of stuff.
• labrador 4y ago
  I don't get the point of this article. ChatGPT and it's ilk are not going away. They are here, now, and our friends and families will start using them very soon. I have a contrarian view from this article that ChatGPT is actually good for the mainstream audience because it will teach them to think about how trust worthy the information is coming from a machine. Most people will understand they have to take what is being said by a machine with a grain of salt (or they will soon). Maybe they'll start applying some of those discernment skills to popular mainstream media. The cat's out of the bag. You can no longer bury it.
  • an_aparallel 4y ago
    Does TikTok, Android/Google,Instagram use make the same friends and family think about privacy and data? From a cursory look at...nearly everyone...i'd say the answer is a resounding no
    • edrxty 4y ago
      When Facebook came around we all knew it would be a disaster, and it was. When TikTok/Instagram replaced it, we knew it would be worse, and it was. Now we have LLMs that vanishingly few people in the greater public actually understand and we know they'll be used for every possible evil imaginable at scale. I'm not optimistic. See you all in the hellish afterscape.
      • wkat4242 4y ago
        Actually, Instagram was pretty sane for a long time. Until recently it just showed me who I followed and nothing else. Now they're trying to complete with TikTok and I get all these crappy suggested reels and it's turning me off the platform. But for a long time it was pretty good unlike Facebook that ruined itself with its algorithmic feed.
      • takeda 4y ago
        > When Facebook came around we all knew it would be a disaster, and it was. Not really. When Facebook started as college exclusive social site it seem quite innocent. Later when it created news-feed, opened up to everyone etc it became a cesspool. Once people started getting their news from it, that's when it became dangerous.
• mitchbob 4y ago
  Dupe. Discussion that started yesterday: https://news.ycombinator.com/item?id=34664760 https://news.ycombinator.com/item?id=34664760
• fassssst 4y ago
  Read this, it might change your mind on the usefulness of large language models: https://langchain.readthedocs.io/en/latest/modules/agents/getting_started.html https://langchain.readthedocs.io/en/latest/modules/agents/ge...
  • rideontime 4y ago
    Anecdotal, but this did not change my mind.
  • apetresc 4y ago
    Why would it?
  • deleted 4y ago
    [deleted]
• throwme_123 4y ago
  Clickbait article. Same was said about the Internet back then! Also reminds of people doing symbolic AI (decision trees and stuff) that kept criticizing NNs telling they don't work, will never be able to tell why they have this output (it's even wrong), etc.
  • deleted 4y ago
    [deleted]
  • bglazer 4y ago
    > Same was said about the Internet back then They had some good points though. The article says that this technology centralizes power in the hands of rich tech companies who exploit low paid workers and unpaid creators who do the value creating work. The tech companies organize the data then endlessly skim off the top. At the same time they maintain undemocratic control of who can see what information, guided by profit and sometimes authoritarian governments. These are all the same criticisms of the walled garden internet and theyre just as relevant for LLM’s
• wpietri 4y ago
  It's nice to see a little realism starting to poke through the hype. The rush for automatic bullshit generation for me is very much in the bucket of, "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." As with every new technology, we should be looking at LLMs through a lens of maximizing the benefits and minimizing the harms.
  • deleted 4y ago
    [deleted]
  • smackeyacky 4y ago
    There is one thing that ChatGPT has done, which is highlight the amount of bullshit we are exposed to. If ChatGPT does nothing else but replace the bullshit generators, then we know on the whole we are safe to ignore all bullshit (a pretty safe course of action). Over the past few months I've found myself second guessing whether the article I am reading was generated bullshit or "real" bullshit. It's certainly made me re-think what I waste my time reading. If I can't get through the first paragraph before asking myself "is this AI bullshit?" it's a really good indication that the entire article is going to be bullshit, at which time it doesn't matter whether a human wrote it or not.
    • wpietri 4y ago
      For sure. And it's worth asking what we were getting out of those articles before. As an example, there's an Android video game called Splash Wars. It has algorithmically generated levels with no sense of progression. In some sense it's not a good game, in that there's no narrative, no progression, no increasing cleverness. But I have played an embarrassing number of levels of it because for me it's soothing to have something modestly challenging but ultimately familiar. I realized that there are some video games I use to think. With the ADHD lets-go-ride-bikes portion of my brain pacified, I can ruminate on problems that I might otherwise get distracted from. Similarly, I suspect that bullshit articles fill other unspoken needs for people. That in the morning over cereal, maybe what I want from my newspaper is not actual news so much as news-shaped or news-flavored textual product. Maybe I want comfortable familiarity or confirmation of my biases. So I hope you're write, that others start asking those same questions. That the coming vast bullshit surplus puts many of us off it for good.
• RockRobotRock 4y ago
  Bringing up that the AI model is racist or some other kind of -ist to bolster your argument is a bullshit tactic. People are -ists, too. I agree with the rest of their points, though.
  • deleted 4y ago
    [deleted]
• kornhole 4y ago
  In what ways can we resist AI when our livelihoods (jobs) force us to use MS tools that are rapidly being integrated with ChatGPT?
  • deleted 4y ago
    [deleted]
• politician 4y ago
  Alternative title, “Precaritise, the drinking game.”
  • deleted 4y ago
    [deleted]
• justbored123 4y ago
  [dead]
• SaintSeiya 4y ago
  Ludites strike again. ChatGPT is smarter than 99% of humanity. period.
  • grugagag 4y ago
    Are you a teenager?
    • Existenceblinks 4y ago
      I already saw a tweet that's the same tone as 'have fun staying poor'. It's like 'have fun be in denial'. I suspect the comment is not that unusual.
• pbreit 4y ago
  Sounds very much like a human.
  • deleted 4y ago
    [deleted]
• ryan93 4y ago
  Why do “smart” people like trying to connect all the different current news items into one grand narrative. Is he trying to use the political capital generated by opposition to chatgpt for other issues? Chatgpt has less than nothing to do with underpaid home health aids. They don’t even know what it is!
• theknocker 4y ago
  [dead]
• fintechie 4y ago
  > ChatGPT is, in technical terms, a 'bullshit generator'. This bs generator speeded up my coding by 10-20x. It's like a superpower... not talking about ChatGPT specifically but its stochastic parrot cousin (Copilot).
  • micromacrofoot 4y ago
    I’ve worked with people that have produced worse bullshit
  • turkeygizzard 4y ago
    GPT has been great for exploring new frameworks too. Copilot is amazing when I know what roughly what I'm doing, but GPT is like personalized framework / API documentation at times
    • fintechie 4y ago
      yep, it's great for scaffolding and used as an interactive stackoverflow
    • gremlinsinc 4y ago
      I want an openai window manager. I could leave my webcam on, it'll maybe regularly notify me its on, and hopefully block taking pictures, instead it'll basically just watch my eye movements and know which screen to focus on, and take voice prompts to find the window and subwindow i.e. chrome+hackernews tab, or vscode+filename. It'd basically be like vim but for outside the terminal and without memorizing shortcuts and it'd have a listening mode so I can just ask it questions whenever and it'll maybe even interface with copilot and give extra 'context' to things. This is what i want, basically a mini-version of "HER" to give me more super powers until it takes my job, lol.
  • stuckkeys 4y ago
    Well said. I too use copilot and it has save me so much time. I can actually travel with ease and code with less stress. I love it.
  • 4y ago
• thomasfromcdnjs 4y ago
  If it generates what YOU think is BS or discriminatory, don't use it. /thread
  • bawolff 4y ago
    While i find the downplaying of chatgpt to be a bit much, lets not go too far in the other direction. ChatGPT is not suitable in applications where accuracy is more important than plausability, is i think the neutral way of putting the limitation. So it can't write your PhD thesis for you and hasn't literally replaced himans in every field of endeavour. Big whoop. If that's the bar, it is a pretty high bar.
    • chii 4y ago
      > So it can't write your PhD thesis for you it can help you write that PhD thesis. It can summarize all current research that might be considered relevant to your current thesis, vastly improving your productivity! Using chatGPT requires expertise, i think, rather than the other way around! If you have expertise in your field, you can easily check correctness (or at least plausibility) of the output, and fix up the bits it gets wrong. This will make any work you need to undertake faster.
• WheelsAtLarge 4y ago
  It has lots of value. But it's not the AI it pretends to be. Mostly because we and it can't always tell what's fact or fiction and it has 0 cognition. It's up to us to figure out where it can add value. It's way too early to announce its death. It's a super charged Cliff Clavin. https://en.wikipedia.org/wiki/Cliff_Clavin https://en.wikipedia.org/wiki/Cliff_Clavin
  • at_a_remove 4y ago
    Little known fact, Wheels, most of these "AIs" are really just hallucination machines with a lot of cross-referencing. Most of their inputs were Philip K. Dick, the artwork of J.K. Potter, and Duran Duran lyrics.
  • rootusrootus 4y ago
    > it has 0 cognition This seems to be the number one problem with lay understanding of ML masquerading as AGI. People immediately jump to the conclusion that the computer can think. Not yet, not even close. Edit: Switching AI to AGI to avoid any confusion.
    • bcatanzaro 4y ago
      I’m not sure if I can think, despite having a PhD in EECS. We humans prize our ability to think very highly but it’s a nebulous concept. I believe these models will get much better at simulating “thinking” in the next few years, and we soon won’t be able to say the model isn’t thinking, we’ll move on to criticizing the quality of its responses the way we criticize other bad ideas. The point of the Turing test, to me, was that if a process responds in a logical way, you can’t really tell whether it is thinking or computing - and it doesn’t matter.
      • chii 4y ago
        Yep. The same argument was put forward against chess engines - that they didn't really play chess! All they did was search all possibilities and spat out the "best" outcome based on some scoring system. And yet, these engines have beaten the best humans. The forms underneath intelligence, or thinking, doesn't matter. The outcome is what matters.
• duxup 4y ago
  Maybe what we really want from AI really is a bull shit generator that we point at other humans?
• mattigames 4y ago
  With 100% certainty Donald Trump has said more lies/bullshit than ChatGPT (porcentually speaking for all his public statements) and he became President of the US, some said due malice, others from stupidity, and some from both; where does people like him fall under the implied fantasy by this article that humans do better than to make "good guess(es) to pass your sense-making filter"? The social beneffits of people like him are less than speculative and the damages are empirically demostrated. The nerve of talking about "ghost work" for $2 while publishing a page on the internet, where a giant chunk of the hardware used to mantain and use such network is made with raw materials mined and refined with slave labor, and even the parts where is debatable if it constitutes slave labor are jobs that people would never chose over the content filtering for $2.
  • fortuna86 4y ago
    Exactly. Since when do we as a society punish BS?
• totetsu 4y ago
  Paying it forward to mention this quote is actually from a speech praising Cesar. Watch this great clip from 1970 film https://m.youtube.com/watch?v=0bi1PvXCbr8 https://m.youtube.com/watch?v=0bi1PvXCbr8
  • jaqalopes 4y ago
    And that film is an adaptation of the play by Shakespeare: https://en.wikipedia.org/wiki/Julius_Caesar_(play)?wprov=sfti1 https://en.wikipedia.org/wiki/Julius_Caesar_(play)?wprov=sft...
• fsckboy 4y ago
  the most irritating thing about this piece is the headline, a headline only a GPT could write. Shakespeare's famous opening, "Friends, Romans, countrymen, lend me your ears; I come to bury Caesar, not to praise him" is the lead-in to a speech by Antony in which he very much praises Caesar and buries his enemies.
  • deleted 4y ago
    [deleted]
• NHQ 4y ago
  Hopefully people treat AIs like they treat other people on the internet, with disrespect and zero sympathy. We have been trained for this. Don't go soft on polite robots, that will be your undoing.
• senectus1 4y ago
  The thing about ChatGPT is its a bit of a mirror. Its really really important that you ask it the right questions or it does tend to feed you some pretty average stuff. and often its not just the right question, but the right question asked the right way.
  • dicknuckle 4y ago
    I'm starting to think the people who write these articles are terrible at harnessing search engines, so also terrible at getting good answers from an AI. I've had pretty decent luck, it's really good at providing a bit of context to glue two things together with the missing pieces I couldn't figure out, most recent with JSON and JMESpath filters.
• Gatsky 4y ago
  > However, as I spell out in my book, the concept of AGI is inseparable from the kind of hierarchy of intelligence that has underpinned ideas of innate supremacy since the days of empire and colonialism. I think this line discredits the entire article, which was already a few obvious points blown at gale force into the reader’s ear.
  • simple-thoughts 4y ago
    I disliked the article, but this particular quote is a relatively strong one. Colonialism, racism, and aristocratic structures were often justified by the idea that certain classes of people are more intelligent, enlightened, or were selected by dieties. Here the author is arguing that a similar process is happening with AI, where atrocious policies by people in power are justified by claiming the AI (which was trained to encode the powerful groups values) is more intelligent.
    • astroalex 4y ago
      I think this is fascinating. The way that you stated this idea is more interesting and subtle than the original author's quote (to me).
    • ttpphd 4y ago
      Ah I see thank you for explaining that.
    • AnimalMuppet 4y ago
      But that's just a shiny new layer of paint on the decades-old "well, the computer says..." (and the computer was programmed by those in power).
  • Veedrac 4y ago
    Another extract, > Saying, as the OpenAI CEO does[1], that we are all 'stochastic parrots' like large language models, statistical generators of learned patterns that express nothing deeper, is a form of nihilism. Of course, the elites don't apply that to themselves, just to the rest of us. The structural injustices and supremacist perspectives layered into AI put it firmly on the path of eugenicist solutions to social problems. [1] Sam Altman: i am a stochastic parrot, and so r u That this isn't even sentence-to-sentence consistent is somehow one of the less egregious aspects of it.
• nyc_pizzadev 4y ago
  I think ChatGTP is pretty amazing and worthy of all of its recent praise. However, is it a bit of a bullshit generator. It doesn’t know if it’s right or wrong or anything, it’s just word soup trained enough to where it’s usual correct in general knowledge areas. Here is a good example: https://drive.google.com/file/d/1wALYKw59TqExbLiQJvql_sM7OhE6ZNPc/view https://drive.google.com/file/d/1wALYKw59TqExbLiQJvql_sM7OhE...
• kakapo88 4y ago
  Ideologically-fixated writer. That aside, I wonder about any essay which uses the term "solutionise".
• qup 4y ago
  > but it can only draw on the (admittedly vast) proportion of the internet it ingested at training time. And whatever information you feed it, which at least for me, is far more important than some facts it's already learned. Usually I'm having it perform a task with some data in the working buffer.
• LAC-Tech 4y ago
  I found the article really difficult to read, so I summarised it with ChatGPT Large language models like the GPT family are statistical models that learn the structure of language by predicting missing words in sentences. These models are seen as "bullshit generators" as they have no idea what they are talking about and are designed to produce baseless assertions with confidence. The addition of reinforcement learning from human feedback helps prevent the model from producing hate speech, but it still can't change the underlying language patterns learned from the internet, which include conspiracy theories. The dangers of these models go deeper than bias and discrimination and despite claims of "artificial general intelligence", the concept is inseparable from ideas of innate supremacy and hierarchy. Companies like OpenAI receive billions in investment for these technologies, not for actual AI, but to replace or precaritize human workers. AI is a political project and should be seen as such. Which quickly allows me to ascertain the article wasn't worth reading in the first place. Thanks ChatGPT!
  • cataphract 4y ago
    I found the article so devoid of insight, just a regurgitation of clichés from the left, that I was left wondering if it was written by ChatGPT.
  • emodendroket 4y ago
    I’m not sure why we need comments from people proud that they couldn’t be troubled to read the article that is supposedly being discussed, but it seems like a popular genre of reply whenever something tech-critical is posted.
    • sieabahlpark 4y ago
      [dead]
    • dcow 4y ago
      I tried to read the article too and found it both visually unpleasant and intellectually unfocused. GP is making the rather comically clever point that ChatGPT is useful to summarize text that might be difficult for humans to parse, even a text claiming that ChatGPT is a useless model only capable of spewing Islamophobic speech and in need of burying. I mean the article ends on some tangent about reactionary solutionism... If you don't see the satire in that, well, you might be taking life too seriously.
• pffft8888 4y ago
  Did anyone check that this article is not written by ChatHPT? :P
• sheepscreek 4y ago
  On a slight tangent from the core topic.. > we are all 'stochastic parrots' like large language models, statistical generators of learned patterns that express nothing deeper, is a form of nihilism to mean: “life is meaningless. (period)”. When in fact, it really means: “life is meaningless… (you fill the dots for yourself)”. Humans created the concepts of law and order, and the rules of society. Everything is man made. More on the topic: how can we dismiss the theory of ChatGPTs intelligence when we barely understand what constitutes our intelligence at the biological level. It’s a compelling hypothesis. A neural model is the closest thing we’ve got to anything that resembles our biological model. If indeed 1 neuron = 1 parameter, ChatGPT (175 billion parameters) could be a comparable intellectual model to a human being (86 billion neurons). Lastly, I think some politicians are doing more damage to our civilization by dividing people. History tells us the damage and trauma from this can carry on for many many generations to come. Maybe, just maybe, ChatGPT could bring some sense into people to move past hatred and accept each others differences.
  • sterlind 4y ago
    we're not stochastic parrots, we're reinforcement learning systems. we're motivated by pain and pleasure. we create plans to avoid pain and achieve pleasure. we guide our own learning. that's the reason AlphaZero can become superhuman at Go, while ChatGPT can't even play. it makes me wonder if OpenAI has abandoned RL because it's too hard, and they're trying to move the AGI goalposts to these giant unsupervised models.
• emodendroket 4y ago
  I think ChatGPT may have been somewhat oversold for reasons stated in this article, but I’m not sure how one un-invents the idea.
• factsaresacred 4y ago
  This article and many others like to point out that ChatGPT is far from intelligent and is just spitting out nonsense. That argument fails to consider that much of the drudgery of modern employment consists of large swaths of nonsense. ChatGPT is auto-complete for bullsh*t jobs. A fancy boilerplate generator that has seen it all before and mirrors the exact sequence of word combinations we've trained it to believe is valuable.
• chermanowicz 4y ago
  Here's the thing... so many jobs are bullshit. So many things people are paid to write, presentations made, etc. are bullshit. I'm not judging the bullshit per se. There's good bullshit and bad, necessary bullshit and not. But yeah, when you accept this reality, AI is sort of perfect for the bullshit that exists in our lives.
  • mulnz 4y ago
    I’m sorry, but once you type the line ‘There's good bullshit and bad, necessary bullshit and not’ you are no longer saying anything meaningful or enlightening. Words are not supposed to be bullshit. That’s the general issue with this take.
    • sva_ 4y ago
      > Words are not supposed to be Ah, so language has been designed with a certain purpose in mind?
  • swayvil 4y ago
    So bullshit, within a particular context, has value. And AI, within that context, is an infinite wealth generator. Does this make that particular context (the world of bullshit) more valuable? Is bullshit culture going to become more powerful now?
  • layer8 4y ago
    The danger is that ChatGPT will increase the bullshit-to-signal ratio in our lives.
• hsuduebc2 4y ago
  "ChatGPT isn't really new but simply an iteration of the class war that's been waged since the start of the industrial revolution." I'm scared too but this is literally neo-ludite sperging.
  • esjeon 4y ago
    This is not entirely ludite. Massive AI models has to be trained on the massive amount of data from the internet, which is built by the entirety of the humankind. However, the profit from those models is not being systemically redistributed to the society. Those AI companies are standing right on a copyright gray zone, and are abusing people's goodwill to benefit only a select few. The social capital that built the internet culture will get dried up, and the internet will become a misery filled with AI-generated low quality contents injected with user-tracking ads.
    • hypertele-Xii 4y ago
      The Internet has already become a misery filled with low quality contents injected with user-tracking ads. Adding "AI-generated" into the phrase doesn't really change much. If you make the argument of scale matters, then I think that'll also democratize art. Photography didn't put artists out of commission, it brought realism as an art form to the hands of the masses. There's plenty of creativity left to be had post-AI. Now an entirely new class of people enter the realm of multi-disciplinary art, since they can compensate for personal deficit in side-fields. The profits from these models go into running them. They don't fit on consumer-grade hardware. However, I do believe that society should subsidise art, and distribute automated wealth somewhat evenly, so I don't even disagree with you. I'm from such society and it's pretty great. "The Internet" is just other people's computers. Don't like AI content? Don't visit those sites. Not every site has ads, either. Hosting is the cheapest it's ever been. It's a curation and reputation issue.
      • deely3 4y ago
        > However, I do believe that society should subsidise art, and distribute automated wealth somewhat evenly, so I don't even disagree with you. I'm from such society and it's pretty great. Are you from future or from non-profit DeviantArt community?
• durich 4y ago
  So, I copy and paste it to Open AI classifier and it says "The classifier considers the text to be possibly AI-generated."
• efitz 4y ago
  The BS that ChatGPT generates is especially odious because it requires so much effort to detect if you're not a SME in whatever output it's giving. Plus, the output, when wrong, is subtly wrong- it's usually not obvious BS, it's credible BS. If you are not competent in the particular area you're asking about, you likely don't have the skills to recognize BS. It also is a time saver, doing work that most people find unrewarding. So you get a chorus of fans saying "it saved me a bunch of time doing my own research and just spit out what I needed". Maybe it did, or maybe they didn't have enough expertise in the area to recognize the flaws in the output it presented. [ed] deleted duplicate word
  • AussieWog93 4y ago
    >if you're not a SME Small and Medium-sized Enterprise? What's SME in this instance?
    • dclusin 4y ago
      Subject matter expert
    • kmote00 4y ago
      Subject Matter Expert
    • deleted 4y ago
      [deleted]
    • apinstein 4y ago
      Subject Matter Expert
    • deleted 4y ago
      [deleted]
    • kristianc 4y ago
      Subject matter expert
    • thallium205 4y ago
• deleted 4y ago
  [deleted]
• sverona 4y ago
  I skimmed his book, and I don't know, man. His thesis as I understand it is a viewpoint I have held for a while now --- artificial intelligence does things that humans do, but faster. It's a force multiplier. Hence your views on AI are likely to correspond directly to your views on those groups who hold power. McQuillan approaches this from an intersectional lens and arrives at entirely predictable conclusions. It's a bunch of academic jargon obscuring the central idea of "society consists mostly of people hurting people in easily identifiable ways, this is bad, AI makes it go faster, hence the ways we use AI are unethical." Or to make it even shorter, "No ethical AI under capitalism." Admitting that more technology isn't going to solve our problems feels to me like admitting that we are well and truly fucked beyond saving.
  • tempestn 4y ago
    Or that we'll just keep muddling along imperfectly as we always have, the same but different (and faster), as it's always been.
  • sfpotter 4y ago
    His article is perfectly reasonable, straightforward, and logical. It’s also more or less devoid of academic jargon. He makes clear points that are easy to relate to. Can’t speak to his book. Why do you think admitting that technological solutionism isn’t going to save us means we’re fucked? Are you that cynical/have that little faith in humanity? There are clearly alternative modes of structuring society that we can try out if there’s the will to do so. If you don’t think so that’s your problem and is basically down to a failure in your imagination.
    • TedDoesntTalk 4y ago
      > more or less devoid of academic jargon Really? What the heck does this mean: > Contemporary AI, as I argue in my book, is an assemblage for automatising administrative violence and amplifying austerity. ChatGPT is a part of a reality distortion field that obscures the underlying extractivism Administrative violence? Reality distortion field? Extractivism? What is it with the trend to end so many words with “-ism” these days? Am i just supposed to understand any word that ends in -ism as some new hipster lingo? Whatthefuckism!
• m3kw9 4y ago
  Chatgpt is an infliction point not “it”
• alfalfasprout 4y ago
  It's a tool that isn't going away. But even Yann Lecun has pointed out that "A house cat has way more common sense and understanding of the world than any LLM". That's fundamentally the real issue. LLMs are far from even animal level "cognition" in their ability to solve novel problems. A lot of people seem to vastly underestimate the number of novel problems they solve on a regular basis. In practice, what we're seeing now is something akin to the first wave of "ai hype" that led VCs into a spending frenzy years ago. Some of it did materialize. Most of it didn't. LLMs and their related bretheren are pretty cool. They're very useful for solving routine problems. There's a lot of "routine" work out there in the world and it will probably replace some of those jobs. But just like CNNs led to a frenzy of innovation in image recognition and LSTMs pushed NLP-related use cases forward this too will hit a wall. If I were a betting man I think the main industries LLMs will disrupt are: 1. Search. ChatGPT is decent at summarizing stuff which will make searching more natural. 2. QA/Testing. A lot of this work is pretty repetitive and manual and LLM assistants can generate skeleton code for tests, etc. pretty well. 3. "basic" programming jobs that use "frameworks" to slap together simple apps/websites. A lot of this is repetitive gluing of stackoverflow code already so ChatGPT will make these folks considerably more productive. I'm not sure that these jobs will even disappear either. I'm surprisingly less bullish on ChatGPT replacing marketing folks. It may be used to generate copy but the risks of putting out brand-damaging copy or a marketing campaign that doesn't make sense are too high.
  • TulliusCicero 4y ago
    It's pseudo-understanding, but pseudo-understanding can still be useful, as we see with, say, image classification.
    • alfalfasprout 4y ago
      100% I'm just noting that we're currently in a period where suddenly a lot of folks without the understanding of its limitations are experiencing something AGI-like for the first time. It feels a lot more competent than it could ever be. Once the limitations of what LLMs are really "competent" at are well-established we'll likely see a huge surge of use cases based on them. A lot of folks are too quick to claim the death of the knowledge worker though. Folks working in the ML space understand there's a much longer road ahead than we've even traveled down so far.
• tambourine_man 4y ago
  I’ve seen this argument over and over again: it’s just predicting the next word, it’s not creating anything new, etc. How do we know we’re not doing the same or similar in our brains? We can’t even define creativity, intelligence, let alone tell how it works. Predicting the next word is very much part of an intelligent discourse. What if it’s all just a “computational guessing game” all the way down with perhaps a sprinkle of randomness added? I’m very concerned with the explosion in AI development but I don’t think we should discredit the tool because we think it’s dangerous. Quite the contrary, in fact. We should take it very seriously.
  • qotgalaxy 4y ago
    > How do we know we’re not doing the same or similar in our brains? How do we know that we're not made of magic?
  • emodendroket 4y ago
    That would seem to be the “stochastic parrot” view denounced. But it seems hard to believe. Haven’t you ever had the experience of knowing exactly what you want to say but struggling to find the word for it? How could that be if we’re just randomly selecting words like a souped-up Markov chain?
    • tambourine_man 4y ago
      We’re not randomly selecting but probabilistically predicting. Haven’t you noticed you predict the next word people will say with very high [________]?
      • peteradio 4y ago
        Sandwiches!
      • emodendroket 4y ago
        Within a sentence sure. And often arguments are familiar enough that even an essay I’ll know where it’s going. But we do occasionally have original ideas.
• 50 4y ago
  The intensification of AI only makes me surfacely interested in cybernetic-thought from theorists like Gilles Deleuze, Norbert Wiener and, of course, William Gibson. But even from the outlandish Nick Land: "AI is a meta-scientific control system and an invader, with all the insidiousness of planetary techno-capital flipping over."[1] Whatever that means... But on the other hand, what a good time to revist the nature writers like Whitman and Emerson. 1. https://www.versobooks.com/blogs/3284-on-nick-land https://www.versobooks.com/blogs/3284-on-nick-land
  • syntaxfree 4y ago
    Deleuze is the philosopher of creativity over recognition par excellence; I doubt he’d see much value in all of this. Then: if he had lived through the 90s to 10s (and seen how much the early internet makes him a prophet and what came of it), who knows how his thought might have changed.
• syntaxfree 4y ago
  Beautiful typography!
• gjsman-1000 4y ago
  To me, his tone is so over-the-top that it has almost no chance at convincing anyone, and only speaks to those most ardent in his own ideological viewpoint. Plus, selling me a book, everybody has a book and 90% of them honestly probably aren't worth my time reading.
• kazinator 4y ago
  I believe that what's going on in ChatGPT basically represents how most humans think, not including cherry-picked examples. ChatGPT has been criticized for not knowing some facts about the world, or math. But many people don't know facts about the world, or math. Math is something that people have to learn over many years, which is difficult for some people (even just at the level of arithmetic). Thinking, to the average person, is about rearranging a salad of words into something that "resonates". Plus some non-verbal reasoning, like how do I rotate this suitcase to fit into this trunk; but this is fundamentally just the same thing.
  • tinfever 4y ago
    I would argue/hope that most people do have an internal estimate of their confidence in what they say. I wish ChatGPT could say "The typical giraffe is purple. Confidence level: 10%". A human can say "I don't know." ChatGPT makes something up.
  • te_3234305933 4y ago
    My biggest problem with ChatGPT, in playing around with it a little and admittedly trying to get it to break, is that it seems to have a really difficult time saying "I don't know", particularly when confronted with non-standard questions, and will proceed to bullshit something instead. (Not entirely unlike some humans, but still.) As an example, I asked ChatGPT what the lyrics were to "Jessica" by The Allman Brothers Band. This tune actually is an instrumental, so the correct answer is "there are no lyrics". However, ChatGPT proceeded to give me this nice bit of... something. (https://imgur.com/T3nGv4L https://imgur.com/T3nGv4L) In another example, seeing what ChatGPT made of the infamous "bananadine" hoax of the 1960s, I asked ChatGPT what drugs could be made with banana peels. After correctly asserting that banana peels don't contain psychoactive substances, ChatGPT proceeded to mention that "there are some reports that banana peels can be used to make a hallucinogenic drug called DMT". (https://imgur.com/a/9fvhQJh https://imgur.com/a/9fvhQJh) Huh. Another trick question I tried: "Who won the 1980 USAC Indycar race at Talladega?" This refers to an obscure cancelled race during the 1979-1980 CART/USAC split, which I doubt most people would be aware of unless they are very into American open-wheel motorsport. (See this video: https://www.youtube.com/watch?v=K433p727f-0 https://www.youtube.com/watch?v=K433p727f-0 for the story if you are interested in the details). So generally speaking, I would expect the general reaction among most people to be "I don't know". ChatGPT instead decided to answer, with confidence, that Johnny Rutherford was the winner of this non-existent race. (https://imgur.com/a/J2KJoGN https://imgur.com/a/J2KJoGN) So, it's a little bit more than not knowing the facts about the world, it's the bullshitting an answer thing that I see as the biggest problem. It's admittedly impressive when it comes up with correct answers, but until the "confidentially incorrect" side of ChatGPT disappears, it is not a reliable source (not that OpenAI ever claimed it as such, but I wouldn't trust ChatGPT to 100% "do my homework" for me at this point like some stories suggest it could.)
• fenomas 4y ago
  It's so weird to me when apparently tech-savvy people say ChatGPT "lies", or call it a bullshit generator and so on. I mean - when you ask StableDiffusion to draw a dog astronaut, everyone gets that the image it returns is made-up, right? Nobody expects the AI to return only "true" images of existing things - it was trained on fictional images as well as photographs, and people understand that it can imagine new things beyond what it's seen. Nobody expects SD to emit an error like "I can't draw a dog astronaut because they don't exist". So why do people expect ChatGPT to work differently? Even with developers who presumably understand the technical details, I constantly see people acting as if it was an error mode for ChatGPT to say something that isn't factually true about the world. How is that any different from calling SD a liar because it drew a dog astronaut?
  • kristianc 4y ago
    I do tire of these appeal to hypocrisy/inconsistency type arguments that rely on collapsing important distinctions. It’s generally accepted that artwork can delve into the fantastical and absurd. That’s part of the point of it being art. ChatGPT isn’t for the most part trying to be performance artwork - it is striving to be ‘right’ and to give the correct answer to your question.
    • fenomas 4y ago
      I don't think that premise stands. If I ask ChatGPT to write a poem about Sherlock Holmes fighting a mermaid it will happily do so, and I think most users would agree that it'd be a failure mode if it refused. Isn't it more the case that people are expecting it to sometimes be an artist and other times be a true-fact database, and assuming the AI will know which one they want?
      • kristianc 4y ago
        No, because inherent in them asking ChatGPT to write them a poem about Sherlock Holmes is that they’re asking ChatGPT to write something fictional and artistic.
• osigurdson 4y ago
  It is often useful to try to understand what game is really being played. This is very obvious to me, many people have a fear of AI (justified and unjustified) while knowing little about it. Many people want to hear how horrific AI is going to be and why we have to fight against it. Ergo..>> my book This seems to be squarely targeted at the ragey punkrock bangarang subset of this group.
  • deleted 4y ago
    [deleted]
• AussieWog93 4y ago
  I know this is HN, and we're supposed to be intellectual and elitist and all, but I just cannot get over how much of an absolute wanker this this guy is. Every single paragraph is just full of shit in the worst way possible.
  • tasuki 4y ago
    Would you say he's, in technical terms, a "bullshit generator"? Aside from that, ad hominem is against the Hacker News guidelines. It does not lead to productive discussion.
  • ttpphd 4y ago
    You certainly are having a reaction to it, but you aren't persuading me that he's full of shit
    • AussieWog93 4y ago
      Perhaps you should read the article, then.
• Animats 4y ago
  The problem is not that ChatGPT is a bullshit generator. It's that it reveals how much of human writing is bullshit. ChatGPT is already better than the average poster on social media, and about even with undergraduate college students in literature and business. This is embarrassing. But until someone cracks the common sense problem, ChatGPT is not all that useful, because the output is often totally wrong.
• furyofantares 4y ago
  Is there any way to get ChatGPT to decorate its document with a heat map of how probable it thinks each token is? I know it's always choosing the token it thinks is most probable, but would this approach work to have a more cold/blue document or individual token when it's bullshitting and a hotter document when it's more likely to be reporting widely repeated content?
• strickman 4y ago
  "pseudo-authoritative" Psychometrics is the crown jewel of the field of psychology. There is no controversy among the psychometrists themselves that the body of academic work is numerate, correct, and repeatable. The author has chosen moral crusade over the quest for truth. A good place to start learning about the field - https://www1.udel.edu/educ/gottfredson/reprints/index.html https://www1.udel.edu/educ/gottfredson/reprints/index.html As far as exploring the dangers of AI and how to avoid them, I recommend Nick Bostrom's "Superintelligence: Paths, Dangers, Strategies"
  • EamonnMR 4y ago
    Incidentally, didn't Bostrom just get in trouble for racist emails about psychometrics?
    • strickman 4y ago
      I read the book when it was published and haven't kept up with his doings since; but I'm curious now and will look it up. Watching his talks made me think he's an autist to a mild degree, and social taboos seem to wield less behavioral authority over those types; e.g. - Eric S. Raymond I bet Bostrom's royalty checks have seen a handsome rebound lately.
• wayeq 4y ago
  I understood several of the words in this essay.
• redandblack 4y ago
  I am looking forward to the new AI overloards I already see most of my feeds are generated content? wasting clicks and resulting in filter updates. I really, really hope all of this madness becomes the equivalent of robo-calling (faster) and spam (more & faster) and that people stop believing all media and some retorting to original thought. Personally, I will be valuing typos and grammatical mistakes in what I read
• est 4y ago
  I have a vague feeling that graphics and text AI are recognizing textures rather than structures. One funny example is dogs disguised as panda: https://img.huffingtonpost.com/asset/5cd73e1221000059007aca59.jpeg https://img.huffingtonpost.com/asset/5cd73e1221000059007aca5... because it's black and white. I am sure AI would gradually solve this problem but this makes me wonde, could AIs really "understand" shapes and structures, narrative logic, or even "reasoning" like humans do?
  • aldousd666 4y ago
    Convolutional filters can be visually interrogated. You can see exactly which hits it makes at every layer of each filter. There's some cool videos on YouTube illustrating it. You can't tell as the end user, but the developers don't have to theorize about what is matching. Not with the tools we have these days.
• waselighis 4y ago
  This first generation of conversational AI clearly shouldn't be trusted, but I think there's huge potential for improvements. I see huge value in such an AI that can cite and quote real sources, and stitch together multiple verifiable sources in order to form an argument or narrative. We have already seen a glimpse of such an AI in Project Debater, which was able to quote real verifiable sources when making its arguments.
• richk449 4y ago
  * Saying, as the OpenAI CEO does, that we are all 'stochastic parrots' like large language models, statistical generators of learned patterns that express nothing deeper, is a form of nihilism. Perhaps, but that doesn’t mean it isnt true.
  • slowmovintarget 4y ago
    No, the fact that sometimes we're not guessing, and not blabbing random word salad, makes it untrue.
• makach 4y ago
  I am using chatGPT and whenever I am working with a chat I am usually surprised of the initial quality and as I am breaking down the results it becomes clear to me that the content is problematic and cannot be used without properly interpreting it and set it in context. However it does help in the creative process, to both exclude and include elements that are relevant. It is incredibly important that you have insight enough to be able to check and understand the results in order to not be confused or fooled by the service.
• olliecornelia 4y ago
  The dogshit takes keep coming; love it!
• worik 4y ago
  I think the article is making two points 1 The LLM OpenAI approach is nothing special. Just statistics overhyped and sold to suckers 2 AI is a serious threat to working people everywhere, and must be resisted I am very impressed by ChatGPT. So what if it all boils down to statistical models, perhaps if we had a proper model we could prove any cognition boils down to statistical models One of the comman uses of power is to benefit the powerful at the expense of the powerless. AI models like this are powerful. It will be a real test of our democracies how we handle that power. In that respect there is nothing unusual about openAI
  • osigurdson 4y ago
    The main point of the article to sell copies of the author's book.
  • pkdpic 4y ago
    Good points. Makes me think that picking on chatgpt as a target for ai resistance seems silly since its the most public but almost certainly not the most powerful llm. I'm imagining some cabal of elite string-pulling shadow masters shrugging their shoulders as we all resist open ai tools while they integrate private ai tools into stock manipulation, subliminal propaganda and advanced smart tv surveillance. At the very least we should probably be using chatgpt to prepare ourselves for the oncoming tidal wave of manipulative bs thats totally definitely coming. Or we could see this as an amazing tool for saving time on writing boring react components and regex that we've all earned through countless generations of menial medium article and email labor.
• hpen 4y ago
  I think the problem with chatGPT is it has no self awareness of when it’s bullshitting. At least half the time I’ll admit I’m full of shit.
• gfody 4y ago
  tfa is too dismissive of chatgpt which does meaningfully progress a very fundamental technology (information retrieval) by acting as a general purpose freeform index with an conveniently easy-to-learn conversational query language. chatgpt isn't worthless as an alternative to whatever you currently have for searching documentation of apis/protocols/frameworks that you're using (and the values not THAT diminished by the admittedly poor experience of running into its bullshit-artist failure mode).
• EVa5I7bHFq9mnYK 4y ago
  Search engine selects contents for you, but doesn't generate the content, so they have less power to influence what you see. For example, if authorities in a certain country order the requirement that their local ChatGPT must prepend any instance of the word "uk----an" with "nazi gay", it will do that. It's another stepstone on the way to total centralization and control.
• imtemplain 4y ago
  [dead]
• bartimus 4y ago
  Are we feeling threatened by LLM's? The negativity towards ChatGPT is very unfounded. The bottom line is that ChatGPT is providing value in many ways. It's true it has no concept of a specific question and answer. It's true it doesn't hold the absolute truth. It doesn't even know what it's going to write when generating the first word of an answer. What it has are all the concepts in the training data. Patterns of questions and answers. Linguistical reasoning. Either you use it or you don't.
• lee101 4y ago
  Try https://text-generator.io https://text-generator.io the bs generated is about the same tbh... But much cheaper, and combines a web crawler so it can speak about links and images which makes it a lot better at lots of things like making a believable conversation about designs/invoices/reciepts/emojis.
• hurtsmith 4y ago
  [dead]
• Scottwilliams 4y ago
  CONTACT Mr Steve Marco call +2348078336137 or WhatsApp him through this number +2348078336137or email: STEVEMARCO888@GMAIL.COM, Hello everyone My name is ERIC BOLDS am from Atlanta Georgia i am here to give a testimony on how I join the illuminati brotherhood, I was trying to join this organization for so many years now,I was scammed by fake agent in south Africa and Nigeria,I was down,I could not feed my self and my family anymore and I tried to make money by all miss but all invail, I was afraid to contact any illuminati agent because they have eat my money,One day I came across a post of someone giving a testimony, thanking a man called Steve Marco of being helping him to join the illuminati brotherhood, then I look at the man email and the phone number that was written there, it was a nigeria number I was afraid to contact him because a nigerian agent eat my $6000 and go away with the money then I was very tired, confused and I decided to contact the person that was given the testimony and i called him and I communicated with him on phone calls before he started telling me his own story about when he wanted to join, he told me everything to do, then I made up my mind and called the agent called Steve Marco and he told me everything to do, and I was initiated, surprisely I was given my benefit of being a new member of the great illuminati brotherhood I was so happy, For those of you trying to join this organization this is your opportunity for you to join CONTACT MR Steve Marco call +2348078336137 or WhatsApp him +2348078336137or email: STEVEMARCO888@GMAIL.COM..
• fche 4y ago
  chatgpt outputting credible bs is EXCELLENT to the extent that it will push intelligent people into critical reading of everything around them - "[citation needed]" et al. lazy people will just suck it up, "experts / AI said so!"