55 ms·

Jaron Lanier on the danger of AI

352 points by atchoo 4y ago

smm11 4y ago
This may kill the WWW as we know it. I can almost see an alternate 'web' finally taking off, but not really, as the majority just want TikTok left alone.
johnaspden 4y ago
Those whom the gods wish to destroy, they first make mad.
PaulHoule 4y ago
It is great to see how Lanier has grown up. Seriously though. once people take the limiters off chatbots you will see "artificial bullshitting" generating duckspeak on subjects from anti-vax and climate denial to cryptocurrencies, longtermism, transsexual maximalism and police abolitionism. And people who believe those ideologies will eat it up.
- pavlov 4y ago
  I think the scariest part is not even the content but the connection that people can develop with these bot personalities. A demagogue who knows your intimate desires and fears and talks just to you, seems to care about you, is available 24/7. Your very own Personal Jesus (or Personal Hitler). Every communications revolution seems to lead to a genocide. Radio and TV in the 20th century (Germany, Rwanda etc.) Social media in the past decade (Myanmar). The upcoming AI-instigated genocide is a terrifying thought. The Rwandans didn’t think they’d start killing their neighbors even as rhetoric on the radio got harder. Chatbot-dependent Americans may not see it coming either.
  - AnimalMuppet 4y ago
    The parent poster's username seems appropriate... (Note well: This is not a "username checks out" drive-by dismissal. I think the point is perfectly valid.)
  - PaulHoule 4y ago
    Look at the parasocial relationships that people have had with L. Ron Hubbard, Eliezer Yudkowsky, and even camgirls. (Particularly followers of LRH and EY frequently speak and write like a chatbot trained on their writings.) I think most "freezoners" (renegade Scientologists) believe they've had telepathic conversations with "Ron". Ron has certainly written enough that a chatbot could be trained to converse in his style, maybe even write the OT levels that Ron never got around to writing. Since Harold Lasswell people people have been hip to the use of content analysis to predict war and genocide which is possible because the radio, television, and newspapers are all public. However the Iranian revolution of 1979 was not televised, instead it was promoted through relatively unobservable cassette tapes passed through the underground. Since social media shows something different to everybody there might not be any overall surveillance that warns us of upcoming danger. It doesn't have to go so far as genocide, individual acts of violence are bad enough. There is no magic technique that can take an average person and make them into a "Manchurian Candidate" but if you find somebody who is directionless, socially isolated and vulnerable you can very much lead them into a rabbit hole and lead them into antisocial behavior. Chatbots could do this with superhuman patience. For instance, a follower of the black pill incel who calls himself "Wheat Waffles" became a mass shooter. If somebody took the "lesswrong" ideology seriously they might attempt to assassinate an A.I. research and I'd go so far to say that they're just cosplaying because if they were serious one of them would have done it already.
- beepbooptheory 4y ago
  With respect, its bad enough not even directly commenting on the article and instead using the space as a soapbox, but at least please save the rest of us from your perverted fixations. There are other forums!
- leokeba 4y ago
  Can you elaborate on what you mean by "transsexual maximalism" as an ideology ?
  - parthianshotgun 4y ago
    Maybe it was for effect, like 'psychic bedouinism'
  - johnaspden 4y ago
    We want to change everyone's gender, as often as possible; and will devote superhuman resources to spreading this philosophy over as much of the available future lightcone as physics permits.
    - leokeba 4y ago
      Yeah right, it really sounds like what most transgender people are advocating for indeed. I asked ChatGPT to enlighten me on the subject, here's what it had to say : The term "transsexual maximalism" is not a widely recognized or established ideology, but it has been used by some individuals to describe a perspective on transgender issues that emphasizes a strong and uncompromising approach to advocating for the rights and recognition of transgender people. At its core, transsexual maximalism is an approach that prioritizes the experiences and perspectives of transgender people, particularly those who have undergone medical interventions such as hormone therapy or gender confirmation surgery. Proponents of transsexual maximalism often believe that these interventions are necessary to fully alleviate the distress and dysphoria experienced by many transgender individuals. Transsexual maximalism can be seen as a response to more moderate or incremental approaches to transgender advocacy, which may focus on incremental policy changes or attempts to work within existing systems of power and institutions. Some proponents of transsexual maximalism may view these more moderate approaches as insufficient or even counterproductive, arguing that they do not go far enough in challenging the underlying structures of cisnormativity and transphobia. However, it's worth noting that the term "transsexual maximalism" has not been widely adopted or embraced by the broader transgender community, and some individuals within the community may have different perspectives or priorities when it comes to advocacy and activism. Me : Why would people consider this a threat ? ChatGPT : Some people might consider transsexual maximalism as a threat because it is a highly assertive and uncompromising approach to advocating for transgender rights and recognition. This approach can challenge existing social norms and institutions that uphold cisgender privilege and transphobia. For example, some individuals who are opposed to transgender rights might view transsexual maximalism as a threat to traditional gender norms and the gender binary. They may feel uncomfortable with the idea that gender identity can be different from the sex assigned at birth, or that medical interventions such as hormone therapy or gender confirmation surgery are necessary for some transgender people to live authentically. Additionally, some people might see transsexual maximalism as a threat to their own power or privilege. Transgender individuals have historically been marginalized and oppressed, and advocating for their rights and recognition can challenge the existing power structures that uphold cisgender privilege. This can cause discomfort or resistance among those who benefit from these power structures. It's important to note that concerns about transsexual maximalism as a "threat" are often rooted in transphobia and a lack of understanding or empathy for the experiences of transgender individuals. Transgender people have the right to advocate for their own rights and recognition, and it is not a threat to anyone else's well-being or identity.
- Turing_Machine 4y ago
  > It is great to see how Lanier has grown up. Indeed. It would be interesting to study how he became one of the go-to guys for hot takes on any new technology, considering that he hasn't developed any new technology in the last 40 years or so. Only Cory Doctorow, perhaps, matches him when it comes to famous for being famous. Sorry, an awesome set of dreads doesn't make you a "guru".
  - sirsinsalot 4y ago
    I'm not sure any of the fundamentals have changed in 40 years, so being able to speak as an authority isn't really dependent on knowing React or that Java 19 got whatever new syntax. That attitude is why ageism is so prevalent in engineering. Maybe read a few of his books before writing someone off because you think they're out-dated.
    - Turing_Machine 4y ago
      > I'm not sure any of the fundamentals have changed in 40 years Really? He's holding forth on GPT and similar technologies. Those represent dramatic changes from the state of the art 40 years ago. P.S. what makes you think I haven't read his work? I certainly haven't read all of it, but the significant portion that I have left me...underwhelmed.
      - sirsinsalot 4y ago
        He is talking about the impact on society of things perfectly imaginable over 40 years ago and detailed across the whole of sci-fi. He isn't talking about the effectiveness of a loss function or something.
  - hoppersoft 4y ago
    First, I have to admit that I was negatively biased by his appearance. I have a hard time believing anything that some guy who looks like he smokes metric tons of weed and dorks around on his guitar all day has to say. That being said, the tech world has a bad habit of letting "visionaries" rest on their laurels. He'd have a lot more credence if he was actively developing AI and had more than gut feel to contribute.
stareatgoats 4y ago
> “This idea of surpassing human ability is silly because it’s made of human abilities.” Shows the level of insight from this "guru". The truth is we don't know how far the work being done on artificial intelligence is going to go. For now it will continue to develop and acquire more and more autonomy, just because that is the nature of our existence: better and more efficient will replace the lesser so. So, we may have potentially given birth to a new sentient being that will go on to live its own "life" (within 100, 500, 1000 years?), or we might be able to constrain it so it so that it will always be in the service of humans. We simply don't know at this stage, but my money is on the former TBH.
- nathanaldensr 4y ago
  "AI" is not currently autonomous; its algorithms that do exactly what their creators tell them to do. They run on binary computers that only do exactly as they are told.
  - circuit10 4y ago
    That’s not true, current machine learning algorithms involve no manual programming past the training and inference code and it’s extremely difficult to predict what they will do without just trying it and seeing. I think this video is a nice introduction to the basic concept to how a computer can figure things out automatically without being manually programmed and without the creators understanding the “why”: https://youtu.be/qv6UVOQ0F44 https://youtu.be/qv6UVOQ0F44 ChatGPT is much more complicated than the AI in that video but it shows some of the basic concepts
    - goatlover 4y ago
      LLMs generate text. They're built to generate text. That they generate some kind of textual output is entirely predictable. Same with image generators. They will generate some kind of image given a prompt. They're not Skynet.
      - circuit10 4y ago
strken 4y ago
From reading this, I don't get the impression that Lanier has any objective reason to believe the world won't be destroyed as the direct result of AI. If he does have a reason, the reporter certainly doesn't devote any space to analysing it, or to explain why dying from AI-induced insanity is different from being destroyed.
BigCryo 4y ago
Can we please put off worrying about dangerous AI for a couple hundred years? Chat GPT is a chatbot and not much else.. it is not the harbinger of an era of dangerous AI
- antibasilisk 4y ago
  Stable Diffusion was used to produce convincing images of Donald Trump being arrested. A guy used GPT to post comments on 4chan without people noticing. A 'mere chatbot' can do much more damage than you think.
- circuit10 4y ago
  Hundred years? If the current rate of progress holds we have to start worrying about it in 5 or 10 years, the earlier we can come up with solutions the better because it’s a very hard problem
  - AnimalMuppet 4y ago
    Naw, it's simple. We're talking about an AI achieving human abilities, well, we can protect against dangerous AIs just as well as we protect against dangerous humans... Oh. Oh dear.
    - javajosh 4y ago
      Remember that AI work with electrons, and we are of atoms. We should focus on where electrons control atoms, and reduce those points. Of particular concern is that AI may be a very strong investor with the right prompts. AI could also figure out how to use any other software. Which can be used to gain access to any marketplace, including the dark ones. Which means AI can use money (electrons) to pay others to modify the world (atoms). Of course, there is already a problem, as you point out. Humans shouldn't have access to these markets either! But yeah to specifically prevent electron-on-atom violence we need to limit AI physical degrees-of-freedom. by limiting marketplaces. National/global security, not personal morality, should guide these new regulations. We need to end all drive-by-wire automobiles, and electronic locks. Too many services are habituated to act on electronic signals without human confirmation - particularly the police. There needs to be an unbroken verbal chain between the person who saw an event and the person doing the law enforcement. Breaks in the human chain should be treated very seriously -they should be treated as firing offenses, at least. There are many other similar changes we're going to need to make. Some folks aren't gonna like this. Regulations are inherently evil, they say. Maybe the mental model should be more like we're the big bad gorilla in the cage. But now there's a tiger in the cage. Regulation restrains the tiger. Also, some folks aren't gonna like it no matter what change you need. The fact of not liking it doesn't mean we don't need it, and it doesn't mean it won't get done. We have to trust that our leaders don't want to die. And besides, the world will adapt. It always does. AI isn't optional, there's no putting the genie back in the bottle - and personally I don't want to. But I also don't want to be stupid about the stakes. Getting our whole species killed for lack of foresight would be deeply, deeply embarrassing.
testHNac 4y ago
Let's fix that - The Danger is that People who are already driving us insane will start using AI to make their job a little easier. Social Media companies will soon be taken to task by News media for allowing AI created content on their platforms. 2024 is going to be interesting.
- omginternets 4y ago
  This. I am definitely worried about the implications of AI, but just like algorithmic advertising and television before it, it’s amplifying something that’s already out there, not creating an ontologically different threat. The issue is that quantity can become a quality.
- pjc50 4y ago
  News media are also going to get into the AI clickbait race. I wonder if the first accidental misrepresentation of a generated image as reality has already happened. And the first intentional one.
- yamtaddle 4y ago
  One aspect I expect to see play out: Any entity interested in either the truth, or in maintaining some kind of reputation, will need to keep humans in the loop when using these systems. Language models might multiply e.g. ad copy output 10x per worker, and allow micro-targeted campaigns that were impractical before, but it won't allow, say, a 1000x increase, until or unless we can trust these systems not to produce undesirable output when not checked by a human. Ads are tied to brands which will hesitate to put their reputations in the hands of language models without a human verifying that the output is OK. Likewise, any entities wishing to use these to help with writing illuminating, factual works, may see a large benefit, but it'll be limited. 2x, 5x, something like that. Propaganda, though? Misinfo campaigns, astroturfing, where you hide behind sockpuppets and shell companies anyway? Who gives a shit if one out of every few hundred messages isn't quite right? Worst case, you burn a sockpuppet account. Those can leverage these to the fullest. 1000x output per person involved, compared with, say, 2016 and 2020, may actually be something we can expect to see.
  - 4y ago
AnimalMuppet 4y ago
How about the danger that AI itself becomes insane? On the HN "new" page as I write this: "Google and Microsoft’s chatbots are already citing one another's misinformation". As AIs produce more and more content, one danger is that they feed back on each other, with AIs training on more and more AI-generated content, and the connection with reality becomes more and more diluted. I don't think that's going to produce some new kind of wisdom; I think it's going to produce raving lunacy.
- celdon25 4y ago
  Hot take: The world (most of the time) already works that way (more or less), but people usually don’t notice for whatever reason. One possible bias here is that we expect people to make mistakes and computers to get it right. Don’t forget that the vast majority of people don’t get it right. Perhaps this is just the latest and most obvious consequences of the Internet’s tendency to give the podium to some of the least qualified to speak on whatever subject. I think if we go back to the drawing board there, we could be in a slightly better situation. Quora made a nice attempt to do this but fell way short of it’s potential.
- dd36 4y ago
  Just like we’ve seen in some human populations.
- ChatGTP 4y ago
  "New from Silicon Valley...AI Therapists"
- deleted 4y ago
  [deleted]
NikkiA 4y ago
From my point of view, it seems that AI is being kept only to those that can afford to pay for it. So it's not much of a threat to me or other people below the poverty line. If it eats the rich for us, I'm fine with that.
- circuit10 4y ago
  What makes you think an AI with the ability to take actions in the real world will only affect the people who pressed the button to start it? This is like not being worried about a nuclear war because you think it will only affect politicians and the military (I’m not trying to make a statement as to whether I think nuclear war is likely or not because I don’t know, just using it as an example)
  - NikkiA 4y ago
    The title of the post, that it's more likely to drive us (assumedly meaning it's users) insane, rather than destroy us.
    - Jensson 4y ago
      You will interact with AI whether you like it or not, when you call for customer support, they will send spam ad messages to you, bots on dating sites etc.
- sh4rks 4y ago
  It's the opposite. Only the rich will be able to afford AI-free spaces. Everyone else will become an AI-assisted drone.
- throwuwu 4y ago
  You haven’t been paying attention
NoZZz 4y ago
I think I've seen this episode of The Orville.
jjulius 4y ago
... so, same difference? I mean, even with his full quote, the outcome is ultimately the same - humans are gone. >... “the danger isn’t that a new alien entity will speak through our technology and take over and destroy us. To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.”
xwdv 4y ago
The real danger will start when one day an AI emerges that is somehow able to crack any encryption in polynomial time, through methods obfuscated to us by its neural network. And if this model somehow becomes widely available, the tech world will be brought to its knees overnight. Not sure what will become of society then.
qikInNdOutReply 4y ago
Technology implies beligerence http://akkartik.name/post/2012-11-21-07-09-03-soc http://akkartik.name/post/2012-11-21-07-09-03-soc The problem i see, that someone might send our "primitive" AI into a hostile environment, were it has to compete against other AI, creating a "take-over" and a "devensive" monster, similar to the go automaton. While the real world training data might be dripping, the speed in which a NN under evolutionary pressure against itself might evolve could go through the roof.
dukeofdoom 4y ago
Alex talked about this 6 years ago. It's not the AI, it's the people controlling the AI developing a God complex https://www.youtube.com/watch?v=UZPCp8SPfOM&t=6610s https://www.youtube.com/watch?v=UZPCp8SPfOM&t=6610s
- throwaway29812 4y ago
  "Alex talked" Alex who?
1970-01-01 4y ago
Tldr: AI will become so good that we're going to forget how to "live, laugh, love" and instead we slowly insane ourselves out of existence. Yes, I used insane as in a verb. That's how he intends this threat will unfold.
sirsinsalot 4y ago
Anyone questioning the author's intention should read one of his books, "Who Owns the Future?" It was written sometime ago, and I think Sam Altman read it as a handbook on power concentration using AI rather than the human-centric approach it was laying out. Personally I wish Lanier wasn't as right about many things as he is, because I lose a little faith in humanity each time.
- ChrisMarshallNY 4y ago
  I have nothing but respect for the chap. I never wanted to respect him, as I always thought he was one of those "too good to be true" people, and was mostly a paper tiger. It turns out that he's the real deal, and has been right about a lot of stuff.
  - dr_dshiv 4y ago
    Such as? I have my skepticism too.
    - ChrisMarshallNY 4y ago
      I’m not particularly interested in going into a back-and-forth on this. He’s sort of like Edward Tufte; lots of ego, but earned, and not for everyone. I like your job title. Always up for more “human,” in our design.
  - jacquesm 4y ago
    There are lots of parallels between Jaron Lanier and Richard Stallman. Cory Doctorow is another one I would put in that list, as well as SF writer Charles Stross. They are all pretty good at looking ahead.
- worldsayshi 4y ago
  I feel that if smart people spent more time writing books about how good outcomes could come about rather than warning about bad outcomes powerful actors wouldn't have so many dystopian handbooks lying around and might reach for those positive books instead.
deleted 4y ago
[deleted]
motohagiography 4y ago
The best analogy would be that AI will do to culture today and over the next decade or so what pop music did to culture in the 1950's onward. The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc, but even as an afficianado for rock'n'roll and alternative music, it really does give you a head full of garbage. Punk was fun, but I could probably do without exalting the sniveling self pity that made it appealing. For example, if your beliefs about love and relationships come from 20th century love songs, you're likely a rube, or worse, needy and codependent. Arguably, the cliche of the boomer narcissist is the direct result of being relentlessly propagandized by the music industry to believe in similarly inferior things. Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. Pre-internet, young people fully tribalized based on their taste in pop music. Sure, it's culture, it's entertaining, and some of it is even beautiful, but it was designed to exploit your sentimentality. If you think pop music tugged at the heart strings, just wait until the kids get ahold of AI platforms. Imo, the products of AI will be at least as ecstatic and even addictive as pop music and social media, and the cultural consequences will likely rhyme. The antidote to all these trends was always the counterfactual that maybe, just maybe, people will find some higher principle to form their identities around and be able to experience these amusements objectively without imagining themselves as "becoming" something as a result of using them, but who are we kidding, they'll believe whatever entertains them. Imagine all the people.
- pjc50 4y ago
  > Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim. > The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc Almost always with a strong racial bias.
  - yamtaddle 4y ago
    > > Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. > This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim. Eh, it's a concise expression of an idea Charlie Brooker (of Black Mirror) and others have been promoting in their work for years. The famous-at-least-on-here Meditations on Moloch covers it, IIRC. Not really out-there or new. Capitalism learned to pull the teeth of any new counterculture, and turn it into a product, and the history of postwar pop culture is basically its learning to do that, then doing it over and over. The same observation dates to at least the '80s, from actual humans, it's not some gibberish mash-up created by an AI.
ReaderView 4y ago
[dead]
costanzaDynasty 4y ago
People have spent the last decade modify their behavior to please algorithms. They've become indistinguishable from bots. Cattle herded into segregated pens. Being more human is the only possible defense, warts and all.
- smartmic 4y ago
  Yeah, I agree, many of us have become bots or zombies, though still being basic humans and communicating as humans. If you were a techie who wants to create a new algorithm to which we shall obey, you had to learn the language of computers to do so. Now this has changed as well. The computers have learned to speak our —human— language. That means they will also adapt to our behavior, which means the spiral into the insanity Jaron Lanier was talking about could possibly go faster… EDIT: So yes, a return to what makes us human, to nature, with an awareness of history and philosophy would be very desirable and quite appropriate in these and future times.
- jimkleiber 4y ago
  I'm really wondering though because I feel kind of at a loss, what are some things you think we can do to be more human?
mikrl 4y ago
Are we not already insane? Do you not see sociopathic tendencies in our industry already? Has our humanity not already been tested by existing and conceptually simpler technologies?
- studentik 4y ago
  > Are we not already insane? As long as we aim to separate objective truth from objective false it seems that sanity is still present.
  - pjc50 4y ago
    The one thing AI does not even attempt to do. (and arguably can't, because it really is a "brain in a jar" being fed an entirely controlled view of the universe)
    - AnimalMuppet 4y ago
      The one thing this form of AI does not even attempt to do.
mydriasis 4y ago
It has already driven me insane. I am the first casualty ;)
RamblingCTO 4y ago
It's already driving me crazy with low-effort spam bullshit powered by anything GPT.
m0llusk 4y ago
Collapse of society: It's generated troll memes all the way down.
boplicity 4y ago
In the future we'll all have AIs listening to everything we listen to. These AIs will also have sensors on our body to detect our emotional state (heck, maybe even inferring our thoughts based on this.) The AIs will then provide contextual advice. We'll generally say what it tells us to say; do what it suggest we do. This will happen for everyone; we'll end up, effectively, being the "mouthpieces" for AIs talking to one another. The technology for all of this already exists. It's just a matter of time. Right?
- jacobr1 4y ago
  Yep, this is coming. But it is unclear how interactive will be in practice and what the feedback mechanisms will look like.
- Tubbe 4y ago
  Who would want that and why?
  - pjc50 4y ago
    Targeted advertising in exchange for entertainment.
  - anon7725 4y ago
    > Who would want that and why? Nobody wanted a perpetual outrage machine, but we have Twitter. Nobody wanted ever-present surveillance, but we have ad networks. Nobody wanted identity theft, but we have a thousand attack vectors.
    - worldsayshi 4y ago
      It's like innovation by obsessive compulsion.
  - deleted 4y ago
    [deleted]
  - NovaDudely 4y ago
    Thats a big part of advertising. Make people feel inadequate if they don't have these things. FOMO in action.
__derek__ 4y ago
> “[...] To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.” It's surprising that the article doesn't mention Infinite Jest.
whalesalad 4y ago
Is going insane not a form of being destroyed?
concinds 4y ago
The interview isn't very intellectual, and even rambles, but blame the reporter for that. Lanier's a great thinker. I'll add my own danger: AI/VR could lead us to each live in our own realities. When you watch the evening news, it'll be specifically written for you, and won't get any of the scrutiny that a broadcast watched by millions would get. Or, you go watch the president's State of the Union, and get served a custom speech written to appeal to your particular psychological profile. This'll be possible on day, and it gives me Deus Ex vibes.
agentultra 4y ago
I'm more concerned that the people using LLM's will drive me insane. I hope that I don't get flooded with LLM-generated emails and documentation from colleagues and co-workers that don't bother to proof-read what they send me. It will be a frustrating day when I reply to them with a question and they don't have an answer because they don't even know what they sent me. Multiply that by all of the boilerplate code I'll end up reading. No pleasant abstractions that hide complexity and are reinforced by laws checked by properties or proofs; abstractions that make the problem easier to understand and communicate. Instead, endless bowls of oatmeal code: similar, bland, and laborious to read. Who knows if the model "hallucinated" this code and whether the author bothered to check it before asking for a review. Who knows how long before I get bored and worn down and stop bothering to read it myself. And then we're supposed to change our job titles to "Prompt Engineers"? How absurd is that. Nothing about what you do is creative, human, or interesting. Now your job is to feed sentences into an LLM to get it to do all that work for you... how long until that dead-end job gets replaced? Maybe this could drive us all insane from burnout and nihilism.
- volkk 4y ago
  i have these exact concerns. the concept of on overload of information that means very little that will get me to stop caring/reading literally anything on the internet at a certain point
  - ahepp 4y ago
    I think it’s more likely human-to-human communication will adapt (dare I say improve?) than cease
- geraneum 4y ago
  My boss (head of engineering) loves ChatGPT and he uses it more and more in work. Recently he just wrote an crucial feedback after one of our meetings and the moment I started reading, it was obvious it's ChatGPT (confirmed by ZeroGPT later) as he does not talk like that at all. It read like all the other ChatGPT responses you read on Internet.
rambojohnson 4y ago
Utterly weary am I of the ceaseless banter regarding the pessimistic, antiquated artificial intelligence discourse perpetuated on Hacker News... boomer doomerism. boring. doesn't help that the dude looks like he crawled out of the last century.
grej 4y ago
https://archive.is/yJaph https://archive.is/yJaph
dahwolf 4y ago
In recent times we've already significantly given up on our humanity. The lack of shared institutes (church, bars, etc.), remote studying, remote work, ecommerce, personal contact via chat, social media, these all point in the same direction of a contactless society where we rarely interact with the physical world and its people. It stands to reason that AI will only accelerate this further. It will be convenience on steroids. Your AI ear piece isn't going to tell you to throw it into the bin and go for a walk in the forest. It's going to tell you that you need to buy more stuff and it knows exactly what it is that you need. It's also going to feed you non-stop ultimate entertainment, custom generated for you and you only. In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice. A long running trend of outsourcing humanity to technology. Good news for philosophers though, they finally might have their moment of actual relevancy. In particular to answer the question: what is the point of anything, really?
- roryisok 4y ago
  > In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about? Soon our AIs will carry on our social lives and we'll just lie in the dark with tubes in us. We become the computers, and the computers become us, and the robots have finally won.
  - red-iron-pine 4y ago
    > I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about? We already have this. Secretaries. Automated Happy Birthday emails. When I was in a sales engineering role our sales team had a admin assistant who would sent out follow-ups, check-ins, and other correspondence (e.g. customer made a big public release, so congratulate them, etc.). This is just another example of robots takin ur jerbs, basically.
gitfan86 4y ago
It is amazing how many people still have not realized what is going on. The AI is going to be vastly superior to humans in MANY ways very soon. And here we are arguing if it is "actually" smart, or "actually" conscious. Do Ants think that humans form "real" colonies or "real" organizations?
codr7 4y ago
I've been saying this for a long time, ever since ML seriously entered the scene: The singularity is the least of our worries, AI will fuck up society beyond repair long before we reach that point because we give it way more trust and responsibility than it deserves. It's the Eliza-syndrome, as long as a computer shows the least sign of possessing human qualities we're more than happy to fill in the blanks with pixie dust and unicorns.
- wslh 4y ago
  Yes, the core issue is trust: when you trust somebody or something your brain stops working. Far for being a ludite myself but blind less trust is very dangerous.
- thundergolfer 4y ago
  The stories of how much trust layman and even certain ‘experts’ put on ELIZA back in that 60s is wild. Given how much better GPT models are, hopefully there isn’t a proportional increase in trust and anthropomorphizing.
- LesZedCB 4y ago
  i like to call it the "stupid singularity"
theptip 4y ago
If you claim “these [AI risk] concerns make no sense” then you either lack imagination, are willfully ignorant, or are selling something. It’s perfectly reasonable to say something like “I think it’s very unlikely because I disagree that [insert causal link in the model] is plausible.” But to claim that the whole set of concerns are nonsensical is lazy thinking. You see this a lot (and Robin Hanson and PG have commented on this dynamic recently) where a small group makes a bunch of very specific claims, which get dismissed by the “mainstream” without actually engaging with or understanding them. So in this case, “[the concerns] make no sense” should be read as “I did not bother to try to understand these claims, but I don’t like the conclusion”, not any particular argument about whether they are logically sound. Ignore.
- CadmiumYellow 4y ago
  It's the Lucille Bluth response: "I don't understand the question and I won't respond to it."
- PoignardAzur 4y ago
  This is very eloquently put.
- deleted 4y ago
  [deleted]
- tootie 4y ago
  That's not what he said and not even what he was asked. He definitely acknowledged the dangers up to and including "human extinction" but wanted to make sure the question was couched in the right context.
deleted 4y ago
[deleted]
timcavel 4y ago
[dead]
bartislartfast 4y ago
In the short term, I'm concerned about the energy required to run things like ChatGPT and how everyone is cramming it into everything. Soon every todolist app on your phone will need a supercomputer running in the background to auto arrange your grocery list, and we'll double or triple our datacenter footprint within a few years.
- throwuwu 4y ago
  Running the models isn’t the expensive part and is subject to a lot more optimization
01100011 4y ago
As I've said for years, the robots don't have to kill us, they just have to convince us to kill ourselves.
skee8383 4y ago
i don't fear this stuff. at the end of the day if the digital world becomes detrimental to my health i can just call up my ISP and cancel my service. good luck rotting my brain when you have no internet connection to my house.
Mobius01 4y ago
I've read this short story about the singularity years ago, written by a scientist from UW-Madison and although the writing isn't great, it has always stayed with me. Recent developments made me think of it, and the premise is precisely that - the group that develops AGI uses it to control the markets and drives everyone else insane through economic disruption, while staying entirely opaque. https://www.ssec.wisc.edu/~billh/g/mcnrsts.html https://www.ssec.wisc.edu/~billh/g/mcnrsts.html
labrador 4y ago
I'm really enjoying how "Imposter Syndrome" is actually justified in the tech industry as many influencer posts or posts by people riding on reputations from past accomplishments are actually imposters and making fools of themselves. Conversely, some very insightful posts are being made by the quiet ones with few followers who aren't necessarily even in the tech industry. I've been through sea changes before, when the old guard falls by the wayside and a new group rises to meet the new challenges. This is certainly one of those times.
- rideontime 4y ago
  I'm struggling to understand what point this comment is trying to make. I thought it was referring to Lanier as one of "the quiet ones with few followers," but then it claimed that the "old guard" is falling by the wayside, and Lanier's hardly new.
  - labrador 4y ago
    I wasn't actually referring to Lanier because he's mostly been a lone voice in the woods, even as he's highly influential in some circles. I was actually referring to the noisy self-promoters currently polluting Hacker News with their AI posts, but now that you mention it, maybe Lanier fits that category of insightful people with few followers. How long has he been telling people to delete social media? How long have people been ignoring him? The world is finally coming around and I for one would be glad for him to become as famous as some of the noisy ones.
    - unconed 4y ago
      Funny, I have mentally filed him as a designated person who is consulted by media to offer opinions on tech despite having no particular claim to familiarity with said tech.
aaroninsf 4y ago
Something I've found myself observing in disparate domains: the only defense we have against applications of AI we don't like, is the application of more AI. This is a natural implication of AI being augmented-intelligence force-multipliers at the moment. When your opponent takes performance enhancing drugs, or wears a mecha suit, the only way to compete with them or defend against them is adoption of similar technology. The Prime Directive in Star Trek was prescient. AI arriving unevenly—and first under the direction and control of hands which have thus far proven disinterested in the common welfare where it interferes with personal benefit—looks like it will be a civilizational exercise indistinguishable from the delivery of disruptive technology from another species. Of particular personal concern is that the synergies where it meets other technological social problems against which we have few or no defenses, specifically, surveillance and its active counterpart, sentiment control. I am less concerned about needing AI tools to validate AI-generated code; and more concerned about the 2024 election cycle being opportunity for an arms race in application of AI to game politics.
caycep 4y ago
what's the difference?
holoduke 4y ago
AI will encapsulate us slowly till the only thing left is your digestive system to provide energy to the AI systems.
polynomial 4y ago
Assuming it's not already just a reflection of a pre-existing lack of sanity, or that it's arguably a symptom of a the descent into madness, not the cause.
audessuscest 4y ago
too late
r1b 4y ago
If you enjoy this kind of thinking I'd recommend [1]. > Electric speed mingles the cultures of prehistory with the dregs of industrial marketeers, the nonliterate with semiliterate and the postliterate. Mental break-down of varying degrees is the very common result of uprooting and inundation with new information and endless new patterns of information. [1] https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf
UberFly 4y ago
This feels a bit like Bill Joy's article about nano-technology. I think it's just a healthy fear the the unknown future. AI will be refined into usable tools, mostly good with some other unintended negative consequences. Sounds like a lot of things.
thomastjeffery 4y ago
> There are two ways this could go. One is that we pretend the bot is a real thing, a real entity like a person, then in order to keep that fantasy going we’re careful to forget whatever source texts were used to have the bot function. Finally someone who, at least nearly, gets it. > The other way is you do keep track of where the sources came from. And in that case a very different world could unfold where if a bot relied on your reporting, you get payment for it, and there is a shared sense of responsibility and liability where everything works better. The term for that is data dignity. Of course, that's an obvious step in the other direction: isn't the original point to breathe life into the data? If we have to track who originated what, that scaffolding is going to really get in the way. Then again, this is a good moment to recognize the futility of text inference models (LLMs). If we can't understand what path goes from input to output, how useful can the thing be? If we can't choose the path, we can't choose the output. Who can choose: the thing itself? If the whole thing is a black box making its own choices, then how do we interact with it except to personify that black box? Remember, that was option 1. The main reason anyone is excited about LLMs is that those models have been personified. That problem began the moment we started calling them AI. --- > We have to say consciousness is a real thing and there is a mystical interiority to people that’s different from other stuff because if we don’t say people are special, how can we make a society or make technologies that serve people? Mystical. It's an important observation. We really don't know how we think. And that is all the evidence we need: AI does not exist. There is no artificial person. We are free to stop personifying technology. Doing so would make all of these conversations a lot less convoluted, and a lot more direct. We are wasting a lot of effort talking past the very understanding about these machines that is necessary to improve them. Instead we are trying to understand "AI personified" a fictional character that we have never met; and trying to improve it.

352 points by atchoo 4y ago

This may kill the WWW as we know it. I can almost see an alternate 'web' finally taking off, but not really, as the majority just want TikTok left alone.

johnaspden 4y ago

Those whom the gods wish to destroy, they first make mad.

PaulHoule 4y ago

It is great to see how Lanier has grown up. Seriously though. once people take the limiters off chatbots you will see "artificial bullshitting" generating duckspeak on subjects from anti-vax and climate denial to cryptocurrencies, longtermism, transsexual maximalism and police abolitionism. And people who believe those ideologies will eat it up.

pavlov 4y ago
I think the scariest part is not even the content but the connection that people can develop with these bot personalities. A demagogue who knows your intimate desires and fears and talks just to you, seems to care about you, is available 24/7. Your very own Personal Jesus (or Personal Hitler). Every communications revolution seems to lead to a genocide. Radio and TV in the 20th century (Germany, Rwanda etc.) Social media in the past decade (Myanmar). The upcoming AI-instigated genocide is a terrifying thought. The Rwandans didn’t think they’d start killing their neighbors even as rhetoric on the radio got harder. Chatbot-dependent Americans may not see it coming either.
- AnimalMuppet 4y ago
  The parent poster's username seems appropriate... (Note well: This is not a "username checks out" drive-by dismissal. I think the point is perfectly valid.)
- PaulHoule 4y ago
  Look at the parasocial relationships that people have had with L. Ron Hubbard, Eliezer Yudkowsky, and even camgirls. (Particularly followers of LRH and EY frequently speak and write like a chatbot trained on their writings.) I think most "freezoners" (renegade Scientologists) believe they've had telepathic conversations with "Ron". Ron has certainly written enough that a chatbot could be trained to converse in his style, maybe even write the OT levels that Ron never got around to writing. Since Harold Lasswell people people have been hip to the use of content analysis to predict war and genocide which is possible because the radio, television, and newspapers are all public. However the Iranian revolution of 1979 was not televised, instead it was promoted through relatively unobservable cassette tapes passed through the underground. Since social media shows something different to everybody there might not be any overall surveillance that warns us of upcoming danger. It doesn't have to go so far as genocide, individual acts of violence are bad enough. There is no magic technique that can take an average person and make them into a "Manchurian Candidate" but if you find somebody who is directionless, socially isolated and vulnerable you can very much lead them into a rabbit hole and lead them into antisocial behavior. Chatbots could do this with superhuman patience. For instance, a follower of the black pill incel who calls himself "Wheat Waffles" became a mass shooter. If somebody took the "lesswrong" ideology seriously they might attempt to assassinate an A.I. research and I'd go so far to say that they're just cosplaying because if they were serious one of them would have done it already.
beepbooptheory 4y ago
With respect, its bad enough not even directly commenting on the article and instead using the space as a soapbox, but at least please save the rest of us from your perverted fixations. There are other forums!
leokeba 4y ago
Can you elaborate on what you mean by "transsexual maximalism" as an ideology ?
- parthianshotgun 4y ago
  Maybe it was for effect, like 'psychic bedouinism'
- johnaspden 4y ago
  We want to change everyone's gender, as often as possible; and will devote superhuman resources to spreading this philosophy over as much of the available future lightcone as physics permits.
  - leokeba 4y ago
    Yeah right, it really sounds like what most transgender people are advocating for indeed. I asked ChatGPT to enlighten me on the subject, here's what it had to say : The term "transsexual maximalism" is not a widely recognized or established ideology, but it has been used by some individuals to describe a perspective on transgender issues that emphasizes a strong and uncompromising approach to advocating for the rights and recognition of transgender people. At its core, transsexual maximalism is an approach that prioritizes the experiences and perspectives of transgender people, particularly those who have undergone medical interventions such as hormone therapy or gender confirmation surgery. Proponents of transsexual maximalism often believe that these interventions are necessary to fully alleviate the distress and dysphoria experienced by many transgender individuals. Transsexual maximalism can be seen as a response to more moderate or incremental approaches to transgender advocacy, which may focus on incremental policy changes or attempts to work within existing systems of power and institutions. Some proponents of transsexual maximalism may view these more moderate approaches as insufficient or even counterproductive, arguing that they do not go far enough in challenging the underlying structures of cisnormativity and transphobia. However, it's worth noting that the term "transsexual maximalism" has not been widely adopted or embraced by the broader transgender community, and some individuals within the community may have different perspectives or priorities when it comes to advocacy and activism. Me : Why would people consider this a threat ? ChatGPT : Some people might consider transsexual maximalism as a threat because it is a highly assertive and uncompromising approach to advocating for transgender rights and recognition. This approach can challenge existing social norms and institutions that uphold cisgender privilege and transphobia. For example, some individuals who are opposed to transgender rights might view transsexual maximalism as a threat to traditional gender norms and the gender binary. They may feel uncomfortable with the idea that gender identity can be different from the sex assigned at birth, or that medical interventions such as hormone therapy or gender confirmation surgery are necessary for some transgender people to live authentically. Additionally, some people might see transsexual maximalism as a threat to their own power or privilege. Transgender individuals have historically been marginalized and oppressed, and advocating for their rights and recognition can challenge the existing power structures that uphold cisgender privilege. This can cause discomfort or resistance among those who benefit from these power structures. It's important to note that concerns about transsexual maximalism as a "threat" are often rooted in transphobia and a lack of understanding or empathy for the experiences of transgender individuals. Transgender people have the right to advocate for their own rights and recognition, and it is not a threat to anyone else's well-being or identity.
Turing_Machine 4y ago
> It is great to see how Lanier has grown up. Indeed. It would be interesting to study how he became one of the go-to guys for hot takes on any new technology, considering that he hasn't developed any new technology in the last 40 years or so. Only Cory Doctorow, perhaps, matches him when it comes to famous for being famous. Sorry, an awesome set of dreads doesn't make you a "guru".
- sirsinsalot 4y ago
  I'm not sure any of the fundamentals have changed in 40 years, so being able to speak as an authority isn't really dependent on knowing React or that Java 19 got whatever new syntax. That attitude is why ageism is so prevalent in engineering. Maybe read a few of his books before writing someone off because you think they're out-dated.
  - Turing_Machine 4y ago
    > I'm not sure any of the fundamentals have changed in 40 years Really? He's holding forth on GPT and similar technologies. Those represent dramatic changes from the state of the art 40 years ago. P.S. what makes you think I haven't read his work? I certainly haven't read all of it, but the significant portion that I have left me...underwhelmed.
    - sirsinsalot 4y ago
      He is talking about the impact on society of things perfectly imaginable over 40 years ago and detailed across the whole of sci-fi. He isn't talking about the effectiveness of a loss function or something.
- hoppersoft 4y ago
  First, I have to admit that I was negatively biased by his appearance. I have a hard time believing anything that some guy who looks like he smokes metric tons of weed and dorks around on his guitar all day has to say. That being said, the tech world has a bad habit of letting "visionaries" rest on their laurels. He'd have a lot more credence if he was actively developing AI and had more than gut feel to contribute.

stareatgoats 4y ago

> “This idea of surpassing human ability is silly because it’s made of human abilities.” Shows the level of insight from this "guru". The truth is we don't know how far the work being done on artificial intelligence is going to go. For now it will continue to develop and acquire more and more autonomy, just because that is the nature of our existence: better and more efficient will replace the lesser so. So, we may have potentially given birth to a new sentient being that will go on to live its own "life" (within 100, 500, 1000 years?), or we might be able to constrain it so it so that it will always be in the service of humans. We simply don't know at this stage, but my money is on the former TBH.

nathanaldensr 4y ago
"AI" is not currently autonomous; its algorithms that do exactly what their creators tell them to do. They run on binary computers that only do exactly as they are told.
- circuit10 4y ago
  That’s not true, current machine learning algorithms involve no manual programming past the training and inference code and it’s extremely difficult to predict what they will do without just trying it and seeing. I think this video is a nice introduction to the basic concept to how a computer can figure things out automatically without being manually programmed and without the creators understanding the “why”: https://youtu.be/qv6UVOQ0F44 https://youtu.be/qv6UVOQ0F44 ChatGPT is much more complicated than the AI in that video but it shows some of the basic concepts
  - goatlover 4y ago
    LLMs generate text. They're built to generate text. That they generate some kind of textual output is entirely predictable. Same with image generators. They will generate some kind of image given a prompt. They're not Skynet.
    - circuit10 4y ago

strken 4y ago

From reading this, I don't get the impression that Lanier has any objective reason to believe the world won't be destroyed as the direct result of AI. If he does have a reason, the reporter certainly doesn't devote any space to analysing it, or to explain why dying from AI-induced insanity is different from being destroyed.

BigCryo 4y ago

Can we please put off worrying about dangerous AI for a couple hundred years? Chat GPT is a chatbot and not much else.. it is not the harbinger of an era of dangerous AI

antibasilisk 4y ago
Stable Diffusion was used to produce convincing images of Donald Trump being arrested. A guy used GPT to post comments on 4chan without people noticing. A 'mere chatbot' can do much more damage than you think.
circuit10 4y ago
Hundred years? If the current rate of progress holds we have to start worrying about it in 5 or 10 years, the earlier we can come up with solutions the better because it’s a very hard problem
- AnimalMuppet 4y ago
  Naw, it's simple. We're talking about an AI achieving human abilities, well, we can protect against dangerous AIs just as well as we protect against dangerous humans... Oh. Oh dear.
  - javajosh 4y ago
    Remember that AI work with electrons, and we are of atoms. We should focus on where electrons control atoms, and reduce those points. Of particular concern is that AI may be a very strong investor with the right prompts. AI could also figure out how to use any other software. Which can be used to gain access to any marketplace, including the dark ones. Which means AI can use money (electrons) to pay others to modify the world (atoms). Of course, there is already a problem, as you point out. Humans shouldn't have access to these markets either! But yeah to specifically prevent electron-on-atom violence we need to limit AI physical degrees-of-freedom. by limiting marketplaces. National/global security, not personal morality, should guide these new regulations. We need to end all drive-by-wire automobiles, and electronic locks. Too many services are habituated to act on electronic signals without human confirmation - particularly the police. There needs to be an unbroken verbal chain between the person who saw an event and the person doing the law enforcement. Breaks in the human chain should be treated very seriously -they should be treated as firing offenses, at least. There are many other similar changes we're going to need to make. Some folks aren't gonna like this. Regulations are inherently evil, they say. Maybe the mental model should be more like we're the big bad gorilla in the cage. But now there's a tiger in the cage. Regulation restrains the tiger. Also, some folks aren't gonna like it no matter what change you need. The fact of not liking it doesn't mean we don't need it, and it doesn't mean it won't get done. We have to trust that our leaders don't want to die. And besides, the world will adapt. It always does. AI isn't optional, there's no putting the genie back in the bottle - and personally I don't want to. But I also don't want to be stupid about the stakes. Getting our whole species killed for lack of foresight would be deeply, deeply embarrassing.

testHNac 4y ago

Let's fix that - The Danger is that People who are already driving us insane will start using AI to make their job a little easier. Social Media companies will soon be taken to task by News media for allowing AI created content on their platforms. 2024 is going to be interesting.

omginternets 4y ago
This. I am definitely worried about the implications of AI, but just like algorithmic advertising and television before it, it’s amplifying something that’s already out there, not creating an ontologically different threat. The issue is that quantity can become a quality.
pjc50 4y ago
News media are also going to get into the AI clickbait race. I wonder if the first accidental misrepresentation of a generated image as reality has already happened. And the first intentional one.
yamtaddle 4y ago
One aspect I expect to see play out: Any entity interested in either the truth, or in maintaining some kind of reputation, will need to keep humans in the loop when using these systems. Language models might multiply e.g. ad copy output 10x per worker, and allow micro-targeted campaigns that were impractical before, but it won't allow, say, a 1000x increase, until or unless we can trust these systems not to produce undesirable output when not checked by a human. Ads are tied to brands which will hesitate to put their reputations in the hands of language models without a human verifying that the output is OK. Likewise, any entities wishing to use these to help with writing illuminating, factual works, may see a large benefit, but it'll be limited. 2x, 5x, something like that. Propaganda, though? Misinfo campaigns, astroturfing, where you hide behind sockpuppets and shell companies anyway? Who gives a shit if one out of every few hundred messages isn't quite right? Worst case, you burn a sockpuppet account. Those can leverage these to the fullest. 1000x output per person involved, compared with, say, 2016 and 2020, may actually be something we can expect to see.
- 4y ago

AnimalMuppet 4y ago

How about the danger that AI itself becomes insane? On the HN "new" page as I write this: "Google and Microsoft’s chatbots are already citing one another's misinformation". As AIs produce more and more content, one danger is that they feed back on each other, with AIs training on more and more AI-generated content, and the connection with reality becomes more and more diluted. I don't think that's going to produce some new kind of wisdom; I think it's going to produce raving lunacy.

celdon25 4y ago
Hot take: The world (most of the time) already works that way (more or less), but people usually don’t notice for whatever reason. One possible bias here is that we expect people to make mistakes and computers to get it right. Don’t forget that the vast majority of people don’t get it right. Perhaps this is just the latest and most obvious consequences of the Internet’s tendency to give the podium to some of the least qualified to speak on whatever subject. I think if we go back to the drawing board there, we could be in a slightly better situation. Quora made a nice attempt to do this but fell way short of it’s potential.
dd36 4y ago
Just like we’ve seen in some human populations.
ChatGTP 4y ago
"New from Silicon Valley...AI Therapists"
deleted 4y ago
[deleted]

NikkiA 4y ago

From my point of view, it seems that AI is being kept only to those that can afford to pay for it. So it's not much of a threat to me or other people below the poverty line. If it eats the rich for us, I'm fine with that.

circuit10 4y ago
What makes you think an AI with the ability to take actions in the real world will only affect the people who pressed the button to start it? This is like not being worried about a nuclear war because you think it will only affect politicians and the military (I’m not trying to make a statement as to whether I think nuclear war is likely or not because I don’t know, just using it as an example)
- NikkiA 4y ago
  The title of the post, that it's more likely to drive us (assumedly meaning it's users) insane, rather than destroy us.
  - Jensson 4y ago
    You will interact with AI whether you like it or not, when you call for customer support, they will send spam ad messages to you, bots on dating sites etc.
sh4rks 4y ago
It's the opposite. Only the rich will be able to afford AI-free spaces. Everyone else will become an AI-assisted drone.
throwuwu 4y ago
You haven’t been paying attention

NoZZz 4y ago

I think I've seen this episode of The Orville.

jjulius 4y ago

... so, same difference? I mean, even with his full quote, the outcome is ultimately the same - humans are gone. >... “the danger isn’t that a new alien entity will speak through our technology and take over and destroy us. To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.”

xwdv 4y ago

The real danger will start when one day an AI emerges that is somehow able to crack any encryption in polynomial time, through methods obfuscated to us by its neural network. And if this model somehow becomes widely available, the tech world will be brought to its knees overnight. Not sure what will become of society then.

qikInNdOutReply 4y ago

Technology implies beligerence http://akkartik.name/post/2012-11-21-07-09-03-soc http://akkartik.name/post/2012-11-21-07-09-03-soc The problem i see, that someone might send our "primitive" AI into a hostile environment, were it has to compete against other AI, creating a "take-over" and a "devensive" monster, similar to the go automaton. While the real world training data might be dripping, the speed in which a NN under evolutionary pressure against itself might evolve could go through the roof.

dukeofdoom 4y ago

Alex talked about this 6 years ago. It's not the AI, it's the people controlling the AI developing a God complex https://www.youtube.com/watch?v=UZPCp8SPfOM&t=6610s https://www.youtube.com/watch?v=UZPCp8SPfOM&t=6610s

throwaway29812 4y ago
"Alex talked" Alex who?

1970-01-01 4y ago

Tldr: AI will become so good that we're going to forget how to "live, laugh, love" and instead we slowly insane ourselves out of existence. Yes, I used insane as in a verb. That's how he intends this threat will unfold.

sirsinsalot 4y ago

Anyone questioning the author's intention should read one of his books, "Who Owns the Future?" It was written sometime ago, and I think Sam Altman read it as a handbook on power concentration using AI rather than the human-centric approach it was laying out. Personally I wish Lanier wasn't as right about many things as he is, because I lose a little faith in humanity each time.

ChrisMarshallNY 4y ago
I have nothing but respect for the chap. I never wanted to respect him, as I always thought he was one of those "too good to be true" people, and was mostly a paper tiger. It turns out that he's the real deal, and has been right about a lot of stuff.
- dr_dshiv 4y ago
  Such as? I have my skepticism too.
  - ChrisMarshallNY 4y ago
    I’m not particularly interested in going into a back-and-forth on this. He’s sort of like Edward Tufte; lots of ego, but earned, and not for everyone. I like your job title. Always up for more “human,” in our design.
- jacquesm 4y ago
  There are lots of parallels between Jaron Lanier and Richard Stallman. Cory Doctorow is another one I would put in that list, as well as SF writer Charles Stross. They are all pretty good at looking ahead.
worldsayshi 4y ago
I feel that if smart people spent more time writing books about how good outcomes could come about rather than warning about bad outcomes powerful actors wouldn't have so many dystopian handbooks lying around and might reach for those positive books instead.

deleted 4y ago

[deleted]

motohagiography 4y ago

The best analogy would be that AI will do to culture today and over the next decade or so what pop music did to culture in the 1950's onward. The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc, but even as an afficianado for rock'n'roll and alternative music, it really does give you a head full of garbage. Punk was fun, but I could probably do without exalting the sniveling self pity that made it appealing. For example, if your beliefs about love and relationships come from 20th century love songs, you're likely a rube, or worse, needy and codependent. Arguably, the cliche of the boomer narcissist is the direct result of being relentlessly propagandized by the music industry to believe in similarly inferior things. Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. Pre-internet, young people fully tribalized based on their taste in pop music. Sure, it's culture, it's entertaining, and some of it is even beautiful, but it was designed to exploit your sentimentality. If you think pop music tugged at the heart strings, just wait until the kids get ahold of AI platforms. Imo, the products of AI will be at least as ecstatic and even addictive as pop music and social media, and the cultural consequences will likely rhyme. The antidote to all these trends was always the counterfactual that maybe, just maybe, people will find some higher principle to form their identities around and be able to experience these amusements objectively without imagining themselves as "becoming" something as a result of using them, but who are we kidding, they'll believe whatever entertains them. Imagine all the people.

pjc50 4y ago
> Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim. > The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc Almost always with a strong racial bias.
- yamtaddle 4y ago
  > > Folk music is mostly dialectic materialist conspiracy theorists singing hymns to their oppressors. > This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim. Eh, it's a concise expression of an idea Charlie Brooker (of Black Mirror) and others have been promoting in their work for years. The famous-at-least-on-here Meditations on Moloch covers it, IIRC. Not really out-there or new. Capitalism learned to pull the teeth of any new counterculture, and turn it into a product, and the history of postwar pop culture is basically its learning to do that, then doing it over and over. The same observation dates to at least the '80s, from actual humans, it's not some gibberish mash-up created by an AI.

ReaderView 4y ago

[dead]

costanzaDynasty 4y ago

People have spent the last decade modify their behavior to please algorithms. They've become indistinguishable from bots. Cattle herded into segregated pens. Being more human is the only possible defense, warts and all.

smartmic 4y ago
Yeah, I agree, many of us have become bots or zombies, though still being basic humans and communicating as humans. If you were a techie who wants to create a new algorithm to which we shall obey, you had to learn the language of computers to do so. Now this has changed as well. The computers have learned to speak our —human— language. That means they will also adapt to our behavior, which means the spiral into the insanity Jaron Lanier was talking about could possibly go faster… EDIT: So yes, a return to what makes us human, to nature, with an awareness of history and philosophy would be very desirable and quite appropriate in these and future times.
jimkleiber 4y ago
I'm really wondering though because I feel kind of at a loss, what are some things you think we can do to be more human?

mikrl 4y ago

Are we not already insane? Do you not see sociopathic tendencies in our industry already? Has our humanity not already been tested by existing and conceptually simpler technologies?

studentik 4y ago
> Are we not already insane? As long as we aim to separate objective truth from objective false it seems that sanity is still present.
- pjc50 4y ago
  The one thing AI does not even attempt to do. (and arguably can't, because it really is a "brain in a jar" being fed an entirely controlled view of the universe)
  - AnimalMuppet 4y ago
    The one thing this form of AI does not even attempt to do.

mydriasis 4y ago

It has already driven me insane. I am the first casualty ;)

RamblingCTO 4y ago

It's already driving me crazy with low-effort spam bullshit powered by anything GPT.

m0llusk 4y ago

Collapse of society: It's generated troll memes all the way down.

boplicity 4y ago

In the future we'll all have AIs listening to everything we listen to. These AIs will also have sensors on our body to detect our emotional state (heck, maybe even inferring our thoughts based on this.) The AIs will then provide contextual advice. We'll generally say what it tells us to say; do what it suggest we do. This will happen for everyone; we'll end up, effectively, being the "mouthpieces" for AIs talking to one another. The technology for all of this already exists. It's just a matter of time. Right?

jacobr1 4y ago
Yep, this is coming. But it is unclear how interactive will be in practice and what the feedback mechanisms will look like.
Tubbe 4y ago
Who would want that and why?
- pjc50 4y ago
  Targeted advertising in exchange for entertainment.
- anon7725 4y ago
  > Who would want that and why? Nobody wanted a perpetual outrage machine, but we have Twitter. Nobody wanted ever-present surveillance, but we have ad networks. Nobody wanted identity theft, but we have a thousand attack vectors.
  - worldsayshi 4y ago
    It's like innovation by obsessive compulsion.
- deleted 4y ago
  [deleted]
- NovaDudely 4y ago
  Thats a big part of advertising. Make people feel inadequate if they don't have these things. FOMO in action.

__derek__ 4y ago

> “[...] To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.” It's surprising that the article doesn't mention Infinite Jest.

whalesalad 4y ago

Is going insane not a form of being destroyed?

concinds 4y ago

The interview isn't very intellectual, and even rambles, but blame the reporter for that. Lanier's a great thinker. I'll add my own danger: AI/VR could lead us to each live in our own realities. When you watch the evening news, it'll be specifically written for you, and won't get any of the scrutiny that a broadcast watched by millions would get. Or, you go watch the president's State of the Union, and get served a custom speech written to appeal to your particular psychological profile. This'll be possible on day, and it gives me Deus Ex vibes.

agentultra 4y ago

I'm more concerned that the people using LLM's will drive me insane. I hope that I don't get flooded with LLM-generated emails and documentation from colleagues and co-workers that don't bother to proof-read what they send me. It will be a frustrating day when I reply to them with a question and they don't have an answer because they don't even know what they sent me. Multiply that by all of the boilerplate code I'll end up reading. No pleasant abstractions that hide complexity and are reinforced by laws checked by properties or proofs; abstractions that make the problem easier to understand and communicate. Instead, endless bowls of oatmeal code: similar, bland, and laborious to read. Who knows if the model "hallucinated" this code and whether the author bothered to check it before asking for a review. Who knows how long before I get bored and worn down and stop bothering to read it myself. And then we're supposed to change our job titles to "Prompt Engineers"? How absurd is that. Nothing about what you do is creative, human, or interesting. Now your job is to feed sentences into an LLM to get it to do all that work for you... how long until that dead-end job gets replaced? Maybe this could drive us all insane from burnout and nihilism.

volkk 4y ago
i have these exact concerns. the concept of on overload of information that means very little that will get me to stop caring/reading literally anything on the internet at a certain point
- ahepp 4y ago
  I think it’s more likely human-to-human communication will adapt (dare I say improve?) than cease
geraneum 4y ago
My boss (head of engineering) loves ChatGPT and he uses it more and more in work. Recently he just wrote an crucial feedback after one of our meetings and the moment I started reading, it was obvious it's ChatGPT (confirmed by ZeroGPT later) as he does not talk like that at all. It read like all the other ChatGPT responses you read on Internet.

rambojohnson 4y ago

Utterly weary am I of the ceaseless banter regarding the pessimistic, antiquated artificial intelligence discourse perpetuated on Hacker News... boomer doomerism. boring. doesn't help that the dude looks like he crawled out of the last century.

grej 4y ago

https://archive.is/yJaph https://archive.is/yJaph

dahwolf 4y ago

In recent times we've already significantly given up on our humanity. The lack of shared institutes (church, bars, etc.), remote studying, remote work, ecommerce, personal contact via chat, social media, these all point in the same direction of a contactless society where we rarely interact with the physical world and its people. It stands to reason that AI will only accelerate this further. It will be convenience on steroids. Your AI ear piece isn't going to tell you to throw it into the bin and go for a walk in the forest. It's going to tell you that you need to buy more stuff and it knows exactly what it is that you need. It's also going to feed you non-stop ultimate entertainment, custom generated for you and you only. In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice. A long running trend of outsourcing humanity to technology. Good news for philosophers though, they finally might have their moment of actual relevancy. In particular to answer the question: what is the point of anything, really?

roryisok 4y ago
> In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about? Soon our AIs will carry on our social lives and we'll just lie in the dark with tubes in us. We become the computers, and the computers become us, and the robots have finally won.
- red-iron-pine 4y ago
  > I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about? We already have this. Secretaries. Automated Happy Birthday emails. When I was in a sales engineering role our sales team had a admin assistant who would sent out follow-ups, check-ins, and other correspondence (e.g. customer made a big public release, so congratulate them, etc.). This is just another example of robots takin ur jerbs, basically.

gitfan86 4y ago

It is amazing how many people still have not realized what is going on. The AI is going to be vastly superior to humans in MANY ways very soon. And here we are arguing if it is "actually" smart, or "actually" conscious. Do Ants think that humans form "real" colonies or "real" organizations?

codr7 4y ago

I've been saying this for a long time, ever since ML seriously entered the scene: The singularity is the least of our worries, AI will fuck up society beyond repair long before we reach that point because we give it way more trust and responsibility than it deserves. It's the Eliza-syndrome, as long as a computer shows the least sign of possessing human qualities we're more than happy to fill in the blanks with pixie dust and unicorns.

wslh 4y ago
Yes, the core issue is trust: when you trust somebody or something your brain stops working. Far for being a ludite myself but blind less trust is very dangerous.
thundergolfer 4y ago
The stories of how much trust layman and even certain ‘experts’ put on ELIZA back in that 60s is wild. Given how much better GPT models are, hopefully there isn’t a proportional increase in trust and anthropomorphizing.
LesZedCB 4y ago
i like to call it the "stupid singularity"

theptip 4y ago

If you claim “these [AI risk] concerns make no sense” then you either lack imagination, are willfully ignorant, or are selling something. It’s perfectly reasonable to say something like “I think it’s very unlikely because I disagree that [insert causal link in the model] is plausible.” But to claim that the whole set of concerns are nonsensical is lazy thinking. You see this a lot (and Robin Hanson and PG have commented on this dynamic recently) where a small group makes a bunch of very specific claims, which get dismissed by the “mainstream” without actually engaging with or understanding them. So in this case, “[the concerns] make no sense” should be read as “I did not bother to try to understand these claims, but I don’t like the conclusion”, not any particular argument about whether they are logically sound. Ignore.

CadmiumYellow 4y ago
It's the Lucille Bluth response: "I don't understand the question and I won't respond to it."
PoignardAzur 4y ago
This is very eloquently put.
deleted 4y ago
[deleted]
tootie 4y ago
That's not what he said and not even what he was asked. He definitely acknowledged the dangers up to and including "human extinction" but wanted to make sure the question was couched in the right context.

deleted 4y ago

[deleted]

timcavel 4y ago

[dead]

bartislartfast 4y ago

In the short term, I'm concerned about the energy required to run things like ChatGPT and how everyone is cramming it into everything. Soon every todolist app on your phone will need a supercomputer running in the background to auto arrange your grocery list, and we'll double or triple our datacenter footprint within a few years.

throwuwu 4y ago
Running the models isn’t the expensive part and is subject to a lot more optimization

01100011 4y ago

As I've said for years, the robots don't have to kill us, they just have to convince us to kill ourselves.

skee8383 4y ago

i don't fear this stuff. at the end of the day if the digital world becomes detrimental to my health i can just call up my ISP and cancel my service. good luck rotting my brain when you have no internet connection to my house.

Mobius01 4y ago

I've read this short story about the singularity years ago, written by a scientist from UW-Madison and although the writing isn't great, it has always stayed with me. Recent developments made me think of it, and the premise is precisely that - the group that develops AGI uses it to control the markets and drives everyone else insane through economic disruption, while staying entirely opaque. https://www.ssec.wisc.edu/~billh/g/mcnrsts.html https://www.ssec.wisc.edu/~billh/g/mcnrsts.html

labrador 4y ago

I'm really enjoying how "Imposter Syndrome" is actually justified in the tech industry as many influencer posts or posts by people riding on reputations from past accomplishments are actually imposters and making fools of themselves. Conversely, some very insightful posts are being made by the quiet ones with few followers who aren't necessarily even in the tech industry. I've been through sea changes before, when the old guard falls by the wayside and a new group rises to meet the new challenges. This is certainly one of those times.

rideontime 4y ago
I'm struggling to understand what point this comment is trying to make. I thought it was referring to Lanier as one of "the quiet ones with few followers," but then it claimed that the "old guard" is falling by the wayside, and Lanier's hardly new.
- labrador 4y ago
  I wasn't actually referring to Lanier because he's mostly been a lone voice in the woods, even as he's highly influential in some circles. I was actually referring to the noisy self-promoters currently polluting Hacker News with their AI posts, but now that you mention it, maybe Lanier fits that category of insightful people with few followers. How long has he been telling people to delete social media? How long have people been ignoring him? The world is finally coming around and I for one would be glad for him to become as famous as some of the noisy ones.
  - unconed 4y ago
    Funny, I have mentally filed him as a designated person who is consulted by media to offer opinions on tech despite having no particular claim to familiarity with said tech.

aaroninsf 4y ago

Something I've found myself observing in disparate domains: the only defense we have against applications of AI we don't like, is the application of more AI. This is a natural implication of AI being augmented-intelligence force-multipliers at the moment. When your opponent takes performance enhancing drugs, or wears a mecha suit, the only way to compete with them or defend against them is adoption of similar technology. The Prime Directive in Star Trek was prescient. AI arriving unevenly—and first under the direction and control of hands which have thus far proven disinterested in the common welfare where it interferes with personal benefit—looks like it will be a civilizational exercise indistinguishable from the delivery of disruptive technology from another species. Of particular personal concern is that the synergies where it meets other technological social problems against which we have few or no defenses, specifically, surveillance and its active counterpart, sentiment control. I am less concerned about needing AI tools to validate AI-generated code; and more concerned about the 2024 election cycle being opportunity for an arms race in application of AI to game politics.

caycep 4y ago

what's the difference?

holoduke 4y ago

AI will encapsulate us slowly till the only thing left is your digestive system to provide energy to the AI systems.

polynomial 4y ago

Assuming it's not already just a reflection of a pre-existing lack of sanity, or that it's arguably a symptom of a the descent into madness, not the cause.

audessuscest 4y ago

too late

r1b 4y ago

If you enjoy this kind of thinking I'd recommend [1]. > Electric speed mingles the cultures of prehistory with the dregs of industrial marketeers, the nonliterate with semiliterate and the postliterate. Mental break-down of varying degrees is the very common result of uprooting and inundation with new information and endless new patterns of information. [1] https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf

UberFly 4y ago

This feels a bit like Bill Joy's article about nano-technology. I think it's just a healthy fear the the unknown future. AI will be refined into usable tools, mostly good with some other unintended negative consequences. Sounds like a lot of things.

thomastjeffery 4y ago

> There are two ways this could go. One is that we pretend the bot is a real thing, a real entity like a person, then in order to keep that fantasy going we’re careful to forget whatever source texts were used to have the bot function. Finally someone who, at least nearly, gets it. > The other way is you do keep track of where the sources came from. And in that case a very different world could unfold where if a bot relied on your reporting, you get payment for it, and there is a shared sense of responsibility and liability where everything works better. The term for that is data dignity. Of course, that's an obvious step in the other direction: isn't the original point to breathe life into the data? If we have to track who originated what, that scaffolding is going to really get in the way. Then again, this is a good moment to recognize the futility of text inference models (LLMs). If we can't understand what path goes from input to output, how useful can the thing be? If we can't choose the path, we can't choose the output. Who can choose: the thing itself? If the whole thing is a black box making its own choices, then how do we interact with it except to personify that black box? Remember, that was option 1. The main reason anyone is excited about LLMs is that those models have been personified. That problem began the moment we started calling them AI. --- > We have to say consciousness is a real thing and there is a mystical interiority to people that’s different from other stuff because if we don’t say people are special, how can we make a society or make technologies that serve people? Mystical. It's an important observation. We really don't know how we think. And that is all the evidence we need: AI does not exist. There is no artificial person. We are free to stop personifying technology. Doing so would make all of these conversations a lot less convoluted, and a lot more direct. We are wasting a lot of effort talking past the very understanding about these machines that is necessary to improve them. Instead we are trying to understand "AI personified" a fictional character that we have never met; and trying to improve it.