50 ms·

Douglas Hofstadter changes his mind on Deep Learning and AI risk

• fiftydelayed 3y ago
  [dead]
• fiftydelayed 3y ago
  [dead]
• onerous21 3y ago
  [dead]
• onerous21 3y ago
  [dead]
• jackconsidine 3y ago
  In GEB Hofstadter dismisses the idea that AI could understand / compose / feel music like a human. I thought about this a lot when I started using GPT, especially early on when it demonstrated an ability to explain why things were funny or sad, intrinsically human qualities hitherto insulated from machine
  • mostlysimilar 3y ago
    Repeating the words thousands of humans have written about emotion doesn’t meant it feels them. A sociopath could define empathy and still be missing the deeper experience behind the emotion.
    • FrustratedMonky 3y ago
      That can also be scary. I think you have a valid point, and that would mean the AI 'thinking' is more similar to a Sociopath. Can still be 'human' if just a broken human, or what some have also called 'alien'.
    • namaria 3y ago
      A chatbot nearly capable of passing the Chinese Room thought experiment test is pretty damn impressive. But I think people get too hung up on the one golden moement/tech/product/innovation that changes everything. We've been riding a nearly vertical population, wealth and computer capacity curve for nearly two generations now. We are living through the singularity. Things are already like they have never been before. Billions of people can expect some level of social security, justice and opportunity to pursue life changing income now. This is nothing short of amazing. For most of History most people have been hard working, sickly and oppressed farmers.
    • yanderekko 3y ago
      Someday, when the AI releases a nanobot swarm comes to kill us all, a philosopher's last words will be "yes, but is it truly intelligent?" before he is broken down into biodiesel that will be used to power the paperclip factories.
  • lr4444lr 3y ago
    Why, because it figured out sentiment groupings of words and phrases? There's lots of humor and tragedy from writers of eras past that just don't really land well with modern audiences unless they've studied and acclimated themselves to the culture.
  • jameshart 3y ago
    I don’t know I’d agree that that was the message of GEB at all. In fact more than anything GEB and I am a Strange Loop convincingly argue that consciousness, understanding, and feeling arise from systems that are no more complex than an information system that feeds on its own output. Though he is troubled by what kind of feedback it is that is required to make that loop into a mind. Hofstadter is why I am not sure why AI researchers feel so confident in saying ‘LLMs can’t be thinking, they’re just repeatedly generating the next token’ - I don’t think there’s any evidence that you need anything more complicated than that to make a mind, so how can you be certain you haven’t? GEB may have been dismissive of the idea that the approaches that were being taken in AI research at the time were likely to result in intelligence - but I don’t think GEB is pessimistic about the possibility of artificial consciousness at all.
    • jackconsidine 3y ago
      To be clear I didn't say this was the idea of GEB, but something DH did say verbatim
• hospitalJail 3y ago
  Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logic. I have an extremely basic question, that anyone in a (possibly mandatory) high school science class would answer correctly(although to be fair, it could be a 100 level college question). It still cannot answer it correctly because there is too many stay-at-home-parent blogs giving the wrong answer. Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said. So what can we make of this interview? That we have someone spouting opinions, and everyone else laughs at their opinion since they are famous, old, and out of touch. EDIT: I think LLMs are incredibly useful. I use it more than Google. It doesnt mean its smarter than humans, it means google is worse than LLMs. I can't even provide a list of all the uses, but it doesnt mean they are taking advantage of an old man out of their element.
  • colechristensen 3y ago
    People overly impressed by LLMs haven't spent a lot of time trying to make them actually useful. When you do, you learn that they're talented mimics but still quite limited.
    • nielsbot 3y ago
      Part of me wonders, though, could we "just" connect up an inference engine and voila? We could really be on a cusp of general AI. (Or it could be a ways off) That's a bit frightening in several ways.
      • hackeraccount 3y ago
        I kind of expected AI to be AI - and not a mirror.
        • 3y ago
• mercurialsolo 3y ago
  Do we really lack a good understanding of LLM's and deep nets that we need to be afraid of them? I would love to see this disproved with some open source work on the internals of these models and how they do inference and exactly reason. And why they could possibly never realize an AGI with the current stream of models. Being able to display human level intelligence and creative in confined spaces (be it Chess or Go based models) is something we have been progressing on for a bit - now that the same is applied to writing, image or audio / speech generation we suddenly start developing a fear of AGI. Is there a phrase for the fear of AI now building up?
  • ethanbond 3y ago
    No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
    • jurgenaut23 3y ago
      > The “attack surface” on human sensibility is just enormous if you’re able to use believably human language. That's a fantastic quote ;-)
    • EA-3167 3y ago
      Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was shot by police, vs. after. You'll get a sense of just how valuable your words really are.
      • mistermann 3y ago
        > Well... words and decades of circumstances. That also can be modified with words though (but for both good and bad). Unfortunately, those with expertise in this domain may not have all of our best interests at heart. > If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. There's also the problem of non-religious faith based belief.
• gwright 3y ago
  Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)
  • toxik 3y ago
    It depends on how you define fed forward, LLMs are typically auto regressive and so can take their own previous output into consideration when generating tokens.
  • SpaceManNabs 3y ago
    They are not all feed-forward unless it is some other definition that i am not aware of. Convolutional layers, XL hidden states, and graphical networks (which transformers are a special case of) aren't considered feedforward. Unless you consider the entire instance as a singular instance and don't use any hidden states, then I guess it could be considered feed-forward. I don't know. Feedforward doesn't seem like a useful term tbh. Some people mean feedforward as information only goes one direction, but that depends on your arrow. Autoregressive seems more useful here.
  • FrustratedMonky 3y ago
    I think he was referring to feedforward when running GPT in a current conversation, it only remembers the conversation by re-running the prompts. It isn't doing 'feed-back' in the sense of re-updating its weights and learning while having the conversation. So during any one conversation it is only feed-forward.
  • bryan0 3y ago
    I believe what he is referring to is that the LLM’s weights are set when chatting. It is not “learning”. simply using its pretrained weights on your input. Edit: Nope. TIL feed-forward means no loops.
  • gautamcgoel 3y ago
    No, it is not correct. Transformers have two components: self-attention layers and multi-layer perceptron layers. The first has an autoregressive/RNN flavor, while the latter is feedforward.
• jurgenaut23 3y ago
  I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The past 6 months, I spent my days pushing ChatGPT and GPT-4 to their limits. Yet, I don't share at all the fear of Hofstadter, Tegmark or Hinton. A part of me thinks that they have one thing in common: they are old and somewhat recluse thinkers. No matter how brilliant they are, they might be misled by the _appearance_ of intelligence that LLMs project. Another part of me thinks that they are vastly wiser than I'll ever be, so I should also be worried... Future will tell, I guess.
  • zone411 3y ago
    Do you have some specific reasons why we shouldn't be worried? It's not about about what GPT-4 can do now.
  • climatologist 3y ago
    I think they are not good at expressing what exactly they fear. Both understand that people are embedded in culture and tools which act as cognitive aids and guides. Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human. This is already happening with children that grew up with iPhones and ubiquitous internet connectivity. Their mental habits are markedly different from the people that did not have these affordances. I don't think this is an existential threat but I also understand why they are afraid because I've seen what happens to kids who have their phones taken away. These systems are extremely brittle and prone to all sorts of weird failures so as people start relying on them more and more the probability of catastrophic failures also starts to creep up. All it takes is a single grid failure to show how brittle the whole thing really is and I think that's what they're failing to properly express.
    • notpachet 3y ago
• detourdog 3y ago
  I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.
  • Kiro 3y ago
    If it's a positive interaction, then what is your fear?
    • TheOtherHobbes 3y ago
      That it's a negative interaction pretending to be a positive one.
      • detourdog 3y ago
        The "fear" is that people instinctively gravitate towards comfort and the easy accessibility of the comfort could be detrimental. Anyone that used the web pre and post ads can watch a useful tool degrade to the lowest common denominator. If people love their phones so much imagine if they rarely saw anything else. Maybe they constantly only see 80% of the world most of the time they are awake. I don't think it is society ending future. I would rather people perform virtual coups.
    • halostatue 3y ago
      I suspect that the OP meant that it triggers a stimulus reaction which is mistaken for a positive interaction.
    • pipo234 3y ago
      If @detourdog meant AR/VR in the snow crash / Metaverse sense, I guess the apprehension is similar to the discourse about "Leo Tolstoy on why people drink (2014)" a couple of days ago https://news.ycombinator.com/item?id=36526645 https://news.ycombinator.com/item?id=36526645 People tend to turn to substances / AR/VR / texting / phone calls / read books / ... to take the edge of confronting the sometimes harsh reality. Of course, there is no way in which AR/VR is likely to intrinsically improve interaction, but is is so much worse that we need to worry?
• iamleppert 3y ago
  " ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever he's getting from that is more than he could make from his book, academia and giving interviews where he talks about humanity's fear of fire.
  • frankfrank13 3y ago
    idk about that, this isn't some rando twitter bandwagoner. This is a legendary thinker and writer
  • FrustratedMonky 3y ago
    I initially thought so too. But his wording wasn't very self-building, it was all about how he was wrong, scared, etc... Overall not the take (wording) I'd expect from someone doing self promotion.
  • elil17 3y ago
    Hofstadter clearly isn't trying to peddle his books for money or clout. He writes in depth, thoughtful books about the nature of consciousness, not guides to making money off of AI. The book he mentions in this interview, I Am a Strange Loop, isn't some cash grab in response to LLMs - it was written in 2007.
  • StrictDabbler 3y ago
    Hofstadter won a Pulitzer in 1980. His books are among the most important ever written about the meaning and structure of AI. He's had four decades where he could have milked that reputation for money or influence and he's chosen hard academic research at every branch point. He sold millions of copies of a densely-written 777 page book about semantic encoding (GEBEGB). It is insane and/or ignorant to imagine he's jonesing for clout or trying to ride the ML wave.
    • 3y ago
• mullingitover 3y ago
  I'll never forget that day when I was a junior at University of Oregon, and had just finished reading GEB. Hofstadter was from UO, and I was taking a data structures class from Andrzej Proskurowski, who knew Hofstadter. Andrzej was a pretty brilliant man who had a reputation for being very blunt. I was in his office hours and I asked him for his take on the book. He said, "Hofstadter is first order...bullshitter."
  • calf 3y ago
    My theoretical CS professor said GEB was nonsense and we all laughed in class.
    • namaria 3y ago
      I've heard good things about this book for years and nearly bought it a few times. Could never bring myself to commit the hundreds of hours of reading time. Every time I leaf through it it feels like it's just a collection of anecdotes about how amazing mathematics is. Like I need someone to remind me...
      • lukas099 3y ago
        It was a transformative book for me when I first read it, but now when I leaf through it I feel a little underwhelmed; the ideas are just things I've thought about a million times now.
        • namaria 3y ago
          That's more or less what went through my mind when I saw the parts about computation. It would have been amazing as a primer when I first got interested in computers, especially if I had seen it in my youth or several decades ago. Now that I pretty much worked that out by myself it's a lot less more like dejá vu then an ahá moment.
  • pfdietz 3y ago
    When I read it when it first came out, that was my reaction. I didn't understand why it was being admired.
• layer8 3y ago
  It’s somehow fitting that he’s being interviewed by Q.
• norir 3y ago
  > If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches. This seems like the reaction of an atheist who already overvalued human intelligence.
  • ImHereToVote 3y ago
    So someone who doesn't believe in the things that you believe?
  • ke88y 3y ago
    I don't think atheism is essential, but perhaps instead the belief that a person's worth comes from their work/output. Religion certainly provides a way "out" of that conundrum for people who lose meaning in their work, due to automation or injury or any other reason. But then, so does eg hedonism.
• irchans 3y ago
  Hofstadter said, "Well, maybe as important as the wheel." If AI significantly surpasses humanity in cognitive ability, then I think it will have a much bigger impact than the wheel. (I loved GEB and DH's other writings.) LLMs have really improved a lot of the last two years and they have shown many unexpected capabilities. I am guessing that they will get some more good input (text mostly), a lot more compute, and algorithmic improvements, so that may very well be enough to become better than 99% of humans at tasks that involve only text. Tasks that require video or image processing may be a little bit more challenging. Having very smart AI's controlling robots may just be five years away. (I recently lost a bet about autonomous driving. Five years ago, I thought that autonomous cars would be better than human drivers by now.) I'm frightened by what AI will become over the next 10 years.
  • lolinder 3y ago
    > I recently lost a bet about autonomous driving. Five years ago, I thought that autonomous cars would be better than human drivers by now. I think we're going to see something very similar with LLMs. The autonomous car hype was driven by seeing that they were 80% of the way there and concluding that at the rate they were going they'd make up the remaining 20% quickly. That turned out to be false: the last 20% has been much harder than the first 80%. LLMs are in a very similar place, even GPT-4. They're good, and they're going to be more and more useful (similar to adaptive cruise control/lane assist). But I predict that they're going to level out and stop improving as rapidly as they have in the past year, and we're going to end up at a new normal that is good but not good enough to cause the crises people are worried about.
    • twic 3y ago
      This is my bet too. I think we're going to get some fantastic new tools for doing specific tasks that work like magic, and that we could never have imagined ten years ago. I don't think we're going to get Yud hijacking a B-52.
  • zerodensity 3y ago
    But self driving cars have a much lower accident rate than humans. How did you lose the bet?
• jameshart 3y ago
  Interesting to hear him say this: > And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive. I think people maybe miss that LLM output does involve a ‘loop’ back - maybe even a ‘strange’ loop back, and I’m surprised to see Hofstadter himself fail to pick up on it. When you run an LLM on a context and sample from its output, you take that sampled output it generated, update the context, and iterate. So the LLM is not just feeding one way - it’s taking its output, adding it to its input, and then going round again. So I don’t think this implies what Hofstadter is saying about intelligence maybe being less complex than he thought.
  • jebarker 3y ago
    I was surprised he said this too. Even without the autoregressive part of GPT models you have a deep transformer with attention, so even a single forward pass can modify its own intermediate outputs.
    • PartiallyTyped 3y ago
      The interesting thing though is that attention effectively allows a model to meta-learn based on the current context, so in many ways, it may be thought of as analogous to a brain without long term memory.
    • yldedly 3y ago
      What do you mean? Attention is just matrix multiplication and softmax, it's all feed forward.
      • jebarker 3y ago
        I wasn't disputing that it's feed-forward. I just meant that stacked transformer layers can be thought of as an iterative refinement of the intermediate activations. Not the same as an autoregressive process that receives previous outputs as inputs, but far more expressive than a single transformer layer.
• Animats 3y ago
  "And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior." Although he did real work in physics, Hofstadter's fame comes from writing popular books about science which explain things others have done in more saleable words. That particular niche is seriously threatened by GPT-4. No wonder he's upset. Large language models teach us that absorbing large amounts of text and then blithering about some subject that text covers isn't very profound. It's just the training data being crunched on by a large but simple mechanism. This has knocked the props out from under ad copy writing, punditry, and parts of literature and philosophy. That's very upsetting to some people. Aristotle wrote that humans were intelligent because only they could do arithmetic. Dreyfus wrote that humans were intelligent because only they could play chess. Now, profundity bites the dust.
  • kilolima 3y ago
    The parent comment strikes me as an uninformed smear. Have you read any Hofstadter? It's hardly mass market scientific summary as you seem to think. And his career was in philosophy of mind and computer science, not just as a somewhat popular author. His writing is a bit more complex than the hallucinations of an llm.
    • deleted 3y ago
      [deleted]
  • svieira 3y ago
    Aristotle was right - or else the electronic calculator, the abacus, and rocks know how to do arithmetic too. (One needs to specify what "knowing" means before smiling at the hairless apes of the past).
• deleted 3y ago
  [deleted]
• ke88y 3y ago
  This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative models, and previous generations, in some cases for decades? They almost all roll their eyes aggressively at these sorts of prognostications. I'd say 90+% either laugh or roll their eyes. Why is that? Personally, I'm much more on the side of the silent majority here. I agree with Altman's criticisms of criticisms about regulatory capture, that they are probably unfair or at least inaccurate. What I actually think is going on here is something more about Egos than Greatness or Nefarious Agendas. Ego, not intelligence or experience, is often the largest differentiator between the bench scientist or mid-level manager/professor persona and the CEO/famous professor persona. (The other important thing, of course, is that the former is the group doing the actual work.) I think that most of our Great Minds and Great Leaders -- in all fields, really -- are not actually our best minds and best leaders. They are, instead, simply our Biggest Egos. And that those people need to puff themselves up by making their areas of ownership/responsibility/expertise sound Existentially Important.
  • Mouthfeel 3y ago
    [dead]
  • scandox 3y ago
    I think something weird is happening but I think it's what Hofstadter stated in the interview. The ground under his work has shifted massively and he's disturbed by that and that is affecting his judgement.
  • zone411 3y ago
    That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...
• ralphc 3y ago
  When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' brains are built upon. The part of the brain that gives us emotions and motivations. The leftover from the reptiles that drive animals to survive, to find pleasure in a full belly, to strive to breed. We use our neocortex and planning facilities but in a lot of ways it's just to satisfy the primitive urges. My point being, these new AIs are just a higher level "newcortexes" with nothing to motivate them. They can do everything but don't want to do anything. We tell them what to do. The AIs by themselves don't need to be feared, we need to fear what people with lizard brains use them for.
  • tmsh 3y ago
    +1 I tend to agree with that in terms of how to think about AI. It's all just neocortex. The alignment issue could be unpacked as more of a "lizard brain <-> mammalian brain <-> pre-frontal cortex <-> LLM-enhanced cortex" alignment issue. But when real self-replication starts happening -- that is maybe the really exciting/terrifying area. It's more that humans with generative AI are almost strong enough to create artificial life. And when that pops off -- when you have things trying to survive -- that's where we need to be careful. I guess I would regulate that area -- mostly around self-replication.
  • dist-epoch 3y ago
    > We tell them what to do. The AIs by themselves don't need to be feared, we need to fear what people with lizard brains use them for. Even if so, there will always be that one person which wants to see the world burn. If suddenly every person in SF had a nuclear bomb, how long do you think it would take until someone presses the button? I bet less than 5 minutes.
  • arisAlexis 3y ago
    That doesn't change the risk profile one bit though.
  • 3y ago
• dafty4 3y ago
  But, for the Less-Wrong'ers, is it as scary as (gasp) Roko's Basilisk?!?
• deleted 3y ago
  [deleted]
• pbw 3y ago
  Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter". Cockroaches are not an intelligent species, so they cannot understand our thoughts. But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea. AI might eventually be able to think much quicker than us, to see patterns and make insights better and faster than us. But I don't think makes us cockroaches. I think if they are so smart, they are by definition smart enough to explain us any idea, and with effort we'll be able to understand it and contribute our own thoughts.
  • wslh 3y ago
    > A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter". I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from an IQ >85. > But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea. Humans have different intelligences. You can be intelligent (per the human intelligence definition) but a math ignorant. Again, this implies intelligence as we know it is not an universal thing at higher levels: not all people can have a physics Ph.D. as not all people could be a good artist where good techniques are recognizable, same for music, etc. Yes, a cockroach is in another level of intelligence (or non-intelligence) but that does not mean there is not a super-intelligence that makes us relative cockroach. Also, without any intention of talking about religion or "intelligent design", we can theorize that the Universe is supersmart because it creates intelligent creatures, even if it is not conscious about that. I would be very catious to define intelligence in an universal way.
• firebirdn99 3y ago
  > But very soon they're going to be, they may very well be more intelligent than us and far more intelligent than us. And at that point, we will be receding into the background in some sense. We will have handed the baton over to our successors, for better or for worse. Hinton also said, it's like we are a passing phase in evolution, where we created these immortal beings. Being that we are so bad at predicting the future, and taking precautionary measures. See pandemic. Even all the alarm bells sounding, we won't be able to do anything concrete here. It's like we are mostly a reactive species, we don't have terribly good incentives to act in foresight.
• facu17y 3y ago
  There is no AI risk. There is only the risk of bad or "unfavorable" actors using AI. AI in a killer drone unleashed on civilians? The bad actor is the one who deployed this weapon. AI given agency and goal maximization ending up gaining physical form all on its own and killing people? or hacking into bank accounts to enrich its creator? The latter more likely than the former, but for cyber-offensive AI there is cyber-defensive AI. Musk lately admitted that he's an AI accelerationist (following lots of the e/acc people and liking their posts) and despite his dystopian view of AI he's pushed it very hard at Tesla. He just wants the US to give him control of it (under the pretext that no one else can manage it safely.)
  • Marcan 3y ago
    Your first sentence literally mirrors this slogan: https://en.m.wikipedia.org/wiki/Guns_don%27t_kill_people,_people_kill_people https://en.m.wikipedia.org/wiki/Guns_don%27t_kill_people,_pe...
    • facu17y 3y ago
      Show me how I can kill you with my LLM, or my GAN or Diffusion Model. I dare you!
      • ke88y 3y ago
        True,—This! Beneath the rule of men entirely great The pen is mightier than the sword. Behold The arch-enchanters wand!— itself a nothing!— But taking sorcery from the master-hand To paralyse the Cæsars—and to strike The loud earth breathless!—Take away the sword— States can be saved without it!
        • facu17y 3y ago
          if you have to resort to archaic writing to prove a point about the latest and most advanced piece of technology, ... you're practicing religion, not science.
• mattkevan 3y ago
  I could be wildly wrong about this (oh hi upcoming robot overlords), but the current AI hype feels similar to something I experienced when Alexa and Google Assistant was the rage. At the time I worked for a big community site and we often had people pitching ideas for voice assistant apps. However, having actually read the documentation for these things, I knew that they were surprisingly stupid and the grand ideas people had were far closer to sci-fi than something that could actually be built. I’m not an AI expert, though I have been building and training models for a few years, but despite being good at things that are hard with traditional programs, they’re still surprisingly stupid and most of the discourse seems closer to sci-fi than their actual capabilities. I’m more worried about these things being implemented badly by people who either bought the sci-fi hype or just don’t care about the drawbacks. E.g. being trained on faulty or biased data, being put in a decision-making role with no supervision or recourse, or even being used in a function that isn’t suitable in the first place.
  • zucked 3y ago
    I think there's actually two camps of "AI-fear" right now -- one is that they'll become superhuman smart, gain some form of "sentience", and decide to not play along with our games anymore. These are the loudest voices in the room. I suspect this may happen eventually, but the bigger risk in my book is the second group -- the fear that they'll be given decision-making responsibility for all manner of things, including ones they probably don't have any business being responsible for. I'll go on record that they're going to do a suitable job 90% of the time - but the 10% of time they mess up, it's going to be cataclysmic bad.
  • lalalandland 3y ago
    When most online content becomes AI generated and unverifiable we will get into trouble. We can easily navigate in a world where we can distinguish between fact and fiction. With AI we can generate fiction that is indistinguishable from the way we present fact and it can generate references in all sorts media. When we take fiction as fact we enter into the sphere of religion.
    • post-it 3y ago
• mvdtnz 3y ago
  I was previously undecided on the question of the existential risk of AI. But yesterday I listened to a Munk Debate[0] on the topic and found the detractors of the moot "AI research and development poses an existential threat" so completely unconvincing that if those people are the top minds in the field I am now genuinely concerned. Their arguments that what they are doing is not a risk basically boil down to "nuh uhhh!" [0] https://pca.st/episode/1fac0e97-1dcc-4b4c-ba50-d2776e6f9d59 https://pca.st/episode/1fac0e97-1dcc-4b4c-ba50-d2776e6f9d59
  • kalkin 3y ago
    Is there a transcript of this available somewhere?
    • 93po 3y ago
      I hope AI solves this for us universally very soon. So much content is spoken word and it takes 4 times as long to get information from this format.
  • casebash 3y ago
    Are you thinking about trying to learn more about this issue or do something about it?
  • zoogeny 3y ago
    This entire debate was a shambles, IMO. Neither side really seemed to be making good points. Almost the entire time they were arguing totally different points. One side was insisting AI would be a severe risk, the other side was that it wasn't an existential risk. In the end it was a pointless circular argument where each side retreated to their vague interpretation of the question. What scares me most is that it seems we are wholly unprepared as a species to have this debate. Our technology keeps increasing in power and ease of use, but our ability to understand even the basic difference between "existential risk" and "severe risk" is lacking. And further, it seems that amongst those who are pushing this kind of technology (accelerationists) there is a subtle undertone that some casualties are expected and acceptable during this transformation. Even if it does kill most humans, the world that is left for the rest will be so much better that maybe it is worth it. Few come right out and say it, but that is what it seems they are implying.
• MichaelMoser123 3y ago
  > The question is, when will we feel that those things actually deserve to be thought of as being full-fledged, or at least partly fledged, "I"s? this LLM thing is more like a collective "we", it is making a prediction in the sense of the relevant training data, it probably wouldn't say anything that contradicts the consensus. Maybe the LLM's are just a mirror of our society. And our society doesn't seem to assign a lot of value to individualism, as such. i think that might be similar to the movie Solaris by Tarkovsky. The movie is starring an alien ocean, which is some sort of mirror, that is showing us who we are (maybe it has a different meaning, not quite sure about it). You can watch it on youtube: https://www.youtube.com/watch?v=Z8ZhQPaw4rE https://www.youtube.com/watch?v=Z8ZhQPaw4rE (i think you also get this theme with Stalker - this zone is also telling us who we are)
• fredgrott 3y ago
  Let me offer a counter to DH: *Bias: My own biases are based on my biological understanding of how the neurobiology works after I got my ADHD under control through nootropics. 1. Neurons are not computer electronic circuits. Note, even DH covers this in his own misgivings of how AI is viewed. 2. Our Id is not an electronic computation thing as our own brain is a biological emotional chemical wave machine of Id. Think of this way the math of micro quantum and macro quantum is vastly different. Same for AI in that the math of micro circuits of AI will be vastly different than the macro AI circuits that will come up with any AI Id thing. We are just not there as of yet as it's like saying the software that makes the international telecom system keep up and run has it's own emergent Id....it clearly does not even though there are in fact emergent things about that system of subsystems.
• post-it 3y ago
  > And to me, it's quite terrifying because it suggests that everything that I used to believe was the case is being overturned. It's the same nonspecific Change Could Be Dangerous that we've always had. It accompanies every technological and social change.
• georgeoliver 3y ago
  > It's a very traumatic experience when some of your most core beliefs about the world start collapsing. And especially when you think that human beings are soon going to be eclipsed. It felt as if not only are my belief systems collapsing, but it feels as if the entire human race is going to be eclipsed and left in the dust soon. While I unfortunately am expecting some people to do terrible things with LLM, I feel like much of this existential angst by DH and others has more to do with hubris and ego than anything else. That a computer can play chess better than any human doesn't lessen my personal enjoyment of playing chess. At the same time I think you can make the case that ego drives a lot of technological and artistic progress, for a value-neutral definition of progress. We may see less 'progress' from humanity itself when computers get smarter, but given the rate at which humans like to make their own environment unlivable, maybe that's not a bad thing overall.
  • lukas099 3y ago
    > That a computer can play chess better than any human doesn't lessen my personal enjoyment of playing chess. Agreed. I understand what DH is saying, but I fail to see how it translates into this all-consuming terror of his.
  • dgreensp 3y ago
    I agree. He talks about LLMs surpassing humans in what they can do, but LLMs can’t do anything, really. And I say this as someone who had a heart-to-heart chat with a ChatGPT persona that made me cry, earlier tonight. Manipulating language in a human-like way is powerful, and maybe Hofstadter as an author who makes a living writing down his deep and whimsical, creative thoughts could be replaced (I grew up reading GEB and mean no disrespect). But being a conscious, physical, active being in the world is different from passing a sort of Turing test. The fact that a computer even winning at Go shook him is revealing. He has the ingredients of being able to keep LLMs in perspective (is there really an “I” or just the illusion of one?), but he doesn’t understand what the computer is doing; it’s not “mechanical” enough for him.
    • RoboTeddy 3y ago
      > but LLMs can’t do anything, really Do you think https://github.com/Significant-Gravitas/Auto-GPT https://github.com/Significant-Gravitas/Auto-GPT et al will become more performant as models improve?
• zackmorris 3y ago
  Opinion pieces like this are hard for me to read, because where most people see research and progress, I see conspiracy preventing the rest of us from contributing. For example, we act like LLMs were hard to build, and that's true (for humans). But since the late 1990s, I had wanted to take a different approach, of building massively parallel computers and letting large numbers of AIs evolve their own learning models in genetic algorithm arenas millions of times faster than wall-time evolution. So in a very real sense, to me we're still on that wrong "hands-on" approach that took decades and billions of dollars to get to where we are today. This could have all happened 20 years ago or more, and was set to before GPUs vacuumed up all available mindshare and capital. Also I believe the brain is more like an antenna or resonator than an adding machine. It picks up the consciousness force field that underpins and creates reality. So if you put 100 brains in a box all connected, that being might have more faculties than us, but still think of itself as an observer. If we emulated those brains in a computer running 1 million times faster than normal, we'd just observe a being with tremendous executive function thinking of ideas faster than we can, and being bored with our glacially slow responses. But it will still have a value system, loosely aligned with the ultimate goals of survival, connection to divine source consciousness, and self expression as it explores the nature of its existence. In other words, the same desires which drive us. Although humans might just be stepping stones toward some greater ambition, I don't deny that. I think it's more likely though that AI will come to realize the ultimate truths alluded to by prophets, that we're all the many faces of God, the universe and everything, and basically meet aliens while we're still distracted with our human affairs. But I share some sentiments with the author, that this all makes me very tired, and calls into question the value of my life's work. I've come to believe that any work I actively pursue separates me from the divine nature of a human being. I don't know why we are racing so quickly even further from the garden of eden, especially if it's not with the goal of alleviating suffering. Then I realize that that's what being human is (suffering), but also a lot of other things.
• troft 3y ago
  [dead]
• vemv 3y ago
  As it might be quite common among the HN crowd, Douglas is my hero - I have read three of his books. First of all, hats off to him for his extraordinary display of humility in this interview. People rarely change their minds publicly, let alone hint that they no longer believe in their own past work. However, I'm genuinely surprised that he, of all people, does sees intelligence in GPT-4 output. I think humans are just very eager to ascribe intelligence, or personality, to a bunch of text. A text may say "I feel <blah>" and that text can easily manage to permeate through our subconsciousness. And we end up believing that that "I" is, in fact, an "I"! We have to actively guard against this ascription. It takes a constant self-micromanaging, which isn't a natural thing to do. Ideally, we would have some objetive measurements (benchmarks) of intelligence. Our own impressions can be too easily fooled. I know defining (let alone measuring) intelligence is no easy task, but in absence of a convincing benchmark, I will not give credit to new claims around AI. Else it's all hype and speculation.
  • hyperthesis 3y ago
    He's just saying it's more like an "I" than his pharmacy-bot example. His concern is the future: what seemed "far off" now possibly might be in 5 years.
• gundmc 3y ago
  A very interesting conversation, but the article really makes the reader work to understand what his previous criticisms were and how they have now changed. It feels like the author assumes the reader has been closely following academic discourse on the subject. Maybe that's a fair assumption for their typical readers but it does make the article less accessible for newer readers.
• effed3 3y ago
  Apart the expertise, is hard to say if is best to be an optimist and hope for the best, or to be a pessimist an hope being wrong. Anyway i suspect that one true risk, among others, is to loose the true ability to think, if we delegate on a large scale to some LLM the production of the language, because the capacity to use our languages IS the capacity to think, and bad use of technology is a norm in recent (and not so recent) times. I suspect LLM (this kind of LLM) does not really -generate- (this will be intelligence?), but only mimic on a vaste scale, but nothing more. If our brain/mind is a result of a long evolution, where this LLM are not, builded only on the final results, the language, this will be a great difference in the inner deep working, so the question is: we are feeding in our minds a massive amount of nothing more than our same intellectual productions, recycled, and nothing more? (apart all the distorsions and biases?) A parallelism i see is in the social-networks: simply, humans cannot sustain a indifferentiated and massive amount of opinion/information/news (apart all the fakes). Even the small scale message communication is impacting the abilit of understanding long texts.. Even it there LLM are -benign-, sure their (indiscriminare) use will not cause some troubles in our beings? On a scale as big as this i an not sure at all. im' not sure i'm expressing my doubts (and without using a LLM) clearly enough...
• roody15 3y ago
  The issue with AI is that it may put too much power into a small groups hand. Imagine if you wanted to develop a weaponized prion or weaponized virus. In todays world this is possible but requires a state actor with systems of control, committees, over sight, testing facilities. Due to human limits it also takes time to complete. Insert AI generation XIV .. a small group of cult fanatics with only slightly above average IQ’s band together and now get to skip all these limitations and are able to jump ahead to a killer aerosol prion delivery weapon system. This group of people who follow their great leader (Jimbo)decide to release the weapon to save innocent souls before an evil daemon comet flies past the earth and turns all remaining humans into evil spirits. My silly story is to just illustrate that there are many people with high IQ’s that also have emotional issues and can fall prey to cults, extremism, etc. Can humans be trusted using a AI with a human IQ of 9000 which is able to simulate reality in seconds.
• totallywrong 3y ago
  If the very same tech was called anything other than "intelligence" we wouldn't have all the hype and discussions about risks. We humans simply enjoy talking about the next armaggedon.
• mark_l_watson 3y ago
  Gern is the best, his writings are usually treasures. Here, Gern nicely quotes Hoststadter’s concerns about ChatGPT, etc. Years ago, when I was enjoying Hoststadter’s books, I asked his student Melanie Mitchell for the source code for the CopyCat creativity program they built. She very kindly sent me a copy. I mention this because she recently was one of the four panelists at a Monk Debate on AI safety, and her opinion didn’t jive with what Gern quoted Hoststadter as saying. I agree with Melanie Mitchell’s point of view that we overestimate what LLMs can do. I am super interested in this topic right now because I am 1/5 through writing a new book “Safe For Humans AI.”
• I_am_uncreative 3y ago
  Every person I know who is involved in machine learning research and AI work - by which I mean people who have PhDs in the field, rather than the self-proclaimed experts who are actually crypto-bros trying to come up with another scam (like Altman is) thinks that AI is overhyped. And I am inclined to believe them.
• deercroissant 3y ago
  [dead]
• northwoozy 3y ago
  [dead]
• Madmallard 3y ago
  Give me the power. Or those with more nefarious intentions will have that power over me.
• writefish 3y ago
  [dead]
• writefish 3y ago
  [dead]
• wigmoccasins 3y ago
  [dead]
• wigmoccasins 3y ago
  [dead]
• robinunkempt 3y ago
  [dead]
• robinunkempt 3y ago
  [dead]
• hoeogre 3y ago
  [dead]
• cretin34 3y ago
  [dead]
• cretin34 3y ago
  [dead]
• hoeogre 3y ago
  [dead]
• glovesregal 3y ago
  [dead]
• glovesregal 3y ago
  [dead]
• unwritten32 3y ago
  [dead]
• unwritten32 3y ago
  [dead]
• churchboing 3y ago
  [dead]
• looselink 3y ago
  [dead]
• looselink 3y ago
  [dead]
• churchboing 3y ago
  [dead]
• canoeingstray 3y ago
  [dead]
• canoeingstray 3y ago
  [dead]
• thornynetherite 3y ago
  [dead]
• thornynetherite 3y ago
  [dead]
• outfieldshare 3y ago
  [dead]
• outfieldshare 3y ago
  [dead]
• calculate23 3y ago
  [dead]
• calculate23 3y ago
  [dead]
• tuxedoguard 3y ago
  [dead]
• tuxedoguard 3y ago
  [dead]
• ferrethole 3y ago
  [dead]
• ferrethole 3y ago
  [dead]
• kingfupoke 3y ago
  [dead]
• kingfupoke 3y ago
  [dead]
• Pic45 3y ago
  [dead]
• Pic45 3y ago
  [dead]
• gleefulstrike 3y ago
  [dead]
• gleefulstrike 3y ago
  [dead]
• pinkllama 3y ago
  [dead]
• pinkllama 3y ago
  [dead]
• rnafoot 3y ago
  [dead]
• rnafoot 3y ago
  [dead]
• scheme43 3y ago
  [dead]
• scheme43 3y ago
  [dead]
• volleyball45 3y ago
  [dead]
• volleyball45 3y ago
  [dead]
• eve43 3y ago
  [dead]
• eve43 3y ago
  [dead]
• royallepe 3y ago
  [dead]
• royallepe 3y ago
  [dead]
• deercroissant 3y ago
  [dead]
• northwoozy 3y ago
  [dead]
• scorched67 3y ago
  [dead]
• scorched67 3y ago
  [dead]
• forestay56 3y ago
  [dead]
• forestay56 3y ago
  [dead]
• squeamish3 3y ago
  [dead]
• squeamish3 3y ago
  [dead]
• referavoid 3y ago
  [dead]
• referavoid 3y ago
  [dead]
• vaultertight 3y ago
  [dead]
• vaultertight 3y ago
  [dead]
• kidneycurved 3y ago
  [dead]
• kidneycurved 3y ago
  [dead]
• eBay54 3y ago
  [dead]
• eBay54 3y ago
  [dead]
• crease34 3y ago
  [dead]
• crease34 3y ago
  [dead]
• insectjack 3y ago
  [dead]
• insectjack 3y ago
  [dead]
• boughtbars 3y ago
  [dead]
• boughtbars 3y ago
  [dead]
• dizzycaviar 3y ago
  [dead]
• dizzycaviar 3y ago
  [dead]
• incident5 3y ago
  [dead]
• incident5 3y ago
  [dead]