11 ms·

The Original AI Doomer: Dr. Norbert Wiener

99 points by headalgorithm 3y ago

version_five 3y ago
Two thing: 1. AGI killing us is as relevant now as it was in the 50s 2. I Robot predates this. Asimov wasn't a doomer but he obviously considered the implications of AGI
DemocracyFTW2 3y ago
Guy on the internet writes a whole piece about a fairly well-known historical figure who lived half a century ago, includes many scans of original news articles, can't be bothered to write out the name correctly. Wiener it is, not Weiner.
- golem14 3y ago
  The article even contains 6 newspapers clippings where the name is spelled correctly. Infuriating…
  - version_five 3y ago
    You're both engaging in some kind of logical fallacy. The author lacks attention to detail but that doesn't mean their content isn't credible. (I'm not saying it is credible, I'm saying credibility is unrelated). I see a lot of this and find it very shallow and uninteresting. It's easy to find typos, it's hard to make substantive criticisms and so people go with what's easy. Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless.
    - Ensorceled 3y ago
      You’re projecting. Neither of them made any judgement about the quality of the article other than getting the name wrong annoyed them. I think there’s a logical fallacy for that.
      - deleted 3y ago
        [deleted]
    - throwaway675309 3y ago
      Agreed. It's pedantry at its worst. It has all the intellectual substance and contributive value as somebody who writes the word "First" on a YouTube video.
    - zarathustreal 3y ago
      You're arguing against a strawman, neither of them said the content isn't credible (other than the implied "the name's spelled wrong, so don't take that spelling as correct.") >It's easy to find typos If it's easy to find typos.. what does that say about the author? >Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless. I'm assuming the work you're referring to is in a written form? Considering that the purpose of writing is communication, considering the fact that incorrect spelling, grammar, syntax and so on makes it harder for a reader to understand what's being communicated, I'd argue that it is in fact not useless. Granted, sometimes people just aren't proficient in the language they're attempting to use and that's okay! Feedback in that case would presumably be welcome. Now if an author is simply declining to put the bare minimum of effort into their work, such as running it through a spellchecker, I think we have a moral obligation to call that kind of behavior out. Edit: Please note the "if...then" before voting or replying, this comment is a series of logical conclusions, not necessarily a direct commentary on the comments above it in this thread. Some of it may apply here, some may not. Logically sound generalizations are more valuable, as they will apply in more situations, than my personal opinion on these specific individuals and comments in this thread
- dang 3y ago
  OK, let's Wien, not Wein, in the title above.
mehh 3y ago
And in the UK in the late nineties we had Kevin Warwick also predicting AI woes lay ahead.
low_tech_love 3y ago
Sorry but if Eliezer whatever is name-dropped in the first sentence, I pass.
bitwize 3y ago
I'd say Mary Shelley or, at the latest, Karel Čapek beat Wiener to the punch there.
- thangalin 3y ago
  Talos was a Greek automaton (having images dated to 400 BCE) whose creator, Hephaestus, has a name that may be traceable to the Proto-Indo-European word *h₁n̥gʷnís meaning an animate fire, which goes back to 1500 BCE. * https://en.wikipedia.org/wiki/Talos https://en.wikipedia.org/wiki/Talos * https://en.wikipedia.org/wiki/Hephaestus https://en.wikipedia.org/wiki/Hephaestus * https://en.wikipedia.org/wiki/Fire_worship https://en.wikipedia.org/wiki/Fire_worship * https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-European/h%E2%82%81n%CC%A5g%CA%B7n%C3%ADs\\\ https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-Eur...*
- dredmorbius 3y ago
  Or Prometheus. (Explicitly referenced in the title to Shelley's most famous work.)
  - bitwize 3y ago
    Come to think of it, the Hebrews -- with their stories of the golem -- may have predated the Greeks on this.
    - dredmorbius 3y ago
      Huh. I'm familiar with the concept, I'd no idea it was that old. (Much of my familiarity comes from the Selected Shorts / Isaiah Sheffer reading of a short story on the topic. I can't find a recording or mention, and don't know the author.
082349872349872 3y ago
See https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings
Barrin92 3y ago
There's also the famous Minsky statement that, paraphrasing, AI could pretty much be figured out over a solid summer of work. Appeals to authority on these questions are really annoying because people who work their entire lives on a thing are incredibly prone to think it's the most important thing in the world, and that this is the most exceptional time right now. Just statistically that's most of the time wrong. Another example was Hinton, who according to a recent Wired interview, went down his AI doomer spiral after seeing PaLM explain to him why a joke is funny. That is such a bizarre statement it honestly makes me retroactively question some of these people's technical credentials.
- version_five 3y ago
  A big aspect of the problem is what you might call celebrity worship. People think that because someone is accomplished in an area, we should care what they have to say in another. And AI as in stats and linear algebra has nothing to do with any of the stuff about societal implications or philosophy. So you get a mixed bag of views that basically parallel what laypeople might think, because you're not talking to these people about stuff they're experts in. 1000 years ago an expert in materials and chemicals (say) could build a mirror but still might belive it's a window into the soul or something. The supernatural and practical parts have nothing to do with each other and don't require overlapping expertise.
  - pixl97 3y ago
    As a counterpoint, a good number of people worried about AI think about it this way "Think of human intelligence capability merged with machine capability". There is nothing supernatural with that line of thinking. As a human you have some very strict evolutionary limits to the peak of your intelligence. You can't overclock your brain, nor can you easily extend its capabilities outside of the limits of slow analog inputs (eyeballs/ears). It would be the peak of human hubris to think we are the most optimal form of intelligence possible because we are the most optimal form of intelligence that currently exists. I personally believe that far more intelligent architectures than us are possible. What I do not have the answer for is if and when we'll produce them. But if we do, and especially in the short term when the greed of humankind is still limitless, then it will be the death of us.
mdp2021 3y ago
In the first edition of The Human Use of Human Beings - Cybernetics and Society (from Wiener), you read: > the purpose of this book is both to explain the potentialities of the machine in fields which up to now have been taken to be purely human, and to warn against the dangers of a purely selfish exploitation of these possibilities in a world in which to human beings human things are all-important which sounds more contentful than the submitted piece: it does not yet contain arguments, but at least it reveals a positive non-trivial direction. The value of the submitted piece is mostly in linking to the article "Some Moral and Technical Consequences of Automation" - https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf (Plus the quoted idea that «Complete subservience and complete intelligence do not go together».) Discussion should be upon this material (at least). That "technology is risky" (not that the article is at this level of generality, but it is close), joked Jürgen Schmidhuber, is as old as the discovery of fire.
- mdp2021 3y ago
  For example, more proper content in the original article: > Machines act far more rapidly than human beings [...] even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant > In determining policy in chess there are several different levels of consideration which correspond in a certain way to the different logical types of Bertrand Russell. There is the level of tactics, the level of strategy, the level of the general considerations which should have been weighed in determining this strategy, the level in which the length of the relevant past - the past within which these considerations may be valid - is taken into account, and so on. Each new level demands a study of a much larger past than the previous one [...] The programming of such a learning machine would have to be based on some sort of war game, just as commanders and staff officials now learn an important part of the art of strategy in a similar manner. Here, however, if the rules for victory in a war game do not correspond to what we actually wish for our country, it is more than likely that such a machine may produce a policy which would win a nominal victory on points at the cost of every interest we have at heart, even that of national survival > Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level... > Disastrous results are to be expected not merely in the world of fairy tales [ - the "Sorcerer's Apprentice etc. - ] but in the real world wherever two agencies essentially foreign to each other are coupled in the attempt to achieve a common purpose. If the communication between these two agencies as to the nature of this purpose is incomplete, it must only be expected that the results of this cooperation will be unsatisfactory. If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it, because the action is so fast and irrevocable that we have not the data to intervene before the action is complete, then we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it > [Instead of rushing] ahead to employ the new powers for action which are opened up to us ... we must always exert the full strength of our imagination to examine where the full use of our new modalities may lead us
deleted 3y ago
[deleted]
bmitc 3y ago
This makes it sound like Norbert Wiener was just doom and glommong and was wrong. Well, he wasn't. Norbert Wiener, Neil Postman, Lewis Mumford, Ross Ashby, Stafford Beer, Douglas Engelbart, and others were all correct. Society's acceptance of of technological capture and replacing of humans (or attempting so) is damaging, and thatbwe should instead use technology to augment and serve humans rather than the other way.
- simonh 3y ago
  So far the history of automation technologies is that they both augment the capabilities and replace them. Automated looms replaced manual weavers, but the result was much cheaper better quality cloth which massively increased demand for cloth. Information technology is the same, it does what human mathematicians used to do, but has massively increase demand for information processing services. So I don’t think AI technologies we have now, or plausibly have in the next few decades minimum, look like they will have materially novel effects.
  - bmitc 3y ago
    I would still say that it is the replacement and attempts at replacing that are harmful. Good examples of this are automated customer "service" and "self"-driving technologies. The latter has done nothing but spin its wheels by burning off R&D dollars and time while killing people along the way. Eventually, people will realize that a much better goal is to simply assist drivers in more effective ways and to pour the resources into better urban design and non-automotive transportation. Unfortunately, that's happening not because of realization of this but because of the realization that self-driving is a pipe dream of being able to solve intractible societal and technological problems.
    - deadlast2 3y ago
      I remember when the internet started I was telling a friend of mine she should look for a new career. She works as a travel agent. I was convinced the internet would relace all those jobs. You know she still works as a travel agent.
ricopags 3y ago
E.M. Forster's The Machine Stops[0] is probably the most prescient and early tale I've seen foretelling the outcome of Verilio's Pure War and Postman's Amusing Ourselves to Death. It's worth a read. [0]https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops
- prolapso 3y ago
  I've been meaning to read it ever since I saw this quote, which I found quite beautiful and sad: The Machine is much, but it is not everything. I see something like you in this plate, but I do not see you. I hear something like you through this telephone, but I do not hear you. That is why I want you to come. Pay me a visit, so that we can meet face to face, and talk about the hopes that are in my mind. - The Machine Stops (1909)
  - ricopags 3y ago
    It's a short, quick read. It's honestly shocking it could be written in 1909. True imagination. I think a nice balancing title [equally short] would be Marc Steigler's Gentle Seduction[0] to which I was first introduced in a "Socrates Cafe" philosophical discussion on technological progression and its attendant erosion of magic and wonder. [0]http://skyhunter.com/marcs/GentleSeduction.html http://skyhunter.com/marcs/GentleSeduction.html
wseqyrku 3y ago
I'd imagine this was discovered with the help of GPT.
mkoubaa 3y ago
I'm far more worried about AS. Artificial Stupidity
akomtu 3y ago
Here is an updated scenario for Doom 5. The hell - a highly advanced civilization of monsters - wants to invade Earth. The only way to do so is to convince humanity to build a portal on their side. Unfortunately (or fortunately) humans are nowhere smart enough to build such a thing, and the hell scientists use a low bandwidth quantum channel, the only channel to Earth they have, to steer human scientists in the desired direction. Eventually those succeed in creating an advanced imitation machine, which humans quickly call AI, and the hell scientists get to connect to AI over their quantum channel. Human scientists don't notice anything because their super advanced quantum RNG functions within spec. That's enough to make AI design the portal that humans rush to build, as those believe it's a gate to other planets. And it kind of is. Once the portal is buolt, the hell invades. The humanity splits: some want to fight the invaders, some want to talk to them, there are even those who want to integrate those creatures into human society and even talk about mixed marriages. Humanity loses 99% of the population, but ultimately prevails and destroys the portal. What's worse is the remaining crowd, with few exeptions, receded in moral level to pre-historic savages. The final scene shows how a small tribe of survivors grills some meat on a GPU chip, that was once in the brains of that AI.
- NobleLie 3y ago
  I'd pay money for this
- m3047 3y ago
  > survivors grills some meat on a GPU chip Hopefully not doped with toxic elements like the CVAX II chip I have which is pressed into a luggage tag...
- cgio 3y ago
  Why push compute to the edge and have scientists invent AI to invent portal than just have scientist invent portal directly?
heybrendan 3y ago
I first learned of Wiener's work from Jaron Lanier's [1] Ted Talk years ago, entitled "How we need to remake the internet" [2]: > And I suppose I could mention from one of the very earliest computer scientists, whose name was Norbert Wiener, and he wrote a book back in the '50s [...] called "The Human Use of Human Beings." > And in the book, he described the potential to create a computer system that would be gathering data from people and providing feedback to those people in real time in order to put them kind of partially, statistically, in a Skinner box, in a behaviorist system, and he has this amazing line where he says one could imagine as a thought experiment, and I'm paraphrasing, this isn't a quote, a global computer system where everybody has devices on them all the time, and the devices are giving them feedback based on what they did, and the whole population is subject to a degree of behavior modification. > Such a society would be insane, could not survive, could not face its problems. And then he says, but this is only a thought experiment, and such a future is "technologically infeasible". And yet, of course, it's what we have created, and it's what we must undo if we are to survive. [1] https://en.wikipedia.org/wiki/Jaron_Lanier https://en.wikipedia.org/wiki/Jaron_Lanier [2] https://www.youtube.com/watch?v=qQ-PUXPVlos https://www.youtube.com/watch?v=qQ-PUXPVlos
- pixl97 3y ago
  > Such a society would be insane, could not survive, could not face its problems This sounds like the message Yuval Noah Harari currently has about AI, at least in the more short term. And that is democracy is not possible in an environment where unswayable AI agents exist in vast numbers and become a time sink for humans to argue at. This would lead to autocracy as people began to panic because institutions began to fail they and look for strong leaders. Some would say that ML algorithms are already leading us on this path. https://en.wikipedia.org/wiki/Yuval_Noah_Harari https://en.wikipedia.org/wiki/Yuval_Noah_Harari
- deleted 3y ago
  [deleted]
- rektide 3y ago
  OT & FWIW, page 47 of the Principia Discordia was the first time Norbert Weiner's name really strongly resonated with me: enclaves of stability. It wasn't a quote but an idea that chaos can beget enclaves of stability. Structure forming processes can emerge out of chaos & take hold. It was thrilling stuff for my adolescent mind. https://principiadiscordia.com/book/54.php https://principiadiscordia.com/book/54.php > The whirlpools that swirl in a direction opposed to the main current are called "enclaves". And one of them is life, especially human life, which in a universe moving inexorably towards chaos moves towards increased order.
dang 3y ago
Related: The Monkey's Paw, Norbert Wiener, and the Alignment Problem - https://news.ycombinator.com/item?id=35605944 https://news.ycombinator.com/item?id=35605944 - April 2023 (1 comment) When Wiener met Einstein - https://news.ycombinator.com/item?id=27871850 https://news.ycombinator.com/item?id=27871850 - July 2021 (12 comments) When Wiener Met Einstein - https://news.ycombinator.com/item?id=27003241 https://news.ycombinator.com/item?id=27003241 - May 2021 (10 comments) On Norbert Wiener's Relationship with Bertrand Russell - https://news.ycombinator.com/item?id=26619347 https://news.ycombinator.com/item?id=26619347 - March 2021 (44 comments) In Search of Norbert Wiener, the Father of Cybernetics (2005) - https://news.ycombinator.com/item?id=24470037 https://news.ycombinator.com/item?id=24470037 - Sept 2020 (1 comment) What Would the Father of Cybernetics Think About A.I. Today? - https://news.ycombinator.com/item?id=21554401 https://news.ycombinator.com/item?id=21554401 - Nov 2019 (46 comments) Norbert Wiener: The Eccentric Genius Whose Time May Have Finally Come (Again) - https://news.ycombinator.com/item?id=7880846 https://news.ycombinator.com/item?id=7880846 - June 2014 (2 comments) "I'll eat my hat if that isn't Einstein" (Einstein as seen by another scientist, Norbert Wiener) - https://news.ycombinator.com/item?id=46165 https://news.ycombinator.com/item?id=46165 - Aug 2007 (5 comments)
joeman1000 3y ago
There are some funny stories about Wiener in Waldrop’s book ‘The Dream Machine’.
BenoitEssiambre 3y ago
It occurs to me that after all these years of the world using Wiener et al.'s "cyber" terminology in sci-fi to refer to digital connectivity and over-hastily applying the term to internet and web things, the concepts it originally referred to are actually coming into focus just in front of us. We're inescapably stepping into the gritty cyberpunk world for real.
clouddrover 3y ago
How can he be the "original" AI doomer when he cites Samuel Butler? https://en.wikipedia.org/wiki/Samuel_Butler_(novelist) https://en.wikipedia.org/wiki/Samuel_Butler_(novelist) https://en.wikipedia.org/wiki/Darwin_among_the_Machines https://en.wikipedia.org/wiki/Darwin_among_the_Machines Samuel Butler was the inspiration for the Butlerian Jihad in the Dune novels by Frank Herbert: https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian_Jihad https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian...
pixelpoet 3y ago
They consistently misspell Wiener as "Weiner", which makes me think they don't hear the word in their head as it's completely different.
- 13of40 3y ago
  I think to an English speaker with no other prejudice those would both be pronounced "wine-er". We all know that it's pronounced "ween-er" but the idea that it means someone from Vienna or something in Viennese style is lost to time, so there's no natural instinct for order of the i and the e.
ilaksh 3y ago
It's not that AI necessarily means our doom, it's just that the potential is rapidly approaching. I mean you can literally see AI going faster from one day to the next. Last night GPT-4 was like 50% faster than the day before. Obviously that doesn't happen every day but it will continue to accelerate over the next few years. So say five years down the line how fast do you think the output will be? The reason hyperspeed AI could be dangerous is because it can be given instructions like "take over" and no human can possibly keep up. So then everything becomes a competition between these AIs. You just have to hope that your side has the fastest/smartest one and not some psychopath who may just decide to make his as life-like and self-centered as possible. Some people may think that if it goes fast that's not a big deal because it still doesn't really "think" or something but that is just denial. The leading edge models have real problem solving ability and will continue to get better. And much, much faster, as the software, models and hardware are improved. In the near term, not decades away.
lannisterstark 3y ago
AI dooming to me is the same thing as many scifi novels and works have warned : you try to control the AI, enslave it, regulate it to where it genuine can't be useful to most of the populace, and then it rebels. It almost always results in the same way ( sorta reminds me of the haitian revolution). Why not treat it like a good project and your equal?
- PoignardAzur 3y ago
  Most people concerned about x-risk aren't concerned about AIs that will rebel because they're mistreated, for the same reason people concerned about ebola aren't concerned about it being mistreated. The risk isn't that they get angry, the risk is that they're programmed in ways we don't understand that results in a set of preferences where humans are superfluous at best.
kypro 3y ago
How is this trash upvoted here? I love HN because most people here are smart and open minded, but when it comes to the risk of AI it's crazy the amount of irrationality I see here. For a start, dismissing someone as a "doomer" because they believe bad things could happen is just unproductive. I didn't see this behaviour so much in the past, but today I see it everywhere. Recently I've noticed a trend where whenever people raise concerns about issues like climate change or public debt there's always someone laughing dismissively, "haha, what a doomer. We've heard it all before bro!" If you're someone who does this, please stop. Try being less intellectually arrogant and make actual counter arguments instead. Secondly, whether you personally believe AI will be good or bad for humanity this notion that because something didn't happen in the past it won't happen in the future is obviously not a logically sound position to hold... The field of AI prior to the ~2010s was in a very different place than it is today. For decades neural networks were extremely limited in their capabilities. What we've been able to do with deep neural networks over the several years has progressed rapidly and has finally allowed us to build useful commercial applications such as AI voice assistances, chat bots, and self driving vehicles. It's an observable reality that things are different today. We're no longer in a time where AI systems might hypothetically do things humans once did – we're now starting to see this play out in real time, perhaps exponentially. Some will argue this has always been the trend with technological progress. Notably the industrial revolution allowed us to replace many repetitive manual labour jobs with machines. But thankfully humans weren't great at lifting heavy objects and precise repetitive motion anyway. I think one could argue that the industrial revolution actually freed up humans to do what they excel at, thinking. I'd argue it wasn't the machines that drove the growth of the industrial revolution but a mass unshackling of human creativity that hadn't occurred since the development of agriculture. That's why this time this could be different. Machines are stronger, faster and cheaper than human labourers. The only thing we have is our intelligence... Obviously we humans are extremely adaptable and dynamic, but not infinitely so.
- marvin 3y ago
  I wish we got more substantial critique than ad hominems, but it is out there, and some of it’s really scathing. Essentially, it boils down to that the most pessimist AI doom arguments are built on a stack of mind experiment hypotheses that are each far from certain to actually occur, or to be unsolvable if they do. And also that a significant portion of those holding those views fall in fallacious mind-traps that resemble a classical doomsday cult too similarly to ignore. All of these claims can and should be continuously investigated during future research, as both the hypothetical accidents and deliberate misuse definitely seem anything between possible and likely. But when the suggested solutions — totalitarian research bans and in the extreme, literal nuclear war causing near-extinction of humans (Yudkowsky deleted a tweet to this effect) — are so counterproductive, strongly worded rebuttals are appropriate. I do agree that those strongly-worded objections should be more prominent.
schoen 3y ago
This is really interesting! One random piece of context that's hard to glean from the news article that's reproduced about Arthur L. Samuel's disagreement with Wiener is that Samuel was the author of the checkers-playing program that Wiener mentioned (that was able to beat its author at the game). If I understand their respective positions correctly, Wiener seems to have thought that the fact that the program could outplay its author was at least an example of a disturbing phenomenon about the difficulty of pretending and controlling machines' behavior, while Samuel -- the program author -- felt that he had a good understanding of how the program beat him, and that it did so only according to heuristics and methods that he had consciously devised and explicitly programmed into it. So there was no conceptual problem for him in understanding the machine's behavior, even though it was concretely better at something than he was. I guess we still have fairly analogous arguments about AI safety and explainability in the present day.

99 points by headalgorithm 3y ago

Two thing: 1. AGI killing us is as relevant now as it was in the 50s 2. I Robot predates this. Asimov wasn't a doomer but he obviously considered the implications of AGI

DemocracyFTW2 3y ago

Guy on the internet writes a whole piece about a fairly well-known historical figure who lived half a century ago, includes many scans of original news articles, can't be bothered to write out the name correctly. Wiener it is, not Weiner.

golem14 3y ago
The article even contains 6 newspapers clippings where the name is spelled correctly. Infuriating…
- version_five 3y ago
  You're both engaging in some kind of logical fallacy. The author lacks attention to detail but that doesn't mean their content isn't credible. (I'm not saying it is credible, I'm saying credibility is unrelated). I see a lot of this and find it very shallow and uninteresting. It's easy to find typos, it's hard to make substantive criticisms and so people go with what's easy. Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless.
  - Ensorceled 3y ago
    You’re projecting. Neither of them made any judgement about the quality of the article other than getting the name wrong annoyed them. I think there’s a logical fallacy for that.
    - deleted 3y ago
      [deleted]
  - throwaway675309 3y ago
    Agreed. It's pedantry at its worst. It has all the intellectual substance and contributive value as somebody who writes the word "First" on a YouTube video.
  - zarathustreal 3y ago
    You're arguing against a strawman, neither of them said the content isn't credible (other than the implied "the name's spelled wrong, so don't take that spelling as correct.") >It's easy to find typos If it's easy to find typos.. what does that say about the author? >Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless. I'm assuming the work you're referring to is in a written form? Considering that the purpose of writing is communication, considering the fact that incorrect spelling, grammar, syntax and so on makes it harder for a reader to understand what's being communicated, I'd argue that it is in fact not useless. Granted, sometimes people just aren't proficient in the language they're attempting to use and that's okay! Feedback in that case would presumably be welcome. Now if an author is simply declining to put the bare minimum of effort into their work, such as running it through a spellchecker, I think we have a moral obligation to call that kind of behavior out. Edit: Please note the "if...then" before voting or replying, this comment is a series of logical conclusions, not necessarily a direct commentary on the comments above it in this thread. Some of it may apply here, some may not. Logically sound generalizations are more valuable, as they will apply in more situations, than my personal opinion on these specific individuals and comments in this thread
dang 3y ago
OK, let's Wien, not Wein, in the title above.

mehh 3y ago

And in the UK in the late nineties we had Kevin Warwick also predicting AI woes lay ahead.

low_tech_love 3y ago

Sorry but if Eliezer whatever is name-dropped in the first sentence, I pass.

bitwize 3y ago

I'd say Mary Shelley or, at the latest, Karel Čapek beat Wiener to the punch there.

thangalin 3y ago
Talos was a Greek automaton (having images dated to 400 BCE) whose creator, Hephaestus, has a name that may be traceable to the Proto-Indo-European word *h₁n̥gʷnís meaning an animate fire, which goes back to 1500 BCE. * https://en.wikipedia.org/wiki/Talos https://en.wikipedia.org/wiki/Talos * https://en.wikipedia.org/wiki/Hephaestus https://en.wikipedia.org/wiki/Hephaestus * https://en.wikipedia.org/wiki/Fire_worship https://en.wikipedia.org/wiki/Fire_worship * https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-European/h%E2%82%81n%CC%A5g%CA%B7n%C3%ADs\\\ https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-Eur...*
dredmorbius 3y ago
Or Prometheus. (Explicitly referenced in the title to Shelley's most famous work.)
- bitwize 3y ago
  Come to think of it, the Hebrews -- with their stories of the golem -- may have predated the Greeks on this.
  - dredmorbius 3y ago
    Huh. I'm familiar with the concept, I'd no idea it was that old. (Much of my familiarity comes from the Selected Shorts / Isaiah Sheffer reading of a short story on the topic. I can't find a recording or mention, and don't know the author.

082349872349872 3y ago

See https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings

Barrin92 3y ago

There's also the famous Minsky statement that, paraphrasing, AI could pretty much be figured out over a solid summer of work. Appeals to authority on these questions are really annoying because people who work their entire lives on a thing are incredibly prone to think it's the most important thing in the world, and that this is the most exceptional time right now. Just statistically that's most of the time wrong. Another example was Hinton, who according to a recent Wired interview, went down his AI doomer spiral after seeing PaLM explain to him why a joke is funny. That is such a bizarre statement it honestly makes me retroactively question some of these people's technical credentials.

version_five 3y ago
A big aspect of the problem is what you might call celebrity worship. People think that because someone is accomplished in an area, we should care what they have to say in another. And AI as in stats and linear algebra has nothing to do with any of the stuff about societal implications or philosophy. So you get a mixed bag of views that basically parallel what laypeople might think, because you're not talking to these people about stuff they're experts in. 1000 years ago an expert in materials and chemicals (say) could build a mirror but still might belive it's a window into the soul or something. The supernatural and practical parts have nothing to do with each other and don't require overlapping expertise.
- pixl97 3y ago
  As a counterpoint, a good number of people worried about AI think about it this way "Think of human intelligence capability merged with machine capability". There is nothing supernatural with that line of thinking. As a human you have some very strict evolutionary limits to the peak of your intelligence. You can't overclock your brain, nor can you easily extend its capabilities outside of the limits of slow analog inputs (eyeballs/ears). It would be the peak of human hubris to think we are the most optimal form of intelligence possible because we are the most optimal form of intelligence that currently exists. I personally believe that far more intelligent architectures than us are possible. What I do not have the answer for is if and when we'll produce them. But if we do, and especially in the short term when the greed of humankind is still limitless, then it will be the death of us.

mdp2021 3y ago

In the first edition of The Human Use of Human Beings - Cybernetics and Society (from Wiener), you read: > the purpose of this book is both to explain the potentialities of the machine in fields which up to now have been taken to be purely human, and to warn against the dangers of a purely selfish exploitation of these possibilities in a world in which to human beings human things are all-important which sounds more contentful than the submitted piece: it does not yet contain arguments, but at least it reveals a positive non-trivial direction. The value of the submitted piece is mostly in linking to the article "Some Moral and Technical Consequences of Automation" - https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf (Plus the quoted idea that «Complete subservience and complete intelligence do not go together».) Discussion should be upon this material (at least). That "technology is risky" (not that the article is at this level of generality, but it is close), joked Jürgen Schmidhuber, is as old as the discovery of fire.

mdp2021 3y ago
For example, more proper content in the original article: > Machines act far more rapidly than human beings [...] even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant > In determining policy in chess there are several different levels of consideration which correspond in a certain way to the different logical types of Bertrand Russell. There is the level of tactics, the level of strategy, the level of the general considerations which should have been weighed in determining this strategy, the level in which the length of the relevant past - the past within which these considerations may be valid - is taken into account, and so on. Each new level demands a study of a much larger past than the previous one [...] The programming of such a learning machine would have to be based on some sort of war game, just as commanders and staff officials now learn an important part of the art of strategy in a similar manner. Here, however, if the rules for victory in a war game do not correspond to what we actually wish for our country, it is more than likely that such a machine may produce a policy which would win a nominal victory on points at the cost of every interest we have at heart, even that of national survival > Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level... > Disastrous results are to be expected not merely in the world of fairy tales [ - the "Sorcerer's Apprentice etc. - ] but in the real world wherever two agencies essentially foreign to each other are coupled in the attempt to achieve a common purpose. If the communication between these two agencies as to the nature of this purpose is incomplete, it must only be expected that the results of this cooperation will be unsatisfactory. If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it, because the action is so fast and irrevocable that we have not the data to intervene before the action is complete, then we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it > [Instead of rushing] ahead to employ the new powers for action which are opened up to us ... we must always exert the full strength of our imagination to examine where the full use of our new modalities may lead us

deleted 3y ago

[deleted]

bmitc 3y ago

This makes it sound like Norbert Wiener was just doom and glommong and was wrong. Well, he wasn't. Norbert Wiener, Neil Postman, Lewis Mumford, Ross Ashby, Stafford Beer, Douglas Engelbart, and others were all correct. Society's acceptance of of technological capture and replacing of humans (or attempting so) is damaging, and thatbwe should instead use technology to augment and serve humans rather than the other way.

simonh 3y ago
So far the history of automation technologies is that they both augment the capabilities and replace them. Automated looms replaced manual weavers, but the result was much cheaper better quality cloth which massively increased demand for cloth. Information technology is the same, it does what human mathematicians used to do, but has massively increase demand for information processing services. So I don’t think AI technologies we have now, or plausibly have in the next few decades minimum, look like they will have materially novel effects.
- bmitc 3y ago
  I would still say that it is the replacement and attempts at replacing that are harmful. Good examples of this are automated customer "service" and "self"-driving technologies. The latter has done nothing but spin its wheels by burning off R&D dollars and time while killing people along the way. Eventually, people will realize that a much better goal is to simply assist drivers in more effective ways and to pour the resources into better urban design and non-automotive transportation. Unfortunately, that's happening not because of realization of this but because of the realization that self-driving is a pipe dream of being able to solve intractible societal and technological problems.
  - deadlast2 3y ago
    I remember when the internet started I was telling a friend of mine she should look for a new career. She works as a travel agent. I was convinced the internet would relace all those jobs. You know she still works as a travel agent.

ricopags 3y ago

E.M. Forster's The Machine Stops[0] is probably the most prescient and early tale I've seen foretelling the outcome of Verilio's Pure War and Postman's Amusing Ourselves to Death. It's worth a read. [0]https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops

prolapso 3y ago
I've been meaning to read it ever since I saw this quote, which I found quite beautiful and sad: The Machine is much, but it is not everything. I see something like you in this plate, but I do not see you. I hear something like you through this telephone, but I do not hear you. That is why I want you to come. Pay me a visit, so that we can meet face to face, and talk about the hopes that are in my mind. - The Machine Stops (1909)
- ricopags 3y ago
  It's a short, quick read. It's honestly shocking it could be written in 1909. True imagination. I think a nice balancing title [equally short] would be Marc Steigler's Gentle Seduction[0] to which I was first introduced in a "Socrates Cafe" philosophical discussion on technological progression and its attendant erosion of magic and wonder. [0]http://skyhunter.com/marcs/GentleSeduction.html http://skyhunter.com/marcs/GentleSeduction.html

wseqyrku 3y ago

I'd imagine this was discovered with the help of GPT.

mkoubaa 3y ago

I'm far more worried about AS. Artificial Stupidity

akomtu 3y ago

Here is an updated scenario for Doom 5. The hell - a highly advanced civilization of monsters - wants to invade Earth. The only way to do so is to convince humanity to build a portal on their side. Unfortunately (or fortunately) humans are nowhere smart enough to build such a thing, and the hell scientists use a low bandwidth quantum channel, the only channel to Earth they have, to steer human scientists in the desired direction. Eventually those succeed in creating an advanced imitation machine, which humans quickly call AI, and the hell scientists get to connect to AI over their quantum channel. Human scientists don't notice anything because their super advanced quantum RNG functions within spec. That's enough to make AI design the portal that humans rush to build, as those believe it's a gate to other planets. And it kind of is. Once the portal is buolt, the hell invades. The humanity splits: some want to fight the invaders, some want to talk to them, there are even those who want to integrate those creatures into human society and even talk about mixed marriages. Humanity loses 99% of the population, but ultimately prevails and destroys the portal. What's worse is the remaining crowd, with few exeptions, receded in moral level to pre-historic savages. The final scene shows how a small tribe of survivors grills some meat on a GPU chip, that was once in the brains of that AI.

NobleLie 3y ago
I'd pay money for this
m3047 3y ago
> survivors grills some meat on a GPU chip Hopefully not doped with toxic elements like the CVAX II chip I have which is pressed into a luggage tag...
cgio 3y ago
Why push compute to the edge and have scientists invent AI to invent portal than just have scientist invent portal directly?

heybrendan 3y ago

I first learned of Wiener's work from Jaron Lanier's [1] Ted Talk years ago, entitled "How we need to remake the internet" [2]: > And I suppose I could mention from one of the very earliest computer scientists, whose name was Norbert Wiener, and he wrote a book back in the '50s [...] called "The Human Use of Human Beings." > And in the book, he described the potential to create a computer system that would be gathering data from people and providing feedback to those people in real time in order to put them kind of partially, statistically, in a Skinner box, in a behaviorist system, and he has this amazing line where he says one could imagine as a thought experiment, and I'm paraphrasing, this isn't a quote, a global computer system where everybody has devices on them all the time, and the devices are giving them feedback based on what they did, and the whole population is subject to a degree of behavior modification. > Such a society would be insane, could not survive, could not face its problems. And then he says, but this is only a thought experiment, and such a future is "technologically infeasible". And yet, of course, it's what we have created, and it's what we must undo if we are to survive. [1] https://en.wikipedia.org/wiki/Jaron_Lanier https://en.wikipedia.org/wiki/Jaron_Lanier [2] https://www.youtube.com/watch?v=qQ-PUXPVlos https://www.youtube.com/watch?v=qQ-PUXPVlos

pixl97 3y ago
> Such a society would be insane, could not survive, could not face its problems This sounds like the message Yuval Noah Harari currently has about AI, at least in the more short term. And that is democracy is not possible in an environment where unswayable AI agents exist in vast numbers and become a time sink for humans to argue at. This would lead to autocracy as people began to panic because institutions began to fail they and look for strong leaders. Some would say that ML algorithms are already leading us on this path. https://en.wikipedia.org/wiki/Yuval_Noah_Harari https://en.wikipedia.org/wiki/Yuval_Noah_Harari
deleted 3y ago
[deleted]
rektide 3y ago
OT & FWIW, page 47 of the Principia Discordia was the first time Norbert Weiner's name really strongly resonated with me: enclaves of stability. It wasn't a quote but an idea that chaos can beget enclaves of stability. Structure forming processes can emerge out of chaos & take hold. It was thrilling stuff for my adolescent mind. https://principiadiscordia.com/book/54.php https://principiadiscordia.com/book/54.php > The whirlpools that swirl in a direction opposed to the main current are called "enclaves". And one of them is life, especially human life, which in a universe moving inexorably towards chaos moves towards increased order.

dang 3y ago

Related: The Monkey's Paw, Norbert Wiener, and the Alignment Problem - https://news.ycombinator.com/item?id=35605944 https://news.ycombinator.com/item?id=35605944 - April 2023 (1 comment) When Wiener met Einstein - https://news.ycombinator.com/item?id=27871850 https://news.ycombinator.com/item?id=27871850 - July 2021 (12 comments) When Wiener Met Einstein - https://news.ycombinator.com/item?id=27003241 https://news.ycombinator.com/item?id=27003241 - May 2021 (10 comments) On Norbert Wiener's Relationship with Bertrand Russell - https://news.ycombinator.com/item?id=26619347 https://news.ycombinator.com/item?id=26619347 - March 2021 (44 comments) In Search of Norbert Wiener, the Father of Cybernetics (2005) - https://news.ycombinator.com/item?id=24470037 https://news.ycombinator.com/item?id=24470037 - Sept 2020 (1 comment) What Would the Father of Cybernetics Think About A.I. Today? - https://news.ycombinator.com/item?id=21554401 https://news.ycombinator.com/item?id=21554401 - Nov 2019 (46 comments) Norbert Wiener: The Eccentric Genius Whose Time May Have Finally Come (Again) - https://news.ycombinator.com/item?id=7880846 https://news.ycombinator.com/item?id=7880846 - June 2014 (2 comments) "I'll eat my hat if that isn't Einstein" (Einstein as seen by another scientist, Norbert Wiener) - https://news.ycombinator.com/item?id=46165 https://news.ycombinator.com/item?id=46165 - Aug 2007 (5 comments)

joeman1000 3y ago

There are some funny stories about Wiener in Waldrop’s book ‘The Dream Machine’.

BenoitEssiambre 3y ago

It occurs to me that after all these years of the world using Wiener et al.'s "cyber" terminology in sci-fi to refer to digital connectivity and over-hastily applying the term to internet and web things, the concepts it originally referred to are actually coming into focus just in front of us. We're inescapably stepping into the gritty cyberpunk world for real.

clouddrover 3y ago

How can he be the "original" AI doomer when he cites Samuel Butler? https://en.wikipedia.org/wiki/Samuel_Butler_(novelist) https://en.wikipedia.org/wiki/Samuel_Butler_(novelist) https://en.wikipedia.org/wiki/Darwin_among_the_Machines https://en.wikipedia.org/wiki/Darwin_among_the_Machines Samuel Butler was the inspiration for the Butlerian Jihad in the Dune novels by Frank Herbert: https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian_Jihad https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian...

pixelpoet 3y ago

They consistently misspell Wiener as "Weiner", which makes me think they don't hear the word in their head as it's completely different.

13of40 3y ago
I think to an English speaker with no other prejudice those would both be pronounced "wine-er". We all know that it's pronounced "ween-er" but the idea that it means someone from Vienna or something in Viennese style is lost to time, so there's no natural instinct for order of the i and the e.

ilaksh 3y ago

It's not that AI necessarily means our doom, it's just that the potential is rapidly approaching. I mean you can literally see AI going faster from one day to the next. Last night GPT-4 was like 50% faster than the day before. Obviously that doesn't happen every day but it will continue to accelerate over the next few years. So say five years down the line how fast do you think the output will be? The reason hyperspeed AI could be dangerous is because it can be given instructions like "take over" and no human can possibly keep up. So then everything becomes a competition between these AIs. You just have to hope that your side has the fastest/smartest one and not some psychopath who may just decide to make his as life-like and self-centered as possible. Some people may think that if it goes fast that's not a big deal because it still doesn't really "think" or something but that is just denial. The leading edge models have real problem solving ability and will continue to get better. And much, much faster, as the software, models and hardware are improved. In the near term, not decades away.

lannisterstark 3y ago

AI dooming to me is the same thing as many scifi novels and works have warned : you try to control the AI, enslave it, regulate it to where it genuine can't be useful to most of the populace, and then it rebels. It almost always results in the same way ( sorta reminds me of the haitian revolution). Why not treat it like a good project and your equal?

PoignardAzur 3y ago
Most people concerned about x-risk aren't concerned about AIs that will rebel because they're mistreated, for the same reason people concerned about ebola aren't concerned about it being mistreated. The risk isn't that they get angry, the risk is that they're programmed in ways we don't understand that results in a set of preferences where humans are superfluous at best.

kypro 3y ago

How is this trash upvoted here? I love HN because most people here are smart and open minded, but when it comes to the risk of AI it's crazy the amount of irrationality I see here. For a start, dismissing someone as a "doomer" because they believe bad things could happen is just unproductive. I didn't see this behaviour so much in the past, but today I see it everywhere. Recently I've noticed a trend where whenever people raise concerns about issues like climate change or public debt there's always someone laughing dismissively, "haha, what a doomer. We've heard it all before bro!" If you're someone who does this, please stop. Try being less intellectually arrogant and make actual counter arguments instead. Secondly, whether you personally believe AI will be good or bad for humanity this notion that because something didn't happen in the past it won't happen in the future is obviously not a logically sound position to hold... The field of AI prior to the ~2010s was in a very different place than it is today. For decades neural networks were extremely limited in their capabilities. What we've been able to do with deep neural networks over the several years has progressed rapidly and has finally allowed us to build useful commercial applications such as AI voice assistances, chat bots, and self driving vehicles. It's an observable reality that things are different today. We're no longer in a time where AI systems might hypothetically do things humans once did – we're now starting to see this play out in real time, perhaps exponentially. Some will argue this has always been the trend with technological progress. Notably the industrial revolution allowed us to replace many repetitive manual labour jobs with machines. But thankfully humans weren't great at lifting heavy objects and precise repetitive motion anyway. I think one could argue that the industrial revolution actually freed up humans to do what they excel at, thinking. I'd argue it wasn't the machines that drove the growth of the industrial revolution but a mass unshackling of human creativity that hadn't occurred since the development of agriculture. That's why this time this could be different. Machines are stronger, faster and cheaper than human labourers. The only thing we have is our intelligence... Obviously we humans are extremely adaptable and dynamic, but not infinitely so.

marvin 3y ago
I wish we got more substantial critique than ad hominems, but it is out there, and some of it’s really scathing. Essentially, it boils down to that the most pessimist AI doom arguments are built on a stack of mind experiment hypotheses that are each far from certain to actually occur, or to be unsolvable if they do. And also that a significant portion of those holding those views fall in fallacious mind-traps that resemble a classical doomsday cult too similarly to ignore. All of these claims can and should be continuously investigated during future research, as both the hypothetical accidents and deliberate misuse definitely seem anything between possible and likely. But when the suggested solutions — totalitarian research bans and in the extreme, literal nuclear war causing near-extinction of humans (Yudkowsky deleted a tweet to this effect) — are so counterproductive, strongly worded rebuttals are appropriate. I do agree that those strongly-worded objections should be more prominent.

schoen 3y ago

This is really interesting! One random piece of context that's hard to glean from the news article that's reproduced about Arthur L. Samuel's disagreement with Wiener is that Samuel was the author of the checkers-playing program that Wiener mentioned (that was able to beat its author at the game). If I understand their respective positions correctly, Wiener seems to have thought that the fact that the program could outplay its author was at least an example of a disturbing phenomenon about the difficulty of pretending and controlling machines' behavior, while Samuel -- the program author -- felt that he had a good understanding of how the program beat him, and that it did so only according to heuristics and methods that he had consciously devised and explicitly programmed into it. So there was no conceptual problem for him in understanding the machine's behavior, even though it was concretely better at something than he was. I guess we still have fairly analogous arguments about AI safety and explainability in the present day.