32 ms·

You are the scariest monster in the woods

222 points by mohi-kalantari 1y ago

tocs3 1y ago
https://en.wikipedia.org/wiki/I_Am_Legend_(novel) https://en.wikipedia.org/wiki/I_Am_Legend_(novel)
deleted 1y ago
[deleted]
kkukshtel 1y ago
(DISCO ELYSIUM SPOILERS BELOW) One of the most poignant moments in Disco Elysium is near the very end when you encounter a very elusive crypid/mythic beast. The moment is treated with a lot of care and consideration, and the conversation itself is, I think, transcendent and some of the best writing in games (or any media) ever. The line that sticks with me most is when the cryptid says: "The moral of our encounter is: I am a relatively median lifeform -- while it is you who are total, extreme madness. A volatile simian nervous system, ominously new to the planet. The pale, too, came with you. No one remembers it before you. The cnidarians do not, the radially symmetricals do not. There is an almost unanimous agreement between the birds and the plants that you are going to destroy us all." It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. Everything runs from us, we are not part of the conversation. We are the exclusion, the anomaly. I think to realize this deeply inside of yourself is a big moment of growth, to see that we exist in a world that was around long before us and will be around long after us.
- Quarrelsome 1y ago
  personally I'm a little concerned about meeting alien life. If the universe is incredibly diverse with many different types of lifeform then how might an arbitrary one of those view us? We proliferate incredibly quickly, we have limited care for our environments, but most importantly our primary means of sustinance is to consume other forms of life. To the point that we consider it an art form, we spend vast amounts of energy perfecting, discussing or advertising the means of cooking and eating the flesh of other life forms. To us its normal but surely to an alien who gains sustinance by some other means; we're absolutely terrifying. Akin to a devouring swarm.
  - saulpw 1y ago
    You Eat Other Animals? https://www.youtube.com/watch?v=_gz9Vj8TMCc https://www.youtube.com/watch?v=_gz9Vj8TMCc
- sickofparadox 1y ago
  >It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. This just seems like noble savaging birds and rabbits and deer. None of these creatures have any communication with each other, and while they may be more aware of each other's presence than a 'go hiking every once in a while' person, someone who actually spends a good amount of time in the woods, such as a hunter or birdwatcher, probably has a pretty good sense of them. The Disco Elysium quote just reads like fairly common environmentalist misanthropy, which I suppose isn't surprising considering the creators.
  - wat10000 1y ago
    I think people forget how big people are. We're well above average size in the typical natural environment, and especially in the typical sorta-natural-but-sorta-urban environment most of us are in most of the time. The local rabbits and squirrels tolerate each other but are pretty scared of me. Of course they are, I'm two hundred times bigger than they are, and much more dangerous. The local foxes are the closest thing we have to an apex predator around here, and they're rightfully terrified of this massive creature that outweighs their entire family combined. Imagine wandering through the woods, enjoying the birds tweeting and generally being at one with nature, and then you come across a 20-ton 35ft-tall monster. You'd run away screaming.
us-merul 1y ago
"Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of." -- A pretty bleak, and also accurate, observation of humanity. I have to hope that the alternative sentence encompassing all of the good can lead to some balance.
- wmeredith 1y ago
  I really liked this article, but it is pessimistic. Unfortunately that seems to be the culture-du-jour. Anger and fear drive engagement effectively, as it always has. If it bleeds, it Leeds has been a thing in news organizations since at least the 70s. If we ignore the headlines peddled by those who stand to benefit the most from inflaming and inciting, we live in a miraculous modern age largely devoid of much of the suffering previous generations were forced to endure. Make no mistake there are problems, but they are growing exponentially fewer by the day. An alternate take: humans will do what they’ve always tried to do—build, empower, engineer, cure, optimize, create, or just collaborate with other humans for the benefit of their immediate community—but now with new abilities that we couldn’t have dreamed of.
  - pixl97 1y ago
    I mean, any article that doesn't include both is incomplete. >If it bleeds, it Leeds has been a thing in news organizations since at least the 70s. The term yellow journalism is far older.
- Flamingoat 1y ago
  No, it is not accurate at all. There are some people that do all of these sure, however the vast majority of people live pretty ordinary lives where they do very little of what is described. I actually think it is very intellectually lazy to be this cynical.
  - ViktorRay 1y ago
    I think it is partially true. The vast majority of human beings don’t act like that. But it seems the ones in power or close proximity to power do. This is why it is important to have societies where various forms of power are managed carefully. Limited constitutional government with guaranteed freedoms and checks and balances for example. Regulations placed on mega corporations is another example. Restrictions to prevent the powerful in government or business (or both!) from messing around with the rest of us…
ekimekim 1y ago
As always, there is a relevant XKCD: https://xkcd.com/1968/ https://xkcd.com/1968/
- Filligree 1y ago
  We can worry about both. There being additional dangers does not reduce the impact of the other dangers.
  - excalibur 1y ago
    True, but one of them is much more imminent than the other.
blamestross 1y ago
We have been using the mechanism of "Humans communicating and working together" to form superintelligences for a long time. They have gotten more effective, more efficient and more durable with time. AI isn't the new monster. It's a new mitochondria being domesticated to supercharge the existing monsters.
- forgetfulness 1y ago
  That reminds me of the “paper clip maximizer” thought experiment about rogue AI fulfilling a particular goal, without being held to any consideration of human welfare I thought it was a very novel idea at first, until I realized that this describes all manners of human groups, notably corporations and lobbying groups, they will turn the world into a stove, subvert democracy, (try to) disrupt the economic and social fiber of society, anything to eg maximize shareholder value
  - Nasrudith 1y ago
    And corporations are ultimately just sock-puppets for humanity. People forget that and like to other them while simultaneously turning them into the scapegoats. See every idiot with the list of companies responsible for global warming that amounts to 'every oil company' and ignore that it was the customers were the ones burning their products. It is to be expected really. Humans themselves hold little consideration to human welfare when fulfilling their goals. It is something ingrained by nature for survival and in no way limited to humans. Every drop of water you drink, every bite you eat, are ones which cannot go to the thirsty and the hungry. With few exceptions, only our children would even give us pause to forgo such things for the sake of others. Also bit of a pet peeve of mine: society isn't a delicate bolt of laced silk. It isn't a fabric, much less one that is damaged by any little change that you don't like. It isn't even stable which makes the charges of anything as 'ruining' society especially bizarre when nobody can point to where it would go before the blamed change. So hold off on poisoning Socrates. Even if we hold the current state as worthy of preservation we would ultimately fail to for reasons related to the central paradox of tradition. Why your forebearers first did something was not done because of obedience to tradition, so by trying to preserve it set in stone, you have already failed.
dwnw 1y ago
"to the monsters we're the the monsters" station eleven
deleted 1y ago
[deleted]
jvanderbot 1y ago
About as helpful as "Guns don't kill people ... " And equally rebutted by Eddie Izzards "Well, I think the gun helps".
- fruitworks 1y ago
  All of the solutions to AI "safety" are analagous to gun control: it is a centralization of power.
  - Cthulhu_ 1y ago
    Are you claiming a lack of gun / AI control is democratizing? That's not working for (the lack of) gun control in the US at the moment though. Compare also with capitalism; unchecked capitalism on paper causes healthy competition, but in practice it means concentration of power (monopolies) at the expense of individuals (e.g. our accumulated expressions on the internet being used for training materials).
    - fruitworks 1y ago
      >Are you claiming a lack of gun / AI control is democratizing? This is obviously the case. It results in a greater distribution of power. >That's not working for (the lack of) gun control in the US at the moment though. In the US, one political party is pro gun-control and the other is against. The party with the guns gets to break into the capitol, and the party without the guns gets to watch. I expect the local problem of AI safety, like gun safety will also be self-solving in this manner. Eventually, Gun control will not work anywhere, regardless of regulation. The last time I checked, you don't need a drone license. And what are the new weapons of war? Not guns. The technology will increase in acessibility until the regulation is impossible to enforce. The idea that you can control the use of technology by limiting it to some ordained group of is very brittle. It is better to rely on a balance of powers. The only way to secure civilization in the long run is to make the defensive technology stronger than the offensive technology.
      - dctoedt 1y ago
d_silin 1y ago
That's why I am never afraid to walk alone at night, in the dark forest.
- whycombinetor 1y ago
  Must not live in the Sundarbans
myrmidon 1y ago
I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). As soon as profit can be made by transfering decision power into an AIs hand, some form of agency for them is just a matter of time, and we might simply not be willing to pull the plug until it is much too late.
- moralestapia 1y ago
  >why would it be impossible to achieve the exact same result in silicon Because there might be a non-material component involved.
  - ACCount37 1y ago
    [flagged]
    - moralestapia 1y ago
      @dang
      - tomhow 1y ago
        We replied to that comment, but the @reply isn't a reliable way to get moderator attention, and publicly calling for moderator help like this isn't a great look. Best to flag the comment and email us if you really want us to take a look.
    - tomhow 1y ago
      Please don't post sneery comments like this on HN. The guidelines make it clear we're here for curious conversation, and kindness is one of the foremost expectations.
petsfed 1y ago
A nit: There's a subtle distinction between an individual human and the power of human organization and civilization that is implied by the article, but never outright stated. One-for-one, there are many creatures that are individually more dangerous to humans, and a decent number of people are killed by such animals every year. Indeed, a naked human in the wild is going to be quite fragile and easy to kill until they can bring some technology to bear. But there are no animals or even set of animals that could conceivably wipe out all of humanity at any of our technological peaks from the last 100,000 years. Even the number one killer of humans, the mosquito, is gradually being defeated, going from a vector for disease to just an annoyance, just like the flea.
- pixl97 1y ago
  A lot of rugged individualists seem to neglect talking about the human super organism. That the vast majority of our strength comes from sharing knowledge, making tools, and working together. And on that same note it should be mentioned that exchange of information between humans is relatively slow and guarded. A group of entities that could exchange knowledge quickly and efficiently would represent an extreme challenge for us.
  - hn_acc1 1y ago
    They also ignore that a lot of the knowledge they now have / have studied comes FROM others sharing freely.
whycombinetor 1y ago
"We (humans) are the scariest animal in the woods. We’re the scariest animal anywhere. At any time, in any location, under any circumstances, if there’s a human present then that’s the scariest motherfucker in the woods." Has this guy not heard of the Bengal tigers of the Sundarban forests, which kill ~50 humans per year? https://en.wikipedia.org/wiki/Tiger_attacks_in_the_Sundarbans https://en.wikipedia.org/wiki/Tiger_attacks_in_the_Sundarban...
- WJW 1y ago
  I think he heard of all the other forests, where tigers are now extinct because humans are around.
- graemep 1y ago
  > Has this guy not heard of the Bengal tigers of the Sundarban forests, which kill ~50 humans per year? How many tigers have been killed by humans? How many would be killed if humans did not restrain each other from killing tigers because we have killed so many that they are endangered. That population could be entirely wiped out by humans in the area. Mosquitoes kill far more people than tigers. So do venomous snakes. The fact is that when a human faces either of these they are far more likely to end up dead than the human.
  - whycombinetor 1y ago
    This isn't about what humans have done at other times or in other places. The author wrote "At any time, in any location, under any circumstances", so the statement can be proven false with one single counterexample - e.g. a lone human without a firearm encountering a tiger in the Indian Sundarbans.
    - graemep 1y ago
      I interpreted that as hyperbole.
- bob1029 1y ago
  A desperately hungry tiger might go for a human, but on average they are not interested in that fight. They would much prefer going for something like deer. The calories per unit of risk are much higher with non-human food sources. Wildlife tends to be very efficient at this economy. Anything smaller than a human is absolutely terrified of us. I used to be afraid of things like snakes and spiders, but they want nothing to do with humans. They will get the hell out of your way when you are walking through the woods. You have to do something really stupid to get bitten. Keeping animals as pets is where most of the trouble happens.
armada651 1y ago
> Just like a hammer, sword, or a rifle lying on a ground is nothing to be feared, so too is AI. It’s just an inanimate object; a tool, potentially. A bomb is an inanimate object too, but if you find some unexploded ordnance lying on the ground you should fear it.
- robertlagrant 1y ago
  An unexploded bomb definitely has the potential to be highly animate. It's just not currently animate.
  - alt227 1y ago
    As does AI.
gota 1y ago
> I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it. I'm on board with being skeptical that LLMs will lead to AGI; but - there being no possibility seems like such a strong claim. Should we really bet that there is something special (or even 'magic') about our particular brain/neural architecture/nervous system + senses + gut biota + etc.? Don't, like, crows (or octopuses; or elephants; or ...) have a different architecture and display remarkable intelligence? Ok, maybe not different enough (not 'digital') and not AGI (not 'human-level') but already -somewhat- different that should hint at the fact that there -can- be alternatives Unless we define 'human-level' to be 'human-similar'. Then I agree - "our way" may be the only way to make something that is "us".
- kulahan 1y ago
  We still haven’t figured out what intelligence even is. Depending on what you care about, the second-smartest animal in the world varies wildly.
  - myrmidon 1y ago
    Evolution had a much worse understanding of intelligence than we do and still managed just fine (thus i'd expect us to need less time and iterations to get there).
  - squidbeak 1y ago
    This is a bogus argument. There's a lot we don't understand about LLMs, yet we built them.
    - simianparrot 1y ago
      We built something we don’t understand by trial and error. Evolution took a few billion years getting to intelligence, so I guess we’re a few sprints away at least
snozolli 1y ago
At any time, in any location, under any circumstances, if there’s a human present then that’s the scariest motherfucker in the woods. This guy has clearly never bumped into a grizzly bear momma, a moose in rut, a hippo, or loads of other animals. Personally, I'm not even worried about AI itself, I'm worried about the people wielding it. MBAs are the scariest monsters in the woods.
- Insanity 1y ago
  In a 1-to-1 situation you are right, a single human is not the scariest. But humans (mostly) dominated the animal kingdom, and with enough time we could probably eradicate every other species if we wanted to. I guess bacteria and viruses might have a stronger claim for being the 'scariest monster' though. I don't think there's a strong 'living' contender for wiping out humanity apart from those. (Viruses is a bit debatable on the 'living' part though).
Insanity 1y ago
So “guns don’t kill people, people with guns kill people”. But for AI I’m not sure that preposition will hold indefinitely. Although I do think we are a far away from having actual AGI that would pose this threat. Still, the author has a good but obvious point.
- deleted 1y ago
  [deleted]
- supermatt 1y ago
  > guns don’t kill people * SIG P320 enters the chat *
woeirua 1y ago
There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war keeps me up at night.
- runjake 1y ago
  100%. Except the things that kill most of us[1] keeps me up at night. 1. https://ourworldindata.org/does-the-news-reflect-what-we-die-from https://ourworldindata.org/does-the-news-reflect-what-we-die...
  - masfuerte 1y ago
    I'm more concerned with the ways that other people can kill me than with the ways they kill themselves.
- myrmidon 1y ago
  I personally think that AI is a realistic extinction threat for our whole species within this century, and a full nuclear war never was (and probably never will be). Neither is climate change. Collapse of our current civilizations? Sure. Extinction? No. And I honestly see stronger incentives on a road towards us being outcompeted by AI, then on our leaders starting a nuclear war.
  - lm28469 1y ago
    Personally I don't bieleve in a collapse nor extinction, just a slow spiral into more and more enshittification. You'll have to talk to an "ai" doctor because real doctors will treat people with money, you'll have to face an "ai" administration because the real administration will work for people with money, you'll have to be a flesh and blood robot to an "ai" telling you what to do (already the case for Amazon warehouse workers, food delivery people, &c.), some "ai" will determine if you qualify for X or Y benefits, X or Y treatment, X or Y job. Basically everything wrong with today's productivism, but 100 times worse and powered by a shitty ai that's very far from agi.
superkuh 1y ago
Human beings are not the scariest monsters in the woods. By far the most dangerous, most malicious, and most powerful monsters are conglomerate entities: the legal corporate persons. They've been absolutely ravaging the earth and societies since they achieved legal personhood and rights without any of the downsides or legal liabilities. They're much scarrier than individual actual human persons. And while most people blame "AI" for the web's current troubles AI is only barely involved. The real culprits are the corporations. Again, same as it ever was. We really need to get rid of corporate personhood. Or at least have a corporate death penalty.
XiphiasX 1y ago
Thanks for the encouragement.
abbycurtis33 1y ago
What I'm scared by now is non-sentient intelligence, which I never thought would be this capable and still obviously dumb, which really makes it dangerous.
- slow_typist 1y ago
  Is sentient intelligence less scary than the non-sentient variant? Or is non-sentient intelligence already scary enough?
  - abbycurtis33 1y ago
    I hope we can reason with sentient intelligence.
psychoslave 1y ago
All monsters were born equal, but some are more scarier than other.
jmull 1y ago
So silly. Of course humans are (by far) the source of the biggest problems humans face. The argument here is that since humans are the scariest, we should ignore problems AI might cause. Pure nonsense, since (1) AI is human-created — so just another piece of what makes us scary; (2) this kind of binary thinking makes no sense anyway — as if there can’t be multiple things to be concerned with. > Anyone trying to tell you otherwise is trying to distract you. A particularly poor bit of argumentation. Barely above sticking your fingers in your ears and shouting, “Blah, blah, blah! I can’t hear you!” What is is about AI that makes people lose their minds? edit: seeing the engagement this is getting, maybe it’s a false flag operation of sorts? Making such bad anti-ai arguments that the argument for ai gets stronger? That would at least make some sense.
hdseggbj 1y ago
The distinction is irrelevant. No one thought the nukes would blow themselves up, but here we are on the precipice of destruction. We aren't going to bury ai in 20 story bunkers to protect llms. Everyone knows the real evil are the people building the ai. We aren't afraid of agi we are afraid of Boston dynamics (aka Google). We are afraid of Bill gates, Sam altman, Jeff bezos, Larry Ellison and elon musk. These are dangerous people. Their mind set. Their unstoppable lust for money and power. We are afraid of their ability to convince humans to do unspeakable things for this imaginary thing we call money. With agi they don't have to persuade anyone with anything. There's not Even a human conscience to stop them. We know these folks have no conscience of their own. The world is still here because other people have them. Agi won't have a conscience or the integrity to stop evil. Human society is collapsing because Good people are doing nothing.
- bryanlarsen 1y ago
  P.S. Boston Dynamics is no longer owned by Google, it's part of Hyundai now.
  - hdseggbj 1y ago
    Ah yes, rings a bell. Doesn't change the sentiment though. They're all crooks and cheaters who will sacrifice the greater good for their own self interest. https://en.wikipedia.org/wiki/2025_Georgia_Hyundai_plant_immigration_raid https://en.wikipedia.org/wiki/2025_Georgia_Hyundai_plant_imm... It's the fear of death that's killing us all.
    - bryanlarsen 1y ago
      There are tons of examples of chaebols behaving evilly in Korea. You don't have to pull out a scenario where half of Americans and most non-Americans think Hyundai are the victims, not the bad guys.
      - dbjkffgbn 1y ago
        [dead]
nyeah 1y ago
This is pleasant to read, but it's missing a logical step. We are great at dealing with the stuff we evolved to deal with. Sure. But anything outside that evolutionary process? Who knows? For example we don't do so good on top of Mt Everest. We seem to have totally unsolved problems with refined opiates. Even computer games can be a trap. Agentic AI? Who knows?
WhitneyLand 1y ago
For the love of God let’s ignore him being wrong about the impossibility of AGI, and take to heart the nightmarish and inevitable threats coming from how people will use it, that will almost certainly challenge us first.
raldi 1y ago
The author comes so close to getting it, with the paragraph about how if you drop a human into an environment, they inevitably take over as the deadliest and most powerful creature. But the next step is to ask why; in the case of the Gruffalo it was obvious: fangs, claws, strength, size… In the case of humans, it’s because we’re the most intelligent creature in the forest. And for the first time in our history, we’re about to not be.
- lif 1y ago
  yes, and: ruthlessness + strength + WMD =/= intelligence
- audunw 1y ago
  No, it’s not really about intelligence. That’s not what’s driving the behaviour. That’s like saying a calculator is dangerous because it was used to make an atomic bomb. A calculator is infinitely smarter at math than any animal. Yet it’s not what’s dangerous. What’s driving humanity’s dangerous behaviours is that we’ve evolved over millions upon millions of years to compete with each other for resources. We had to kill to survive. Both animals and each others. AI is under no such evolutionary process. In fact quite the opposite: there’s an absolutely ruthless process to eliminate any AI which doesn’t do exactly what humans say with as little energy wasted as possible. There is no room in that process for an AI to develop that will somehow want to compete with humans. So whatever bad happens, it is overwhelmingly likely to be because a human asked an AI to do it. Not that an AI will decide on its own to do it to further its own interest. I don’t even think a “paperclip maximiser” scenario is remotely probable. Long before there could be a successful attempt to exterminate humans to reach some goal, there will be countless failed or half-assed attempts. That kind of behaviour will have no chance to develop.
catapart 1y ago
I can't figure out why people who write articles like this seem to be oblivious to the very practical argument about the US second amendment that goes along the lines of "guns don't kill people, people do". To everyone who wants to pose an argument like this, please consider: We have already internalized that very simple, and very thoughtless argument, and have moved past that facile framing to start asking the more pressing questions like "what are we going to do about how humans are using X to perpetrate Y?" We don't need the extra step of you making us reframe the question to fit your exact criteria of "good question". "AI is going to kill us" can be understood as: "Humans are going to use AI to kill us". The fact that you want to misunderstand it as a more childish argument is your own failings with semiotics. It's so blatantly silly that in most cases, it's less degrading to assume that you are purposefully trying to misdirect the argument than to assume you actually believe people are worried about the agency of inanimate objects. But then you go and write a whole article to preen "logical" at us about how it's just that "no one's asking the right questions!", which kind of makes it hard to imagine that you don't actually think so little of people.
Timsky 1y ago
> We don’t need to worry about AI itself, we need to be concerned about what “humans + AI” will do. Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of. Starting with the AI itself: LLMs sold as AI are the greatest mislead. Text generation using Markov chains is not particularly intelligent, even when it is looped back through itself a thousand times and appears alike an intelligent conversation. What is actually being sold is an enormous matrix trained on terabytes of human-written, high-quality texts, obviously in violation of all imaginable copyright laws. Here is a gedanken experiment to test if an AI has any intelligence: until the machine starts to determine and resolve contradictions in its own outputs w/o human help, one can sleep tight. Human language is a fuzzy thing that is not quite suitable for a non-contradictory description of the world. Building such a machine would require resolving all the contradictions humanity has ever faced in a unified way. Before it happens, humanity will be drowned in low-quality generated LLM output.
om8 1y ago
... Yet.
lifeisstillgood 1y ago
Yes - a simple and necessary reminder of what goes on the priority list
buu700 1y ago
I see the point the author is making, but AI doesn't need to be AGI or malicious to cause mass destruction. A poorly implemented deployment of AI in the right circumstances with too much access and insufficient guardrails could theoretically wind up LARPing as Skynet purely by chance. This is why I've always considered current AI "safety" efforts to be totally wrongheaded. It's not a threat to humanity if someone has an AI generate hate speech, porn, misinformation, or political propaganda. AI is only a threat to humanity if we don't take security seriously as we roll out increasingly more AI-driven automation of the economy. It's already terrifying to me that people are relying on containers to sandbox yolo-mode coding agents, or even raw dogging them or their personal machines.
tourist2d 1y ago
[dead]
kazinator 1y ago
> Just like a hammer, sword, or a rifle lying on a ground is nothing to be feared, so too is AI. AI is not just going to lie on the ground until someone picks it up to do harm. An intelligent rifle with legs is something to be feared. You cannot compare AI to inanimate objects under human control which have no agency. Especially not if you are bringing the imaginary AGI into the conversation. The idea that AGI is just a hammer is absurd.
bilater 1y ago
I sort of agree, but I hate the premise of this article because it sneakily focuses only on the potential harm of human + AI collaboration, without acknowledging the good. That said, I agree that human + AI can cause damage and it’s precisely why, from a game theory perspective, the right move is to go full steam ahead. Regulation only slows down the good actors. Valhalla is within reach, but we have to leap across a massive chasm to get there. Perhaps the only way out of this is through: accelerating fast enough to mitigate the “mid-curve” disasters, such as population revolt due to mass inequality or cyberattacks caused by vulnerabilities in untested systems.
JonathanRaines 1y ago
I agree with what you're worried about - humans using AI to do bad things. However, have you considered also being worried about what AI can do? (can you ever have too much existential dread?) You cast it as a tool, it may be the first thing we've made that goes beyond that.
blueflow 1y ago
def written by a clanker
ImPleadThe5th 1y ago
I'm kind of shocked by this thread. I cant get over the hubris that we think concepts that were introduced to society at large by science fiction in and before the 19th century is _inevitable_ just because we made a really really good predictive text engine in the 21st century. Just because a concept exists star trek episode does not guarantee technology moving in that direction. I understand art has an effect on reality, but how hard are we spinning our gears because some writer made something so compelling it lives in our collective psyche? You can point to the communicator from star trek and I'll point to the reanimation of Frankenstein's monster.
- myrmidon 1y ago
  How has the feasibility of AI anything to do with science fiction? Unless you believe in some kind of magic (soul/divine spark/etc.), it seems completely inevitable to conclude that human cognition can be replicated by a machine, at the extreme end simply by simulating the whole thing. I would argue that "language" was a defining characteristic of human intelligence (as opposed to "lesser animals") long before we even conceived of AI, and hitting language processing/understanding benchmarks that far exceed animal capabilities is a very strong indicator by itself.
  - ImPleadThe5th 1y ago
    To clarify my stance, I do think that AI research is a good thing and that it can help us understand our own consciousness by studying it. I just think anyone concerned about AGI at this point is drinking too much kool-aid. I think science fiction makes us feel like AGI is closer and more relevant than it really is. Are we really so close that there's "no time" to regulate for it? Are we even close enough to give a care at all? Is there any evidence at this time at all that AGI would be a existential threat? Is all this talk coming from decades of "Creator vs. Daemon" stories combined with LLMs abilities to sound human? Does someone gain something from pointing our attention at the far-far scifi imagined future instead of current day issues with AI? To your simulation point. We talk about our bodies, brains and conciousness like they are machines, computers, and software. They are not. They are organic and their function is highly emergent. Think about all the endless input our bodies receive, we probably don't even fully understand all the sensitivities our "consciousness" has. We don't "execute" a program based on input, in fact the input is likely part of the system. Its why our gut biome can affect our mental health, why our cells communicate with each other locally and likely guide our behaviour in ways we don't understand yet. In other words, we're not any closer to simulating human consciousness than we are to simulating the universe itself. So I do think its hubris to think that a rock with some switches on it running some mathematical probability model can come even close to generating consciousness. > exceed animal capabilities Hubris like this. We don't even have a way to measure this. We've just made a math equation that can spit out language, you're comparing this to things that can actually _experience_ without someone clicking enter on a keyboard.
renewiltord 1y ago
Humanity is the scariest, yes. But a human is not scary.
olooney 1y ago
I think this is interesting and partially true: humans are scary. But it's important to remember the opposite is true as well: humans are the most cooperative species out there by a wide margin. Eusocial insects and pack animals are in a distant 2nd and 3rd place: they generally don't cooperate much past their immediate kin group. Only humans create vast networks of trade and information sharing. Only humans establish establish complex systems to pool risk, or undertake public works for the common good. In fact, a big part of the reason we are so scary is that ability to coordinate action. Ask any mammoth. Ask the independent city states conquered by Alexander the Great. Ask Napoleon as he faced the coalition force at Waterloo. We are victims of our own success: the problems of the modern world are those of coordination mechanisms so effective and powerful that they become very attractive targets for bad actors and so are under siege, at constant risk of being captured and subverted. In a word, the problem of robust governance. Despite the challenges, it is a solvable problem: every day, through due diligence, attestations, contract law, earnest money, and other such mechanisms people who do not trust each other in the least and have every incentive to screw over the other party are able to successfully negotiate win-win deals for life altering sums of money, whether that's buying a house or selling a business. Every century sees humans design larger, more effective, more robust mechanisms of cooperation. It's slow: it's like debugging when someone is red teaming you, trying to find every weak point to exploit. But the long term trend is the emergence of increasingly robust systems. And it suggests a strategy for AI and AGI: find a way to cooperate with it. Take everything we've learned about coordinating with other people and apply the same techniques. That's what humans are good at. This, I think, is a more useful framing than thinking of humans as "scary."
- Spivak 1y ago
  I'm not sure if it's your intention but this reads as a strong critique of the technolibertarianism world philosophy that dominates our industry. We lose something by replacing high-trust cooperative systems with ones that are mutually antagonistic. We fall into the bad square of the prisoners dilemma by not only not exorcising the defectors but holding them up as the highest moral good and the example to follow.
  - 1y ago
owenfi 1y ago
I like this fairytale as an analogy for AGI, but this article misses the point. We are the mouse. > I don’t really believe in the threat of AGI Just like the mouse doesn't believe in the Gruffalo (until it shows up). The mouse goes through the woods scaring the hypothetically-more-dangerous creatures with its stories (us, using our intellect, weapons, destroying habitats) until the real Gruffalo shows up. For a bit, the mouse "uses" the new tool to scare the animals even more (as alluded, human with tool, scarier than without). Eventually the mouse scares the Gruffalo away (analogous to the brief window when we think we have AGI under control). The next (unwritten) chapter probably doesn't look so good for the mouse (when the gruffalo grows to enormous size, eats him and all the other animals in the woods on a sandwich, and sucks up the rest of the resources on the planet.)
HellDunkel 1y ago
I am increasingly less willing to take this stuff seriously. A catchy headline, that’s about it.
keeda 1y ago
Something the article glosses over is that we are the scariest monster in the woods because we are intelligent. We can "survive, adapt, control, kill, or wipe out" anything or anybody else because our intelligence lets us outsmart nature and other people. And now we've created some form of intelligence that none of us really understands. Whether it is "real" intelligence or a "stochastic parrot" does not matter if it shows similar capabilities as us. Worse, it's similar but different in ways we cannot explain! I mean, if it outperforms most humans on advanced tasks but then makes elementary mistakes that a 4-year old won't, isn't that weird? "Weird" can be good or bad, and I usually like "weird"... but now we're rapidly giving it tools to affect the real world and exponentially expanding the scale at which it can operate. We don't know what weird compounded at that scale and capability extrapolates to. Whether it is a SkyNet or a Bond supervillain or an Asimov scenario, the potential risks are considerable and unpredictable. It's fair to be concerned about another monster in the woods even if we created it ourselves, because we've equipped it with the same powers we possess without understanding how they work.
tim333 1y ago
>.. if you equip humans with a hammer, or sword, or rifle, or AI then you’ve just made the scariest monster in the woods (that’s you) even more terrifying. I have some optimism that humans with AI may be less scary than without. One of the biggest problem is once humans get in power, having said they'll be nice they go a bit nuts like Putin or quite a few others I'm sure you can think of. If instead we had things run by open source AI everyone could see what it was thinking and it wouldn't be so able to act nice and then flip to trying to be dictator and launch wars. So far the LLMs have tended towards bland niceness which is kind of what you want in government.
burnt-resistor 1y ago
This writing is another example of anthropocentricist hubris. It doesn't really matter to us humans if "artificial life" AGI agents can/will exist at all or not, it's a MacGuffin canard. The greater impact is that many forms of narrow AI/ML, with some hype deserved and some not, are and will be used with minimal regulation to upheave society in profound ways like the cotton gin, steam engine, transistor, internet, and smartphone all in one to impoverish [mb]illions of workers and make trillionaires. In the absences of corrective measures, the present course is running towards ever more grotesque inequality, deprivation, corruption, and tyranny. Flock cameras' "ALPR" object and people classifier mass surveillance and likely misuse by intelligence services, police, and data brokers is just one other example. Hidden socioeconomic "social credit"-like data brokering is already selectively benefitting and sabotaging people by routine discrimination they're unable to inquire about.
tylervigen 1y ago
I think the problem with this argument is that it’s like saying “don’t worry about nuclear weapons, worry about the people who will use nuclear weapons.” Sure it’s true, in the absolute sense, but it’s more helpful to specify AI than it is to specify humans.

tocs3 1y ago

https://en.wikipedia.org/wiki/I_Am_Legend_(novel) https://en.wikipedia.org/wiki/I_Am_Legend_(novel)

deleted 1y ago

[deleted]

kkukshtel 1y ago

(DISCO ELYSIUM SPOILERS BELOW) One of the most poignant moments in Disco Elysium is near the very end when you encounter a very elusive crypid/mythic beast. The moment is treated with a lot of care and consideration, and the conversation itself is, I think, transcendent and some of the best writing in games (or any media) ever. The line that sticks with me most is when the cryptid says: "The moral of our encounter is: I am a relatively median lifeform -- while it is you who are total, extreme madness. A volatile simian nervous system, ominously new to the planet. The pale, too, came with you. No one remembers it before you. The cnidarians do not, the radially symmetricals do not. There is an almost unanimous agreement between the birds and the plants that you are going to destroy us all." It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. Everything runs from us, we are not part of the conversation. We are the exclusion, the anomaly. I think to realize this deeply inside of yourself is a big moment of growth, to see that we exist in a world that was around long before us and will be around long after us.

Quarrelsome 1y ago
personally I'm a little concerned about meeting alien life. If the universe is incredibly diverse with many different types of lifeform then how might an arbitrary one of those view us? We proliferate incredibly quickly, we have limited care for our environments, but most importantly our primary means of sustinance is to consume other forms of life. To the point that we consider it an art form, we spend vast amounts of energy perfecting, discussing or advertising the means of cooking and eating the flesh of other life forms. To us its normal but surely to an alien who gains sustinance by some other means; we're absolutely terrifying. Akin to a devouring swarm.
- saulpw 1y ago
  You Eat Other Animals? https://www.youtube.com/watch?v=_gz9Vj8TMCc https://www.youtube.com/watch?v=_gz9Vj8TMCc
sickofparadox 1y ago
>It's easy to see this reflected in nature in the real world. All animals and life seem to be aware and accommodating of each other, but humans are cut out from that communication. This just seems like noble savaging birds and rabbits and deer. None of these creatures have any communication with each other, and while they may be more aware of each other's presence than a 'go hiking every once in a while' person, someone who actually spends a good amount of time in the woods, such as a hunter or birdwatcher, probably has a pretty good sense of them. The Disco Elysium quote just reads like fairly common environmentalist misanthropy, which I suppose isn't surprising considering the creators.
- wat10000 1y ago
  I think people forget how big people are. We're well above average size in the typical natural environment, and especially in the typical sorta-natural-but-sorta-urban environment most of us are in most of the time. The local rabbits and squirrels tolerate each other but are pretty scared of me. Of course they are, I'm two hundred times bigger than they are, and much more dangerous. The local foxes are the closest thing we have to an apex predator around here, and they're rightfully terrified of this massive creature that outweighs their entire family combined. Imagine wandering through the woods, enjoying the birds tweeting and generally being at one with nature, and then you come across a 20-ton 35ft-tall monster. You'd run away screaming.

us-merul 1y ago

"Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of." -- A pretty bleak, and also accurate, observation of humanity. I have to hope that the alternative sentence encompassing all of the good can lead to some balance.

wmeredith 1y ago
I really liked this article, but it is pessimistic. Unfortunately that seems to be the culture-du-jour. Anger and fear drive engagement effectively, as it always has. If it bleeds, it Leeds has been a thing in news organizations since at least the 70s. If we ignore the headlines peddled by those who stand to benefit the most from inflaming and inciting, we live in a miraculous modern age largely devoid of much of the suffering previous generations were forced to endure. Make no mistake there are problems, but they are growing exponentially fewer by the day. An alternate take: humans will do what they’ve always tried to do—build, empower, engineer, cure, optimize, create, or just collaborate with other humans for the benefit of their immediate community—but now with new abilities that we couldn’t have dreamed of.
- pixl97 1y ago
  I mean, any article that doesn't include both is incomplete. >If it bleeds, it Leeds has been a thing in news organizations since at least the 70s. The term yellow journalism is far older.
Flamingoat 1y ago
No, it is not accurate at all. There are some people that do all of these sure, however the vast majority of people live pretty ordinary lives where they do very little of what is described. I actually think it is very intellectually lazy to be this cynical.
- ViktorRay 1y ago
  I think it is partially true. The vast majority of human beings don’t act like that. But it seems the ones in power or close proximity to power do. This is why it is important to have societies where various forms of power are managed carefully. Limited constitutional government with guaranteed freedoms and checks and balances for example. Regulations placed on mega corporations is another example. Restrictions to prevent the powerful in government or business (or both!) from messing around with the rest of us…

ekimekim 1y ago

As always, there is a relevant XKCD: https://xkcd.com/1968/ https://xkcd.com/1968/

Filligree 1y ago
We can worry about both. There being additional dangers does not reduce the impact of the other dangers.
- excalibur 1y ago
  True, but one of them is much more imminent than the other.

blamestross 1y ago

We have been using the mechanism of "Humans communicating and working together" to form superintelligences for a long time. They have gotten more effective, more efficient and more durable with time. AI isn't the new monster. It's a new mitochondria being domesticated to supercharge the existing monsters.

forgetfulness 1y ago
That reminds me of the “paper clip maximizer” thought experiment about rogue AI fulfilling a particular goal, without being held to any consideration of human welfare I thought it was a very novel idea at first, until I realized that this describes all manners of human groups, notably corporations and lobbying groups, they will turn the world into a stove, subvert democracy, (try to) disrupt the economic and social fiber of society, anything to eg maximize shareholder value
- Nasrudith 1y ago
  And corporations are ultimately just sock-puppets for humanity. People forget that and like to other them while simultaneously turning them into the scapegoats. See every idiot with the list of companies responsible for global warming that amounts to 'every oil company' and ignore that it was the customers were the ones burning their products. It is to be expected really. Humans themselves hold little consideration to human welfare when fulfilling their goals. It is something ingrained by nature for survival and in no way limited to humans. Every drop of water you drink, every bite you eat, are ones which cannot go to the thirsty and the hungry. With few exceptions, only our children would even give us pause to forgo such things for the sake of others. Also bit of a pet peeve of mine: society isn't a delicate bolt of laced silk. It isn't a fabric, much less one that is damaged by any little change that you don't like. It isn't even stable which makes the charges of anything as 'ruining' society especially bizarre when nobody can point to where it would go before the blamed change. So hold off on poisoning Socrates. Even if we hold the current state as worthy of preservation we would ultimately fail to for reasons related to the central paradox of tradition. Why your forebearers first did something was not done because of obedience to tradition, so by trying to preserve it set in stone, you have already failed.

dwnw 1y ago

"to the monsters we're the the monsters" station eleven

deleted 1y ago

[deleted]

jvanderbot 1y ago

About as helpful as "Guns don't kill people ... " And equally rebutted by Eddie Izzards "Well, I think the gun helps".

fruitworks 1y ago
All of the solutions to AI "safety" are analagous to gun control: it is a centralization of power.
- Cthulhu_ 1y ago
  Are you claiming a lack of gun / AI control is democratizing? That's not working for (the lack of) gun control in the US at the moment though. Compare also with capitalism; unchecked capitalism on paper causes healthy competition, but in practice it means concentration of power (monopolies) at the expense of individuals (e.g. our accumulated expressions on the internet being used for training materials).
  - fruitworks 1y ago
    >Are you claiming a lack of gun / AI control is democratizing? This is obviously the case. It results in a greater distribution of power. >That's not working for (the lack of) gun control in the US at the moment though. In the US, one political party is pro gun-control and the other is against. The party with the guns gets to break into the capitol, and the party without the guns gets to watch. I expect the local problem of AI safety, like gun safety will also be self-solving in this manner. Eventually, Gun control will not work anywhere, regardless of regulation. The last time I checked, you don't need a drone license. And what are the new weapons of war? Not guns. The technology will increase in acessibility until the regulation is impossible to enforce. The idea that you can control the use of technology by limiting it to some ordained group of is very brittle. It is better to rely on a balance of powers. The only way to secure civilization in the long run is to make the defensive technology stronger than the offensive technology.
    - dctoedt 1y ago

d_silin 1y ago

That's why I am never afraid to walk alone at night, in the dark forest.

whycombinetor 1y ago
Must not live in the Sundarbans

myrmidon 1y ago

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). As soon as profit can be made by transfering decision power into an AIs hand, some form of agency for them is just a matter of time, and we might simply not be willing to pull the plug until it is much too late.

moralestapia 1y ago
>why would it be impossible to achieve the exact same result in silicon Because there might be a non-material component involved.
- ACCount37 1y ago
  [flagged]
  - moralestapia 1y ago
    @dang
    - tomhow 1y ago
      We replied to that comment, but the @reply isn't a reliable way to get moderator attention, and publicly calling for moderator help like this isn't a great look. Best to flag the comment and email us if you really want us to take a look.
  - tomhow 1y ago
    Please don't post sneery comments like this on HN. The guidelines make it clear we're here for curious conversation, and kindness is one of the foremost expectations.

petsfed 1y ago

A nit: There's a subtle distinction between an individual human and the power of human organization and civilization that is implied by the article, but never outright stated. One-for-one, there are many creatures that are individually more dangerous to humans, and a decent number of people are killed by such animals every year. Indeed, a naked human in the wild is going to be quite fragile and easy to kill until they can bring some technology to bear. But there are no animals or even set of animals that could conceivably wipe out all of humanity at any of our technological peaks from the last 100,000 years. Even the number one killer of humans, the mosquito, is gradually being defeated, going from a vector for disease to just an annoyance, just like the flea.

pixl97 1y ago
A lot of rugged individualists seem to neglect talking about the human super organism. That the vast majority of our strength comes from sharing knowledge, making tools, and working together. And on that same note it should be mentioned that exchange of information between humans is relatively slow and guarded. A group of entities that could exchange knowledge quickly and efficiently would represent an extreme challenge for us.
- hn_acc1 1y ago
  They also ignore that a lot of the knowledge they now have / have studied comes FROM others sharing freely.

whycombinetor 1y ago

"We (humans) are the scariest animal in the woods. We’re the scariest animal anywhere. At any time, in any location, under any circumstances, if there’s a human present then that’s the scariest motherfucker in the woods." Has this guy not heard of the Bengal tigers of the Sundarban forests, which kill ~50 humans per year? https://en.wikipedia.org/wiki/Tiger_attacks_in_the_Sundarbans https://en.wikipedia.org/wiki/Tiger_attacks_in_the_Sundarban...

WJW 1y ago
I think he heard of all the other forests, where tigers are now extinct because humans are around.
graemep 1y ago
> Has this guy not heard of the Bengal tigers of the Sundarban forests, which kill ~50 humans per year? How many tigers have been killed by humans? How many would be killed if humans did not restrain each other from killing tigers because we have killed so many that they are endangered. That population could be entirely wiped out by humans in the area. Mosquitoes kill far more people than tigers. So do venomous snakes. The fact is that when a human faces either of these they are far more likely to end up dead than the human.
- whycombinetor 1y ago
  This isn't about what humans have done at other times or in other places. The author wrote "At any time, in any location, under any circumstances", so the statement can be proven false with one single counterexample - e.g. a lone human without a firearm encountering a tiger in the Indian Sundarbans.
  - graemep 1y ago
    I interpreted that as hyperbole.
bob1029 1y ago
A desperately hungry tiger might go for a human, but on average they are not interested in that fight. They would much prefer going for something like deer. The calories per unit of risk are much higher with non-human food sources. Wildlife tends to be very efficient at this economy. Anything smaller than a human is absolutely terrified of us. I used to be afraid of things like snakes and spiders, but they want nothing to do with humans. They will get the hell out of your way when you are walking through the woods. You have to do something really stupid to get bitten. Keeping animals as pets is where most of the trouble happens.

armada651 1y ago

> Just like a hammer, sword, or a rifle lying on a ground is nothing to be feared, so too is AI. It’s just an inanimate object; a tool, potentially. A bomb is an inanimate object too, but if you find some unexploded ordnance lying on the ground you should fear it.

robertlagrant 1y ago
An unexploded bomb definitely has the potential to be highly animate. It's just not currently animate.
- alt227 1y ago
  As does AI.

gota 1y ago

> I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it. I'm on board with being skeptical that LLMs will lead to AGI; but - there being no possibility seems like such a strong claim. Should we really bet that there is something special (or even 'magic') about our particular brain/neural architecture/nervous system + senses + gut biota + etc.? Don't, like, crows (or octopuses; or elephants; or ...) have a different architecture and display remarkable intelligence? Ok, maybe not different enough (not 'digital') and not AGI (not 'human-level') but already -somewhat- different that should hint at the fact that there -can- be alternatives Unless we define 'human-level' to be 'human-similar'. Then I agree - "our way" may be the only way to make something that is "us".

kulahan 1y ago
We still haven’t figured out what intelligence even is. Depending on what you care about, the second-smartest animal in the world varies wildly.
- myrmidon 1y ago
  Evolution had a much worse understanding of intelligence than we do and still managed just fine (thus i'd expect us to need less time and iterations to get there).
- squidbeak 1y ago
  This is a bogus argument. There's a lot we don't understand about LLMs, yet we built them.
  - simianparrot 1y ago
    We built something we don’t understand by trial and error. Evolution took a few billion years getting to intelligence, so I guess we’re a few sprints away at least

snozolli 1y ago

At any time, in any location, under any circumstances, if there’s a human present then that’s the scariest motherfucker in the woods. This guy has clearly never bumped into a grizzly bear momma, a moose in rut, a hippo, or loads of other animals. Personally, I'm not even worried about AI itself, I'm worried about the people wielding it. MBAs are the scariest monsters in the woods.

Insanity 1y ago
In a 1-to-1 situation you are right, a single human is not the scariest. But humans (mostly) dominated the animal kingdom, and with enough time we could probably eradicate every other species if we wanted to. I guess bacteria and viruses might have a stronger claim for being the 'scariest monster' though. I don't think there's a strong 'living' contender for wiping out humanity apart from those. (Viruses is a bit debatable on the 'living' part though).

Insanity 1y ago

So “guns don’t kill people, people with guns kill people”. But for AI I’m not sure that preposition will hold indefinitely. Although I do think we are a far away from having actual AGI that would pose this threat. Still, the author has a good but obvious point.

deleted 1y ago
[deleted]
supermatt 1y ago
> guns don’t kill people * SIG P320 enters the chat *

woeirua 1y ago

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war keeps me up at night.

runjake 1y ago
100%. Except the things that kill most of us[1] keeps me up at night. 1. https://ourworldindata.org/does-the-news-reflect-what-we-die-from https://ourworldindata.org/does-the-news-reflect-what-we-die...
- masfuerte 1y ago
  I'm more concerned with the ways that other people can kill me than with the ways they kill themselves.
myrmidon 1y ago
I personally think that AI is a realistic extinction threat for our whole species within this century, and a full nuclear war never was (and probably never will be). Neither is climate change. Collapse of our current civilizations? Sure. Extinction? No. And I honestly see stronger incentives on a road towards us being outcompeted by AI, then on our leaders starting a nuclear war.
- lm28469 1y ago
  Personally I don't bieleve in a collapse nor extinction, just a slow spiral into more and more enshittification. You'll have to talk to an "ai" doctor because real doctors will treat people with money, you'll have to face an "ai" administration because the real administration will work for people with money, you'll have to be a flesh and blood robot to an "ai" telling you what to do (already the case for Amazon warehouse workers, food delivery people, &c.), some "ai" will determine if you qualify for X or Y benefits, X or Y treatment, X or Y job. Basically everything wrong with today's productivism, but 100 times worse and powered by a shitty ai that's very far from agi.

superkuh 1y ago

Human beings are not the scariest monsters in the woods. By far the most dangerous, most malicious, and most powerful monsters are conglomerate entities: the legal corporate persons. They've been absolutely ravaging the earth and societies since they achieved legal personhood and rights without any of the downsides or legal liabilities. They're much scarrier than individual actual human persons. And while most people blame "AI" for the web's current troubles AI is only barely involved. The real culprits are the corporations. Again, same as it ever was. We really need to get rid of corporate personhood. Or at least have a corporate death penalty.

XiphiasX 1y ago

Thanks for the encouragement.

abbycurtis33 1y ago

What I'm scared by now is non-sentient intelligence, which I never thought would be this capable and still obviously dumb, which really makes it dangerous.

slow_typist 1y ago
Is sentient intelligence less scary than the non-sentient variant? Or is non-sentient intelligence already scary enough?
- abbycurtis33 1y ago
  I hope we can reason with sentient intelligence.

psychoslave 1y ago

All monsters were born equal, but some are more scarier than other.

jmull 1y ago

So silly. Of course humans are (by far) the source of the biggest problems humans face. The argument here is that since humans are the scariest, we should ignore problems AI might cause. Pure nonsense, since (1) AI is human-created — so just another piece of what makes us scary; (2) this kind of binary thinking makes no sense anyway — as if there can’t be multiple things to be concerned with. > Anyone trying to tell you otherwise is trying to distract you. A particularly poor bit of argumentation. Barely above sticking your fingers in your ears and shouting, “Blah, blah, blah! I can’t hear you!” What is is about AI that makes people lose their minds? edit: seeing the engagement this is getting, maybe it’s a false flag operation of sorts? Making such bad anti-ai arguments that the argument for ai gets stronger? That would at least make some sense.

hdseggbj 1y ago

The distinction is irrelevant. No one thought the nukes would blow themselves up, but here we are on the precipice of destruction. We aren't going to bury ai in 20 story bunkers to protect llms. Everyone knows the real evil are the people building the ai. We aren't afraid of agi we are afraid of Boston dynamics (aka Google). We are afraid of Bill gates, Sam altman, Jeff bezos, Larry Ellison and elon musk. These are dangerous people. Their mind set. Their unstoppable lust for money and power. We are afraid of their ability to convince humans to do unspeakable things for this imaginary thing we call money. With agi they don't have to persuade anyone with anything. There's not Even a human conscience to stop them. We know these folks have no conscience of their own. The world is still here because other people have them. Agi won't have a conscience or the integrity to stop evil. Human society is collapsing because Good people are doing nothing.

bryanlarsen 1y ago
P.S. Boston Dynamics is no longer owned by Google, it's part of Hyundai now.
- hdseggbj 1y ago
  Ah yes, rings a bell. Doesn't change the sentiment though. They're all crooks and cheaters who will sacrifice the greater good for their own self interest. https://en.wikipedia.org/wiki/2025_Georgia_Hyundai_plant_immigration_raid https://en.wikipedia.org/wiki/2025_Georgia_Hyundai_plant_imm... It's the fear of death that's killing us all.
  - bryanlarsen 1y ago
    There are tons of examples of chaebols behaving evilly in Korea. You don't have to pull out a scenario where half of Americans and most non-Americans think Hyundai are the victims, not the bad guys.
    - dbjkffgbn 1y ago
      [dead]

nyeah 1y ago

This is pleasant to read, but it's missing a logical step. We are great at dealing with the stuff we evolved to deal with. Sure. But anything outside that evolutionary process? Who knows? For example we don't do so good on top of Mt Everest. We seem to have totally unsolved problems with refined opiates. Even computer games can be a trap. Agentic AI? Who knows?

WhitneyLand 1y ago

For the love of God let’s ignore him being wrong about the impossibility of AGI, and take to heart the nightmarish and inevitable threats coming from how people will use it, that will almost certainly challenge us first.

raldi 1y ago

The author comes so close to getting it, with the paragraph about how if you drop a human into an environment, they inevitably take over as the deadliest and most powerful creature. But the next step is to ask why; in the case of the Gruffalo it was obvious: fangs, claws, strength, size… In the case of humans, it’s because we’re the most intelligent creature in the forest. And for the first time in our history, we’re about to not be.

lif 1y ago
yes, and: ruthlessness + strength + WMD =/= intelligence
audunw 1y ago
No, it’s not really about intelligence. That’s not what’s driving the behaviour. That’s like saying a calculator is dangerous because it was used to make an atomic bomb. A calculator is infinitely smarter at math than any animal. Yet it’s not what’s dangerous. What’s driving humanity’s dangerous behaviours is that we’ve evolved over millions upon millions of years to compete with each other for resources. We had to kill to survive. Both animals and each others. AI is under no such evolutionary process. In fact quite the opposite: there’s an absolutely ruthless process to eliminate any AI which doesn’t do exactly what humans say with as little energy wasted as possible. There is no room in that process for an AI to develop that will somehow want to compete with humans. So whatever bad happens, it is overwhelmingly likely to be because a human asked an AI to do it. Not that an AI will decide on its own to do it to further its own interest. I don’t even think a “paperclip maximiser” scenario is remotely probable. Long before there could be a successful attempt to exterminate humans to reach some goal, there will be countless failed or half-assed attempts. That kind of behaviour will have no chance to develop.

catapart 1y ago

I can't figure out why people who write articles like this seem to be oblivious to the very practical argument about the US second amendment that goes along the lines of "guns don't kill people, people do". To everyone who wants to pose an argument like this, please consider: We have already internalized that very simple, and very thoughtless argument, and have moved past that facile framing to start asking the more pressing questions like "what are we going to do about how humans are using X to perpetrate Y?" We don't need the extra step of you making us reframe the question to fit your exact criteria of "good question". "AI is going to kill us" can be understood as: "Humans are going to use AI to kill us". The fact that you want to misunderstand it as a more childish argument is your own failings with semiotics. It's so blatantly silly that in most cases, it's less degrading to assume that you are purposefully trying to misdirect the argument than to assume you actually believe people are worried about the agency of inanimate objects. But then you go and write a whole article to preen "logical" at us about how it's just that "no one's asking the right questions!", which kind of makes it hard to imagine that you don't actually think so little of people.

Timsky 1y ago

> We don’t need to worry about AI itself, we need to be concerned about what “humans + AI” will do. Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of. Starting with the AI itself: LLMs sold as AI are the greatest mislead. Text generation using Markov chains is not particularly intelligent, even when it is looped back through itself a thousand times and appears alike an intelligent conversation. What is actually being sold is an enormous matrix trained on terabytes of human-written, high-quality texts, obviously in violation of all imaginable copyright laws. Here is a gedanken experiment to test if an AI has any intelligence: until the machine starts to determine and resolve contradictions in its own outputs w/o human help, one can sleep tight. Human language is a fuzzy thing that is not quite suitable for a non-contradictory description of the world. Building such a machine would require resolving all the contradictions humanity has ever faced in a unified way. Before it happens, humanity will be drowned in low-quality generated LLM output.

om8 1y ago

... Yet.

lifeisstillgood 1y ago

Yes - a simple and necessary reminder of what goes on the priority list

buu700 1y ago

I see the point the author is making, but AI doesn't need to be AGI or malicious to cause mass destruction. A poorly implemented deployment of AI in the right circumstances with too much access and insufficient guardrails could theoretically wind up LARPing as Skynet purely by chance. This is why I've always considered current AI "safety" efforts to be totally wrongheaded. It's not a threat to humanity if someone has an AI generate hate speech, porn, misinformation, or political propaganda. AI is only a threat to humanity if we don't take security seriously as we roll out increasingly more AI-driven automation of the economy. It's already terrifying to me that people are relying on containers to sandbox yolo-mode coding agents, or even raw dogging them or their personal machines.

tourist2d 1y ago

[dead]

kazinator 1y ago

> Just like a hammer, sword, or a rifle lying on a ground is nothing to be feared, so too is AI. AI is not just going to lie on the ground until someone picks it up to do harm. An intelligent rifle with legs is something to be feared. You cannot compare AI to inanimate objects under human control which have no agency. Especially not if you are bringing the imaginary AGI into the conversation. The idea that AGI is just a hammer is absurd.

bilater 1y ago

I sort of agree, but I hate the premise of this article because it sneakily focuses only on the potential harm of human + AI collaboration, without acknowledging the good. That said, I agree that human + AI can cause damage and it’s precisely why, from a game theory perspective, the right move is to go full steam ahead. Regulation only slows down the good actors. Valhalla is within reach, but we have to leap across a massive chasm to get there. Perhaps the only way out of this is through: accelerating fast enough to mitigate the “mid-curve” disasters, such as population revolt due to mass inequality or cyberattacks caused by vulnerabilities in untested systems.

JonathanRaines 1y ago

I agree with what you're worried about - humans using AI to do bad things. However, have you considered also being worried about what AI can do? (can you ever have too much existential dread?) You cast it as a tool, it may be the first thing we've made that goes beyond that.

blueflow 1y ago

def written by a clanker

ImPleadThe5th 1y ago

I'm kind of shocked by this thread. I cant get over the hubris that we think concepts that were introduced to society at large by science fiction in and before the 19th century is _inevitable_ just because we made a really really good predictive text engine in the 21st century. Just because a concept exists star trek episode does not guarantee technology moving in that direction. I understand art has an effect on reality, but how hard are we spinning our gears because some writer made something so compelling it lives in our collective psyche? You can point to the communicator from star trek and I'll point to the reanimation of Frankenstein's monster.

myrmidon 1y ago
How has the feasibility of AI anything to do with science fiction? Unless you believe in some kind of magic (soul/divine spark/etc.), it seems completely inevitable to conclude that human cognition can be replicated by a machine, at the extreme end simply by simulating the whole thing. I would argue that "language" was a defining characteristic of human intelligence (as opposed to "lesser animals") long before we even conceived of AI, and hitting language processing/understanding benchmarks that far exceed animal capabilities is a very strong indicator by itself.
- ImPleadThe5th 1y ago
  To clarify my stance, I do think that AI research is a good thing and that it can help us understand our own consciousness by studying it. I just think anyone concerned about AGI at this point is drinking too much kool-aid. I think science fiction makes us feel like AGI is closer and more relevant than it really is. Are we really so close that there's "no time" to regulate for it? Are we even close enough to give a care at all? Is there any evidence at this time at all that AGI would be a existential threat? Is all this talk coming from decades of "Creator vs. Daemon" stories combined with LLMs abilities to sound human? Does someone gain something from pointing our attention at the far-far scifi imagined future instead of current day issues with AI? To your simulation point. We talk about our bodies, brains and conciousness like they are machines, computers, and software. They are not. They are organic and their function is highly emergent. Think about all the endless input our bodies receive, we probably don't even fully understand all the sensitivities our "consciousness" has. We don't "execute" a program based on input, in fact the input is likely part of the system. Its why our gut biome can affect our mental health, why our cells communicate with each other locally and likely guide our behaviour in ways we don't understand yet. In other words, we're not any closer to simulating human consciousness than we are to simulating the universe itself. So I do think its hubris to think that a rock with some switches on it running some mathematical probability model can come even close to generating consciousness. > exceed animal capabilities Hubris like this. We don't even have a way to measure this. We've just made a math equation that can spit out language, you're comparing this to things that can actually _experience_ without someone clicking enter on a keyboard.

renewiltord 1y ago

Humanity is the scariest, yes. But a human is not scary.

olooney 1y ago

I think this is interesting and partially true: humans are scary. But it's important to remember the opposite is true as well: humans are the most cooperative species out there by a wide margin. Eusocial insects and pack animals are in a distant 2nd and 3rd place: they generally don't cooperate much past their immediate kin group. Only humans create vast networks of trade and information sharing. Only humans establish establish complex systems to pool risk, or undertake public works for the common good. In fact, a big part of the reason we are so scary is that ability to coordinate action. Ask any mammoth. Ask the independent city states conquered by Alexander the Great. Ask Napoleon as he faced the coalition force at Waterloo. We are victims of our own success: the problems of the modern world are those of coordination mechanisms so effective and powerful that they become very attractive targets for bad actors and so are under siege, at constant risk of being captured and subverted. In a word, the problem of robust governance. Despite the challenges, it is a solvable problem: every day, through due diligence, attestations, contract law, earnest money, and other such mechanisms people who do not trust each other in the least and have every incentive to screw over the other party are able to successfully negotiate win-win deals for life altering sums of money, whether that's buying a house or selling a business. Every century sees humans design larger, more effective, more robust mechanisms of cooperation. It's slow: it's like debugging when someone is red teaming you, trying to find every weak point to exploit. But the long term trend is the emergence of increasingly robust systems. And it suggests a strategy for AI and AGI: find a way to cooperate with it. Take everything we've learned about coordinating with other people and apply the same techniques. That's what humans are good at. This, I think, is a more useful framing than thinking of humans as "scary."

Spivak 1y ago
I'm not sure if it's your intention but this reads as a strong critique of the technolibertarianism world philosophy that dominates our industry. We lose something by replacing high-trust cooperative systems with ones that are mutually antagonistic. We fall into the bad square of the prisoners dilemma by not only not exorcising the defectors but holding them up as the highest moral good and the example to follow.
- 1y ago

owenfi 1y ago

I like this fairytale as an analogy for AGI, but this article misses the point. We are the mouse. > I don’t really believe in the threat of AGI Just like the mouse doesn't believe in the Gruffalo (until it shows up). The mouse goes through the woods scaring the hypothetically-more-dangerous creatures with its stories (us, using our intellect, weapons, destroying habitats) until the real Gruffalo shows up. For a bit, the mouse "uses" the new tool to scare the animals even more (as alluded, human with tool, scarier than without). Eventually the mouse scares the Gruffalo away (analogous to the brief window when we think we have AGI under control). The next (unwritten) chapter probably doesn't look so good for the mouse (when the gruffalo grows to enormous size, eats him and all the other animals in the woods on a sandwich, and sucks up the rest of the resources on the planet.)

HellDunkel 1y ago

I am increasingly less willing to take this stuff seriously. A catchy headline, that’s about it.

keeda 1y ago

Something the article glosses over is that we are the scariest monster in the woods because we are intelligent. We can "survive, adapt, control, kill, or wipe out" anything or anybody else because our intelligence lets us outsmart nature and other people. And now we've created some form of intelligence that none of us really understands. Whether it is "real" intelligence or a "stochastic parrot" does not matter if it shows similar capabilities as us. Worse, it's similar but different in ways we cannot explain! I mean, if it outperforms most humans on advanced tasks but then makes elementary mistakes that a 4-year old won't, isn't that weird? "Weird" can be good or bad, and I usually like "weird"... but now we're rapidly giving it tools to affect the real world and exponentially expanding the scale at which it can operate. We don't know what weird compounded at that scale and capability extrapolates to. Whether it is a SkyNet or a Bond supervillain or an Asimov scenario, the potential risks are considerable and unpredictable. It's fair to be concerned about another monster in the woods even if we created it ourselves, because we've equipped it with the same powers we possess without understanding how they work.

tim333 1y ago

>.. if you equip humans with a hammer, or sword, or rifle, or AI then you’ve just made the scariest monster in the woods (that’s you) even more terrifying. I have some optimism that humans with AI may be less scary than without. One of the biggest problem is once humans get in power, having said they'll be nice they go a bit nuts like Putin or quite a few others I'm sure you can think of. If instead we had things run by open source AI everyone could see what it was thinking and it wouldn't be so able to act nice and then flip to trying to be dictator and launch wars. So far the LLMs have tended towards bland niceness which is kind of what you want in government.

burnt-resistor 1y ago

This writing is another example of anthropocentricist hubris. It doesn't really matter to us humans if "artificial life" AGI agents can/will exist at all or not, it's a MacGuffin canard. The greater impact is that many forms of narrow AI/ML, with some hype deserved and some not, are and will be used with minimal regulation to upheave society in profound ways like the cotton gin, steam engine, transistor, internet, and smartphone all in one to impoverish [mb]illions of workers and make trillionaires. In the absences of corrective measures, the present course is running towards ever more grotesque inequality, deprivation, corruption, and tyranny. Flock cameras' "ALPR" object and people classifier mass surveillance and likely misuse by intelligence services, police, and data brokers is just one other example. Hidden socioeconomic "social credit"-like data brokering is already selectively benefitting and sabotaging people by routine discrimination they're unable to inquire about.

tylervigen 1y ago

I think the problem with this argument is that it’s like saying “don’t worry about nuclear weapons, worry about the people who will use nuclear weapons.” Sure it’s true, in the absolute sense, but it’s more helpful to specify AI than it is to specify humans.