16 ms·

AI Safety and the Age of Dislightenment

124 points by wskinner 3y ago

jononomo 3y ago
It is wild how seriously people are taking the development of these LLMs. There is talk of them leading to the extinction of life while neglecting to consider that we can't even build self-driving cars, let alone have a robot clear the table after dinner. Also, many people seem to be mistaking "artificial intelligence" for "actual intelligence". You know that these LLMs are not actually intelligent, right? Right???
- CamperBob2 3y ago
  "Intelligence" is a word we made up. It doesn't mean much of anything at the end of the day. If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent." Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. Assuming that this is not the case, don't hold machines to standards you're not willing to hold humans to.
  - throw47474777j 3y ago
    “X is a word we made up. It doesn't mean much of anything at the end of the day.” This has to be a candidate for the most unhelpful statement possible. If you are sincere about it, then why not apply it to every word in your reply and see where that leaves you?
    - SpicyLemonZest 3y ago
      I can't speak for the original commenter, but I would definitely say that "is this actually X" is a pointless debate for most values of X. If I understand what people mean by X, there's no need to quibble about whether it satisfies my personal definition; if I don't understand, I should try to clarify rather than just saying "it's not really X" and crossing my arms.
  - bumby 3y ago
    I don't want to speak for the OP, but one of the issues they may be poking at is the transference of intelligence. E.g., you are "trained" to play a game, but at some point someone decides to change the rules of the game. Human children can transfer their previous knowledge to the new rule set fairly well. The most intelligent children can do so very quickly without supplemental "training" on the game played under the new rules. Humans certainly have flaws when it comes to this. I've heard some discussions about the success and failures of AI in this regard. Can someone in this domain elaborate on the current state-of-the-art performance in this regard?
    - famouswaffles 3y ago
      1. LLMs are general pattern machines. They can generalize and complete in context to a wide variety of complex non linguistic patterns.https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ 2. LLMs see positive transfer in multi lingual capability. For example, an LLM trained on 500B tokens of English and 10B tokens of French will not speak in french like a model trained on only 10B tokens of French. What will happen is that the model will be nearly as competent in French as it is in English https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349 3. Language models of code reason better even if the benchmarks have nothing to do with code. https://arxiv.org/abs/2210.07128 https://arxiv.org/abs/2210.07128
      - bumby 3y ago
        This is interesting in the context of the other response that links to poor performance in terms of counterfactuals. I wonder if it is related to how well one domain maps to another? E.g., can they transfer to english to french well because both share a similar word classes (nouns, verbs, etc.). But I believe other languages change more based on social context (e.g., Japanese) than English does. Would a LLM transfer just as well to the latter? In that case, my guess is humans would find it more difficult to transfer as well, so it may not be a good measure. (Apologies to any linguists. Please correct anything above if I'm off).
        famouswaffles 3y ago
  - goatlover 3y ago
    "All words are made up." - Thor The word was made up to cover a range of cognitive abilities that humans and animals (to varying degrees) possess. And we're gradually figuring out how to design machines with similar abilities. The general idea of intelligence being you can figure out how to do things that aren't just instinctual. a generalized intelligence can do this across any number of domains without some fixed limit.
  - AlexandrB 3y ago
    > Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. So you're saying humans, collectively, are gods? After all, as a species we started with nothing - no "clean" training data, no paintings to replicate, not even language itself. And here we are - arguing about whether one creation of ours, trained on a bunch of other stuff we created, is as intelligent as us.
    - famouswaffles 3y ago
      We didn't continue with nothing. Evolution is training. LLMs "start with nothing" too
  - prhn 3y ago
    > "Intelligence" is a word we made up So is all human knowledge. They're all things we "made up" to describe the world around us. That's a really cheap way to reduce something. There's a mountain of psychological literature that intends to describe and measure intelligence. Saying it doesn't mean much of anything is a bit of a postmodern hot take. > If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent." Painting pictures and composing poems aren't good ways to measure general intelligence. That's creativity. They're correlated but not the same. There are specific tests for measuring general intelligence, and there's absolutely no way current gen LLM would score highly on them because they don't even accept visual input. Yes I've seen that they can score highly on verbal-only tests. That's obvious anyway because they have seen, written down, and have access to the answers during the test. Even if they did accept visual input they would only be able to solve problems for which they've previously been given the answers (trained on the dataset). Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec.
    - CamperBob2 3y ago
      Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec. One problem is that ChatGPT4 has been degrading over time -- and no, I don't GAF about any opinions or assurances to the contrary. So we're seeing more and more people look at it for the first time and ask what the big deal is. Maybe the Code Interpreter feature will reverse that trend, we'll see. Those of us who were around (read: paying $20/month) when they first enabled GPT4 support are left waving our hands fecklessly, muttering "Yeah, but you should've seen it back in the old days, back in March of '23. Now you kids get off my lawn."
  - Keegs 3y ago
    > If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. Poetry is inspired in a way that’s hard to replicate with a prompt. Artists also take inspiration from other artists, but the “distance” between their influences and their work is so much greater than the distance between an LLM’s that I don’t fault anyone who think it’ll take time for LLMs to catch up.
    - CamperBob2 3y ago
      An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. And you don't grasp the fundamentally-incremental nature of this point?
      - jononomo 3y ago
        No. After you've asked ChatGPT to write 5-6 poems you'll get a sense for how profoundly uncreative it is as a poet. It's not going to "incrementally" be anything other than a dead-as-a-doornail algorithm.
        Aeolos 3y ago
        That's ChatGPT version 3. Version 4 is already significantly better. What about version 5? 6? 10?
  - mdp2021 3y ago
    > to be called "intelligent" quality is required.
- disadvantage 3y ago
  > we can't even build self-driving cars, let alone have a robot clear the table after dinner I think once you give AI working limbs and the ability to manipulate matter like humans, that's concerning, because if the AI becomes a runaway AI with no upper limits on the types of behavior allowed, we get into Paperclip Maximizer[0] territory very fast. [0] https://tanzanite.ai/paperclip-maximizer-experiment/ https://tanzanite.ai/paperclip-maximizer-experiment/
- gnulinux 3y ago
  I'm completely with you on this one. Sci-fi and AI hype are extremely strong on these threads. I bet we're not any closer to human level intelligence with ML than we were in 1500s with mechanical clocks. Both are impressive nonetheless.
- thiago_fm 3y ago
  Just as in finance everyday you see a hedge fund manager making some weird big bet of a stock or market for the next 20 years (like a prophet). Everyone wants to pose as the front(wo)man of that innovation, thought-leaders. Whatever. It's all marketing, for their own careers, their own personal agenda. The whole text looks like written by ChatGPT in a prompt like "Write about AI safety and relates it to the Age of Reason". This is a good reason why AI should replace us, so we no longer have human beings writing that stuff to self-promote themselves. The funniest part is in the end of the article: "Eric Ries has been my close collaborator throughout the development of this article" So the guy who wrote "The lean startup" is helping him with that article. drops mic
- marvin 3y ago
  A pretty influential and very well-financed subculture of AI philosophers are currently performing a comprehensive lobbying campaign for AI regulation. They generally go by Rationalists or Effective Altruism, and have a philosophy stemming from MIRI, Eliezer Yudkowsky, Nate Soares and others. They've had considerable success getting traction for these views with politicians in Europe, and that's the reason you see so much of this in the media currently. I'm hoping more sane voices will soon get organized, because significant parts of this are reminiscent of a well-funded doomsday cult. I think this article is an early attempt at creating a politically viable counterpoint.
- PaulStatezny 3y ago
  This reminds me of when I attended the first official conference for Facebook's popular React.js library. (2014 I believe?) Got to hang out with some "hotshot" Facebook developers after hours. I remember Lee Byron (of React, GraphQL, etc) stating very confidently that the roads would be teeming with self driving cars by 2020.
- sp332 3y ago
  When AIs kill people, it is not because they are intelligent, but because they are given lethal powers without appropriate judgement. The first autonomous weapons platform with a kill count was an accident. https://www.wired.com/2007/10/robot-cannon-ki/ https://www.wired.com/2007/10/robot-cannon-ki/ Self-driving cars kill people all the time. These things happen because the AIs are not super-intelligent, but people keep giving them too much power.
  - cypress66 3y ago
    > Self-driving cars kill people all the time. Maybe L2 systems. But those are just a driver assistance, the driver must supervise and intervene. Actual L4/L5 systems haven't killed anyone as far as I know, except for the Uber accident, and even then the driver was supposed to be supervising it, I'm not sure if it counts as L4/L5
- famouswaffles 3y ago
  What testable definition of general intelligence does GPT-4 fail that a big chunk of humans don't also fail?
  - SirMaster 3y ago
    Self reflection? Emotional intelligence? Empathy?
    - famouswaffles 3y ago
      I don't know how we'd go about comparing reflection to human baselines but GPT-4 can reflect just fine https://arxiv.org/abs/2303.11366 https://arxiv.org/abs/2303.11366 GPT-4's emotional intelligence seems to be very high by the looks of things https://arxiv.org/abs/2304.11490 https://arxiv.org/abs/2304.11490 Empathy =/ Intelligence
      - SirMaster 3y ago
        I really mean self-awareness and consciousness. Isn’t that a metric we use to determine the intelligence of animals and such? Does GPT love its creators the way we love our parents? Does it mourn a loss the way we do? Elephants are known to be of the more intelligent animals and apparently they mourn loss as well. So apparently there is something there linked with intelligence. Maybe it's not directly intelligence related, but what makes a human have a spontaneous emotional response like laughter for instance? The self-reflection link you sent isn't what I was thinking. I was thinking about things like self reflection on my existence. Like am I happy doing what I'm doing? Do I feel I am making a positive difference in the world? If not what should I start doing to work toward becoming what I want to be? The fact that I want to be something seems different. Nobody tells me what I want to be. But we still need to tell computers an awful lot about what they should be and what they should want to be. When will AI be able to decide for itself what it wants to be and what it wants to do? And to do all that sort of self reflection on it's own without us specifying a reward system that "makes" it want to do that? I don't strive to become a better in the world person for some reward.
- ben_w 3y ago
  > You know that these LLMs are not actually intelligent, right? Right??? Can a submarine swim? Does it matter if an LLM — which is far from the only kind of AI, and trivial to include in another more complex AI — does or doesn't match your definition of "intelligent" when it can write fluently in any language (human and code), and also getting near top-of-class results in the Bar exam and the biology olympiad? Corporations — sometimes used as another example of an unaligned optimising agent — can't drive cars, they have to hire humans to do that for them, but corporations can still be prosecuted for dangerous chemicals (3M); and AI, just like other computer programs, can have dangerous output that you should not rely on (e.g. Thule early warning radar incident where someone forgot to tell it that it's OK for the moon to not respond to IFF pings and it nearly started WW3).
  - hoxtrader 3y ago
    [dead]
kramerger 3y ago
From ealier today: https://news.ycombinator.com/item?id=36674930 https://news.ycombinator.com/item?id=36674930
zeryx 3y ago
Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon. 2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong. I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day. I am happy politicians and think-tanks are taking this seriously, you should be too.
- waihtis 3y ago
  Can you provide some tangible examples of existential risk materializing?
  - deleted 3y ago
    [deleted]
  - zeryx 3y ago
    Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today). Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world. I foresee a depression/suicide spiral never before seen in our history unless we do something radical
    - tlb 3y ago
      Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.
    - waihtis 3y ago
      I disagree with the last part in your scenario. It's going to be waves of murder and anarchy, not depression and suicide
deleted 3y ago
[deleted]
300bps 3y ago
The next big trend in AI is it dying the death of a thousand cuts in the form of copyright infringement lawsuits. Example: https://www.theverge.com/2023/7/9/23788741/sarah-silverman-openai-meta-chatgpt-llama-copyright-infringement-chatbots-artificial-intelligence-ai https://www.theverge.com/2023/7/9/23788741/sarah-silverman-o... Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it.
- ausbah 3y ago
  I mean they already do that right now.
  - 300bps 3y ago
    Sure. "Easier to say sorry than please". Until you get sued into bankruptcy by millions of litigants.
    - stale2002 3y ago
      Or, instead of that, everyone gets away with it, under broad "transformative/fair use" protections and nobody gets sued. That's what's happening now. But sure, people will always make these unfalsifiable arguments about some hypothetical doomsday that is far enough in the future that they can never get called out when it doesn't arrive.
- api 3y ago
  > Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it. That's one of the main business models of the Internet, only usually it's not even scraped. People upload it willingly in exchange for free hosting and free social media. The other big business model of the Internet is mass surveillance driven targeted advertising.
  - 300bps 3y ago
    It's hard to guess who you're talking about but that is not what Google does. They scrape public web servers, respecting robots.txt directives and then they provide links back to the original site. And if someone hosts user-submitted copyrighted information without a license, the copyright holder can submit a complaint via DCMA and the hoster has safe harbor from liability. OpenAI is literally scraping copyrighted works, packing it up and selling it without a license. Art, books, magazines, everything. No safe harbor from DMCA for doing that.
waihtis 3y ago
Whether or not the existential risk around AI is real, governments, corporations, NGO's and others will twist this to their own purpose to build censorship, entry barriers, and other unpleasantries. We should be mindful of that.
- wil421 3y ago
  What I said is 100% incorrect.
  - wskinner 3y ago
    Are you confusing dang with Sam Altman?
  - ru552 3y ago
    Pretty sure it was Sam that went to Congress to get a moat built.
  - mdp2021 3y ago
    That - that somebody would be at the same time an HN admin and an OpenAI founder - is a statement (a remarkably extraordinary statement) that requires some links to show it.
    - jimwhite42 3y ago
      It's either a simple mistake, or a deliberate troll. Sam Altman used to do a role on Hacker News very similar to what dang does today.
  - deleted 3y ago
    [deleted]
- jimwhite42 3y ago
  Is it too cynical to claim the the noise about existential risks is a convenient distraction from these unquestionably real issues we will face, maybe even a deliberate part of helping it happen? Here's a random bit of fun prescience: From Dune, by Frank Herbert: > JIHAD, BUTLERIAN: (see also Great Revolt) — the crusade against computers, thinking machines, and conscious robots begun in 201 B.G. and concluded in 108 B.G. Its chief commandment remains in the O.C. Bible as “Thou shalt not make a machine in the likeness of a human mind.” The usual interpretation is something along the lines that AIs took over the thinking for the leaders, making the leaders lazy, impotent, and society fragile and chaotic. This fits with a lot of Herbert's themes. But, from Children of Dune: > Three — Planetary feudalism remained in constant danger from a large technical class, but the effects of the Butlerian Jihad continued as a damper on technological excesses. Ixians, Tleilaxu, and a few scattered outer planets were the only possible threat in this regard, and they were planet-vulnerable to the combined wrath of the rest of the Imperium. The Butlerian Jihad would not be undone. Mechanized warfare required a large technical class. The Atreides Imperium had channeled this force into other pursuits. No large technical class existed unwatched. And the Empire remained safely feudalist, naturally, since that was the best social form for spreading over widely dispersed wild frontiers — new planets. Arguable paints a different picture - that the AI in the hands of the masses would be too threatening to the status quo.
valine 3y ago
The author seems to be entirely unaware of the current crop of open source / public language models like LLaMA and Falcon. People are already engaging in the behavior that, according to the article, might present an existential threat to humanity. The existential threat argument is a ridiculous notion to begin with. Pretending open source LLMs don’t already exist makes the argument even more silly.
- AndrewKemendo 3y ago
  The author Jeremy Howard, is very well aware of the state of the LLMs in the market and this is precisely WHY he wrote this: https://twitter.com/jeremyphoward/status/1678558165712113664 https://twitter.com/jeremyphoward/status/1678558165712113664 Whether it's a convincing argument is different question, but it's not for lack of knowledge about the state of art in AI.
  - valine 3y ago
    Seems strange to me not to mention LLaMA once in the article then. Why tiptoe around the subject?
    - throwaway1777 3y ago
      Probably not wanting to give them airtime. Some kind of Streisand effect idea.
      - valine 3y ago
        It’s intellectually dishonest in my opinion.
        deleted 3y ago
        [deleted]
        posterboy 3y ago
        An inadvertent slip, the worry is being left behind in the competition.
tyre 3y ago
I'm empathetic to the desire for innovative freedom and democratization of technology, but the benefit has to be balanced with the scale of impact and the possibility of mitigating/reparative response. The capabilities of current generation AI—LLMs + audio/image/video generative models—are far enough already that wide-scale distribution is extremely dangerous. Social media allowed for broadcasting from 1:many and 1:1 with troll armies, but LLMs are on a whole other planet of misinformation. A scalable natural language interface can target people at the one-to-one conversation level and generate misinformation ad hoc across text, voice, images, and video. The trouble with this approach is that harm will rack up way faster than liability can catch up, with irreversible and sometimes immeasurable long term effects. We have yet to contend with the downstream effects of social media and engagement hacking, for example, and suing Facebook out of existence wouldn't put a dent in it. Current generation AI has enough capabilities to swing the 2024 US presidential election. It can be used today for ransom pleas trained on your childrens' instagram posts. It doesn't seem farfetched that AI could start a war. Not because it's SkyNet, but because we've put together the tools to influence already-powerful-enough human forces into civilization-level catastrophe.
- nawgz 3y ago
  Seems like a ridiculous standard to use US elections. Arguably Facebook showed the ability to do the same in 2016, so it’s also uninteresting. The only reason an LLM would even have such access to people to do the hyperbolic damage you claim is thru ad networks and social media anyways - yet you are calling to behead public access to tech instead of regulating propaganda platforms. Both seem facile but your selection is the one that seems subservient
  - ben_w 3y ago
    Hmm. I wasn't convinced, but I did the maths and, well, looks like you're right: Hiring people in to read and respond in a politically biased way to each person in the USA would take, let's say 2 messages per day for a month, so 60 messages per person, let's say 100 million actual voters you care about so 6 billion messages, let's say 15 seconds per response so 25 million hours, let's say you're hiring Kenyans because I happen to have the GDP/capita and workforce participation rate to hand, so it's about 2 USD/hour on average, so 50 million USD. With GPT-4, and let's approximate each message as 100 input tokens and 100 output tokens, that's 0.1 * ($0.03 + $0.06) = $0.009, times the same 6e9 messages = 54 million USD. So, yeah, looks like you're right. 3.5 is cheaper, but tends to be noticeable; from what I've been hearing (fast moving target and all that) the OSS LLMs tend to be 90% of the quality of 3.5.
arisAlexis 3y ago
"The reason this distinction is critical is because these models are, in fact, nothing but mathematical functions. They take as input a bunch of numbers, and calculate and return a different bunch of numbers. They don’t do anything themselves — they can only calculate numbers." This is entirely false, sounds like Andersen that seems incapable of understanding that agency and goals will be a thing.
- tsunamifury 3y ago
  This is almost a textbook example of “reductionism ad absurdum”
- deleted 3y ago
  [deleted]
- cubefox 3y ago
  Yeah. He seems to assume int and float are the only possible data types. But computers themselves have nothing to do with numbers in particular, they manipulate bits, and those can be used for far more than representing numbers.
- JieJie 3y ago
  Yeah, the "it's just math" argument isn't really convincing anymore. Previous to this, math never had opinions about Taylor Swift songs.
  - CharlesW 3y ago
    > Previous to this, math never had opinions about Taylor Swift songs. Nor do they now, of course.
    - arisAlexis 3y ago
      Actually they do. You need to make a philosophical argument as how this different than us humans manipulating bits and neurons to come up with opinions. Good luck with that.
arisAlexis 3y ago
"What does it look like to embrace the belief in progress and the rationality of all humans when we respond to the threat of AI mis-use?" This is more utopian than communism even. "There will still be Bad Guys looking to use them to hurt others or unjustly enrich themselves. But most people are not Bad Guys" Failure to understand that even one by guy can kill everyone with a sufficiently advanced AGI
earthboundkid 3y ago
"We made a new kind of hammer. We're worried that bad people might use the hammer to build a house, but we're even more worried that the hammer might build a hammer factory and bash all our brains in." Okay, GLHF.
cubefox 3y ago
Here is different article arguing for the same thesis, i.e. that AI regulation increases AI risk, but IMHO with better arguments: https://www.lesswrong.com/posts/6untaSPpsocmkS7Z3/ways-i-expect-ai-regulation-to-increase-extinction-risk https://www.lesswrong.com/posts/6untaSPpsocmkS7Z3/ways-i-exp...
- tedivm 3y ago
  I don't think the thesis of this piece is that "AI regulation increases AI risk". In fact it explicitly talks about increasing the regulation on the usage of AI, rather than the development of it.
  - cubefox 3y ago
    Quote: > Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways
    - tedivm 3y ago
      "AI regulation increases AI risk" is not the same as "Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways". The first refers to all regulation, while the second is talking about a specific attempt to regulate. This is like saying "I don't like beef" and "we should ban all meat" are the same statements. They're clearly not, the scope is vastly different.
      - cubefox 3y ago
        Okay, but the LessWrong article clearly goes in a similar direction.
changeslitl 3y ago
[dead]
FrustratedMonky 3y ago
Netflix documentaries aren't always the most 'academic', or 'robust'. But the one that was just released about AI for autonomous drones, is pretty shocking. The military side is farther along than I thought, AND, you know what is appearing in a Netflix documentary is old, it isn't the latest things that are still secret. In any case, why it relates to this article. We are in a race with other nation states, so all of this discussion about 'curtailing' or limiting AI, is all just grand standing, smoke screen. Nothing will slow down development because other countries are also developing AI, and everyone firmly believes they have to be first or will be wiped out. Hence no one will slow down.
- GTP 3y ago
  Or, we try to set common rules before the thing goes out of hand. We already have UN rules about fire weapons, and your argument applies to those as well. But just as we have widely agreed upon rules for weapons, if there's the chance of AI being used used for war (which indeed seems to be the case, see e.g. UAVs) then we should regulate it as well. This leaves out civil uses of AI, but as we have safety regulations for every other technology, I don't see why AI should be an exception. Then we can discuss which regulations make sense and which not, but this is a different topic. PS: I don't thik AI will pose an existential risk to our species anytime soon, if ever. But it can already cause damages if used improperly, like any other technology. So it has to be regulated, like any other technology.
  - FrustratedMonky 3y ago
    On one hand, I agree. Nuclear weapons are a risk to everyone, so the world(US, Russia) did come together in various treaties to limit production, development, etc... So, this could apply to AI also, everyone sees the risk and comes together to curb their us and development. On other hand: Nukes are very difficult to make. Iran and North Korea do not abide by the UN resolutions, they are continuing trying to develop them. We are just lucky that is not so easy. With AI, the barrier to entry is much lower. You can find things that can be weaponized for free on github. It isn't like they are going to bother reading the licenses and not use them. And for compute power, sure, they have harder time getting the best latest CPU's, and there is an embargo. But still, I think, far easier to get illegal gotten CPU's, than developing Nukes, the barrier is still lower.
guy98238710 3y ago
> Artificial Intelligence is moving fast We wish. AI is hardware-limited and hardware is not moving fast. We are very, very far from matching raw compute power of the human brain. Robots are even more limited compared to human body.
- waffletower 3y ago
  "We are very, very far from matching raw compute power of the human brain." This has been said for decades with viral conviction. While not equivalent -- we don't understand how these numbers relate -- the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. At some point we might have to re-evaluate the value of this dogmatic comparison between the human brain and existing AI systems.
  - GTP 3y ago
    > the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. But that's not an apples-to-apples comparison, as the artificial neurons of neural networks are just a rough approximation of our neurons. They are also connected in a much simpler way too (nicely divided into layers, where each layer can only pass signals to the very next layer), so it could be that it isn't just a matter of how many parameters you have.
    - FrustratedMonky 3y ago
      And, it isn't just number of neurons, it is connections between neurons. I think brain still outnumbers on a count of connections (sorry, don't have source handy).
    - waffletower 3y ago
      You didn't read what I said. Did I need to use a specific fruit metaphor so you would understand that I claimed they weren't directly comparable?
      - 3y ago
stOneskull 3y ago
elon musk said he gave 50 million dollars to get OpenAI started, wanting it to be non-profit and open-source, and they kinda betrayed him. i wonder if he would consider doing it again but making sure the organization stayed open-source and non-profit.
waffletower 3y ago
"Existential risk" truly is a misnomer at this time. Please use the better fitting: "existential fear". AI "risk" could instead be applied to global information infrastructure, a collapse of the internet by rogue AI bots etc. There are too many technological challenges for any AI to overcome in robotics, mobile energy, material and bio printing/fabrication, and supply chain dominance for an imminent existential risk to humanity to be a credible near-term threat. LLM performance is a gift of data availability of the internet. There aren't equivalent datasets available for the challenges articulated above. How likely can they materialize, particularly by machines that do not yet have a tactile interface and understanding of the physical universe? Perhaps a malevolent machine sentience is possible at some time in the future-- but there are many undeveloped technologies required to pose a threat to human existence.
- waffletower 3y ago
  Downvoting without providing meaningful dialogue is a cowardly privilege.
  - deleted 3y ago
    [deleted]
- highstep 3y ago
  i think you might be focusing too much on ai risk presented in fantasy (killer robots), meanwhile we can already clearly see how LLMs negatively impact society via disruption or popular opinion, politics (recommendation algorithms), and rapid uncontrolled scientific discovery. Such disruptions could potentially result in nuclear war, human created plagues, etc. You might be getting down voted without comment because you're message comes across like cheerleading that only examines one future distant risk. Your framing as "extential fear" is particularly dismissive and doesn't seem to be in good faith for such a serious and subject.
  - c_crank 3y ago
    It's hard to take doomers seriously when they think that things that exist and are clearly not causing nuclear war or plagues are only a few steps from doing so. If you want to talk seriously, talk about actual problems AI is causing right now, like job loss or malpractice.
waffletower 3y ago
"Historically, large power differentials have led to violence and subservience of whole societies." I agree, the calls for regulation by large corporate entities who quite transparently wish to control AI development for themselves, is a much greater near-term "existential threat" than AI itself.
phillipcarter 3y ago
IMO this paragraph in the summary is the most important one: > There are interventions we can make now, including the regulation of “high-risk applications” proposed in the EU AI Act. By regulating applications we focus on real harms and can make those most responsible directly liable. Another useful approach in the AI Act is to regulate disclosure, to ensure that those using models have the information they need to use them appropriately.
guy98238710 3y ago
I don't get this scare about AI. Imagine, you are a genius with IQ 200. You have studied hard your entire life and you are now an expert in several fields. One day you decide to conquer the world. Will you succeed or not?
- dwaltrip 3y ago
  On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control. Assuming you have 200 IQ, if you could make thousands or millions of clones of yourself quite cheaply, you still might not succeed in taking over the world, but it is no longer a laughable idea. Overall, I’m somewhat ambivalent about the possibility of x-risk, especially if we work reasonably hard to prevent it. But it shouldn’t be ignored. AI is moving very quickly and it is unclear how powerful its capabilities will be in the next 10-20 years. Of course, there are many other risks presented by AI that we need to stay on top of as well.
  - guy98238710 3y ago
    > On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control. Copying AIs just results in effectively higher intelligence (bigger brain). Things would be different with self-replicating robots of course, but duplicating robots is not that fast and one could argue that the robots are dangerous rather than the AI controlling them, because they would be dangerous even if controlled by stupid non-AI program.
    - dwaltrip 3y ago
      Copying itself means it can do more “work” in the same amount of time. Earning money, strategizing, researching, smooth-talking humans, playing politics, and so on. Potentially even improving itself. These activities could give it power and influence.
      - guy98238710 3y ago
Tenoke 3y ago
>As we’ll see, if AI turns out to be powerful enough to be a catastrophic threat, the proposal may not actually help. In fact it could make things much worse, by creating a power imbalance.. Power imbalance is not 'much worse' than extinction. It's very clear how people start with the conclusion - "openess and progress have been good and will keep being the best choice" and then just contort their thinking and arguments to match the conclusion without really critically examining the counter arguments.

124 points by wskinner 3y ago

It is wild how seriously people are taking the development of these LLMs. There is talk of them leading to the extinction of life while neglecting to consider that we can't even build self-driving cars, let alone have a robot clear the table after dinner. Also, many people seem to be mistaking "artificial intelligence" for "actual intelligence". You know that these LLMs are not actually intelligent, right? Right???

CamperBob2 3y ago
"Intelligence" is a word we made up. It doesn't mean much of anything at the end of the day. If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent." Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. Assuming that this is not the case, don't hold machines to standards you're not willing to hold humans to.
- throw47474777j 3y ago
  “X is a word we made up. It doesn't mean much of anything at the end of the day.” This has to be a candidate for the most unhelpful statement possible. If you are sincere about it, then why not apply it to every word in your reply and see where that leaves you?
  - SpicyLemonZest 3y ago
    I can't speak for the original commenter, but I would definitely say that "is this actually X" is a pointless debate for most values of X. If I understand what people mean by X, there's no need to quibble about whether it satisfies my personal definition; if I don't understand, I should try to clarify rather than just saying "it's not really X" and crossing my arms.
- bumby 3y ago
  I don't want to speak for the OP, but one of the issues they may be poking at is the transference of intelligence. E.g., you are "trained" to play a game, but at some point someone decides to change the rules of the game. Human children can transfer their previous knowledge to the new rule set fairly well. The most intelligent children can do so very quickly without supplemental "training" on the game played under the new rules. Humans certainly have flaws when it comes to this. I've heard some discussions about the success and failures of AI in this regard. Can someone in this domain elaborate on the current state-of-the-art performance in this regard?
  - famouswaffles 3y ago
    1. LLMs are general pattern machines. They can generalize and complete in context to a wide variety of complex non linguistic patterns.https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ 2. LLMs see positive transfer in multi lingual capability. For example, an LLM trained on 500B tokens of English and 10B tokens of French will not speak in french like a model trained on only 10B tokens of French. What will happen is that the model will be nearly as competent in French as it is in English https://arxiv.org/abs/2108.13349 https://arxiv.org/abs/2108.13349 3. Language models of code reason better even if the benchmarks have nothing to do with code. https://arxiv.org/abs/2210.07128 https://arxiv.org/abs/2210.07128
    - bumby 3y ago
      This is interesting in the context of the other response that links to poor performance in terms of counterfactuals. I wonder if it is related to how well one domain maps to another? E.g., can they transfer to english to french well because both share a similar word classes (nouns, verbs, etc.). But I believe other languages change more based on social context (e.g., Japanese) than English does. Would a LLM transfer just as well to the latter? In that case, my guess is humans would find it more difficult to transfer as well, so it may not be a good measure. (Apologies to any linguists. Please correct anything above if I'm off).
      - famouswaffles 3y ago
- goatlover 3y ago
  "All words are made up." - Thor The word was made up to cover a range of cognitive abilities that humans and animals (to varying degrees) possess. And we're gradually figuring out how to design machines with similar abilities. The general idea of intelligence being you can figure out how to do things that aren't just instinctual. a generalized intelligence can do this across any number of domains without some fixed limit.
- AlexandrB 3y ago
  > Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. So you're saying humans, collectively, are gods? After all, as a species we started with nothing - no "clean" training data, no paintings to replicate, not even language itself. And here we are - arguing about whether one creation of ours, trained on a bunch of other stuff we created, is as intelligent as us.
  - famouswaffles 3y ago
    We didn't continue with nothing. Evolution is training. LLMs "start with nothing" too
- prhn 3y ago
  > "Intelligence" is a word we made up So is all human knowledge. They're all things we "made up" to describe the world around us. That's a really cheap way to reduce something. There's a mountain of psychological literature that intends to describe and measure intelligence. Saying it doesn't mean much of anything is a bit of a postmodern hot take. > If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent." Painting pictures and composing poems aren't good ways to measure general intelligence. That's creativity. They're correlated but not the same. There are specific tests for measuring general intelligence, and there's absolutely no way current gen LLM would score highly on them because they don't even accept visual input. Yes I've seen that they can score highly on verbal-only tests. That's obvious anyway because they have seen, written down, and have access to the answers during the test. Even if they did accept visual input they would only be able to solve problems for which they've previously been given the answers (trained on the dataset). Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec.
  - CamperBob2 3y ago
    Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec. One problem is that ChatGPT4 has been degrading over time -- and no, I don't GAF about any opinions or assurances to the contrary. So we're seeing more and more people look at it for the first time and ask what the big deal is. Maybe the Code Interpreter feature will reverse that trend, we'll see. Those of us who were around (read: paying $20/month) when they first enabled GPT4 support are left waving our hands fecklessly, muttering "Yeah, but you should've seen it back in the old days, back in March of '23. Now you kids get off my lawn."
- Keegs 3y ago
  > If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. Poetry is inspired in a way that’s hard to replicate with a prompt. Artists also take inspiration from other artists, but the “distance” between their influences and their work is so much greater than the distance between an LLM’s that I don’t fault anyone who think it’ll take time for LLMs to catch up.
  - CamperBob2 3y ago
    An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. And you don't grasp the fundamentally-incremental nature of this point?
    - jononomo 3y ago
      No. After you've asked ChatGPT to write 5-6 poems you'll get a sense for how profoundly uncreative it is as a poet. It's not going to "incrementally" be anything other than a dead-as-a-doornail algorithm.
      - Aeolos 3y ago
        That's ChatGPT version 3. Version 4 is already significantly better. What about version 5? 6? 10?
- mdp2021 3y ago
  > to be called "intelligent" quality is required.
disadvantage 3y ago
> we can't even build self-driving cars, let alone have a robot clear the table after dinner I think once you give AI working limbs and the ability to manipulate matter like humans, that's concerning, because if the AI becomes a runaway AI with no upper limits on the types of behavior allowed, we get into Paperclip Maximizer[0] territory very fast. [0] https://tanzanite.ai/paperclip-maximizer-experiment/ https://tanzanite.ai/paperclip-maximizer-experiment/
gnulinux 3y ago
I'm completely with you on this one. Sci-fi and AI hype are extremely strong on these threads. I bet we're not any closer to human level intelligence with ML than we were in 1500s with mechanical clocks. Both are impressive nonetheless.
thiago_fm 3y ago
Just as in finance everyday you see a hedge fund manager making some weird big bet of a stock or market for the next 20 years (like a prophet). Everyone wants to pose as the front(wo)man of that innovation, thought-leaders. Whatever. It's all marketing, for their own careers, their own personal agenda. The whole text looks like written by ChatGPT in a prompt like "Write about AI safety and relates it to the Age of Reason". This is a good reason why AI should replace us, so we no longer have human beings writing that stuff to self-promote themselves. The funniest part is in the end of the article: "Eric Ries has been my close collaborator throughout the development of this article" So the guy who wrote "The lean startup" is helping him with that article. drops mic
marvin 3y ago
A pretty influential and very well-financed subculture of AI philosophers are currently performing a comprehensive lobbying campaign for AI regulation. They generally go by Rationalists or Effective Altruism, and have a philosophy stemming from MIRI, Eliezer Yudkowsky, Nate Soares and others. They've had considerable success getting traction for these views with politicians in Europe, and that's the reason you see so much of this in the media currently. I'm hoping more sane voices will soon get organized, because significant parts of this are reminiscent of a well-funded doomsday cult. I think this article is an early attempt at creating a politically viable counterpoint.
PaulStatezny 3y ago
This reminds me of when I attended the first official conference for Facebook's popular React.js library. (2014 I believe?) Got to hang out with some "hotshot" Facebook developers after hours. I remember Lee Byron (of React, GraphQL, etc) stating very confidently that the roads would be teeming with self driving cars by 2020.
sp332 3y ago
When AIs kill people, it is not because they are intelligent, but because they are given lethal powers without appropriate judgement. The first autonomous weapons platform with a kill count was an accident. https://www.wired.com/2007/10/robot-cannon-ki/ https://www.wired.com/2007/10/robot-cannon-ki/ Self-driving cars kill people all the time. These things happen because the AIs are not super-intelligent, but people keep giving them too much power.
- cypress66 3y ago
  > Self-driving cars kill people all the time. Maybe L2 systems. But those are just a driver assistance, the driver must supervise and intervene. Actual L4/L5 systems haven't killed anyone as far as I know, except for the Uber accident, and even then the driver was supposed to be supervising it, I'm not sure if it counts as L4/L5
famouswaffles 3y ago
What testable definition of general intelligence does GPT-4 fail that a big chunk of humans don't also fail?
- SirMaster 3y ago
  Self reflection? Emotional intelligence? Empathy?
  - famouswaffles 3y ago
    I don't know how we'd go about comparing reflection to human baselines but GPT-4 can reflect just fine https://arxiv.org/abs/2303.11366 https://arxiv.org/abs/2303.11366 GPT-4's emotional intelligence seems to be very high by the looks of things https://arxiv.org/abs/2304.11490 https://arxiv.org/abs/2304.11490 Empathy =/ Intelligence
    - SirMaster 3y ago
      I really mean self-awareness and consciousness. Isn’t that a metric we use to determine the intelligence of animals and such? Does GPT love its creators the way we love our parents? Does it mourn a loss the way we do? Elephants are known to be of the more intelligent animals and apparently they mourn loss as well. So apparently there is something there linked with intelligence. Maybe it's not directly intelligence related, but what makes a human have a spontaneous emotional response like laughter for instance? The self-reflection link you sent isn't what I was thinking. I was thinking about things like self reflection on my existence. Like am I happy doing what I'm doing? Do I feel I am making a positive difference in the world? If not what should I start doing to work toward becoming what I want to be? The fact that I want to be something seems different. Nobody tells me what I want to be. But we still need to tell computers an awful lot about what they should be and what they should want to be. When will AI be able to decide for itself what it wants to be and what it wants to do? And to do all that sort of self reflection on it's own without us specifying a reward system that "makes" it want to do that? I don't strive to become a better in the world person for some reward.
ben_w 3y ago
> You know that these LLMs are not actually intelligent, right? Right??? Can a submarine swim? Does it matter if an LLM — which is far from the only kind of AI, and trivial to include in another more complex AI — does or doesn't match your definition of "intelligent" when it can write fluently in any language (human and code), and also getting near top-of-class results in the Bar exam and the biology olympiad? Corporations — sometimes used as another example of an unaligned optimising agent — can't drive cars, they have to hire humans to do that for them, but corporations can still be prosecuted for dangerous chemicals (3M); and AI, just like other computer programs, can have dangerous output that you should not rely on (e.g. Thule early warning radar incident where someone forgot to tell it that it's OK for the moon to not respond to IFF pings and it nearly started WW3).
- hoxtrader 3y ago
  [dead]

kramerger 3y ago

From ealier today: https://news.ycombinator.com/item?id=36674930 https://news.ycombinator.com/item?id=36674930

zeryx 3y ago

Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon. 2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong. I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day. I am happy politicians and think-tanks are taking this seriously, you should be too.

waihtis 3y ago
Can you provide some tangible examples of existential risk materializing?
- deleted 3y ago
  [deleted]
- zeryx 3y ago
  Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today). Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world. I foresee a depression/suicide spiral never before seen in our history unless we do something radical
  - tlb 3y ago
    Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.
  - waihtis 3y ago
    I disagree with the last part in your scenario. It's going to be waves of murder and anarchy, not depression and suicide

deleted 3y ago

[deleted]

300bps 3y ago

The next big trend in AI is it dying the death of a thousand cuts in the form of copyright infringement lawsuits. Example: https://www.theverge.com/2023/7/9/23788741/sarah-silverman-openai-meta-chatgpt-llama-copyright-infringement-chatbots-artificial-intelligence-ai https://www.theverge.com/2023/7/9/23788741/sarah-silverman-o... Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it.

ausbah 3y ago
I mean they already do that right now.
- 300bps 3y ago
  Sure. "Easier to say sorry than please". Until you get sued into bankruptcy by millions of litigants.
  - stale2002 3y ago
    Or, instead of that, everyone gets away with it, under broad "transformative/fair use" protections and nobody gets sued. That's what's happening now. But sure, people will always make these unfalsifiable arguments about some hypothetical doomsday that is far enough in the future that they can never get called out when it doesn't arrive.
api 3y ago
> Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it. That's one of the main business models of the Internet, only usually it's not even scraped. People upload it willingly in exchange for free hosting and free social media. The other big business model of the Internet is mass surveillance driven targeted advertising.
- 300bps 3y ago
  It's hard to guess who you're talking about but that is not what Google does. They scrape public web servers, respecting robots.txt directives and then they provide links back to the original site. And if someone hosts user-submitted copyrighted information without a license, the copyright holder can submit a complaint via DCMA and the hoster has safe harbor from liability. OpenAI is literally scraping copyrighted works, packing it up and selling it without a license. Art, books, magazines, everything. No safe harbor from DMCA for doing that.

waihtis 3y ago

Whether or not the existential risk around AI is real, governments, corporations, NGO's and others will twist this to their own purpose to build censorship, entry barriers, and other unpleasantries. We should be mindful of that.

wil421 3y ago
What I said is 100% incorrect.
- wskinner 3y ago
  Are you confusing dang with Sam Altman?
- ru552 3y ago
  Pretty sure it was Sam that went to Congress to get a moat built.
- mdp2021 3y ago
  That - that somebody would be at the same time an HN admin and an OpenAI founder - is a statement (a remarkably extraordinary statement) that requires some links to show it.
  - jimwhite42 3y ago
    It's either a simple mistake, or a deliberate troll. Sam Altman used to do a role on Hacker News very similar to what dang does today.
- deleted 3y ago
  [deleted]
jimwhite42 3y ago
Is it too cynical to claim the the noise about existential risks is a convenient distraction from these unquestionably real issues we will face, maybe even a deliberate part of helping it happen? Here's a random bit of fun prescience: From Dune, by Frank Herbert: > JIHAD, BUTLERIAN: (see also Great Revolt) — the crusade against computers, thinking machines, and conscious robots begun in 201 B.G. and concluded in 108 B.G. Its chief commandment remains in the O.C. Bible as “Thou shalt not make a machine in the likeness of a human mind.” The usual interpretation is something along the lines that AIs took over the thinking for the leaders, making the leaders lazy, impotent, and society fragile and chaotic. This fits with a lot of Herbert's themes. But, from Children of Dune: > Three — Planetary feudalism remained in constant danger from a large technical class, but the effects of the Butlerian Jihad continued as a damper on technological excesses. Ixians, Tleilaxu, and a few scattered outer planets were the only possible threat in this regard, and they were planet-vulnerable to the combined wrath of the rest of the Imperium. The Butlerian Jihad would not be undone. Mechanized warfare required a large technical class. The Atreides Imperium had channeled this force into other pursuits. No large technical class existed unwatched. And the Empire remained safely feudalist, naturally, since that was the best social form for spreading over widely dispersed wild frontiers — new planets. Arguable paints a different picture - that the AI in the hands of the masses would be too threatening to the status quo.

valine 3y ago

The author seems to be entirely unaware of the current crop of open source / public language models like LLaMA and Falcon. People are already engaging in the behavior that, according to the article, might present an existential threat to humanity. The existential threat argument is a ridiculous notion to begin with. Pretending open source LLMs don’t already exist makes the argument even more silly.

AndrewKemendo 3y ago
The author Jeremy Howard, is very well aware of the state of the LLMs in the market and this is precisely WHY he wrote this: https://twitter.com/jeremyphoward/status/1678558165712113664 https://twitter.com/jeremyphoward/status/1678558165712113664 Whether it's a convincing argument is different question, but it's not for lack of knowledge about the state of art in AI.
- valine 3y ago
  Seems strange to me not to mention LLaMA once in the article then. Why tiptoe around the subject?
  - throwaway1777 3y ago
    Probably not wanting to give them airtime. Some kind of Streisand effect idea.
    - valine 3y ago
      It’s intellectually dishonest in my opinion.
      - deleted 3y ago
        [deleted]
      - posterboy 3y ago
        An inadvertent slip, the worry is being left behind in the competition.

tyre 3y ago

I'm empathetic to the desire for innovative freedom and democratization of technology, but the benefit has to be balanced with the scale of impact and the possibility of mitigating/reparative response. The capabilities of current generation AI—LLMs + audio/image/video generative models—are far enough already that wide-scale distribution is extremely dangerous. Social media allowed for broadcasting from 1:many and 1:1 with troll armies, but LLMs are on a whole other planet of misinformation. A scalable natural language interface can target people at the one-to-one conversation level and generate misinformation ad hoc across text, voice, images, and video. The trouble with this approach is that harm will rack up way faster than liability can catch up, with irreversible and sometimes immeasurable long term effects. We have yet to contend with the downstream effects of social media and engagement hacking, for example, and suing Facebook out of existence wouldn't put a dent in it. Current generation AI has enough capabilities to swing the 2024 US presidential election. It can be used today for ransom pleas trained on your childrens' instagram posts. It doesn't seem farfetched that AI could start a war. Not because it's SkyNet, but because we've put together the tools to influence already-powerful-enough human forces into civilization-level catastrophe.

nawgz 3y ago
Seems like a ridiculous standard to use US elections. Arguably Facebook showed the ability to do the same in 2016, so it’s also uninteresting. The only reason an LLM would even have such access to people to do the hyperbolic damage you claim is thru ad networks and social media anyways - yet you are calling to behead public access to tech instead of regulating propaganda platforms. Both seem facile but your selection is the one that seems subservient
- ben_w 3y ago
  Hmm. I wasn't convinced, but I did the maths and, well, looks like you're right: Hiring people in to read and respond in a politically biased way to each person in the USA would take, let's say 2 messages per day for a month, so 60 messages per person, let's say 100 million actual voters you care about so 6 billion messages, let's say 15 seconds per response so 25 million hours, let's say you're hiring Kenyans because I happen to have the GDP/capita and workforce participation rate to hand, so it's about 2 USD/hour on average, so 50 million USD. With GPT-4, and let's approximate each message as 100 input tokens and 100 output tokens, that's 0.1 * ($0.03 + $0.06) = $0.009, times the same 6e9 messages = 54 million USD. So, yeah, looks like you're right. 3.5 is cheaper, but tends to be noticeable; from what I've been hearing (fast moving target and all that) the OSS LLMs tend to be 90% of the quality of 3.5.

arisAlexis 3y ago

"The reason this distinction is critical is because these models are, in fact, nothing but mathematical functions. They take as input a bunch of numbers, and calculate and return a different bunch of numbers. They don’t do anything themselves — they can only calculate numbers." This is entirely false, sounds like Andersen that seems incapable of understanding that agency and goals will be a thing.

tsunamifury 3y ago
This is almost a textbook example of “reductionism ad absurdum”
deleted 3y ago
[deleted]
cubefox 3y ago
Yeah. He seems to assume int and float are the only possible data types. But computers themselves have nothing to do with numbers in particular, they manipulate bits, and those can be used for far more than representing numbers.
JieJie 3y ago
Yeah, the "it's just math" argument isn't really convincing anymore. Previous to this, math never had opinions about Taylor Swift songs.
- CharlesW 3y ago
  > Previous to this, math never had opinions about Taylor Swift songs. Nor do they now, of course.
  - arisAlexis 3y ago
    Actually they do. You need to make a philosophical argument as how this different than us humans manipulating bits and neurons to come up with opinions. Good luck with that.

arisAlexis 3y ago

"What does it look like to embrace the belief in progress and the rationality of all humans when we respond to the threat of AI mis-use?" This is more utopian than communism even. "There will still be Bad Guys looking to use them to hurt others or unjustly enrich themselves. But most people are not Bad Guys" Failure to understand that even one by guy can kill everyone with a sufficiently advanced AGI

earthboundkid 3y ago

"We made a new kind of hammer. We're worried that bad people might use the hammer to build a house, but we're even more worried that the hammer might build a hammer factory and bash all our brains in." Okay, GLHF.

cubefox 3y ago

Here is different article arguing for the same thesis, i.e. that AI regulation increases AI risk, but IMHO with better arguments: https://www.lesswrong.com/posts/6untaSPpsocmkS7Z3/ways-i-expect-ai-regulation-to-increase-extinction-risk https://www.lesswrong.com/posts/6untaSPpsocmkS7Z3/ways-i-exp...

tedivm 3y ago
I don't think the thesis of this piece is that "AI regulation increases AI risk". In fact it explicitly talks about increasing the regulation on the usage of AI, rather than the development of it.
- cubefox 3y ago
  Quote: > Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways
  - tedivm 3y ago
    "AI regulation increases AI risk" is not the same as "Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways". The first refers to all regulation, while the second is talking about a specific attempt to regulate. This is like saying "I don't like beef" and "we should ban all meat" are the same statements. They're clearly not, the scope is vastly different.
    - cubefox 3y ago
      Okay, but the LessWrong article clearly goes in a similar direction.

changeslitl 3y ago

[dead]

FrustratedMonky 3y ago

Netflix documentaries aren't always the most 'academic', or 'robust'. But the one that was just released about AI for autonomous drones, is pretty shocking. The military side is farther along than I thought, AND, you know what is appearing in a Netflix documentary is old, it isn't the latest things that are still secret. In any case, why it relates to this article. We are in a race with other nation states, so all of this discussion about 'curtailing' or limiting AI, is all just grand standing, smoke screen. Nothing will slow down development because other countries are also developing AI, and everyone firmly believes they have to be first or will be wiped out. Hence no one will slow down.

GTP 3y ago
Or, we try to set common rules before the thing goes out of hand. We already have UN rules about fire weapons, and your argument applies to those as well. But just as we have widely agreed upon rules for weapons, if there's the chance of AI being used used for war (which indeed seems to be the case, see e.g. UAVs) then we should regulate it as well. This leaves out civil uses of AI, but as we have safety regulations for every other technology, I don't see why AI should be an exception. Then we can discuss which regulations make sense and which not, but this is a different topic. PS: I don't thik AI will pose an existential risk to our species anytime soon, if ever. But it can already cause damages if used improperly, like any other technology. So it has to be regulated, like any other technology.
- FrustratedMonky 3y ago
  On one hand, I agree. Nuclear weapons are a risk to everyone, so the world(US, Russia) did come together in various treaties to limit production, development, etc... So, this could apply to AI also, everyone sees the risk and comes together to curb their us and development. On other hand: Nukes are very difficult to make. Iran and North Korea do not abide by the UN resolutions, they are continuing trying to develop them. We are just lucky that is not so easy. With AI, the barrier to entry is much lower. You can find things that can be weaponized for free on github. It isn't like they are going to bother reading the licenses and not use them. And for compute power, sure, they have harder time getting the best latest CPU's, and there is an embargo. But still, I think, far easier to get illegal gotten CPU's, than developing Nukes, the barrier is still lower.

guy98238710 3y ago

> Artificial Intelligence is moving fast We wish. AI is hardware-limited and hardware is not moving fast. We are very, very far from matching raw compute power of the human brain. Robots are even more limited compared to human body.

waffletower 3y ago
"We are very, very far from matching raw compute power of the human brain." This has been said for decades with viral conviction. While not equivalent -- we don't understand how these numbers relate -- the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. At some point we might have to re-evaluate the value of this dogmatic comparison between the human brain and existing AI systems.
- GTP 3y ago
  > the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. But that's not an apples-to-apples comparison, as the artificial neurons of neural networks are just a rough approximation of our neurons. They are also connected in a much simpler way too (nicely divided into layers, where each layer can only pass signals to the very next layer), so it could be that it isn't just a matter of how many parameters you have.
  - FrustratedMonky 3y ago
    And, it isn't just number of neurons, it is connections between neurons. I think brain still outnumbers on a count of connections (sorry, don't have source handy).
  - waffletower 3y ago
    You didn't read what I said. Did I need to use a specific fruit metaphor so you would understand that I claimed they weren't directly comparable?
    - 3y ago

stOneskull 3y ago

elon musk said he gave 50 million dollars to get OpenAI started, wanting it to be non-profit and open-source, and they kinda betrayed him. i wonder if he would consider doing it again but making sure the organization stayed open-source and non-profit.

waffletower 3y ago

"Existential risk" truly is a misnomer at this time. Please use the better fitting: "existential fear". AI "risk" could instead be applied to global information infrastructure, a collapse of the internet by rogue AI bots etc. There are too many technological challenges for any AI to overcome in robotics, mobile energy, material and bio printing/fabrication, and supply chain dominance for an imminent existential risk to humanity to be a credible near-term threat. LLM performance is a gift of data availability of the internet. There aren't equivalent datasets available for the challenges articulated above. How likely can they materialize, particularly by machines that do not yet have a tactile interface and understanding of the physical universe? Perhaps a malevolent machine sentience is possible at some time in the future-- but there are many undeveloped technologies required to pose a threat to human existence.

waffletower 3y ago
Downvoting without providing meaningful dialogue is a cowardly privilege.
- deleted 3y ago
  [deleted]
highstep 3y ago
i think you might be focusing too much on ai risk presented in fantasy (killer robots), meanwhile we can already clearly see how LLMs negatively impact society via disruption or popular opinion, politics (recommendation algorithms), and rapid uncontrolled scientific discovery. Such disruptions could potentially result in nuclear war, human created plagues, etc. You might be getting down voted without comment because you're message comes across like cheerleading that only examines one future distant risk. Your framing as "extential fear" is particularly dismissive and doesn't seem to be in good faith for such a serious and subject.
- c_crank 3y ago
  It's hard to take doomers seriously when they think that things that exist and are clearly not causing nuclear war or plagues are only a few steps from doing so. If you want to talk seriously, talk about actual problems AI is causing right now, like job loss or malpractice.

waffletower 3y ago

"Historically, large power differentials have led to violence and subservience of whole societies." I agree, the calls for regulation by large corporate entities who quite transparently wish to control AI development for themselves, is a much greater near-term "existential threat" than AI itself.

phillipcarter 3y ago

IMO this paragraph in the summary is the most important one: > There are interventions we can make now, including the regulation of “high-risk applications” proposed in the EU AI Act. By regulating applications we focus on real harms and can make those most responsible directly liable. Another useful approach in the AI Act is to regulate disclosure, to ensure that those using models have the information they need to use them appropriately.

guy98238710 3y ago

I don't get this scare about AI. Imagine, you are a genius with IQ 200. You have studied hard your entire life and you are now an expert in several fields. One day you decide to conquer the world. Will you succeed or not?

dwaltrip 3y ago
On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control. Assuming you have 200 IQ, if you could make thousands or millions of clones of yourself quite cheaply, you still might not succeed in taking over the world, but it is no longer a laughable idea. Overall, I’m somewhat ambivalent about the possibility of x-risk, especially if we work reasonably hard to prevent it. But it shouldn’t be ignored. AI is moving very quickly and it is unclear how powerful its capabilities will be in the next 10-20 years. Of course, there are many other risks presented by AI that we need to stay on top of as well.
- guy98238710 3y ago
  > On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control. Copying AIs just results in effectively higher intelligence (bigger brain). Things would be different with self-replicating robots of course, but duplicating robots is not that fast and one could argue that the robots are dangerous rather than the AI controlling them, because they would be dangerous even if controlled by stupid non-AI program.
  - dwaltrip 3y ago
    Copying itself means it can do more “work” in the same amount of time. Earning money, strategizing, researching, smooth-talking humans, playing politics, and so on. Potentially even improving itself. These activities could give it power and influence.
    - guy98238710 3y ago

Tenoke 3y ago

>As we’ll see, if AI turns out to be powerful enough to be a catastrophic threat, the proposal may not actually help. In fact it could make things much worse, by creating a power imbalance.. Power imbalance is not 'much worse' than extinction. It's very clear how people start with the conclusion - "openess and progress have been good and will keep being the best choice" and then just contort their thinking and arguments to match the conclusion without really critically examining the counter arguments.