duvenaud

7 ms·

31.
by duvenaud 3y ago
Because I think being permanently disempowered is likely to lead to extinction eventually. I also think there are other plausible, faster routes to extinction.
32.
by duvenaud 3y ago
I'm not totally sure, but humans are better than our ancestors at basically every important job. It would be surprising if human brains were the pinnacle of computational power, and even if they were, digital brains could be run faste
33.
by duvenaud 3y ago
I think it's very unlikely that any of those would lead to human extinction, especially since most of those take decades to unfold, and would still leave large parts of the earth habitable. Toby Ord wrote a book trying to estimate each
34.
by duvenaud 3y ago
I'm confused. You agree that we're surround by naturally-arising, self-organizing agents, both biological and institutional. People are constantly experimenting with agentic AIs of all kinds. There are tons of theoretical chara
35.
by duvenaud 3y ago
Good point. Although in some ways it's a lower bar, since agents that can control organizations can delegate most of the difficult tasks. Most rulers don't invent their own societies from scratch, they simply co-opt existing powe
36.
by duvenaud 3y ago
Sorry, my gun analogy was unclear. I meant that, just because some agents on a planet have an ability, doesn't mean that everyone on that planet benefits. I agree that we probably won't see human extinction before robotics gets m
37.
by duvenaud 3y ago
This is a great point. But I'd say that capable entities have a habit of turning themselves into agents. A great example is totalitarian governments. Even if every single citizen hates the regime, they're still forced to suppor
38.
by duvenaud 3y ago
I guess so, but the difference between "humans are extinct" and "a small population of powerless humans survive in the margins as long as they don't cause trouble" seems pretty small to me. Most non-human primates
39.
by duvenaud 3y ago
Yes, I agree real life is fuzzy, I just chose chess as an example because it's unambiguous that machines dominate humans in that domain. As for being competitive with humans: Again, how about running a scan of a human brain, but faste
40.
by duvenaud 3y ago
Again, are you claiming that it's impossible for a machine to invent anything that a human could? Right now a large chunk of humanity's top talent and capital are working on exactly this problem. As for your second point, human c
41.
by duvenaud 3y ago
I see what you mean - for a while, the best chess was played by humans aided by chess engines. But that era has passed, and now having a human trying to aid the best chess engines just results in worse chess (or the same, if the human does
42.
by duvenaud 3y ago
AI Safety hasn't been divorced from tech companies, at least not from Deepmind, OpenAI, and Anthropic. They were all founded by people who said explicitly that AGI will probably mean the end of human civilization as we know it. All th
43.
by duvenaud 3y ago
Here's one of my concrete worries: At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an
44.
by duvenaud 3y ago
Well, Rees co-founded the Centre for the Study of Existential Risk, and has at least been thinking and writing about these issues for years now.
45.
by duvenaud 3y ago
I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old per
46.
by duvenaud 3y ago
Thanks for asking. I mean, my brain is just a machine, and eventually we'll make machines that can do everything brains can (even if it's just by scanning human brains). And once we build one that's about as capable as me,
47.
by duvenaud 3y ago
I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old
48.
by duvenaud 4y ago
Yeah, I agree that there'd probably be ways around this patch such as the ones you suggest.
49.
by duvenaud 4y ago
I agree, and this makes my proposed patch a weak solution. I was imagining that the specialness of the token would be reinforced during fine-tuning, but even that wouldn't provide any sort of guarantee.
50.
by duvenaud 4y ago
Here's a potential patch for that particular issue: Use a special token for "AI Instruction" that is always stripped from user text before it's shown to the model.
51.
by duvenaud 4y ago
Perhaps a better place to find algorithmic details is this related paper, also with Hinton as a co-author, which implements similar ideas in more standard networks: Scaling Forward Gradient With Local Losses Mengye Ren, Simon Kornblith, Ren
52.
by duvenaud 4y ago
Thanks for the explanation, but I'm not sure how it relates to my comment. It sounds like you're saying regulations help because they force banks to keep reserves. I think this helps in the short term, but in the long run create
53.
by duvenaud 4y ago
I mean, banking was already hundreds of years old in the 2000s. So recent bubbles and crashes in traditional markets seem like evidence that regulation, at least the kind that ends up being implemented, isn't doing much to prevent ver
54.
by duvenaud 4y ago
It is a parody.
55.
by duvenaud 4y ago
>We also know that the diffrence between salaries of rank and file staff and C-suit executives have increased 15x in the past 20 years, without any corresponding increase in productivity Seems like evidence against the claim that status
56.
by duvenaud 4y ago
Thanks for providing more context. However, the original claim I was reacting to was that "half the population ... [are] beneficiaries of affirmative action", i.e. that all men in college are beneficiaries of AA. I agree there a
57.
by duvenaud 4y ago
> While I believe there is belief that Harvard is accepting C-students over asians A+ students I think the reality is much closer; they are rejecting students with perfect scores and accepting students who scored in the 98th percentile.
58.
by duvenaud 4y ago
As a STEM academic, I can say that I've never seen or heard of ongoing AA for men in any STEM program in any institution whose policies I'm acquainted with. So it seems to me to be misleading to claim, or at least hard to believe
59.
by duvenaud 4y ago
Thanks for elaborating. I'm still not convinced about the strengths of claims that can be made in this instance, but I like the analogy to the 4-minute mile. However, I think that people didn't determine that a 4-minute mile was
60.
by duvenaud 4y ago
I think mirror neurons are a good example. I still don't understand what difference it makes which neurons are involved in empathy. Pretty much every signal in the brain can be tuned out or strengthened, right? In what sense does the

More ›