6 ms·

> I think it is fundamentally a crisis of trust. I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that

> I think it is fundamentally a crisis of trust. I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. [...] > I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing. > We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that. Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc. Rather we'll just do all of that even harder and the miracles ("curing cancer", lol) we've so far failed to deliver are bound to arrive in short order!

amarcheschi 1mo ago
rapture but for tech people
- spwa4 1mo ago
  [flagged]
  - trentor 1mo ago
    The only factual point you made is solved on February second. All Model providers have to provide offline tooling to check for invisible watermarking.
    - disgruntledphd2 1mo ago
      Yeah, it's particularly common when the EU does stuff.
    - spwa4 1mo ago
      There is nothing in the AI act that requires offline tooling, nor is there anything about requiring the SynthID secrets and/or API to work for every model, only for the people you are providing the model to. “The detection solution may be made available in the Union as one or more of the following: (i) a public, ideally standardised, specification allowing any third party to implement a detection mechanism; (ii) a piece of software (e.g., a standalone executable or library); (iii) a cloud-based service accessible to users in the Union through an API.” Option 3: "a cloud-based service". And accessible "to users". NOT to the public. Note: this is not the actual law, it is the code of practice, ie. guidance for model providers. The start of that document clearly states that you can comply with the law in other ways if you want, you'll just have to justify yourself. So you have a choice to not even do this. The actual law is here: https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50 https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-5... As to who decides the law clearly states who decides if someone is legal, like in most EU legislation. It's not the courts, it's "National market surveillance authorities designated by each EU Member State", and there is an EU office as well (and it is explicitly stated that they are not allowed to override each other). So every EU country has the right to provide exceptions to the law, just like they do for the GPDR. You do not have any rights under this legislation as an individual. Only these "National market surveillance authorities" get rights under this legislation. Secondary: article 7 additionally gives the EU commission the power to declare any code of conduct they want that declares what compliance with the AI act actually means. (and, of course, this is yet another attempt at declaring math illegal. The only way to actually enforce this legislation is for all models to comply with this, all over the internet. Obviously the EU does not remotely have the power to make that happen)
  - RandomLensman 1mo ago
    >The watermarking algorithm itself has a VERY specific property that should have set off everyone's alarm bells: it is NOT the case that you can "check text for AI watermark". What it technically mandates is that if you provide access to a model, those people should be able to check if text is generated by that model. Not by any other model. It is NOT a general "is this AI?" Where do I find that in the Act (or code of practice etc.)? IANAL, but Article 50 reads different to me but if there is a comment or guide how to read it - also fair enough.
    - rvba 1mo ago
      It sounds like anti EU propaganda.
      - jacquesm 1mo ago
        That's that accounts main contribution to HN and it's a miracle to me that they are still here.
  - pjc50 1mo ago
    While there is a large element of paranoia here, we're already seeing serious problems in e.g. planning consultations that people can just spam them with AI. AI makes a lot of traditional social processes stop working properly. > puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming), .. do we? > You get a mail from the government about taxes. You want to check if that text is generated by an AI No, actually, I want to know whether it's correct and whether it's legally binding. Using AI makes it less likely to be correct, sure, but ultimately whether it's human, spreadsheet, or AI the important thing is the legal right to correct process.
    - cucumber3732842 1mo ago
      >AI makes a lot of traditional social processes stop working properly. Which is mostly a good thing because most of those processes were cooked up to be exclusionary but with plausible deniability God forbid some upstart get a big contract they "shouldn't" have gotten because AI lets them create a submission that's on par with a big guys. God forbid Joe Schmo be able to prepare the paperwork and dot his I's and cross his T's and be able to take advantage of some stupid zoning exemption that you spec'd out to essentially be only available to big moneyed interests. The ability to suffer one's way through beurocratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI. But people don't frame it like for obvious reasons.
      - cindyllm 1mo ago
        [dead]
      - pjc50 1mo ago
        > The ability to suffer one's way through bureaucratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI These are adversarial processes. You might get a win in the short term, but everyone else is playing the same game with access to the same tools, and developing a game-breaking strategy in a PVP multiplayer game is going to both escalate and get patched out. A colleague of mine gave the example of using AI to snipe the UK driving test queuing system, which currently has a massive backlog. Which is great .. for them. Someone else gets pushed down by automated sniping. Then everyone else will have to start sniping. Then the website falls over because it wasn't designed to deal with 100m requests from the few hundred thousand people who are trying to bot their way to victory. (I can see one likely patch out of AI spam being "you must submit requests in person 9-5, online system is closed down"!)
  - watwut 1mo ago
    I trust EU massively more then I trust to Amodei.
dgellow 1mo ago
The effective altruist and longtermist crowd are into it as a religion. They would happily sacrifice anything human to satisfy their dreams of AI. They care way more about what the AI needs than their what their fellow humans need. We will end up with an AGI having better living and working conditions than humans and the whole AI industry will be clapping
- vrganj 1mo ago
  It's an ideological thing. (We just had a conversation about ideology actually, remember?) Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions. For the longest time, the only power labor has had to get those things was the power of collective bargaining. No agreement with your workers meant no production happened. Now, there's finally a chance at salvation. The capitalist Messiah is AGI and it will finally deliver them from those annoying laborers. This ideological bent is why they're putting everything they have into AI. It's why VCs and their fellow capitalists are going so crazy. They see it as a way to finally solve the contradictions of their ideology, but in reality it'd only create a new one: If everyone's out of a job, who will buy their products?
  - inigyou 1mo ago
    I wish I was smart enough to credibly promise capitalists they could replace all their workers. It doesn't matter that I can't do it. They'll pay me lots of money if they think I can.
  - danqqqq 1mo ago
    [dead]
  - eloisius 1mo ago
    If they had AGI, why would they need consumers anymore? Just use your drone army to secure all natural resources and produce whatever you need for yourself. Other humans are just pets.
A_D_E_P_T 1mo ago
This is actually crazy, because biochemistry is where LLMs are weakest. It's one of science's most fuzzy and unpredictable domains, in general, and a lot of published works which an LLM might take at face value might be unreplicable or subtly flawed. (See e.g. the entire history of Alzheimer's.) It is also where real-world lab and clinical trial work is most important. So Anthropic are going to spin up a medicinal chemistry lab and start mouse experiments? I mean, it would be nice, but I don't think that the guys at Anthropic know what they're talking about here, or what they might be getting themselves into. (If indeed this is more than just PR.)
- discreteevent 1mo ago
  See also this article by a genomics PhD about how intelligence is not the bottleneck in medical research. https://www.writingruxandrabio.com/p/intelligence-is-not-the-main-bottleneck https://www.writingruxandrabio.com/p/intelligence-is-not-the...
- kilpikaarna 1mo ago
  AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards. "Oh, we'll just set up fully automated robotic research labs everywhere!" Where will you get the raw materials? Oh we'll mine the asteroids! How will you find the energy to get there? Oh, the AI will invent new physics! (again without needing to run experiments). Et.c. Didn't and don't mean to disparage anyone's medical struggles, but "a cure for cancer" is a well-worn strawman. One that's been achieved for the low hanging fruit, the higher ones are seeing steady progress (already before LLM chatbots, even!), the bottleneck isn't "intelligence" and to the extent there are socioeconomic (access to screening, treatment) or environmental/lifestyle factors involved the AI boom is likely just making things worse! I'm sure Dario knows this, and it's anyway too pedestrian compared to the usual list of fruits of ASI. The text probably originally read "nanobots eating you alive and uploading to the cloud" or something, but they figured that wouldn't go over with the intended audience. "What do the peasants care about? Oh I know! Curing cancer!"
  - 1mo ago
pj_mukh 1mo ago
> The thing that will work is actually curing cancer. I think typical bubble behavior the leaders have set up the whole promise to fail. Everyone is expecting some faux super intelligence to come and find a cancer solution everyone else missed. However, it is just as likely that vanilla current LLM's will create enough of a productivity boost for back office automations in research heavy hospitals to create the space for regular humans to create these breakthroughs, but LLM's won't be able to claim that for themselves and inevitably “fail”.
- anon-3988 1mo ago
  I mean they did solve the protein folding problem did they? NOt too far fetch to think it can automate cancer research or problem finding in some way.
  - disgruntledphd2 1mo ago
    > I mean they did solve the protein folding problem did they? They made huge progress, but I would say that the vast majority of work on this problem was designing the harness for the model. That's a lot of work for each and every domain.
  - RandomLensman 1mo ago
    There is already (partial) automation there, for example for finding new drug candidates. Isn't that so far only static folding?
    - krige 1mo ago
      As usual with LLM era craze, finding candidates was never the bottleneck. And, so far, it failed to actually provide any tangible results. [0] [0] https://www.science.org/content/blog-post/so-how-ai-drug-discovery-doing-really https://www.science.org/content/blog-post/so-how-ai-drug-dis...
  - inigyou 1mo ago
    The field of AI did very well in protein folding, but it was completely different from LLMs.
mindwok 1mo ago
I think it's more reasonable than it sounds on the surface. People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own. Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides.
- disgruntledphd2 1mo ago
  > Curing cancer sounds insane, but it's also a research problem, I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer. Without that, I can't see much (if any) progress being made on domains like biology.
  - XorNot 1mo ago
    The real issue is that biology needs actual experiments done in the physical world, which isn't nice and orderly and well behaved and easily loadable onto a 19" rectangular box.
    - bananaflag 1mo ago
      On the other hand, AI means actual experiments done in the physical world but coordinated by an entity that never sleeps and never gets depressed and can multiply itself manifold and always comes up with new ideas.
      - mohamedkoubaa 1mo ago
        That fanatical tireless entity will need access to human tissue.
        1mo ago
traceroute66 1mo ago
> The thing that will work is actually curing cancer. This is honestly hilarious. Dario must think people are stupid. First, cancer research charities are some of the most well funded on this planet. They all obtain donations on the basis of "together, we will cure cancer" messaging. They all fund the best science they can. Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different. Why do you think student doctors have to spend so many years "doing the rounds" Dario ? They are all academically smart, they are all capable of memorizing text books ... but there is no substitute for seeing and doing the reality. I barely trust Claude to write code, let alone find a cure to cancer.
- eloisant 1mo ago
  Besides cancer isn't really "a disease" but rather "a family of diseases" that can vary a lot. Even breast cancer can be one of many different cancers. It's like saying "we'll cure infectious diseases"
  - traceroute66 1mo ago
    > It's like saying "we'll cure infectious diseases" Don't worry, I'm sure that's his next blog post. Right after Claude has finished solving famine and poverty. ;)
    - Applejinx 1mo ago
      Technically, killing everybody is one way of solving famine and poverty. How weird that all those humans seem not to trust the logic of their betters ;P
      - subygan 1mo ago
        you're right. it coincidentally solves cancer and infectious diseases as well. the ultimate utopia
sigbottle 1mo ago
Honestly, it kind of reminds me of my life. I need to start going to the gym, stop being a slob, start showing up to my job not like I'm a college freshman attending classes but as a respectable young man. But instead of doing that, I'm going to drop everything and become a linux kernel maintainer. Learning about the internals of how RCU concurrency is implemented across subsystems is surely what I need to do right now! (/j before I get crucified)
ironqcold 1mo ago
I really can't disagree. What's the point of all this talk if for most ordinary people these technologies do more harm than good?
- simoncion 1mo ago
  > What's the point of all this talk... It signals to people who have money and/or influence that the speaker is "committed to doing the right thing" and "eager to find a workable path forward" that "meaningfully addresses the 'very real' concerns and misunderstandings that non-experts have about what the company the speaker is associated with are doing". Once the speaker has signaled that, what more could they possibly do? They've signaled their willingness to open the channels of dialog with the rabble who are non-experts and -thus- cannot possibly be engaged with as equals. By doing this, the speaker tried their best, didn't they? Or, to put it another way, it's PR and damage control... soothing theater for the folks who can actually affect them.
ookblah 1mo ago
i'll give him this, in every single "breakthrough" you either go down in flames as a fucking idiot or a genius lol. i know the stakes are way lower but look at every other HN post of something novel and it's just filled with takes like you see here, very polarizing.
- jacquesm 1mo ago
  You could easily be an idiot and a genius at the same time and I think that has a better chance of covering this than either exclusively. Amodei is very smart, maybe even well intentioned (though I'm not sure about that) and the long term effects of all this may well end up making the lot of them look like idiots. The way he's coming off here though gives me SBF vibes.
nsagent 1mo ago
After listening to Dario's interview with Dwarkesh, I really started wondering if Dario's been drinking too much of Claude's kool-aid. I know people in positions of power often see lots filtered information that skews their perceptions of reality, but it seems like the level of pure hype Dario's pushing exceeds even the typical sycophantic filter bubble. Cure cancer indeed.
patcon 1mo ago
> ordinary people [...] always suspect that we are cooking up some new way to screw them over Anything big enough always is, no? The next scale of order in the hierarchy starts concerned itself with what's good for it, at the expense of the smaller parts. The fact that government is like a giant organization that everyone lives in, the fact that it mostly doesn't fuck us all over in terrible ways, that's a testament to the amazing alignment of regulation and law imho
txrx0000 1mo ago
Suppose the tech is real, I do not want Anthropic/Dario to hoard it and be my cancer-curing messiah. Release the weights and do research in the open. People will save themselves if you give them the means. The bio/cyber/alignment risk rhetoric is nonsense. Somewhat counterintuitively, all of those risks are lower/non-issues if frontier labs release their weights and do all of their research in the open, as explained in a past comment: https://news.ycombinator.com/item?id=49313739 https://news.ycombinator.com/item?id=49313739 Again: if we were to regulate anything, it would be to force frontier companies to release their model weights and do all of their research in the open. This is a necessary yet insufficient condition for the intelligence explosion to benefit the vast majority of people.
stellamariesays 1mo ago
[flagged]
pavel_lishin 1mo ago
"Saying we'll cure cancer is a cliche. So I'm here to announce that we're going to try to cure cancer!"
mm263 1mo ago
> Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc No snark here from my side, just curious about your vision - which concrete actions by Anthropic could improve trust for the concerns you listed?
- kilpikaarna 1mo ago
  Well, is ”not actively make things worse in the short term?” too ambitious a goal? Considering the vast amount of money, world-class talent and political and industry vonnections Anthropic has access to, it feels a bit unfair to ask me to solve this for them.
  - mm263 1mo ago
    I’m not asking you to solve it. Just curious what are the solutions that Anthropic can do right now from your perspective. The concerns you have are pretty well-defined, so the solutions for them must also flow naturally, I think?
- nshshbr 1mo ago
  You're not asking me, but I want to say that there is nothing Anthropic or any other AI company can do to make me trust them other than to dissolve and give up their ambitions and do something else. Then maybe, after seeing genuine contrition and humility I would trust them to do something else. But likely not. After all they've done, they should accept that a permanent loss of trust is a reasonable consequence. They should lose their influence, position, and money. They should live the rest of their lives as ordinary people, unable to ever again cause mass harm to others, and they should be grateful to be forgiven, and allowed such a humble opportunity. My answer to their proposal for what our society should look like, and what kind of technologies we should use is "no". It's not a trust issue, as in I would not accept the proposal from anyone, even if I did otherwise trust them. And if someone I trusted insistently proposed this, I would stop trusting them. In any case, they don't accept refusal, but try this grotesque and condescending maneuver about "trust", as if I and many others were not really just saying "no". No one who refuses to take "no" for an answer can ever deserve trust. If you are committed to convincing me to go along with your desires, and have no intent of accepting my refusal, you will never deserve trust, and I will never trust you. These companies may have the power to do what they want anyway. But they will never have a clear conscience, or trust and support from me or anyone like me.
deleted 1mo ago
[deleted]
blmarket 1mo ago
They should not waste their time on curing cancers. They should waste their time winning the stock market - by doing HFT, market making, or whatever. AI please predict butterfly effect and reap all market money! Surely it will be quite dystopian future - AI will dictate how the money allocated, where there is only one private company left - Anthropic (as per Dario says)
bigbadfeline 1mo ago
D.A. >> Anthropic is ramping up its efforts very quickly in biology and medicine At the same time, the best Anthro models refuse to answer biology questions, keeping the public in the dark not only about new AI-assisted results but also about conventional and health-protective research. D.A. >> we hope to have incredible results in the coming years and some early glimmers in the coming months. So, what changed? He went from a promise of AI bonanza any minute now to a promise to maybe cure cancer in the coming years, that's right, an unspecified number of years. Until that time, we're supposed to eat glimmers of hope because the AI-caused inflation will make food unaffordable. D.A. >> When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that. And don't forget to stock on ear protection, because when something happens, it's going to be "as laud as possible", you've got Amo's word too - to marvel at in the meantime.
overgard 1mo ago
Totally agree, like: > I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. Well, yeah Dario. You keep PUBLICLY TALKING about ways you want to fuck over people and everyone has noticed. I'm sorry, but when a lot of people are living paycheck-to-paycheck and watching layoffs everywhere, being like "well it'll take all the jobs and maybe kill humanity" is going to set off a LOT of alarm bells, despite your attempts to make concerned looking faces. You're trying to solve hypothetical problems in unproven ways, while worsening real problems people actually face. These people are SO disconnected from the society they live in.
ModernMech 1mo ago
Yeah this is like SpaceX saying people’s problems with us is that we haven’t gotten to Mars yet, and that to solve that we need to double our efforts to get people to Mars. People have a problem with the efforts, not that they haven’t reached utopia yet. Doubling the efforts will not solve the problem, it will only deepen it.
dragonwriter 1mo ago
> at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. Other than the fact I doubt that will happen, if it did, I think it would just underline the trust issue. If some independent group cures cancer using Anthropic’s models as critical support, that would be great—but the nature of biology and medicine is that guardrails to prevent misuse will also cripple research. If someone with an tight special relationship cures cancer with special privileged access to Anthropic’s models cures cancer making significant use of the models, well, the cure for cancer is nice, but people are going to notice that Anthropic has the control over who gets to try, and the power to set terms, which underlines the stakes for the trust issue.

amarcheschi 1mo ago

rapture but for tech people

spwa4 1mo ago
[flagged]
- trentor 1mo ago
  The only factual point you made is solved on February second. All Model providers have to provide offline tooling to check for invisible watermarking.
  - disgruntledphd2 1mo ago
    Yeah, it's particularly common when the EU does stuff.
  - spwa4 1mo ago
    There is nothing in the AI act that requires offline tooling, nor is there anything about requiring the SynthID secrets and/or API to work for every model, only for the people you are providing the model to. “The detection solution may be made available in the Union as one or more of the following: (i) a public, ideally standardised, specification allowing any third party to implement a detection mechanism; (ii) a piece of software (e.g., a standalone executable or library); (iii) a cloud-based service accessible to users in the Union through an API.” Option 3: "a cloud-based service". And accessible "to users". NOT to the public. Note: this is not the actual law, it is the code of practice, ie. guidance for model providers. The start of that document clearly states that you can comply with the law in other ways if you want, you'll just have to justify yourself. So you have a choice to not even do this. The actual law is here: https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50 https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-5... As to who decides the law clearly states who decides if someone is legal, like in most EU legislation. It's not the courts, it's "National market surveillance authorities designated by each EU Member State", and there is an EU office as well (and it is explicitly stated that they are not allowed to override each other). So every EU country has the right to provide exceptions to the law, just like they do for the GPDR. You do not have any rights under this legislation as an individual. Only these "National market surveillance authorities" get rights under this legislation. Secondary: article 7 additionally gives the EU commission the power to declare any code of conduct they want that declares what compliance with the AI act actually means. (and, of course, this is yet another attempt at declaring math illegal. The only way to actually enforce this legislation is for all models to comply with this, all over the internet. Obviously the EU does not remotely have the power to make that happen)
- RandomLensman 1mo ago
  >The watermarking algorithm itself has a VERY specific property that should have set off everyone's alarm bells: it is NOT the case that you can "check text for AI watermark". What it technically mandates is that if you provide access to a model, those people should be able to check if text is generated by that model. Not by any other model. It is NOT a general "is this AI?" Where do I find that in the Act (or code of practice etc.)? IANAL, but Article 50 reads different to me but if there is a comment or guide how to read it - also fair enough.
  - rvba 1mo ago
    It sounds like anti EU propaganda.
    - jacquesm 1mo ago
      That's that accounts main contribution to HN and it's a miracle to me that they are still here.
- pjc50 1mo ago
  While there is a large element of paranoia here, we're already seeing serious problems in e.g. planning consultations that people can just spam them with AI. AI makes a lot of traditional social processes stop working properly. > puts in it's regulations that you're not allowed to use AI to communicate with them (we all know this is coming), .. do we? > You get a mail from the government about taxes. You want to check if that text is generated by an AI No, actually, I want to know whether it's correct and whether it's legally binding. Using AI makes it less likely to be correct, sure, but ultimately whether it's human, spreadsheet, or AI the important thing is the legal right to correct process.
  - cucumber3732842 1mo ago
    >AI makes a lot of traditional social processes stop working properly. Which is mostly a good thing because most of those processes were cooked up to be exclusionary but with plausible deniability God forbid some upstart get a big contract they "shouldn't" have gotten because AI lets them create a submission that's on par with a big guys. God forbid Joe Schmo be able to prepare the paperwork and dot his I's and cross his T's and be able to take advantage of some stupid zoning exemption that you spec'd out to essentially be only available to big moneyed interests. The ability to suffer one's way through beurocratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI. But people don't frame it like for obvious reasons.
    - cindyllm 1mo ago
      [dead]
    - pjc50 1mo ago
      > The ability to suffer one's way through bureaucratic process without having to suffer through paying the lawyers and accountants and engineers and whatnot to get you through that process is a huge part of the value proposition of AI These are adversarial processes. You might get a win in the short term, but everyone else is playing the same game with access to the same tools, and developing a game-breaking strategy in a PVP multiplayer game is going to both escalate and get patched out. A colleague of mine gave the example of using AI to snipe the UK driving test queuing system, which currently has a massive backlog. Which is great .. for them. Someone else gets pushed down by automated sniping. Then everyone else will have to start sniping. Then the website falls over because it wasn't designed to deal with 100m requests from the few hundred thousand people who are trying to bot their way to victory. (I can see one likely patch out of AI spam being "you must submit requests in person 9-5, online system is closed down"!)
- watwut 1mo ago
  I trust EU massively more then I trust to Amodei.

dgellow 1mo ago

The effective altruist and longtermist crowd are into it as a religion. They would happily sacrifice anything human to satisfy their dreams of AI. They care way more about what the AI needs than their what their fellow humans need. We will end up with an AGI having better living and working conditions than humans and the whole AI industry will be clapping

vrganj 1mo ago
It's an ideological thing. (We just had a conversation about ideology actually, remember?) Capitalism has always been a tension between capitalists, who want maximum return on capital and workers, who want pesky things like a living wage, sick leave or safe work conditions. For the longest time, the only power labor has had to get those things was the power of collective bargaining. No agreement with your workers meant no production happened. Now, there's finally a chance at salvation. The capitalist Messiah is AGI and it will finally deliver them from those annoying laborers. This ideological bent is why they're putting everything they have into AI. It's why VCs and their fellow capitalists are going so crazy. They see it as a way to finally solve the contradictions of their ideology, but in reality it'd only create a new one: If everyone's out of a job, who will buy their products?
- inigyou 1mo ago
  I wish I was smart enough to credibly promise capitalists they could replace all their workers. It doesn't matter that I can't do it. They'll pay me lots of money if they think I can.
- danqqqq 1mo ago
  [dead]
- eloisius 1mo ago
  If they had AGI, why would they need consumers anymore? Just use your drone army to secure all natural resources and produce whatever you need for yourself. Other humans are just pets.

A_D_E_P_T 1mo ago

This is actually crazy, because biochemistry is where LLMs are weakest. It's one of science's most fuzzy and unpredictable domains, in general, and a lot of published works which an LLM might take at face value might be unreplicable or subtly flawed. (See e.g. the entire history of Alzheimer's.) It is also where real-world lab and clinical trial work is most important. So Anthropic are going to spin up a medicinal chemistry lab and start mouse experiments? I mean, it would be nice, but I don't think that the guys at Anthropic know what they're talking about here, or what they might be getting themselves into. (If indeed this is more than just PR.)

discreteevent 1mo ago
See also this article by a genomics PhD about how intelligence is not the bottleneck in medical research. https://www.writingruxandrabio.com/p/intelligence-is-not-the-main-bottleneck https://www.writingruxandrabio.com/p/intelligence-is-not-the...
kilpikaarna 1mo ago
AI bros consistently handwave away any notions of the material world imposing limits, because (spoiler!) it's a religion where you start from the miracle and work backwards. "Oh, we'll just set up fully automated robotic research labs everywhere!" Where will you get the raw materials? Oh we'll mine the asteroids! How will you find the energy to get there? Oh, the AI will invent new physics! (again without needing to run experiments). Et.c. Didn't and don't mean to disparage anyone's medical struggles, but "a cure for cancer" is a well-worn strawman. One that's been achieved for the low hanging fruit, the higher ones are seeing steady progress (already before LLM chatbots, even!), the bottleneck isn't "intelligence" and to the extent there are socioeconomic (access to screening, treatment) or environmental/lifestyle factors involved the AI boom is likely just making things worse! I'm sure Dario knows this, and it's anyway too pedestrian compared to the usual list of fruits of ASI. The text probably originally read "nanobots eating you alive and uploading to the cloud" or something, but they figured that wouldn't go over with the intended audience. "What do the peasants care about? Oh I know! Curing cancer!"
- 1mo ago

pj_mukh 1mo ago

> The thing that will work is actually curing cancer. I think typical bubble behavior the leaders have set up the whole promise to fail. Everyone is expecting some faux super intelligence to come and find a cancer solution everyone else missed. However, it is just as likely that vanilla current LLM's will create enough of a productivity boost for back office automations in research heavy hospitals to create the space for regular humans to create these breakthroughs, but LLM's won't be able to claim that for themselves and inevitably “fail”.

anon-3988 1mo ago
I mean they did solve the protein folding problem did they? NOt too far fetch to think it can automate cancer research or problem finding in some way.
- disgruntledphd2 1mo ago
  > I mean they did solve the protein folding problem did they? They made huge progress, but I would say that the vast majority of work on this problem was designing the harness for the model. That's a lot of work for each and every domain.
- RandomLensman 1mo ago
  There is already (partial) automation there, for example for finding new drug candidates. Isn't that so far only static folding?
  - krige 1mo ago
    As usual with LLM era craze, finding candidates was never the bottleneck. And, so far, it failed to actually provide any tangible results. [0] [0] https://www.science.org/content/blog-post/so-how-ai-drug-discovery-doing-really https://www.science.org/content/blog-post/so-how-ai-drug-dis...
- inigyou 1mo ago
  The field of AI did very well in protein folding, but it was completely different from LLMs.

mindwok 1mo ago

I think it's more reasonable than it sounds on the surface. People's jobs, the bubble, etc are societal level problems and well beyond what Anthropic could even hope to influence on their own. Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides.

disgruntledphd2 1mo ago
> Curing cancer sounds insane, but it's also a research problem, I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer. Without that, I can't see much (if any) progress being made on domains like biology.
- XorNot 1mo ago
  The real issue is that biology needs actual experiments done in the physical world, which isn't nice and orderly and well behaved and easily loadable onto a 19" rectangular box.
  - bananaflag 1mo ago
    On the other hand, AI means actual experiments done in the physical world but coordinated by an entity that never sleeps and never gets depressed and can multiply itself manifold and always comes up with new ideas.
    - mohamedkoubaa 1mo ago
      That fanatical tireless entity will need access to human tissue.
      - 1mo ago

traceroute66 1mo ago

> The thing that will work is actually curing cancer. This is honestly hilarious. Dario must think people are stupid. First, cancer research charities are some of the most well funded on this planet. They all obtain donations on the basis of "together, we will cure cancer" messaging. They all fund the best science they can. Second, the human body is a complicated thing. You can feed your fancy LLM as many textbooks and academic papers as you like, but the reality on the hospital ward will always be different. Why do you think student doctors have to spend so many years "doing the rounds" Dario ? They are all academically smart, they are all capable of memorizing text books ... but there is no substitute for seeing and doing the reality. I barely trust Claude to write code, let alone find a cure to cancer.

eloisant 1mo ago
Besides cancer isn't really "a disease" but rather "a family of diseases" that can vary a lot. Even breast cancer can be one of many different cancers. It's like saying "we'll cure infectious diseases"
- traceroute66 1mo ago
  > It's like saying "we'll cure infectious diseases" Don't worry, I'm sure that's his next blog post. Right after Claude has finished solving famine and poverty. ;)
  - Applejinx 1mo ago
    Technically, killing everybody is one way of solving famine and poverty. How weird that all those humans seem not to trust the logic of their betters ;P
    - subygan 1mo ago
      you're right. it coincidentally solves cancer and infectious diseases as well. the ultimate utopia

sigbottle 1mo ago

Honestly, it kind of reminds me of my life. I need to start going to the gym, stop being a slob, start showing up to my job not like I'm a college freshman attending classes but as a respectable young man. But instead of doing that, I'm going to drop everything and become a linux kernel maintainer. Learning about the internals of how RCU concurrency is implemented across subsystems is surely what I need to do right now! (/j before I get crucified)

ironqcold 1mo ago

I really can't disagree. What's the point of all this talk if for most ordinary people these technologies do more harm than good?

simoncion 1mo ago
> What's the point of all this talk... It signals to people who have money and/or influence that the speaker is "committed to doing the right thing" and "eager to find a workable path forward" that "meaningfully addresses the 'very real' concerns and misunderstandings that non-experts have about what the company the speaker is associated with are doing". Once the speaker has signaled that, what more could they possibly do? They've signaled their willingness to open the channels of dialog with the rabble who are non-experts and -thus- cannot possibly be engaged with as equals. By doing this, the speaker tried their best, didn't they? Or, to put it another way, it's PR and damage control... soothing theater for the folks who can actually affect them.

ookblah 1mo ago

i'll give him this, in every single "breakthrough" you either go down in flames as a fucking idiot or a genius lol. i know the stakes are way lower but look at every other HN post of something novel and it's just filled with takes like you see here, very polarizing.

jacquesm 1mo ago
You could easily be an idiot and a genius at the same time and I think that has a better chance of covering this than either exclusively. Amodei is very smart, maybe even well intentioned (though I'm not sure about that) and the long term effects of all this may well end up making the lot of them look like idiots. The way he's coming off here though gives me SBF vibes.

nsagent 1mo ago

After listening to Dario's interview with Dwarkesh, I really started wondering if Dario's been drinking too much of Claude's kool-aid. I know people in positions of power often see lots filtered information that skews their perceptions of reality, but it seems like the level of pure hype Dario's pushing exceeds even the typical sycophantic filter bubble. Cure cancer indeed.

patcon 1mo ago

> ordinary people [...] always suspect that we are cooking up some new way to screw them over Anything big enough always is, no? The next scale of order in the hierarchy starts concerned itself with what's good for it, at the expense of the smaller parts. The fact that government is like a giant organization that everyone lives in, the fact that it mostly doesn't fuck us all over in terrible ways, that's a testament to the amazing alignment of regulation and law imho

txrx0000 1mo ago

Suppose the tech is real, I do not want Anthropic/Dario to hoard it and be my cancer-curing messiah. Release the weights and do research in the open. People will save themselves if you give them the means. The bio/cyber/alignment risk rhetoric is nonsense. Somewhat counterintuitively, all of those risks are lower/non-issues if frontier labs release their weights and do all of their research in the open, as explained in a past comment: https://news.ycombinator.com/item?id=49313739 https://news.ycombinator.com/item?id=49313739 Again: if we were to regulate anything, it would be to force frontier companies to release their model weights and do all of their research in the open. This is a necessary yet insufficient condition for the intelligence explosion to benefit the vast majority of people.

stellamariesays 1mo ago

[flagged]

pavel_lishin 1mo ago

"Saying we'll cure cancer is a cliche. So I'm here to announce that we're going to try to cure cancer!"

mm263 1mo ago

> Turns out "winning back trust" doesn't have anything to do with any actual concerns people may have re: employment, electricity prices, stock market bubble, intellectual property, scams, cybersecurity, environmental issues etc No snark here from my side, just curious about your vision - which concrete actions by Anthropic could improve trust for the concerns you listed?

kilpikaarna 1mo ago
Well, is ”not actively make things worse in the short term?” too ambitious a goal? Considering the vast amount of money, world-class talent and political and industry vonnections Anthropic has access to, it feels a bit unfair to ask me to solve this for them.
- mm263 1mo ago
  I’m not asking you to solve it. Just curious what are the solutions that Anthropic can do right now from your perspective. The concerns you have are pretty well-defined, so the solutions for them must also flow naturally, I think?
nshshbr 1mo ago
You're not asking me, but I want to say that there is nothing Anthropic or any other AI company can do to make me trust them other than to dissolve and give up their ambitions and do something else. Then maybe, after seeing genuine contrition and humility I would trust them to do something else. But likely not. After all they've done, they should accept that a permanent loss of trust is a reasonable consequence. They should lose their influence, position, and money. They should live the rest of their lives as ordinary people, unable to ever again cause mass harm to others, and they should be grateful to be forgiven, and allowed such a humble opportunity. My answer to their proposal for what our society should look like, and what kind of technologies we should use is "no". It's not a trust issue, as in I would not accept the proposal from anyone, even if I did otherwise trust them. And if someone I trusted insistently proposed this, I would stop trusting them. In any case, they don't accept refusal, but try this grotesque and condescending maneuver about "trust", as if I and many others were not really just saying "no". No one who refuses to take "no" for an answer can ever deserve trust. If you are committed to convincing me to go along with your desires, and have no intent of accepting my refusal, you will never deserve trust, and I will never trust you. These companies may have the power to do what they want anyway. But they will never have a clear conscience, or trust and support from me or anyone like me.

deleted 1mo ago

[deleted]

blmarket 1mo ago

They should not waste their time on curing cancers. They should waste their time winning the stock market - by doing HFT, market making, or whatever. AI please predict butterfly effect and reap all market money! Surely it will be quite dystopian future - AI will dictate how the money allocated, where there is only one private company left - Anthropic (as per Dario says)

bigbadfeline 1mo ago

D.A. >> Anthropic is ramping up its efforts very quickly in biology and medicine At the same time, the best Anthro models refuse to answer biology questions, keeping the public in the dark not only about new AI-assisted results but also about conventional and health-protective research. D.A. >> we hope to have incredible results in the coming years and some early glimmers in the coming months. So, what changed? He went from a promise of AI bonanza any minute now to a promise to maybe cure cancer in the coming years, that's right, an unspecified number of years. Until that time, we're supposed to eat glimmers of hope because the AI-caused inflation will make food unaffordable. D.A. >> When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that. And don't forget to stock on ear protection, because when something happens, it's going to be "as laud as possible", you've got Amo's word too - to marvel at in the meantime.

overgard 1mo ago

Totally agree, like: > I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. Well, yeah Dario. You keep PUBLICLY TALKING about ways you want to fuck over people and everyone has noticed. I'm sorry, but when a lot of people are living paycheck-to-paycheck and watching layoffs everywhere, being like "well it'll take all the jobs and maybe kill humanity" is going to set off a LOT of alarm bells, despite your attempts to make concerned looking faces. You're trying to solve hypothetical problems in unproven ways, while worsening real problems people actually face. These people are SO disconnected from the society they live in.

ModernMech 1mo ago

Yeah this is like SpaceX saying people’s problems with us is that we haven’t gotten to Mars yet, and that to solve that we need to double our efforts to get people to Mars. People have a problem with the efforts, not that they haven’t reached utopia yet. Doubling the efforts will not solve the problem, it will only deepen it.

dragonwriter 1mo ago

> at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. Other than the fact I doubt that will happen, if it did, I think it would just underline the trust issue. If some independent group cures cancer using Anthropic’s models as critical support, that would be great—but the nature of biology and medicine is that guardrails to prevent misuse will also cripple research. If someone with an tight special relationship cures cancer with special privileged access to Anthropic’s models cures cancer making significant use of the models, well, the cure for cancer is nice, but people are going to notice that Anthropic has the control over who gets to try, and the power to set terms, which underlines the stakes for the trust issue.