4 ms·

Karpathy will start this week on Anthropic's pre-training team, which is responsible for the massive training runs that give Claude its core knowledge and capab

Karpathy will start this week on Anthropic's pre-training team, which is responsible for the massive training runs that give Claude its core knowledge and capabilities, according to Anthropic. Source: https://www.axios.com/2026/05/19/anthropic-openai-karpathy-andrej-claude https://www.axios.com/2026/05/19/anthropic-openai-karpathy-a...

ollin 5mo ago
Specifically it looks like he's planning to extend the ideas from https://github.com/karpathy/autoresearch https://github.com/karpathy/autoresearch into a larger effort towards recursive training improvement [1]: > Excited to welcome Andrej to the Pretraining team! He'll be building a team focused on using Claude to accelerate pretraining research itself. I can’t think of anyone better suited to do it — looking forward to what we build together! [1] https://x.com/nickevanjoseph/status/2056760504949842219 https://x.com/nickevanjoseph/status/2056760504949842219
- deleted 5mo ago
  [deleted]
- triyambakam 5mo ago
  I guess we must expect it at this point. But funny that has model written tokens like ’ instead of '
- stingraycharles 5mo ago
  Am I the only one who wasn’t particularly impressed by AutoResearch? If you looked at what the agent was actually doing, it was just tuning parameters mostly, not really trying different novel approaches. I couldn’t help myself but consider this mostly a very inefficient variant of hyperparameter optimization, but someone correct me if I’m wrong, I may be looking at this too pessimistic.
  - lacker 5mo ago
    I didn't dig into what the actual repository was doing, but personally, I took some inspiration from the idea after reading about it and realizing that I might have been underestimating the ability of LLMs. I put a bit more work into a performance harness I was using locally and just set some agents to brainstorming and they did seem to find some great stuff. So I don't really have a stance one way or another on this specific repo, but the general idea seems like a really good one.
    - delis-thumbs-7e 5mo ago
      Could you elaborate in specifics how you had been underestimating models? Ypu mean just using more tighter harnessing to make them work in structured agentic eay or something else?
      - lacker 5mo ago
        The specific code I was working on, I had a general idea of the sort of performance improvement that would be possible. I just thought that it would be too hard for the models to figure out without a lot of hand-holding. But it ended up being not "too hard ever", but more like, in 1 out of every 5 tries, the model did in fact manage to get a large refactoring to the point where it improved performance. So once I set it up to try something, use the perf test, see if it worked, if not, throw it away, repeat. Then it started, slowly, finding some useful things.
      - inciampati 5mo ago
        Just remember that the will do clever but useless things to improve. Like changing the random seed as per autoresearch's hero image. lol! imo, out of the box thinking is needed.
  - clbrmbr 5mo ago
    Karpathy embedded within an organization is way more impressive than him out on his own with hot takes and little projects. I hope he does great things for Anthropic.
- 4ashz 5mo ago
  More like he'll blog and tweet about using Claude and get gullible software engineers to buy Claude subscriptions and work on their own obsolescence while paying for it. Many people are still deluded and think he is the same person who wrote the informal AI tutorials in plain html. He isn't, he is selling stuff now.
  - bonoboTP 5mo ago
    What is he selling? How is this time different compared to when he was at OpenAI or at Tesla? You could say he was shilling those products too. I don't see any shift. He's still posted free in depth YouTube videos recently.
    - 23998h 5mo ago
      > What is he selling? Is that a serious question? He already promoted vibe coding and AI hype. Now he is literally there to promote Anthropic and its IPO price. When he was at OpenAI it wasn't overtly commercial yet. At Tesla he had a way lower profile. Now he is the vibe coding Jesus for deluded software engineers. The impact is much larger.
      - bonoboTP 5mo ago
        I think he's just genuinely excited about the capabilities. (I do understand that for Anthropic it's a brand boost as well, just like signing other prominent researchers, as it was with LeCun and Meta etc).
      - sho_hn 5mo ago
        > At Tesla he had a way lower profile. ? He was literally rolled out in front of camera as Tesla's AI prodigy at multiple streamed events designed to appeal to techy consumers and dev recruitment. He's definitely been one of AI's public personas for a long time now, and his employers have regularly aided/directed/utilized him accordingly.
    - deleted 5mo ago
      [deleted]
- zmmmmm 5mo ago
  So he's working on the singularity
ed_elliott_asc 5mo ago
Why do they need this when they have the next gen mythos? Surely that can manage everything?
- cyanydeez 5mo ago
  You don't understand: no ones ever reading more than 1% of the training material; so they need someone who has reduced that to 0.1%. The less you know, the more you know!
the_arun 5mo ago
This is good branding move for Anthropic. Karpathy is well respected among ML crowd.
- zachncst 5mo ago
  Minor celebrity fwiw - deserved though.
  - fakedang 5mo ago
    He's a celebrity all right - promoting bunk just like the rest of that lot, along with his Lord and Savior Elon Musk.
- wodenokoto 5mo ago
  Speaking of, how did he not lose credibility at “full self driving next year, better buy it now”-Tesla? It might be Elon who went and said that and said they don’t need lidar, but as director of AI and auto vision Karpathy bears the responsibility for those features.
  - joe_mamba 5mo ago
    >Speaking of, how did he not lose credibility at “full self driving next year, better buy it now”-Tesla? That I also want to know. He bailed out of Tesla right when the limitations of his "LIDAR-less cameras only self driving" system were becoming obvious, and nobody asked him about the hindsight of this obvious fuckup. >but as director of AI and auto vision Karpathy bears the responsibility for those features. Exactly. You lead the R&D, so it's on you. If your boss makes stupid decisions in public overriding your best judgement, the leave and go somewhere where your decisions be respected. The ML market was red hot for people like him back then so it's not like he didn't have alternatives. Although I doubt Elon forced that idea on him, since he's the one who was confidently claiming that vision only is better since Lidar pollutes the sensor fusion data.
    - kopirgan 5mo ago

ollin 5mo ago

Specifically it looks like he's planning to extend the ideas from https://github.com/karpathy/autoresearch https://github.com/karpathy/autoresearch into a larger effort towards recursive training improvement [1]: > Excited to welcome Andrej to the Pretraining team! He'll be building a team focused on using Claude to accelerate pretraining research itself. I can’t think of anyone better suited to do it — looking forward to what we build together! [1] https://x.com/nickevanjoseph/status/2056760504949842219 https://x.com/nickevanjoseph/status/2056760504949842219

deleted 5mo ago
[deleted]
triyambakam 5mo ago
I guess we must expect it at this point. But funny that has model written tokens like ’ instead of '
stingraycharles 5mo ago
Am I the only one who wasn’t particularly impressed by AutoResearch? If you looked at what the agent was actually doing, it was just tuning parameters mostly, not really trying different novel approaches. I couldn’t help myself but consider this mostly a very inefficient variant of hyperparameter optimization, but someone correct me if I’m wrong, I may be looking at this too pessimistic.
- lacker 5mo ago
  I didn't dig into what the actual repository was doing, but personally, I took some inspiration from the idea after reading about it and realizing that I might have been underestimating the ability of LLMs. I put a bit more work into a performance harness I was using locally and just set some agents to brainstorming and they did seem to find some great stuff. So I don't really have a stance one way or another on this specific repo, but the general idea seems like a really good one.
  - delis-thumbs-7e 5mo ago
    Could you elaborate in specifics how you had been underestimating models? Ypu mean just using more tighter harnessing to make them work in structured agentic eay or something else?
    - lacker 5mo ago
      The specific code I was working on, I had a general idea of the sort of performance improvement that would be possible. I just thought that it would be too hard for the models to figure out without a lot of hand-holding. But it ended up being not "too hard ever", but more like, in 1 out of every 5 tries, the model did in fact manage to get a large refactoring to the point where it improved performance. So once I set it up to try something, use the perf test, see if it worked, if not, throw it away, repeat. Then it started, slowly, finding some useful things.
    - inciampati 5mo ago
      Just remember that the will do clever but useless things to improve. Like changing the random seed as per autoresearch's hero image. lol! imo, out of the box thinking is needed.
- clbrmbr 5mo ago
  Karpathy embedded within an organization is way more impressive than him out on his own with hot takes and little projects. I hope he does great things for Anthropic.
4ashz 5mo ago
More like he'll blog and tweet about using Claude and get gullible software engineers to buy Claude subscriptions and work on their own obsolescence while paying for it. Many people are still deluded and think he is the same person who wrote the informal AI tutorials in plain html. He isn't, he is selling stuff now.
- bonoboTP 5mo ago
  What is he selling? How is this time different compared to when he was at OpenAI or at Tesla? You could say he was shilling those products too. I don't see any shift. He's still posted free in depth YouTube videos recently.
  - 23998h 5mo ago
    > What is he selling? Is that a serious question? He already promoted vibe coding and AI hype. Now he is literally there to promote Anthropic and its IPO price. When he was at OpenAI it wasn't overtly commercial yet. At Tesla he had a way lower profile. Now he is the vibe coding Jesus for deluded software engineers. The impact is much larger.
    - bonoboTP 5mo ago
      I think he's just genuinely excited about the capabilities. (I do understand that for Anthropic it's a brand boost as well, just like signing other prominent researchers, as it was with LeCun and Meta etc).
    - sho_hn 5mo ago
      > At Tesla he had a way lower profile. ? He was literally rolled out in front of camera as Tesla's AI prodigy at multiple streamed events designed to appeal to techy consumers and dev recruitment. He's definitely been one of AI's public personas for a long time now, and his employers have regularly aided/directed/utilized him accordingly.
  - deleted 5mo ago
    [deleted]
zmmmmm 5mo ago
So he's working on the singularity

ed_elliott_asc 5mo ago

Why do they need this when they have the next gen mythos? Surely that can manage everything?

cyanydeez 5mo ago
You don't understand: no ones ever reading more than 1% of the training material; so they need someone who has reduced that to 0.1%. The less you know, the more you know!

the_arun 5mo ago

This is good branding move for Anthropic. Karpathy is well respected among ML crowd.

zachncst 5mo ago
Minor celebrity fwiw - deserved though.
- fakedang 5mo ago
  He's a celebrity all right - promoting bunk just like the rest of that lot, along with his Lord and Savior Elon Musk.
wodenokoto 5mo ago
Speaking of, how did he not lose credibility at “full self driving next year, better buy it now”-Tesla? It might be Elon who went and said that and said they don’t need lidar, but as director of AI and auto vision Karpathy bears the responsibility for those features.
- joe_mamba 5mo ago
  >Speaking of, how did he not lose credibility at “full self driving next year, better buy it now”-Tesla? That I also want to know. He bailed out of Tesla right when the limitations of his "LIDAR-less cameras only self driving" system were becoming obvious, and nobody asked him about the hindsight of this obvious fuckup. >but as director of AI and auto vision Karpathy bears the responsibility for those features. Exactly. You lead the R&D, so it's on you. If your boss makes stupid decisions in public overriding your best judgement, the leave and go somewhere where your decisions be respected. The ML market was red hot for people like him back then so it's not like he didn't have alternatives. Although I doubt Elon forced that idea on him, since he's the one who was confidently claiming that vision only is better since Lidar pollutes the sensor fusion data.
  - kopirgan 5mo ago