3 ms·

As always Dwarkesh gives the best interview in tech. The whole episode is worth a listen, but some of the highlights for me were: Learning that an earlier Opus

As always Dwarkesh gives the best interview in tech. The whole episode is worth a listen, but some of the highlights for me were: Learning that an earlier Opus was highly committed to animal welfare and they’re not sure why; other versions and models have not been Hearing that their mechanistic interpretability groups recently beat an internal red team quickly, using Claude’s help Speculation on de-risking AI growth for nation states that aren’t building frontier models Real real profound comment near the end noting that even if we get no more algorithmic improvements ever, current tech is sufficient to replace most white collar jobs, given enough data. That last one is a super profound and interesting point to me. We’re at the inflection point where white collar jobs will at the very least be implemented by this tech and overseen by humans, and the current tech and economics make it desirable to do so for companies that pay for a lot of white collar work. Feels like it’s time to buckle up. Not AI2027 buckle up, but still time to buckle up.

mritchie712 1y ago
> an earlier Opus was highly committed to animal welfare and they’re not sure why; other versions and models have not been I caught that too, very interesting. Many people treat their pets as part of their family. Many people make statements online like: "I like dogs / cats more than most people". So it's not surprising that this sentiment would be picked up in training data. I'd wager animals on average are thought of more highly on a site like Reddit than humans are. i.e. the average sentiment of a post on cats is more positive than a post about a human. Cats, in the training data, may be perceived as "more important" or "better" than humans. It is surprising Opus holds on to this tighter than other models.
- vessenes 1y ago
  The MechInterp guys talk a lot about concept overloading. I wonder if “kind to animals” randomly overlaid with “helpful and not harmful” concepts during training and they got locked in together. All highly speculative obv.
SubiculumCode 1y ago
I recently asked an AI to write a scientific review for me via the deep research feature on a field of study with which I have some familiarity. While the review did not reveal new deep insights, the review was accurate (!!) and cross-cutting, well written, as requested, only sourced from pubmed, and the review was perfectly styled in the form I wanted...writing this would have taken me a couple of days. I don't know how I feel about it. Writing basic literature reviews is tedious, on the other hand, it is approaching the line where I can feel the temptation to let it work while I sip a cocktail. At what point will I be replaced by the open science literature we made?
- vessenes 1y ago
  Think of it like this - you just got a research team added to your group. I’d stay on top of tasking them though, because soon you’ll get a research manager added, and you’ll work through it. But you’ll still occasionally need to dig in and see how that manager got it wrong.
barrenko 1y ago
We are going to be entering a hockey-stick-graph level of explosion of intelligence (well knowledge), and we're gonna be in that spot for so much time it will feel like an event horizon.

mritchie712 1y ago

> an earlier Opus was highly committed to animal welfare and they’re not sure why; other versions and models have not been I caught that too, very interesting. Many people treat their pets as part of their family. Many people make statements online like: "I like dogs / cats more than most people". So it's not surprising that this sentiment would be picked up in training data. I'd wager animals on average are thought of more highly on a site like Reddit than humans are. i.e. the average sentiment of a post on cats is more positive than a post about a human. Cats, in the training data, may be perceived as "more important" or "better" than humans. It is surprising Opus holds on to this tighter than other models.

vessenes 1y ago
The MechInterp guys talk a lot about concept overloading. I wonder if “kind to animals” randomly overlaid with “helpful and not harmful” concepts during training and they got locked in together. All highly speculative obv.

SubiculumCode 1y ago

I recently asked an AI to write a scientific review for me via the deep research feature on a field of study with which I have some familiarity. While the review did not reveal new deep insights, the review was accurate (!!) and cross-cutting, well written, as requested, only sourced from pubmed, and the review was perfectly styled in the form I wanted...writing this would have taken me a couple of days. I don't know how I feel about it. Writing basic literature reviews is tedious, on the other hand, it is approaching the line where I can feel the temptation to let it work while I sip a cocktail. At what point will I be replaced by the open science literature we made?

vessenes 1y ago
Think of it like this - you just got a research team added to your group. I’d stay on top of tasking them though, because soon you’ll get a research manager added, and you’ll work through it. But you’ll still occasionally need to dig in and see how that manager got it wrong.

barrenko 1y ago

We are going to be entering a hockey-stick-graph level of explosion of intelligence (well knowledge), and we're gonna be in that spot for so much time it will feel like an event horizon.