3 ms·

This article is uncannily apropos for HN. The spectrum of opinions towards emerging technologies is so starkly colored by the eras during which practitioners d

This article is uncannily apropos for HN. The spectrum of opinions towards emerging technologies is so starkly colored by the eras during which practitioners did their primary work. It reminds me of the Heinlein quote from “A Door into Summer”, “When railroading time comes you can railroad—but not before.”

wpietri 5y ago
For sure. The older I get, the more I try to watch out for this effect. It's a pain, but one of the nice parts is that it drives you to look for deeper principles. E.g., when my dad started coding, machine time was really expensive as was storage space, so everybody optimized for those things. Hello, Y2K! Now the opposite is true: developer time has gotten more expensive, and computation is cheap. In one sense, that invalidates a lot of old expertise. But on the other, optimization is still usually important, we're just optimizing for different things. A little reframing of habits and you can end up with a more flexible and subtle model.
- dougabug 5y ago
  We are in violent agreement! Optimization never grows old, but it’s the more general form of optimization of which you speak, rather than hyper-specific instances of optimization which can overfit to specific eras.
2sk21 5y ago
I was thinking about this in the context of large transformer based language models like GPT-3. Based on my years of experience in the field, I keep thinking that this is a dead-end on the path to full AI but I have to force myself to keep an open mind and to not to fall into that trap. Incidentally, Morgan Housel, the author of this essay is a very good financial writer - worth reading his other essays too.
- dougabug 5y ago
  Perhaps this was a cherry picked example on the part of the authors, but the network generating relevant novel images conditioned upon the text prompt “an illustration of a baby daikon radish in a tutu walking a dog” … made me take a (not entirely permanent) break from on ripping on GPT-3. https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/

wpietri 5y ago

For sure. The older I get, the more I try to watch out for this effect. It's a pain, but one of the nice parts is that it drives you to look for deeper principles. E.g., when my dad started coding, machine time was really expensive as was storage space, so everybody optimized for those things. Hello, Y2K! Now the opposite is true: developer time has gotten more expensive, and computation is cheap. In one sense, that invalidates a lot of old expertise. But on the other, optimization is still usually important, we're just optimizing for different things. A little reframing of habits and you can end up with a more flexible and subtle model.

dougabug 5y ago
We are in violent agreement! Optimization never grows old, but it’s the more general form of optimization of which you speak, rather than hyper-specific instances of optimization which can overfit to specific eras.

2sk21 5y ago

I was thinking about this in the context of large transformer based language models like GPT-3. Based on my years of experience in the field, I keep thinking that this is a dead-end on the path to full AI but I have to force myself to keep an open mind and to not to fall into that trap. Incidentally, Morgan Housel, the author of this essay is a very good financial writer - worth reading his other essays too.

dougabug 5y ago
Perhaps this was a cherry picked example on the part of the authors, but the network generating relevant novel images conditioned upon the text prompt “an illustration of a baby daikon radish in a tutu walking a dog” … made me take a (not entirely permanent) break from on ripping on GPT-3. https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/