4 ms·

Of course superhuman intelligence might be dismissed as outlandish science fiction technology which is far away. But three years ago ChatGPT-4 or Midjourney V5

Of course superhuman intelligence might be dismissed as outlandish science fiction technology which is far away. But three years ago ChatGPT-4 or Midjourney V5 were also outlandish science fiction technology. I actually really question this, was it truly outlandish to imagine what we have now? We had Google, Stackoverflow etc, yes there new elements to it, the data was there and accessible, but I don't think LLMs are unimaginable?

stevenhuang 4y ago
It's not wrong to say current LLM capability were indeed unimaginable back then. A lot about the transformer model was driven by experimentation, there was no underlying theory per se that said we'd get as good results as we did. Everyone and the researchers themselves were shocked iirc; all this higher order reasoning emerged from being a next token predictor, yes that was the outlandish part.
- gitfan86 4y ago
  Right, general intelligence has been here all along. Humans are the ones that are specialized. Pylomeic mistake 2.0
cubefox 4y ago
If we are honest and look at our memories of three years ago, there was nothing remotely like the current technology. There were pattern recognizers and AlphaGo, but nothing which looked remotely generally intelligent in the way ChatGPT does. Very few people even knew about GPT-2, as it was easily dismissed as just creating an surface illusion of meaningful text, not much different from old text generation approaches from a few decades ago. There were GANs which e.g. produced realistic faces, but these were restricted to one subject matter only, not something like Midjourney V5 which makes basically every picture whatsoever based on a text description, just with occasionally one to many limb. People were so impressed with Dall-E 2 that it is hard to imagine how extremely impressed they would have been with Midjourney V5 (without having seen Dall-E 2 first). If you told HN commenter three years ago (for context, that was when COVID took off) that we would very soon have technologies like ChatGPT-4 then, I'm sure, that would not have been taken seriously.

stevenhuang 4y ago

It's not wrong to say current LLM capability were indeed unimaginable back then. A lot about the transformer model was driven by experimentation, there was no underlying theory per se that said we'd get as good results as we did. Everyone and the researchers themselves were shocked iirc; all this higher order reasoning emerged from being a next token predictor, yes that was the outlandish part.

gitfan86 4y ago
Right, general intelligence has been here all along. Humans are the ones that are specialized. Pylomeic mistake 2.0

cubefox 4y ago

If we are honest and look at our memories of three years ago, there was nothing remotely like the current technology. There were pattern recognizers and AlphaGo, but nothing which looked remotely generally intelligent in the way ChatGPT does. Very few people even knew about GPT-2, as it was easily dismissed as just creating an surface illusion of meaningful text, not much different from old text generation approaches from a few decades ago. There were GANs which e.g. produced realistic faces, but these were restricted to one subject matter only, not something like Midjourney V5 which makes basically every picture whatsoever based on a text description, just with occasionally one to many limb. People were so impressed with Dall-E 2 that it is hard to imagine how extremely impressed they would have been with Midjourney V5 (without having seen Dall-E 2 first). If you told HN commenter three years ago (for context, that was when COVID took off) that we would very soon have technologies like ChatGPT-4 then, I'm sure, that would not have been taken seriously.