2 ms·

> LLMs are also getting to be pretty old. BERT and GPT-2 are both more than 5 years old You sound like you have no clue that five years are the blink of an eye

> LLMs are also getting to be pretty old. BERT and GPT-2 are both more than 5 years old You sound like you have no clue that five years are the blink of an eye.

talldayo 2y ago
I'm someone that used both BERT and GPT-2 5 years ago, and am incredibly disappointed by how little progress the industry has made in that time. GPT-J and GPT-NEO are still arguably competitive with the latest AI models you can use today; that's sad. Cryptocurrency apologists also used this line of logic. "Yes, today the value of crypto is nothing... but imagine where it will be in five years!" Then we all wait 5 years, and cryptocurrency is still the victim of it's own mindset. Like cryptocurrency, I think LLMs have "technically" solved what they set out to do; generate readable text. But you need more than a solution in search of a problem; there isn't really that much demand for marginally truthful text in the same way completely decentralized currency isn't really necessary for the average well-meaning civilian. I'd even go further and argue that introducing AI into our daily lives requires you to replace something else, something that was probably more accurate and better-designed than the AI replacing it. If 5 years is a "blink of an eye" for the industry, the entire field will be dead within 18 months. VC just moves that fast.
- gaganyaan 2y ago
  "Arguably competitive" is just plain wrong. I know it's fun to be dismissive of things, but making silly claims doesn't help comments like this read seriously.
  - talldayo 2y ago
    I would legitimately argue they are competitive. GPT-J and GPT-Neo are both fundamentally quite similar to modern LLMs, and when compared against other 4b and 8b parameter models I would genuinely champion the quality of their responses over their contemporaries.
    - textninja 2y ago
      > when compared against other 4b and 8b parameter models I would genuinely champion the quality of their responses You clearly have some very specific models in mind. Even if the latest 4B and 8B models don’t move the needle on the “results you would champion” metric, this does not advance your argument that the state of the art hasn’t significantly progressed from 5 years ago. > I would legitimately argue I’ll bet you would!
- throw310822 2y ago
  > generate readable text Because this to you is "generate readable text"? https://m.youtube.com/watch?v=MirzFk_DSiI https://m.youtube.com/watch?v=MirzFk_DSiI Sorry to be so direct, but you're in denial.
  - talldayo 2y ago
    That is a party trick, that Hatsune Miku does every year for thousands of loving fans. The fact that an American company figured out vocaloid is not making any nerds lose their minds. There's a reason why GPT-4o is not taking over YouTube and social media with it's incredible capabilities; nobody cares.

> LLMs are also getting to be pretty old. BERT and GPT-2 are both more than 5 years old You sound like you have no clue that five years are the blink of an eye.

talldayo 2y ago

I'm someone that used both BERT and GPT-2 5 years ago, and am incredibly disappointed by how little progress the industry has made in that time. GPT-J and GPT-NEO are still arguably competitive with the latest AI models you can use today; that's sad. Cryptocurrency apologists also used this line of logic. "Yes, today the value of crypto is nothing... but imagine where it will be in five years!" Then we all wait 5 years, and cryptocurrency is still the victim of it's own mindset. Like cryptocurrency, I think LLMs have "technically" solved what they set out to do; generate readable text. But you need more than a solution in search of a problem; there isn't really that much demand for marginally truthful text in the same way completely decentralized currency isn't really necessary for the average well-meaning civilian. I'd even go further and argue that introducing AI into our daily lives requires you to replace something else, something that was probably more accurate and better-designed than the AI replacing it. If 5 years is a "blink of an eye" for the industry, the entire field will be dead within 18 months. VC just moves that fast.

gaganyaan 2y ago
"Arguably competitive" is just plain wrong. I know it's fun to be dismissive of things, but making silly claims doesn't help comments like this read seriously.
- talldayo 2y ago
  I would legitimately argue they are competitive. GPT-J and GPT-Neo are both fundamentally quite similar to modern LLMs, and when compared against other 4b and 8b parameter models I would genuinely champion the quality of their responses over their contemporaries.
  - textninja 2y ago
    > when compared against other 4b and 8b parameter models I would genuinely champion the quality of their responses You clearly have some very specific models in mind. Even if the latest 4B and 8B models don’t move the needle on the “results you would champion” metric, this does not advance your argument that the state of the art hasn’t significantly progressed from 5 years ago. > I would legitimately argue I’ll bet you would!
throw310822 2y ago
> generate readable text Because this to you is "generate readable text"? https://m.youtube.com/watch?v=MirzFk_DSiI https://m.youtube.com/watch?v=MirzFk_DSiI Sorry to be so direct, but you're in denial.
- talldayo 2y ago
  That is a party trick, that Hatsune Miku does every year for thousands of loving fans. The fact that an American company figured out vocaloid is not making any nerds lose their minds. There's a reason why GPT-4o is not taking over YouTube and social media with it's incredible capabilities; nobody cares.