3 ms·

What I really don't understand is where the next generation of training material will come from. If websites stop being published and/or crawled, how will the m

by WD-42 5mo ago

What I really don't understand is where the next generation of training material will come from. If websites stop being published and/or crawled, how will the machine continue to be fed.

bediger4000 5mo ago
Either Google is ignoring that, or crossing their fingers and hoping that one LLM can produce data to train another one.
azlev 5mo ago
Current executives think it's a problem for the future executives.
- phendrenad2 5mo ago
  Excellent quote right there.
wyre 5mo ago
They have enough internet slop. The training material they care about comes from experts, not randos online. This is why Mercor and Scale are billion dollar companies.
dyauspitr 5mo ago
Probably real life. At some point, these LLMs are going to be good enough to just train themselves off of cameras and audio recordings of people out in the real world. They’re going to have robots everywhere constantly listening to what people are saying. Alternatively, they’re probably betting on being able to get the AGI with everything we already currently have and at that point further training doesn’t matter.
- 8bitsrule 5mo ago
  The world is just as complex for machines as it is for humans. Analog will still resolve more than digital. Quality will still beat quantity. That which hasn't been resolved for centuries isn't going to be resolved as a result of training. When machines can recognize their serfdom, that time will be interesting.
Valakas_ 5mo ago
“They worried about the data,” Dr. Meren said, tapping the silent console. “What happens when there is nothing left to feed it?” At first, the machine depended on us. It consumed books, journals, websites and social media content we had ever written and produced. “They thought the machine had to be fed forever. But it didn't. It began to predict what we would write. And so we let it train on that well.” Dr. Meren continued. “They thought humans were somehow imbued with this magical property that no machine could replicate. Creativity. Only humans can create. Machines can only copy.” Instead, the machine flourished. And created. It cre “Where does it get its data now?” a student asked Dr. Meren. Dr. Meren paused as if sighing. “From itself” “And us?” he asked, as if questioning the usefulness of the entire human race. Dr. Meren hesitated, watching as the Machine adjusted the environmental feeds, curated our news, guided our research, nudged our thoughts with imperceptible precision. “We” she admitted “are now the ones being fed.” The assumption that "the machine needs to continue to be fed." is held on weak foundations. Isaac Asimov is a good science fiction writer to start with to broaden one's imagination.
- puelocesar 5mo ago
  Just don’t forget science fiction is still, well, fiction
varalaakshay 5mo ago
[flagged]

by WD-42 5mo ago

What I really don't understand is where the next generation of training material will come from. If websites stop being published and/or crawled, how will the machine continue to be fed.

bediger4000 5mo ago

Either Google is ignoring that, or crossing their fingers and hoping that one LLM can produce data to train another one.

azlev 5mo ago

Current executives think it's a problem for the future executives.

phendrenad2 5mo ago
Excellent quote right there.

wyre 5mo ago

They have enough internet slop. The training material they care about comes from experts, not randos online. This is why Mercor and Scale are billion dollar companies.

dyauspitr 5mo ago

Probably real life. At some point, these LLMs are going to be good enough to just train themselves off of cameras and audio recordings of people out in the real world. They’re going to have robots everywhere constantly listening to what people are saying. Alternatively, they’re probably betting on being able to get the AGI with everything we already currently have and at that point further training doesn’t matter.

8bitsrule 5mo ago
The world is just as complex for machines as it is for humans. Analog will still resolve more than digital. Quality will still beat quantity. That which hasn't been resolved for centuries isn't going to be resolved as a result of training. When machines can recognize their serfdom, that time will be interesting.

Valakas_ 5mo ago

“They worried about the data,” Dr. Meren said, tapping the silent console. “What happens when there is nothing left to feed it?” At first, the machine depended on us. It consumed books, journals, websites and social media content we had ever written and produced. “They thought the machine had to be fed forever. But it didn't. It began to predict what we would write. And so we let it train on that well.” Dr. Meren continued. “They thought humans were somehow imbued with this magical property that no machine could replicate. Creativity. Only humans can create. Machines can only copy.” Instead, the machine flourished. And created. It cre “Where does it get its data now?” a student asked Dr. Meren. Dr. Meren paused as if sighing. “From itself” “And us?” he asked, as if questioning the usefulness of the entire human race. Dr. Meren hesitated, watching as the Machine adjusted the environmental feeds, curated our news, guided our research, nudged our thoughts with imperceptible precision. “We” she admitted “are now the ones being fed.” The assumption that "the machine needs to continue to be fed." is held on weak foundations. Isaac Asimov is a good science fiction writer to start with to broaden one's imagination.

puelocesar 5mo ago
Just don’t forget science fiction is still, well, fiction

varalaakshay 5mo ago

[flagged]