4 ms·

Information decay has started around the year 2000. I'm not a Hipster, I have experienced the internet pre 2000, which was an semi exclusive club for educated a

Information decay has started around the year 2000. I'm not a Hipster, I have experienced the internet pre 2000, which was an semi exclusive club for educated and intelligent people with accurate search results. After the masses poured onto it the results became more half truths and opinions. And now you have "AI" feeding on those half truths and opinions. Whatever could go wrong? However imagine if it was fed with facts only.

jvanderbot 4y ago
Well, you can't train LLM on "facts alone", since they are basically throwing darts at the next word, not using information to craft a message. But I hear your larger point: it'd be nice to have an agent that was trained on wikipedia information, but already spoke a language
- r3trohack3r 4y ago
  GPT is trained on Wikipedia though. It’s speculated libgen is in there as well.
  - artificial 4y ago
    Let’s not forget Twitter was a training source. Early Twitter was like 4chan.
  - jvanderbot 4y ago
    I think GGP comment started this thread about a knowledge engine, not a "word generator" if that makes sense. Training on Wikipedia in this case means "training to put together words like wikipedia authors do" right? Not "learning the facts of wikipedia to provide useful synthesis and cross referencing." though it probably does OK by accident.
ethbr0 4y ago
Imho, info decay took off when social metastasized, which is to say ~2010+. It was an interesting time, because (from memory) 2000 was still "It's pretty complicated for average person to create and post content on the web" and transitioned to 2010's "There are highly polished, easy to use, free SaaS solutions that a trained dog could use." With mobile versions of same in the next few years. (All dates mass market penetration, not first release) There was a direct correlation between ease-of-publishing and less intellectual discourse on the web. "Info decay" was definitely a good way to put it. It wasn't that people were necessarily dumb or angry, but that there was a lot more honestly incorrect information online. Before, there were crackpots, but it was pretty easy to tell they were crazy. Having seen both, the banal wrongness is far worse. I think that's one reason wikipedia really took off ~2005+. People were craving an easier way to get even roughly-right information.

jvanderbot 4y ago

Well, you can't train LLM on "facts alone", since they are basically throwing darts at the next word, not using information to craft a message. But I hear your larger point: it'd be nice to have an agent that was trained on wikipedia information, but already spoke a language

r3trohack3r 4y ago
GPT is trained on Wikipedia though. It’s speculated libgen is in there as well.
- artificial 4y ago
  Let’s not forget Twitter was a training source. Early Twitter was like 4chan.
- jvanderbot 4y ago
  I think GGP comment started this thread about a knowledge engine, not a "word generator" if that makes sense. Training on Wikipedia in this case means "training to put together words like wikipedia authors do" right? Not "learning the facts of wikipedia to provide useful synthesis and cross referencing." though it probably does OK by accident.

ethbr0 4y ago

Imho, info decay took off when social metastasized, which is to say ~2010+. It was an interesting time, because (from memory) 2000 was still "It's pretty complicated for average person to create and post content on the web" and transitioned to 2010's "There are highly polished, easy to use, free SaaS solutions that a trained dog could use." With mobile versions of same in the next few years. (All dates mass market penetration, not first release) There was a direct correlation between ease-of-publishing and less intellectual discourse on the web. "Info decay" was definitely a good way to put it. It wasn't that people were necessarily dumb or angry, but that there was a lot more honestly incorrect information online. Before, there were crackpots, but it was pretty easy to tell they were crazy. Having seen both, the banal wrongness is far worse. I think that's one reason wikipedia really took off ~2005+. People were craving an easier way to get even roughly-right information.