2 ms·
(a) makes little sense to me. All the pristine data, i.e. Reddit up to late 2022, is already packaged up, mirrored, and available to download. Everything afterw
by TeMPOraL 3y ago
(a) makes little sense to me. All the pristine data, i.e. Reddit up to late 2022, is already packaged up, mirrored, and available to download. Everything afterwards - and especially everything from the last two months onward - is increasingly becoming adulterated with LLM-generated content, which so far is proving poisonous to LLMs when fed to them in training. In other words: the widespread access to LLMs is rapidly destroying most of the ML training value Reddit data could offer.
- throwaway9274 3y agoThe theory would be stop the bleeding and try to clawback the old data with copyright enforcement against hosts of datasets and ML companies. Get the ML companies to license the content to settle the litigation, and include all new data as part of the deal. Then offer a similar “training only” content license to all other ML companies. Simultaneously you have to cut down on LLM-generated content, which may prove impossible. Where they botched it was the execution. It should have started with community engagement strategy: (a) buy out the 3P app developers, (b) fix the official app, and (c) offer low cost API access for the community by application, (d) shower mods and content creators with honorifics and if necessary rev share on their subs.