3 ms·

Plenty of shortsighted people have done things that are stupid in the long term for short term gains. It's the modus operandi of the US economy since 1981. I

Plenty of shortsighted people have done things that are stupid in the long term for short term gains. It's the modus operandi of the US economy since 1981. I don't believe that the people who train models have a secret way of identifying and filtering out bot-generated content that no one else (email spam filters, search engines, etc.) have identified. I do believe that they feel their models need to have up-to-date information on a variety of topics that require regularly ingesting new data. So no, I don't think they have a good way to avoid their inputs rotting from their outputs.

eru 7mo ago
What's so special about 1981? And what do you mean about short term gains? If you are training a model, and you see model collapse, where's the short term gain? I don't get it. The incentive for the person who trains a model, even in the short run, is for them to avoid model collapse. > I don't believe that the people who train models have a secret way of identifying and filtering out bot-generated content that no one else (email spam filters, search engines, etc.) have identified. Huh, why would they need a secret filter? A filter is only one way thing you can try. You can also look into using different models, different training, making your approach more resistant to model collapse; training multi-modal models, using approaches to economise on training data; and thousands of other ideas I can't think of in thirty seconds. > So no, I don't think they have a good way to avoid their inputs rotting from their outputs. You lack imagination. People can be remarkably clever if its in their (short term!) interest to find solutions.
- Arainach 7mo ago
  > What's so special about 1981? Two very significant things happened in 1981. After years of claiming the government couldn't help people, Ronald Reagan was elected and Republicans have been working hard to make that statement more true ever since. A big part of that was deregulation of the financial markets. That same year, Jim Welch became Chairman and CEO of General Electric. He juiced the stock prices by selling off the company's prize jewels, real estate, and future, and for a while (before the utter collapse of the company) artificially raised the stock price so high that executives around the country copied him, and and an entire industry of vultures like Mitt Romney started private equity firms to cannibalize healthy companies for their personal profit.

eru 7mo ago

What's so special about 1981? And what do you mean about short term gains? If you are training a model, and you see model collapse, where's the short term gain? I don't get it. The incentive for the person who trains a model, even in the short run, is for them to avoid model collapse. > I don't believe that the people who train models have a secret way of identifying and filtering out bot-generated content that no one else (email spam filters, search engines, etc.) have identified. Huh, why would they need a secret filter? A filter is only one way thing you can try. You can also look into using different models, different training, making your approach more resistant to model collapse; training multi-modal models, using approaches to economise on training data; and thousands of other ideas I can't think of in thirty seconds. > So no, I don't think they have a good way to avoid their inputs rotting from their outputs. You lack imagination. People can be remarkably clever if its in their (short term!) interest to find solutions.

Arainach 7mo ago
> What's so special about 1981? Two very significant things happened in 1981. After years of claiming the government couldn't help people, Ronald Reagan was elected and Republicans have been working hard to make that statement more true ever since. A big part of that was deregulation of the financial markets. That same year, Jim Welch became Chairman and CEO of General Electric. He juiced the stock prices by selling off the company's prize jewels, real estate, and future, and for a while (before the utter collapse of the company) artificially raised the stock price so high that executives around the country copied him, and and an entire industry of vultures like Mitt Romney started private equity firms to cannibalize healthy companies for their personal profit.