2 ms·

> properly-licensed and verified datasets is immensely more difficult, expensive Arguably the bigger problem is that many of those datasets e.g. WSJ articles a

> properly-licensed and verified datasets is immensely more difficult, expensive Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. So we end up with in a situation where competition is simply not possible.

piva00 2y ago
> Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. > So we end up with in a situation where competition is simply not possible. Exactly, and Technofeudalism advances a little more into a new feud. OpenAI is trying to create its moat by shoring up training data, probably attempting to not allow competitors to train on the same datasets they've been licencing, at least for a while. Training data is the only possible moat for LLMs, models seem to be advancing quite well between different companies but as mentioned here a tidy training dataset is the actual gold.
- gessha 2y ago
  It's landmines no matter how you approach the problem. If you treat the web as a free-for-all and you scrape freely, you get sued by the content platforms for copyright or term of service violation. If you license the content, you let the highest bidder get the content. No matter what happens, capital wins.
motohagiography 2y ago
the irony is that if large media providers aren't represented in the training sets, my comments on internet forums over the decades will be over-represented, which is kind of great, really.
- pennomi 2y ago
  Right? I always ask people - Hypothetically if someone created a superintelligent AI that took over the world, wouldn’t you WANT it to share your opinions and morals? Every tiny bit of text you write is a vote in the election of our future AI overlords.

piva00 2y ago

> Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. > So we end up with in a situation where competition is simply not possible. Exactly, and Technofeudalism advances a little more into a new feud. OpenAI is trying to create its moat by shoring up training data, probably attempting to not allow competitors to train on the same datasets they've been licencing, at least for a while. Training data is the only possible moat for LLMs, models seem to be advancing quite well between different companies but as mentioned here a tidy training dataset is the actual gold.

gessha 2y ago
It's landmines no matter how you approach the problem. If you treat the web as a free-for-all and you scrape freely, you get sued by the content platforms for copyright or term of service violation. If you license the content, you let the highest bidder get the content. No matter what happens, capital wins.

motohagiography 2y ago

the irony is that if large media providers aren't represented in the training sets, my comments on internet forums over the decades will be over-represented, which is kind of great, really.

pennomi 2y ago
Right? I always ask people - Hypothetically if someone created a superintelligent AI that took over the world, wouldn’t you WANT it to share your opinions and morals? Every tiny bit of text you write is a vote in the election of our future AI overlords.