4 ms·

Also, taking Elon's word at face value for a second... is Twitter really worth scraping for AI training or whatever? Its a hive of misinformation, disinformati

Also, taking Elon's word at face value for a second... is Twitter really worth scraping for AI training or whatever? Its a hive of misinformation, disinformation and toxicity. Its succinct I guess, but nothing is eloquent or descriptive because of the character limit. And its full of repetitive "filler" information. Who wants that in a foundational LLM dataset? Maybe its OK for finding labeled images... But that still seems kidna iffy.

kitsunesoba 3y ago
The effectiveness of this sort of lockdown is questionable anyway, because the cat's already out of the bag and there's no getting it back in. Same for Reddit. The bulk of the data's already out there and nothing these companies can do will change that.
afterburner 3y ago
Maybe someone is trying to make a disinformation bot. (half-serious) I mean as far as uses for LLMs go that seems to me a pretty realistic one. Mass quick propaganda with little effort. Go for immediate impact, doesn't matter if people look deeper, you're just looking to get a swell of emotional reactions.
- brucethemoose2 3y ago
  Yeah, I guess its a way to make an "engagement optimization" bot using follows/likes from posts as criteria. ... That is horrifying.
Hoasi 3y ago
> Also, taking Elon's word at face value for a second... is Twitter really worth scraping for AI training or whatever? Maybe... if you build a LLM scrapping for the lulz?
TillE 3y ago
It's useful if you want your LLM to be able to generate tweet-like microblogging text. That does have some value. Or maybe you want to get an aggregate idea of what people are currently talking about in the world, stuff that doesn't rise to the level of capital-n News. There aren't a lot of alternatives for that.
- brucethemoose2 3y ago
  Output formatting or a quick finetune/LORA can do microblogging very easily. Yeah, lots of general chat is unfortunately stuck in Twitter (or difficult -to-scrape siloed off platforms.
ben_w 3y ago
The thing that LLMs bring to the table isn't factual knowledge — we already had that, even some AI projects specifically dedicated to that — but rather the ability to correctly interact with natural language. Twitter is great for examples of that, and the toxicity and disinformation doesn't get in the way. Conversely, a training set doesn't need to be up to date to be useful for that. I don't know if anyone really was trying to scrape it (examples of Musk disagreeing with his own engineers come to mind), but I assume it's possible, and given the quality of code ChatGPT spits out I can easily believe a really bad scraper has been produced by someone who thought they could do without hiring a software developer. If so, they might think they can get hot stock tips or forewarning of a pandemic from which emoji people post or something — not really what an LLM is for, but loads of people (even here!) conflate all the different kinds of AI into one thing.
deleted 3y ago
[deleted]
epistasis 3y ago
While there may be huge sections of Twitter content that are like what you describe, I haven't encountered that. Instead I see tons of hyper-focused discussion from very specialized scientists that I wouldn't see otherwise. I see lots of discussion if obscure housing policy, that I wouldn't see otherwise. Now, this has been severely degraded by the changes that Musk has made. The spam in direct messages is off the charts now, whereas in the past I would get maybe a spam per year. And when one of my areas of interest has a post that gets popular, I have to scroll past all the insipid clout-chasing replies from blue check marks which get floated to the top of replies in an attempt to reward some of the worst people on the internet. Also the long form tweets that need to be expanded are a big deflation of user experience, as reading and replying to those are suboptimal compared to a tweet thread. But this is also the general internet: 99% spam plus 1% quality. And the quality of the 1% of good Twitter is some of the very best of timer material out there. And since LLMs have been trained on this same mix... they seem to be mostly good at filtering. But they do lie an awful lot.
- rvba 3y ago
  As someone who doesnt use twitter, I dont understand how can you have any sort of a real discussion with a 140 character limit. The best discussion platform is IMHO the older version of reddit / i.reddit with the nested comments + possibility to be indexed by google + possibility to reply to old posts. The super-nesting comments feature is great.
  - epistasis 3y ago
    It's a 280 character per message limit, with replies. This is actually hugely beneficial to discussion as it makes people focus on the most salient point first, and further points go below, and each are easy to address individually. Longer form material goes to outside links, sometimes, but Twitter threads are also great for long form content. At least for executive summaries that link out to the detailed bits for each primary point. Once the UI for Twitter prioritized threading, it became quite easy to express extremely long chains of evidence.
    - 3y ago
exo-pla-net 3y ago
Don't write Elon off. If your goal is to create a toxic misinformation bot, Twitter is indispensable.
muixoozie 3y ago
I once got paid $20 as an undergrad to go through hundreds of thousands of tweets and convert slang into plain english for training data. The only thing I took away from the experience, aside from finally getting good with vim macros, is the average tweet is really low effort an uninteresting. I don't recall reading a single thing that I would imagine someone retweeting (think that's what it's called). Maybe I was given only replies. Anyway, not sure if there's value there for LLMs, but I'd be skeptical.

kitsunesoba 3y ago

The effectiveness of this sort of lockdown is questionable anyway, because the cat's already out of the bag and there's no getting it back in. Same for Reddit. The bulk of the data's already out there and nothing these companies can do will change that.

afterburner 3y ago

Maybe someone is trying to make a disinformation bot. (half-serious) I mean as far as uses for LLMs go that seems to me a pretty realistic one. Mass quick propaganda with little effort. Go for immediate impact, doesn't matter if people look deeper, you're just looking to get a swell of emotional reactions.

brucethemoose2 3y ago
Yeah, I guess its a way to make an "engagement optimization" bot using follows/likes from posts as criteria. ... That is horrifying.

Hoasi 3y ago

> Also, taking Elon's word at face value for a second... is Twitter really worth scraping for AI training or whatever? Maybe... if you build a LLM scrapping for the lulz?

TillE 3y ago

It's useful if you want your LLM to be able to generate tweet-like microblogging text. That does have some value. Or maybe you want to get an aggregate idea of what people are currently talking about in the world, stuff that doesn't rise to the level of capital-n News. There aren't a lot of alternatives for that.

brucethemoose2 3y ago
Output formatting or a quick finetune/LORA can do microblogging very easily. Yeah, lots of general chat is unfortunately stuck in Twitter (or difficult -to-scrape siloed off platforms.

ben_w 3y ago

The thing that LLMs bring to the table isn't factual knowledge — we already had that, even some AI projects specifically dedicated to that — but rather the ability to correctly interact with natural language. Twitter is great for examples of that, and the toxicity and disinformation doesn't get in the way. Conversely, a training set doesn't need to be up to date to be useful for that. I don't know if anyone really was trying to scrape it (examples of Musk disagreeing with his own engineers come to mind), but I assume it's possible, and given the quality of code ChatGPT spits out I can easily believe a really bad scraper has been produced by someone who thought they could do without hiring a software developer. If so, they might think they can get hot stock tips or forewarning of a pandemic from which emoji people post or something — not really what an LLM is for, but loads of people (even here!) conflate all the different kinds of AI into one thing.

deleted 3y ago

[deleted]

epistasis 3y ago

While there may be huge sections of Twitter content that are like what you describe, I haven't encountered that. Instead I see tons of hyper-focused discussion from very specialized scientists that I wouldn't see otherwise. I see lots of discussion if obscure housing policy, that I wouldn't see otherwise. Now, this has been severely degraded by the changes that Musk has made. The spam in direct messages is off the charts now, whereas in the past I would get maybe a spam per year. And when one of my areas of interest has a post that gets popular, I have to scroll past all the insipid clout-chasing replies from blue check marks which get floated to the top of replies in an attempt to reward some of the worst people on the internet. Also the long form tweets that need to be expanded are a big deflation of user experience, as reading and replying to those are suboptimal compared to a tweet thread. But this is also the general internet: 99% spam plus 1% quality. And the quality of the 1% of good Twitter is some of the very best of timer material out there. And since LLMs have been trained on this same mix... they seem to be mostly good at filtering. But they do lie an awful lot.

rvba 3y ago
As someone who doesnt use twitter, I dont understand how can you have any sort of a real discussion with a 140 character limit. The best discussion platform is IMHO the older version of reddit / i.reddit with the nested comments + possibility to be indexed by google + possibility to reply to old posts. The super-nesting comments feature is great.
- epistasis 3y ago
  It's a 280 character per message limit, with replies. This is actually hugely beneficial to discussion as it makes people focus on the most salient point first, and further points go below, and each are easy to address individually. Longer form material goes to outside links, sometimes, but Twitter threads are also great for long form content. At least for executive summaries that link out to the detailed bits for each primary point. Once the UI for Twitter prioritized threading, it became quite easy to express extremely long chains of evidence.
  - 3y ago

exo-pla-net 3y ago

Don't write Elon off. If your goal is to create a toxic misinformation bot, Twitter is indispensable.

muixoozie 3y ago

I once got paid $20 as an undergrad to go through hundreds of thousands of tweets and convert slang into plain english for training data. The only thing I took away from the experience, aside from finally getting good with vim macros, is the average tweet is really low effort an uninteresting. I don't recall reading a single thing that I would imagine someone retweeting (think that's what it's called). Maybe I was given only replies. Anyway, not sure if there's value there for LLMs, but I'd be skeptical.