13 ms·

Update on Twitter's Rate Limits

52 points by tech234a 3y ago

comprev 3y ago
TL;DR - no change
WheatMillington 3y ago
This justification makes zero sense to me. Why would they need to use rate limits to detect bots? They can trivially observe account behaviour without using rate limits.
- toomim 3y ago
  Turing test.
- tvbusy 3y ago
  It's corporate talk for "we f up and systems are failing". Musk's act of not paying people now bites him back. I guess they don't have enough servers to sustain the service.
  - immibis 3y ago
    The new "you need to log in to view this tweet" page has a bug that causes it to repeatedly try to load the tweet, about 10 times per second. This explains the increased server load.
- NovaDudely 3y ago
  About the only thing I could think of is, limit the rate and see what accounts keep trying to post even though they fail? But there would be so many false positives on that idea. Would punish many that are legitimate users and probably harden those that have more sophisticated posting techniques. Like you said, there are better ways.
- shanks512 3y ago
  > Why would they need to use rate limits to detect bots? Accounts that hit rate limits are more likely to be bots. Any ideas on how to do this more trivially than toggling a global rate limit flag?
  - somethingreen 3y ago
    The post itself says that publicly announcing the policy in advance would be detrimental. The policy is now public and they clearly notify the offender of violation. It probably took bot maintainers less than an hour to update their scripts to pause attempts/switch accounts on first failed request. It's the humans who keep trying. Any heuristic they'd want to apply to detect bots would be applied to action history, tuned to acceptable false positive rates and silently rolled out. This circus doesn't align with the statements at all.
- rsynnott 3y ago
  I'm pretty sure this is provided solely for the benefit of Musk's weird fans, who can now go "see? Dear Leader broke the site as part of the 4D chess!" Normal people probably shouldn't take much notice of it.
choeger 3y ago
Twitter is the first social Network to publicly commit ritual suicide. It's somewhat fascinating to watch. Would be very interesting to see how much the login wall affected their traffic and thus ad revenues. But the real damage will be visible in Q4 when ads campaigns simply won't be booked anymore for Twitter because everyone begins to write it off.
- tropicalbeach 3y ago
  Why Twitter is better now than before. I used to never use it and now i use it more than any other platform. Don't really get why people are hating so much. Finally there is somewhere where people can post without being banned or having posts removed. Why do you want filtered social media it's fake.
  - HlessClaudesman 3y ago
    Nice try Elon.
    - tropicalbeach 3y ago
      Genuine question you like your social media filtered by someone who says what you're allowed to see?
      - dale_glass 3y ago
        Genuine answer: Yes. If I wanted an unfiltered experience, there's 4chan and the like. Notice that I'm not there, but somewhere I don't have to wade through mountains of crap. HN is very much moderated, that's what keeps it useful.
      - threeseed 3y ago
        There have been a large number of accounts that have been censored by Twitter. Musk even admitted doing so at the behest of foreign governments something Twitter 1.0 used to aggressively fight and prevent from happening.
      - 3y ago
williamsmj 3y ago
This is content-free gibberish, but a particular highlight for me is the promise that "we will provide an update when the work is complete" that is immediately followed by "this work will never be done".
raverbashing 3y ago
At this point I'm taking Elon to the 'politician' levels of lie detection Where "verified" means you paid $8 or something, I see no reason to trust pretty much anything else coming from them
etchalon 3y ago
“Bill in IT says we need to reboot the router but no one knows the router password and Comcast says the password is on the router but Bill says Jim must have torn off the sticker so we called Best Buy and the soonest the Geek Squad can get here is Thursday. We tried unplugging it and plugging it back in but it’s not turning on.”
wg0 3y ago
It feels like that the above comes directly written by His Royal Highness, just copy pasted in place by the site admins.
deleted 3y ago
[deleted]
ethcat 3y ago
To be honest, it doesn't bother me because I've moved to other platforms. The main reason I log on to twitter is to see if anyone I follow has moved to another platform.
resolutebat 3y ago
> To ensure the authenticity of our user base we must take extreme measures to remove spam and bots from our platform. That’s why we temporarily limited usage so we could detect and eliminate bots and other bad actors that are harming the platform. Any advance notice on these actions would have allowed bad actors to alter their behavior to evade detection. Ah, so they're intentionally making everybody suffer to punish the bad guys, and promising that everything will be peachy keen afterwards. This sounds essentially identical to the Indian government's justification for suddenly demonetizing the 500 and 1000 rupee notes in 2016, leading to massive cash shortages. This directly led to 1.5M people losing their jobs, a stock market crash, and GDP dropping by at least 1%. The silver lining was that it also kick-started the Indian digital payments system.
threeseed 3y ago
This looks to be complete nonsense. Scraping services work by going through thousands of intermediate proxies run on unsuspecting people's compromised computers. Each scraping request will be a unique IP with a normal looking header. They may be able to look at the number of unique requests per account but given the ridiculous amount of spam/bots that exist clearly creating new accounts isn't an impediment. The far more likely scenario is that they were throttled by AWS/GCP for not paying bills and are behind on their migration project.
- supsep2 3y ago
  So all scrapers are viruses? Need to me.
  - DistractionRect 3y ago
    Almost all "residential" proxies are run without the knowledge of the user used as the proxy. Either it's slipped in the ToS/install of other legitimate product or straight up malware. A popular method, iirc, was bundling with free VPNs. Last I looked into the discussion, the consensus was there was no particularly reputable residential proxy service. Edit: I probably should explain the distinction for residential proxies. Basically, if you run a site that isn't B2B, you'll find basically no organic traffic comes from data centers/server farms. So traffic from IP blocks like owned by Microsoft (Azure), Amazon (AWS), Google (GCP), etc are all highly likely to be bot traffic and throttled or outright blocked by default. If you're running a scraper or a bot, you want to get your traffic to appear to come from residential IP blocks (owned by residential ISPs).
nmstoker 3y ago
Strange they should mention spam when these last few days I've seen a steady stream of near identical advertisers appear in spam-like ads. The advertisers have very similar profile pictures which look generated, use a particular colour palette, and all sell the same sort of unwanted junk. Despite a mix of blocking/muting, a new one pops up every few hours!
dathinab 3y ago
> Currently, the restrictions affect a small percentage of people using the platform, and we will provide an update when the work is complete. are they sure, it seem to have hit literally everyone I know who still uses twitter frequently
- ignoramous 3y ago
  This is a CMA post. Nothing much to see other than mea culpa and the palpable disappointment of other (not their own) AI bots winning.
drumhead 3y ago
So I'm assuming this is because he thinks the data is so valuable for LLM training he doesn't want anyone getting it for free. Does he really think there will be that much demand for it that it outweighs the fundamental purpose of Twitter?
- Balgair 3y ago
  As I'm sure you've probably guessed, it's not about the bots. It's that his contract with Google for servers ran out on July 1st. So now they have to pay a lot more. Absurdly, this is a cost saving measure. And yes, I've no idea how it ends up saving either.
nirui 3y ago
> To ensure the authenticity of our user base we must take extreme measures to remove spam and bots from our platform. Well about that, I noticed heavy inflow of spam and bots traffic right after Mr Musk starts selling the blue patch for money. These spammers purchased and utilized the "first-class citizen" privilege to bend the display priority to their favor. Now days, you open a Tweet, the top replies are almost all spams. You know, at this point, it is obviously apparent that the spammers knows better about Twitter than Mr Musk do. But all and all, this (as well as the recent events related to Reddit) is a good thing. Because it serves a good reminder that we (the users) are products, sometimes the self-paying type. So maybe next time you when plan to spend some chill time on some services, think how much you can get out if it for exchange.
DaSHacka 3y ago
As the twitter users would say, #TwitterIsOverParty

52 points by tech234a 3y ago

TL;DR - no change

This justification makes zero sense to me. Why would they need to use rate limits to detect bots? They can trivially observe account behaviour without using rate limits.

toomim 3y ago
Turing test.
tvbusy 3y ago
It's corporate talk for "we f up and systems are failing". Musk's act of not paying people now bites him back. I guess they don't have enough servers to sustain the service.
- immibis 3y ago
  The new "you need to log in to view this tweet" page has a bug that causes it to repeatedly try to load the tweet, about 10 times per second. This explains the increased server load.
NovaDudely 3y ago
About the only thing I could think of is, limit the rate and see what accounts keep trying to post even though they fail? But there would be so many false positives on that idea. Would punish many that are legitimate users and probably harden those that have more sophisticated posting techniques. Like you said, there are better ways.
shanks512 3y ago
> Why would they need to use rate limits to detect bots? Accounts that hit rate limits are more likely to be bots. Any ideas on how to do this more trivially than toggling a global rate limit flag?
- somethingreen 3y ago
  The post itself says that publicly announcing the policy in advance would be detrimental. The policy is now public and they clearly notify the offender of violation. It probably took bot maintainers less than an hour to update their scripts to pause attempts/switch accounts on first failed request. It's the humans who keep trying. Any heuristic they'd want to apply to detect bots would be applied to action history, tuned to acceptable false positive rates and silently rolled out. This circus doesn't align with the statements at all.
rsynnott 3y ago
I'm pretty sure this is provided solely for the benefit of Musk's weird fans, who can now go "see? Dear Leader broke the site as part of the 4D chess!" Normal people probably shouldn't take much notice of it.

choeger 3y ago

Twitter is the first social Network to publicly commit ritual suicide. It's somewhat fascinating to watch. Would be very interesting to see how much the login wall affected their traffic and thus ad revenues. But the real damage will be visible in Q4 when ads campaigns simply won't be booked anymore for Twitter because everyone begins to write it off.

tropicalbeach 3y ago
Why Twitter is better now than before. I used to never use it and now i use it more than any other platform. Don't really get why people are hating so much. Finally there is somewhere where people can post without being banned or having posts removed. Why do you want filtered social media it's fake.
- HlessClaudesman 3y ago
  Nice try Elon.
  - tropicalbeach 3y ago
    Genuine question you like your social media filtered by someone who says what you're allowed to see?
    - dale_glass 3y ago
      Genuine answer: Yes. If I wanted an unfiltered experience, there's 4chan and the like. Notice that I'm not there, but somewhere I don't have to wade through mountains of crap. HN is very much moderated, that's what keeps it useful.
    - threeseed 3y ago
      There have been a large number of accounts that have been censored by Twitter. Musk even admitted doing so at the behest of foreign governments something Twitter 1.0 used to aggressively fight and prevent from happening.
    - 3y ago

williamsmj 3y ago

This is content-free gibberish, but a particular highlight for me is the promise that "we will provide an update when the work is complete" that is immediately followed by "this work will never be done".

raverbashing 3y ago

At this point I'm taking Elon to the 'politician' levels of lie detection Where "verified" means you paid $8 or something, I see no reason to trust pretty much anything else coming from them

etchalon 3y ago

“Bill in IT says we need to reboot the router but no one knows the router password and Comcast says the password is on the router but Bill says Jim must have torn off the sticker so we called Best Buy and the soonest the Geek Squad can get here is Thursday. We tried unplugging it and plugging it back in but it’s not turning on.”

wg0 3y ago

It feels like that the above comes directly written by His Royal Highness, just copy pasted in place by the site admins.

deleted 3y ago

[deleted]

ethcat 3y ago

To be honest, it doesn't bother me because I've moved to other platforms. The main reason I log on to twitter is to see if anyone I follow has moved to another platform.

resolutebat 3y ago

> To ensure the authenticity of our user base we must take extreme measures to remove spam and bots from our platform. That’s why we temporarily limited usage so we could detect and eliminate bots and other bad actors that are harming the platform. Any advance notice on these actions would have allowed bad actors to alter their behavior to evade detection. Ah, so they're intentionally making everybody suffer to punish the bad guys, and promising that everything will be peachy keen afterwards. This sounds essentially identical to the Indian government's justification for suddenly demonetizing the 500 and 1000 rupee notes in 2016, leading to massive cash shortages. This directly led to 1.5M people losing their jobs, a stock market crash, and GDP dropping by at least 1%. The silver lining was that it also kick-started the Indian digital payments system.

threeseed 3y ago

This looks to be complete nonsense. Scraping services work by going through thousands of intermediate proxies run on unsuspecting people's compromised computers. Each scraping request will be a unique IP with a normal looking header. They may be able to look at the number of unique requests per account but given the ridiculous amount of spam/bots that exist clearly creating new accounts isn't an impediment. The far more likely scenario is that they were throttled by AWS/GCP for not paying bills and are behind on their migration project.

supsep2 3y ago
So all scrapers are viruses? Need to me.
- DistractionRect 3y ago
  Almost all "residential" proxies are run without the knowledge of the user used as the proxy. Either it's slipped in the ToS/install of other legitimate product or straight up malware. A popular method, iirc, was bundling with free VPNs. Last I looked into the discussion, the consensus was there was no particularly reputable residential proxy service. Edit: I probably should explain the distinction for residential proxies. Basically, if you run a site that isn't B2B, you'll find basically no organic traffic comes from data centers/server farms. So traffic from IP blocks like owned by Microsoft (Azure), Amazon (AWS), Google (GCP), etc are all highly likely to be bot traffic and throttled or outright blocked by default. If you're running a scraper or a bot, you want to get your traffic to appear to come from residential IP blocks (owned by residential ISPs).

nmstoker 3y ago

Strange they should mention spam when these last few days I've seen a steady stream of near identical advertisers appear in spam-like ads. The advertisers have very similar profile pictures which look generated, use a particular colour palette, and all sell the same sort of unwanted junk. Despite a mix of blocking/muting, a new one pops up every few hours!

dathinab 3y ago

> Currently, the restrictions affect a small percentage of people using the platform, and we will provide an update when the work is complete. are they sure, it seem to have hit literally everyone I know who still uses twitter frequently

ignoramous 3y ago
This is a CMA post. Nothing much to see other than mea culpa and the palpable disappointment of other (not their own) AI bots winning.

drumhead 3y ago

So I'm assuming this is because he thinks the data is so valuable for LLM training he doesn't want anyone getting it for free. Does he really think there will be that much demand for it that it outweighs the fundamental purpose of Twitter?

Balgair 3y ago
As I'm sure you've probably guessed, it's not about the bots. It's that his contract with Google for servers ran out on July 1st. So now they have to pay a lot more. Absurdly, this is a cost saving measure. And yes, I've no idea how it ends up saving either.

nirui 3y ago

> To ensure the authenticity of our user base we must take extreme measures to remove spam and bots from our platform. Well about that, I noticed heavy inflow of spam and bots traffic right after Mr Musk starts selling the blue patch for money. These spammers purchased and utilized the "first-class citizen" privilege to bend the display priority to their favor. Now days, you open a Tweet, the top replies are almost all spams. You know, at this point, it is obviously apparent that the spammers knows better about Twitter than Mr Musk do. But all and all, this (as well as the recent events related to Reddit) is a good thing. Because it serves a good reminder that we (the users) are products, sometimes the self-paying type. So maybe next time you when plan to spend some chill time on some services, think how much you can get out if it for exchange.

DaSHacka 3y ago

As the twitter users would say, #TwitterIsOverParty