42 ms·
Tell HN: t.co is adding a five-second delay to some domains
Go to Twitter and click on a link going to any url on "NYTimes.com" or "threads.net" and you'll see about a ~5 second delay before t.co forwards you to the right address.
Twitter won't ban domains they don't like but will waste your time if you visit them.
I've been tracking the NYT delay ever since it was added (8/4, roughly noon Pacific time), and the delay is so consistent it's obviously deliberate.
- CodeWriter23 3y agoIt seems “it’s a private platform and they can do what they want” only applies when the mob wants [insert social network]’s undesirable behavior.
- downWidOutaFite 3y agowhen does "it's a public square" apply?
- CodeWriter23 3y agoOnly when the mob says so, apparently
- tmaly 3y agoI also noticed for certain threads, if you open in the browser, it does not let you read the other part of the thread. You have to open in the app.
- wonderfuly 3y agoIs it possible to build a browser extension to bypass this redirection?
- archo 3y ago-- Related -- X has started reversing the throttling on some of the sites, including NYTimes Discussions on HN: (61-comments - 2023-08-16) : https://news.ycombinator.com/item?id=37141478 https://news.ycombinator.com/item?id=37141478 Twitter post archive: https://archive.is/PW3eG https://archive.is/PW3eG
- the_real_one 3y ago[flagged]
- midasz 3y agoIf you cannot abide by the very simple rules of HN, please don't bother making an account and commenting.
- NiloCK 3y agoI'm outing myself as an ignoramus* here, but could the title be edited to indicate that this belongs to twitter? I glossed past this on first read because "some url shortener has shitty behavior" wasn't interesting to me. Hearing heard about twitter's throttling someplace else made me come back here because I was surprised not to have heard it about it here first. *: stalwart radical who doesn't use twitter
- sevenc_nanashi 3y agoI just took a benchmark using hyperfine. Not threads.net, cURL User-Agent: 224.3 ms Not threads.net, Firefox User-Agent: 227.4 ms threads.net, cURL User-Agent: 223.9 ms threads.net, Firefox User-Agent: 2743 ms Twitter is trying to hide this fact? (As they don't make delay w/o browser User-Agent) (Full log: https://gist.github.com/sevenc-nanashi/c77d18df6a5f326b0d2921d9553623ba https://gist.github.com/sevenc-nanashi/c77d18df6a5f326b0d292...)
- hn1986 3y agothanks for sharing this!
- sposeray 3y ago[dead]
- chasing 3y agoYup. It's super-easy to test and, yes, any nytimes.com link takes exactly a five-second count to open.
- deleted 3y ago[deleted]
- c0t300 3y agoyou could maybe show some testing? if that's true, then it should be measurable
- stilwelldotdev 3y agoTest yourself, they link to their site from their profile and it's definitely delayed. https://twitter.com/nytimes https://twitter.com/nytimes
- djbusby 3y agoA good test might include a bunch of domains. And checking the timing on each. Could we demonstrate the delay is on t.co and not on NYT?
- stilwelldotdev 3y agoI mean, a good test is to go to go to their website in another browser tab and click around. You're over-complicating this.
- cornel_io 3y agoI went through my own Twitter feed, and found 10 non-NYT links. All redirected almost immediately through t.co via wget, only lagging on the destination sites. I also tried 5 NYT links. All had a very consistent 5 second delay through wget. I could do more, but I don't care to. Everyone knows Elon has gone redpill, so it wouldn't surprise me if he's "owning the libs", but there also could be a dozen other reasons Twitter might do something like this (including plenty that are not nefarious). I just don't care to dig more... Edit: I suppose I could have given the specific URLs, but I don't know if/how much t.co links leak info, so I'm not keen to do that. But the delay is absolutely on t.co and not the destination sites, at least as far as external users are concerned. It's possible that t.co queries the sites first before redirecting, and if e.g. the NYT is throttling their traffic that's what's delaying things. I don't know how to disambiguate that, but it's definitely a theory worth considering...
- pizza 3y agoRemember when people were excoriating Google AMP for encouraging walled gardens? If true, this seems in so much worse faith than that.
- lemper 3y agoi mean, you can stop visiting the site, no? just leave, bro. it's not that hard. there are other means to connect to people.
- MattRix 3y agoYou can both stop using the site as a user and still call out the awful stuff they do.
- dymk 3y agoAt some point though, the complaining becomes nonconstructive whining. It was already time to move on by the time it changed to X.
- chasing 3y agoYou're right! Which is why making Twitter's product worse when there are active competitors taking big bites out of their business seems... dumb?
- lemper 3y agoyep, it also seems to me that the helmsman of that site is dumb. good thing i left that site many years ago.
- dlgeek 3y agoUnfortunately not. All of my local government agencies - Police, Fire, DOT, Weather Service, Emergency Management updates, etc. are exclusively on twitter - they frequently post things there they don't even post to their own websites.
- ehPReth 3y agoI'm in the same boat :/.
- extraduder_ire 3y agoAt the very least, I can get at the info if it's on twitter. Anything that's exclusively on a facebook page may as well not exist to me.
- 88913527 3y agoThe speed at which enshitification is being unleashed surprises me each and every day.
- blowski 3y agoEnshitification is different. It’s when companies destroy a product with hundreds of changes that prioritise internal politics above what end users want. This is something else - just the ego of one rich guy petulantly satisfying his inner demons.
- xg15 3y ago> just the ego of one rich guy petulantly satisfying his inner demons. As that rich guy happens to be the CEO, how is this not the prime example of "prioritising internal politics above what end users want"?
- Terr_ 3y ago"Porque no los dos?" A five-second delay may be enough to cause a measurably increase in the "stickiness" of Twitter if some people wait <5 seconds before clicking or scrolling onwards to something else. Then they spend more time generating ad-revenue for Twitter than if they had gone off to the New York Times or something and started browsing over there.
- blowski 3y agoIt’s not doing it on foxnews.com. Take everything else Musk is saying and doing, and it’s hard to believe it’s not some grudgelist.
- jonny_eh 3y ago> that prioritise internal politics I thought it was about increasing short-term revenue.
- mplewis 3y agoAnother great reason to get off Xitter.
- kbaker 3y agoIs that pronounced like "Shitter"? Huh.
- spaceman_2020 3y agoThe X is pronounced as “Shi”
- siva7 3y agocan confirm from europe. shouldn’t this be illegal in america?
- craftkiller 3y agoNet neutrality was repealed in 2018.
- Terr_ 3y agoNetwork Neutrality has historically been more about malfeasance by ISP networks which have higher/more-concrete barriers to entry, as opposed to the websites.
- astrange 3y agoAnd then reinstated again at the state level. It never went away, this just doesn't apply because Twitter isn't an ISP (in that sense).
- craftkiller 3y agoTIL! Looks like its only some of the states, and that is a good point about Twitter not being an ISP. Twitter is incorporated in Delaware which has not passed state-level net neutrality, so I don't know if it would have mattered anyway.
- astrange 3y agoIt basically just matters what California does. There's enough customers there that it's easier to apply those policies nationwide. Incorporated in Delaware hardly ever affects anything except corporate law.
- deleted 3y ago[deleted]
- catlover76 3y agoIt is probably not illegal in America. Would it be illegal in Europe? Because (at least w/r/t Threads) it is an anti-competitive practice?
- deleted 3y ago[deleted]
- kens 3y agoI can confirm. NYT shows a five-second redirect delay: "wget https://t.co/4fs609qwWt https://t.co/4fs609qwWt". It redirects to gov.uk immediately: "wget https://t.co/iigzas6QBx https://t.co/iigzas6QBx"
- XCSme 3y agoThey both load instantly for me.
- ActivePattern 3y agoIt's already been reverted.
- jquery 3y agoI almost didn't believe OP, because it's so comically inept and petty. But, I can also confirm in some private testing there is a deliberate delay.
- TheRealSteel 3y ago"because it's so comically inept and petty" This is precisely why I did believe OP. This is Elon Musk we're talking about.
- adhesive_wombat 3y agoConsidering how common it is to deliberately break (not just with a "you log in now" modal, but not loading content, spinning forever, breaking layouts, etc) websites for non-logged-in mobile users, that's exactly how petty in imagine them to be. Twitter and Reddit do it, and Imgur comes and goes so I can't decide if for them it's deliberate or just incompetence.
- deepserket 3y agoYou can add Instagram and YouTube to the list of websites that are painful to use from a mobile browser
- abigail95 3y agoI wouldn't expect an SLA on a service without one. This is like cogent/he drama
- mutant_glofish 3y agoI think that HN itself also shadow flags submissions from a list of domains it doesn't like. Try submitting a URL from the following domains, and it will be automatically flagged (but you can't see it's flagged unless you log out): - archive.is - watcher.guru - stacker.news - zerohedge.com - freebeacon.com - thefederalist.com - breitbart.com
- iguana_lawyer 3y agoYeah. HN bans your favorite white supremacist blogs. I don’t see a problem with that.
- chasing 3y agoGood. Hacker News isn't an open-ended political site for people to post weird propaganda.
- deleted 3y ago[deleted]
- mutant_glofish 3y agoHow's archive.is "weird propaganda"?
- chasing 3y agoDid you read the rest of your own post?
- dang 3y agoIt isn't banned in comments - https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22archive.is%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que..., https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22archive.ph%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que..., etc. We probably banned it for submissions because we want original sources at the top level.
- ctrlp 3y agoIf true and intentional, then this is a strong move by Musk against his ideological opponents. Hard to believe he has the cognizance to recognize them as such but maybe he purged more of the 3-letter agency folks from X than it seemed.
- throwawayvo 3y ago[flagged]
- Timon3 3y agoUnless you can prove it it's a baseless assertion.
- yodon 3y ago>I bet rightwing sites had this issue under former management. If Twitter's previous administration had been doing intentionally bad things as your baseless assertion implies, Musk would be posting the code or database tables and showing it everywhere to prove to the world how bad they were. The fact that Musk is quiet here demonstrates your assertion is not simply baseless, it's provably false.
- rhaksw 3y agoThat sounds like a play on the "slow ban": > Selective downtime, where the troll finds that the website is down (or really slow) quite often. Not all of the time, because that would tip them off. Trolls are impatient by nature, so they eventually find a more reliable forum to troll. https://ask.metafilter.com/117775/What-was-the-first-website-to-hide-trolls-activity-to-everyone-but-the-troll-himself?ref=blog.codinghorror.com https://ask.metafilter.com/117775/What-was-the-first-website...
- minimaxir 3y agoUsually, slow bans aren't done for people who pay you money and add value to your platform.
- rhaksw 3y agoGreat point! I just wanted to name the technique. I am actually an active speaker and developer against all secretive content moderation actions.
- BHSPitMonkey 3y agoNYT and others are still adding value to the platform even if users become discouraged from clicking those links and navigating away (choosing instead to just stay on Twitter and read the summary and reactions there). Nudges like this help Twitter have their cake and eat it, too.
- minimaxir 3y agoOnce referral pageviews take a hit, I suspect the NYT will be reevaluating their options.
- prawn 3y agoA variation on this is returning an error message of sorts.
- juxtapose 3y agoFunny they haven't copied the full Chinese model yet. In China, the forwarding system can ban any website they don't like, and you have to copy the URL manually to visit it. 5 second delay is a mercy. :-)
- 1vuio0pswjnm7 3y agoWhy not try a Nitter instance. It offers RSS and none of the URLs are shortened with "t.co".
- ajb 3y agoThey just broke nitter. Hopefully it will be fixed soon but in the long term Musk can cut nitter off if he wants.
- 1vuio0pswjnm7 3y agoIt's working for me.
- ajb 3y agoWhich instance? None of the ones I'm trying are working. Maybe it's regional
- JeremyNT 3y agoYou can refer to the full list on the github page[0] (which I find to be pretty accurate). I personally use La Contre-Voie[1] for no particular reason. nitter.net was historically a little less reliable for me due to rate limiting, which is why I initially switched. They worked around the rate limiting issue now, so that may no longer be the case. [0] https://github.com/zedeus/nitter/wiki/Instances https://github.com/zedeus/nitter/wiki/Instances [1] https://nitter.lacontrevoie.fr/ https://nitter.lacontrevoie.fr/
- ajb 3y agoThanks. Unfortunately, not having any luck with any of them, including lacontrevoie. Nitter.net gives a nim error, the others don't find any tweets
- 1vuio0pswjnm7 3y ago
- BLKNSLVR 3y agoI'd like to invoke Hanlon's razor here to at least cut down on the potential for over-the-top reactions that can be somewhat embarassing in retrospect: "Never attribute to malice that which is adequately explained by stupidity."
- kobalsky 3y agoif I had to guess: ugly synchronous code for statistics that hits a dead shard for the target domain and times out.
- hackerlight 3y agoHanlon's razor needs to be inverted when the person you're talking about has NPD. The assumption should be that there are sinister intentions until proven otherwise. You'll make better predictions about reality if you do that inversion.
- jonny_eh 3y agoOr both
- minimaxir 3y agoThis is a case where Occam's Razor trumps Hanlon's Razor.
- 0cf8612b2e1e 3y agoAbsolutely not applicable to corporations. Bad faith actions are rampant, and I am unwilling to give the benefit of the doubt to billion dollar companies.
- justin101 3y agoThere are a number of reasons this could be that are not necessarily nefarious. It's odd to jump straight to "something evil is going on" Tell me this, does Twitter have some kind of "play nice" code that slows down inbound clicks through to a site so it doesn't DDOS other sites? I can easily imagine a scenario where anti DDOS caode would allow small sites to pass through quickly, yet sites under heavy "click through" load are being slightly throttled.
- archo 3y agoMany words about site moderation have been posted in the thread.. Who here has actually moderate(s|d) a forum/chat site?
- JdeBP 3y agoSome of us have, you will find. (-: But, that said, I'm more interested in the discussions about verification, neutrality, and the reasons that people have for still clinging on for grim death that, in a few hours, will likely be pushed down onto a second page by that huge comment thread currently in the middle of this page and above them.
- prawn 3y agoI have, for about 20 years. I empathise with @dang's methods and explanations here. e.g., don't disclose a blocklist, use shadowbanning, don't get dragged down by pedantic responses to explanations. You're often one person against many trolls finding grey areas for sport, or spammers (automated or otherwise) probing defences.
- archo 3y agoI am in total agreement with you, keeping the noise/damage of nefarious scammers, spammers, hacker/pranksters, attempting to penetrating the site requires ALL the tools in the toolbox.
- DamonHD 3y agoMe also! Though I have tried to avoid it, somehow I get to be admin for many many years! And I'm feeling HN's position, even though I occasionally trip some of the mechanisms here.
- aaron695 3y ago[dead]
- 1000bestlives 3y ago[flagged]
- hlandau 3y agoWorth pointing out that t.co has always been an instance of an annoying and seemingly unjustified practice I named "nonsemantic redirect". Rather than legitimately redirecting using an HTTP Location header, it instead is an HTML page with a META refresh tag on it. You don't see this with curl/wget because they use user agent sniffing. If they don't think you're a browser they _will_ give you a Location header. To see it, capture a request in Firefox developer tools, right click on the request, copy as CURL. (May need to remove the Accept-Encoding tag and add -i to see the headers).
- mzs 3y agoNo, in fact now t.co even returns an empty body with it's 301 response: % curl -vgsSw'< HTTP/size %{size_download}\n' https://t.co/DzIiCFp7Ti 2>&1 | grep '^< \(HTTP/\)\|\(location: \)' < HTTP/2 301 < location: https://www.threads.net/@chaco_mmm_room < HTTP/size 0
- flutas 3y agoFirefox: <head><noscript><META http-equiv="refresh" content="0;URL=https://www.threads.net/@chaco_mmm_room"></noscript><title>https://www.threads.net/@chaco_mmm_room</title></head><script>window.opener = null; location.replace("https:\/\/www.threads.net\/@chaco_mmm_room")</script>
- mzs 3y agoTurns-out it depends on the User-Agent: https://news.ycombinator.com/item?id=37139425 https://news.ycombinator.com/item?id=37139425
- mlyle 3y agoYou didn't read the second paragraph of the comment you replied to-- which explained this exact issue before you replied "no": > You don't see this with curl/wget because they use user agent sniffing. If they don't think you're a browser they _will_ give you a Location header. To see it, capture a request in Firefox developer tools, right click on the request, copy as CURL.
- Khaos_Machine 3y ago[flagged]
- smcleod 3y agoShouldn't it be x.co now? Also doesn't t.co embed some tracking information as well?
- steve1977 3y agoPeople are still using Twitter?
- reportgunner 3y agoReddit too, it's crazy to me.
- riddley 3y agoThe justifications I've seen go along the lines of "I can't go elsewhere because my followers are here." .... But if you left, they'd likely follow. It's like a microcosm of capitalism. The users don't realize they hold all of the power, I guess.
- KolenCh 3y agoIt is called the Tail Call Optimization for a reason. /s
- paulcapewell 3y agoThe redirect has always been a bit annoying and I found on mobile if you click a link, see it load the t.co URL, then quickly go back and click again, it instead loads the actual linked URL. So I just instinctively do this now.
- ChrisArchitect 3y agoThe link doesn't magically change. It might be partially cached when you go back and so it seems immediately to be loading the final url but it's not. It's always going to t.co first. C'mon.
- PenguinCoder 3y agoAdditional details I wrangled for this rabbit hole. I don't think it's t.co doing this intentionally, but rather poor handling of 'do you have our cookies or not'. Everyone in this thread _proving things_ without taking into account the complexity of the modern web. man curl -b, --cookie <data|filename> (HTTP) Pass the data to the HTTP server in the Cookie header. It is supposedly the data previously received from the server in a "Set-Cookie:" line. ---- Add that option to your curl tests. --- $ time curl -s -b -A "curl/8.2.1" -e ";auto" -L https://t.co/4fs609qwWt -o /dev/null | sha256sum eb9996199e81c3b966fa3d2e98e126516dfdd31f214410317f5bdcc3b241b6a2 - real 0m1.245s user 0m0.087s sys 0m0.034s --- $ time curl -s -b -e ";auto" -L https://t.co/4fs609qwWt -o /dev/null | sha256sum eb9996199e81c3b966fa3d2e98e126516dfdd31f214410317f5bdcc3b241b6a2 - real 0m1.265s user 0m0.103s sys 0m0.023s --- $ time curl -s -b -A "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" -e ";auto" -L https://t.co/4fs609qwWt -o /dev/null | sha256sum eb9996199e81c3b966fa3d2e98e126516dfdd31f214410317f5bdcc3b241b6a2 - real 0m1.254s user 0m0.100s sys 0m0.018 ---
- mzs 3y agooh boy... -b takes an option which in your examples is -A and -e, then what follows is interpreted as a URL and you throw away the warnings: % curl -vgsSIw'> %{time_total}\n' -b -A "curl/8.2.1" https://t.co/DzIiCFp7Ti 2>&1 | grep '^\(* WARNING: \)\|\(Could not resolve host: \)\|>' * WARNING: failed to open cookie file "-A" * Could not resolve host: curl curl: (6) Could not resolve host: curl * WARNING: failed to open cookie file "-A" > HEAD /DzIiCFp7Ti HTTP/2 > Host: t.co > User-Agent: curl/8.1.2 > Accept: */* > > 0.013309 > 0.112494
- PenguinCoder 3y agoAlright thanks for explaining that . Here's what I see explicitly setting the cookiejar $ time curl -s -b cookies.txt -c cookies.txt -A "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" -e ";auto" -L https://t.co/DzIiCFp7Ti [t.co meta refresh page src] real 0m4.635s user 0m0.004s sys 0m0.008s $ time curl -b cookies.txt -c cookies.txt -A "wget/1.23" -e ";auto" -L https://t.co/DzIiCFp7Ti curl: (7) Failed to connect to www.threads.net port 443: Connection refused real 0m4.635s user 0m0.011s sys 0m0.005s $ time curl -b cookies.txt -c cookies.txt -e ";auto" -L https://t.co/DzIiCFp7Ti curl: (7) Failed to connect to www.threads.net port 443 Connection refused real 0m0.129s user 0m0.000s sys 0m0.013s The failed to connects are threads.net likely blocking those user agents but the timing is there which is different than the first UA attempt.
- cfillekes 3y agoXitler
- epistasis 3y agoThis is what I have come to expect from every person that calls themselves a "free speech absolutist." What they actually believe is that they should be able to say whatever they want and do whatever they want, personally, without any consequences for themselves. There is no grander principle than "my ability to do what I want and exert power over others however I want, without critique or criticism." I really wish the term hadn't been polluted this way.
- epistasis 3y agoUpdate: hours after being exposed and publicized in the Washington Post, the behavior has stopped: > On Tuesday afternoon, hours after this story was first published, X began reversing the throttling on some of the sites, dropping the delay times back to zero. It was unknown if all the throttled websites had normal service restored. https://archive.is/2023.08.15-210250/https://www.washingtonpost.com/technology/2023/08/15/twitter-x-links-delayed/ https://archive.is/2023.08.15-210250/https://www.washingtonp...
- rhaksw 3y agoI still see a roughly 2 second delay on first grab. The second is immediate.
- raxxorraxor 3y agoI call myself a free speech absolutist (or advocate at least, absolutist is more of a slur). False compromises belong in the past. What X is doing isn't free speech at all and they have stated that advertisers will dictate what content will be seen, there is no commitment to freedom of speech at all. But at least I can hold them responsible for violating their own stated values. The former Twitter leadership just hid content that didn't fit theirs or third parties sensitivities and told me they are doing me a favor. Restricting speech is always in the interests of those that have the power to shape discussions, so limiting speech is always counter productive.
- oneeyedpigeon 3y ago
- ChrisArchitect 3y ago[flagged]
- diebeforei485 3y ago[flagged]
- minimaxir 3y ago"Journalist bad" is apparently your prior.
- diebeforei485 3y agoI'm not sure how you reach this conclusion considering the other posts on this thread show this is related to cookies, which is why clicking the same link twice doesn't have the delay the second time.
- graybeardhacker 3y agoThe solution to X (Twitter) sucking is to stop using it. It will either: get fixed or go out of business and be replaced. It seems we've become a society that rewards bad practices with attention which is all any company on the web is trying to get, your attention.
- hackerlight 3y ago> It seems we've become a society that rewards bad practices with attention I have a very different way of looking at this. It's not us that gives attention. It is them that take it via exploiting our evolved inflexible cognitive systems for attention/reward/desire/anger/lust. We are moths to a flame. The moth's free will isn't to blame for its inability to avoid it. Our cognitive systems are fixed, we can't just turn them off. If a sufficiently powerful dopamine-inducing technology is made, you can't just "opt out". It is not as simple as that. Any individual variation in the ability to opt out likely comes down to variation in genetics or other extraneous factors not inside one's immediate control. This is where regulation needs to come in. Once you accept the reality that opting out is a comforting yet false illusion, you can then do something about it.
- altacc 3y agoTim Wu makes a similar point in his book The Attention Merchants. Humans are interested in things and throughout time various people and media (which tends to be controlled by a small number of people) have been working to capture our attention. It is very hard to totally opt out of something that is so pervasive, like fish trying to ignore water.
- ravetcofx 3y agoMastodon has been a breath of fresh air and you can get a really interesting feed going when you follow the right people and hashtags
- RMPR 3y agoAnd it can handle Twitter scale now https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...