5 ms·
I can confirm. NYT shows a five-second redirect delay: "wget https://t.co/4fs609qwWt https://t.co/4fs609qwWt". It redirects to gov.uk immediately: "wget https:
by kens 3y ago
I can confirm. NYT shows a five-second redirect delay: "wget https://t.co/4fs609qwWt https://t.co/4fs609qwWt". It redirects to gov.uk immediately: "wget https://t.co/iigzas6QBx https://t.co/iigzas6QBx"
- XCSme 3y agoThey both load instantly for me.
- ActivePattern 3y agoIt's already been reverted.
- jquery 3y agoI almost didn't believe OP, because it's so comically inept and petty. But, I can also confirm in some private testing there is a deliberate delay.
- TheRealSteel 3y ago"because it's so comically inept and petty" This is precisely why I did believe OP. This is Elon Musk we're talking about.
- adhesive_wombat 3y agoConsidering how common it is to deliberately break (not just with a "you log in now" modal, but not loading content, spinning forever, breaking layouts, etc) websites for non-logged-in mobile users, that's exactly how petty in imagine them to be. Twitter and Reddit do it, and Imgur comes and goes so I can't decide if for them it's deliberate or just incompetence.
- deepserket 3y agoYou can add Instagram and YouTube to the list of websites that are painful to use from a mobile browser
- comprev 3y agoit's no doubt intentional if the site has an app. They can scoop up significantly more data about you from their app
- mbernstein 3y agoAgree/confirmed - just recorded a number of different nytimes urls that pass through t.co, all 4.7s+. various cnbc and google articles through t.co were ~130-200ms response time from t.co specifically (not total redirect->page load).
- _a9 3y agoIm not getting the same time delay with curl - `time wget https://t.co/4fs609qwWt https://t.co/4fs609qwWt` -> `0m5.389s` - `time curl -L https://t.co/4fs609qwWt https://t.co/4fs609qwWt` -> `0m1.158s`
- djvdq 3y agoAnd now add browser user-agent to the curl request and watch how slow it gets. - `time curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0" -L https://t.co/4fs609qwW https://t.co/4fs609qwW` -> 4.730 total - `time curl -L https://t.co/4fs609qwWt https://t.co/4fs609qwWt` -> 1.313 total Same request, the only difference is user-agent.
- zagrebian 3y agoIt’s about 4.5 seconds for me https://imgur.com/a/qege0O9 https://imgur.com/a/qege0O9
- lamontcg 3y agoAny DNS resolver libraries have a 4.5 second timeout? Maybe their infrastructure is just rotting.
- minimaxir 3y agoIt'd be weird that it's limited to specific outbound domains in that case.
- lamontcg 3y agodomains load balanced across different servers in some overly complicated distributed topology with only one of them busted? although seems unlikely it just happens to be the NYT.
- djvdq 3y agoI don't think it's a problem with rotting infrastructure. Using curl those requests are quick, unless you pass user-agent from browser. 4.7s with firefox user-agent and 1.3s without it. Using twitter link to the same NYT article.
- runlevel1 3y agoglibc defaults to 5 sec, but the server wouldn't need to resolve the redirect domain -- that'd be the client's job. Unless it's doing so for some proprietary reason, of course. Or did you mean failing to resolve some internal service's hostname?
- lamontcg 3y ago> Or did you mean failing to resolve some internal service's hostname? Yeah, something more like that where the internal service is somehow 'sharded' due to some overly complicated distributed database nonsense, and there's a DNS lookup that is failing. Of course that'd mean the DNS lookup wasn't cached, so you're taking that normal latency on every single hit, which would be terrible architecture. The curl-vs-wget performance isn't explained by that though (although that's a bit weird in and of itself, and might suggest that they had to allow that for some internal tool that they didn't want to punish). > glibc defaults to 5 sec, The timeout being close to 5 seconds is what made me wonder about it. Its just off though.
- craftkiller 3y agoOddly enough the delay is reduced to 1 second by using curl's useragent string (wget --user-agent='curl/8.2.1' https://t.co/4fs609qwWt https://t.co/4fs609qwWt)
- ilikehurdles 3y agoSeeing this makes me wonder if it's some sort of server-side header bidding ad server gone haywire, rather than something nefarious. Why would they only delay browser agents otherwise?
- derefr 3y agoBrowsers are generally tolerant of long TTFB. Automation, on the other hand, is sometimes quite brittle.
- londons_explore 3y agoPerhaps their own internal tooling also relies on the t.co redirector?
- nvm0n2 3y agoProbably a phishing/malware scan gone wrong then. NYTimes has Twitterbot in its robots.txt which might be related? Even if it's deliberate, I don't see how people can complain. Google has outright blocked Breitbart for years. They prevent results from that domain from appearing at all unless you specifically force it with site: and apparently HN does the same. Politically motivated censorship and restricting "reach" is just how Silicon Valley rolls. Pre-Musk Twitter did freeze the New York Post's account and many other much worse things. It'd be a shame for Musk to be doing this deliberately, even though it seems unlikely. But that's the problem with creating a culture where that sort of behavior is tolerated, isn't it? One day it might be turned around on you.
- bookmark1231 3y ago[dead]
- 3y ago
- snake_doc 3y agoCould it just be rotting infrastructure? I.e there is some logic on most visited domains to allow ease of moderation, that logic is read heavy and is now bucking under skew. Or even like some junior dev removed an index
- joecool1029 3y agoA few years ago I remember their URL shortener on android app directing somewhere that my hostfile adblocker would catch (like an analytics domain or something). This made it so first click on certain twitter links would fail, but if I clicked it again it would go successfully. Ultimately I never researched it deeply enough but my guess is they had some sort of handler that would log whether loading their analytics service failed and serve up the direct link on the second attempt.
- tech234a 3y agoI've experienced similar behavior on iOS, not sure if it still does that though.
- swissspidy 3y agoSeeing that right now with NYT links. First click takes a few seconds to redirect, second click is almost instant
- praisewhitey 3y agoI tested some substack.com links and there's a delay on those too.
- yohannesk 3y agoIs there some cache going on? On my first attempt, there is a 5 second delay. When I try it second time immediately it works without the 5 second delay. But if I try again after an hour, 5 second delay again!
- hrrsn 3y agoSafari seems to be caching it for me, but I can reproduce the delay every time with curl - so long as the user agent doesn't include the string "curl".
- PenguinCoder 3y agoNow do additional testing by adding and setting the http referer to t.co or twitter. Is it Twitter, or is it NYtimes doing this?
- kens 3y agoYou can do that if you want; I don't take orders.
- mzs 3y agoI don't see it: % curl -gsSIw'foo %{time_total}\n' -- https://t.co/4fs609qwWt https://t.co/iigzas6QBx | grep '^\(HTTP/\)\|\(location: \)\|\(foo \)' HTTP/2 301 location: https://nyti.ms/453cLzc foo 0.119295 HTTP/2 301 location: https://www.gov.uk/government/news/uk-acknowledges-acts-of-genocide- committed-by-daesh-against-yazidis foo 0.037376
- lapcat 3y agoI think Twitter, err, X, just turned off the delay now that it's getting big media attention. I could reproduce it over and over again a little earlier, but now I can't anymore: https://news.ycombinator.com/item?id=37138161 https://news.ycombinator.com/item?id=37138161 [Edit:] I'm still seeing it with threads.net: curl -v -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15' https://t.co/DzIiCFp7Ti
- mzs 3y agoI don't see it with your URL either: % curl -gsSIw'foo %{time_total}\n' https://t.co/DzIiCFp7Ti | grep '^\(HTTP/\)\|\(location: \)\|\(foo \)' HTTP/2 301 location: https://www.threads.net/@chaco_mmm_room foo 0.123137 Doesn't matter if I do a HTTP/2 HEAD or GET: % curl -gsSw'%{time_total}\n' https://t.co/DzIiCFp7Ti 0.121503 HTTP/1.1 also shows no delay: % curl -gsSw'%{time_total}\n' --http1.1 https://t.co/DzIiCFp7Ti 0.120044 I chalk this up to rot at X/twitter that is being fixed now that it was noticed.
- lapcat 3y ago> I don't see it with your URL either That's because you're not spoofing the User-Agent to be a browser rather than curl.
- mzs 3y agoOh that's it, thanks! In fact it it returns a 200 not 301 then: % curl -gsSw'%{time_total}\n' -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15' https://t.co/DzIiCFp7Ti <head><noscript><META http-equiv="refresh" content="0;URL=https://www.threads.net/@chaco_mmm_room"></noscript><title>https://www.threads.net/@chaco_mmm_room</title></head><script>window.opener = null; location.replace("https:\/\/www.threads.net\/@chaco_mmm_room")</script>4.690000 % curl -gsSIw'%{time_total}\n' -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15' https://t.co/DzIiCFp7Ti HTTP/2 200 ... content-length: 272 ... x-response-time: 4524 ... 4.660211 The delay is not there for nyti.ms (anymore) but once you use the Safari UA it's handled as 200 response: % curl -gsSIw'foo %{time_total}\n' -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15' https://t.co/4fs609qwWt https://t.co/iigzas6QBx | grep '^\(HTTP/\)\|\(location: \)\|\(foo \)' HTTP/2 200 foo 0.126043 HTTP/2 200 foo 0.037255 It really does seem that twitter is adding a 4.5s delay to some sites from web browsers. Could be malicious, could be rot...