11 ms·
Nitter is working again
- eterps 3y agoDisplaying a timeline with multiple accounts seems still broken, but other than that it works fine.
- zedeus 3y agoSearch, which is required for the multi-timeline feature, isn't implemented right now. I have a fix in the works using a legacy endpoint.
- meepmorp 3y agoSo, is the AI Scrapocalypse no longer a critically important issue?
- cmrdporcupine 3y agoProbably he loudly threatened to fire his workshop elves so they worked through the nights and slept on old mattresses in the office so they could Make Twitter Scale Again and now all is well in Load Balancer land?
- brokenbyclouds 3y agoI'm skeptical that was ever the rain for the rationing.
- INTPenis 3y agoIs this really permanent? I'd love to know more about this bearer token. Because in the other github issue thread it seemed like every time they found a way around Twitter's safeguards, it was shutdown. It seems like they've literally hard coded a token into the source code. Meaning thousands of nitter-instances, thousands of users, around the world, will use the same token. And potentially so will the AI companies. So I just don't see how this can work.
- cmeacham98 3y agoMy guess is that this token is used in the official Twitter web/mobile app - making it hard for Twitter to just straight up disable/ban.
- soraminazuki 3y agoI wouldn't bet on it. After all, this is Elon Musk we're talking about here. When given a choice, he'll choose the most disruptive option.
- foderking 3y ago"the most entertaining outcome is most likely"
- NelsonMinar 3y agothat's what happened last time we went around this merry-go-round, back in January or so. Those keys were working for many months. https://www.reddit.com/r/fossdroid/comments/10b0krt/comment/j48xn4h/ https://www.reddit.com/r/fossdroid/comments/10b0krt/comment/...
- DeathArrow 3y agoIt seems the old API is working again.
- hospitalJail 3y agoI think this is why NewPipe doesn't work most of the time. Or if it does work, its absurdly slow.
- lolinder 3y agoNewPipe has been working for me for years with occasional breaking changes on YouTube's side that required an urgent update. I watch pretty much everything at 2x speed 1080p and have no issues with buffering.
- rompic 3y agoRSS does not work?
- zedeus 3y agoIt's disabled on nitter.net to reduce load while this new solution is being tested. Check the instance list for updated instances, most of them have RSS enabled: https://github.com/zedeus/nitter/wiki/Instances https://github.com/zedeus/nitter/wiki/Instances
- browningstreet 3y agoThanks for your work on this.
- sheepscreek 3y agoThis is the most impressive project in Nim I’ve seen yet. Rewriting any major front-end, complete with working authentication and handling idiosyncrasies of the private API is a herculean task. For context, Twitter would have a team of two dozen or more supporting what this does, effectively. Kudos to the author for accomplishing this feat!
- unsupp0rted 3y ago> For context, Twitter would have a team of two dozen or more supporting what this does Of course, because if Nitter goes down nobody bats an eye.
- Macha 3y agoAnd the flagship instance of Nitter is unusable more often than not - the fact that it's working today is likely an artifact of people having given up given the week long total outage.
- zedeus 3y agoMaybe you don't use it very often? For months before the recent chaos it was very reliable. The fact that it's working today is only because I fixed it, Twitter didn't change anything.
- lewantmontreal 3y agoWe weren’t supposed to talk about fight club
- snarkyturtle 3y agoI mean, nitter has pages indexed on Google so it's not exactly secret
- joos3 3y agoThis is absolutely amazing, as you still can’t view profiles or replies on Twitter without logging in. I’ve been a nitter user for 4 years now and will be as long as it works.
- DavideNL 3y agoNote that you can use RSS too - just append `/rss` to the url…
- synthoidzeta 3y agoTo the Nitter URL? Not working for me
- DavideNL 3y agoYea, there’s also an icon/link in the upper right corner of the page, just use that. It’s disabled on some instances…
- tourmalinetaco 3y agoUnfortunately RSS has been disabled on the main instance. It had been available before, but presumably given the rate limiting has been disabled.
- linusg789 3y agoThe alternate instances may still have them.
- I_am_tiberius 3y agoThis seems like Elon distributed free API tokens to specific 3rd party vendors?
- Macha 3y agoWhat reason could he possibly have to give them to Nitter, which has at its primary purpose undermining their conversion and engagement attempts? The old token was the twitter web token. I suspect the new one is one of the mobile clients. Maybe new tweetdeck. Though probably the iOS client token makes the most sense, being the hardest to rotate on a whim with app store review.
- I_am_tiberius 3y agoOk thanks.
- tgv 3y agoThey don't have indivual tokens? Well, let's not give them ideas.
- Macha 3y agoThe clients currently work logged out (exempting rate limits) and it's not like you can upload a per user copy of the app to play store/app store, so that root of trust needs to start somewhere which is what the nitter team can extract.
- jakear 3y agoHmmm... I was thinking you could limit based on the unique identifierForVendor[1], but without a way to verify that a given id is legit this would be easily circumvented. An API whereby Apple cryptographically signs a vendor/device-specific ID so you can effectively rate limit without needing any personal info whatsoever would be nice. 1: https://developer.apple.com/documentation/uikit/uidevice/1620059-identifierforvendor https://developer.apple.com/documentation/uikit/uidevice/162...
- nicolas-siplis 3y agoHey there, just wanted to thank you because you also fixed my Twitter Spaces downloader app[0]! After the API changes the default bearer token I was using (same as yours) stopped working, but after changing the same way you all's back to normal :D 0: https://github.com/Chiplis/moonbird https://github.com/Chiplis/moonbird
- tough 3y agoHey thanks I didn't know this app you built existed, will try it out.
- maxlin 3y agoAwesome, now my light personal-use scraper works again. Didn't even take a restart! Hope Twitter soon lets go of the temporary login restriction too. Given that this isn't completely blocked without a login, I'd expect that to be not far from now on. From what I've collected, I hope Twitter'd start selling dumps of their public data for a bit of a win/win with AI companies and Twitter itself.
- costco 3y agoI think this exists: https://developer.twitter.com/en/docs/twitter-api/enterprise/historical-powertrack-api/overview https://developer.twitter.com/en/docs/twitter-api/enterprise... but it is rumored to be absurdly expensive. Unless prices come down to at least the same order of magnitude as scraping I suspect people will be willing to risk the legal uncertainty involved with scraping.
- costco 3y agoWarning: long somewhat related story that is basically humblebragging, but the summary is that bypassing Twitter ratelimits is not very hard. I didn't feel like playing around with Twitter's annoying certificate pinning so I just uploaded the Twitter APK to Corellium, turned on what they call the "network monitor", opened the Twitter app since it lets you use Twitter without signing in. I clicked around, searched and viewed tweets. Then I looked at the requests in the log and saw it has a similar guest token process to the website but with a few differences. Anyways, if you recreate these requests, with one IP address you can generate a few OAuth tokens with no expiry per day. These tokens are for unauthenticated users so obviously they have no write privileges but that's not what was needed here. So if you have a proxy provider with a large pool of IPs where you can buy like 1GB of bandwidth you can use a very small percent of your bandwidth allowance and get thousands of tokens/secrets easily, all with their own separate rate limits. It doesn't even matter what IP you end up using the tokens on. Then I followed https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW4A5RAQW2he7qIpTmG-M/edit https://docs.google.com/document/d/1xVrPoNutyqTdQ04DXBEZW4ZW... and the fact that /statuses/lookup.json still allows you to return 100 (!) tweets at once to reconstruct something close to what the 50% Twitter firehose would look like. And Twitter doesn't even block datacenter IP addresses! Was going to display the data at https://firehose.lol https://firehose.lol but the fact that it required a few hundred requests a second made me feel bad so I didn't end up running the program for more than a few minutes at a time and shut it down. Looking at (a fraction of) the Firehose for a few minutes was interesting, originally I accidentally forgot to not display tweets labelled possibly_sensitive so I saw some pretty salacious material for a few seconds. Lots of Chinese gambling ads even though Twitter is blocked there, dubious investment promoters, accounts with usernames like FirstnameLastname3781264872 who would tweet three random words at each other every couple of seconds, and a handful of funny tweets.
- matteoraso 3y agoNice. This isn't nearly as efficient, but a simpler way to bypass the ratelimit is to use archive.md, which is immune to the ratelimit. It's useful if you don't have an account and just want to see a few tweets here and there.
- naillo 3y agoFunny how the effect of the rate limit has barely affected bots and scrapers at all but severely damaged the userbase of the site.
- MicropenisMike 3y agoIt's true for a lot of cases. Adding almost any restriction only hurts casual users, and attackers are rarely casual users.
- malermeister 3y agotinfoil hat time: what if that was the point all along? twitter was a vital space for organizing protests - think arab spring, occupy wall street. one of the richest guys in the world (who's also very anti-union, btw!) buys it up in a time where inequality is getting worse and worse and social fabrics are starting to tear and makes it unusable. no more space for organizing. one fewer threat to capital.
- klardotsh 3y agoGiven the dude paid $44B for a site clearly worth significantly less than that, and then promptly ran it into the ground, your "tinfoil hat time" answer honestly seems like the only rational answer. It checks out on more levels than any of his actions have.
- coldpie 3y agoYes, it seems plausible. It was also funded in large part by the Saudis, American banks, and other wealthy individuals who are all strongly incentivized to hinder activist communication networks. Now everyone who used Twitter has to re-form their networks elsewhere. Mission accomplished. https://www.aljazeera.com/news/2022/10/28/saudis-kingdom-holding-company-to-maintain-twitter-stake https://www.aljazeera.com/news/2022/10/28/saudis-kingdom-hol...
- immibis 3y agoWhy is this downvoted? It makes a lot of sense, and it's not some outlandish conspiracy theory (it's very inlandish) as we've seen rich people do similar things before, more than once, and dictators do it very frequently.
- sylware 3y agoIt means anonymous access has been restored. Nitter does not use twitter API which is a goner anyway (but was restored for a little while). Twitter should provide a noscript/basic (x)html interop www portal.
- ilikehurdles 3y agoAnyone remember threads? Whatever happened to that app?
- barbazoo 3y agoDidn't they release a beta last week?
- zirgs 3y agoStill not available in the EU. Probably because of privacy issues.
- sangnoir 3y agoLast I heard, it had 100M sign-up by its 5th day, nbd.
- immibis 3y agoThat number doesn't mean anything though. That's the number of people who looked at the app at least once. Give it a month and we'll see how much traffic it gets per day.
- sangnoir 3y ago100M users in 5 days doesn't mean anything? I understand waiting to see MAU & stickiness - but having the fastest-adopted platform of all time is something. Instagram could not have hoped for a better launch - all they have to do now is not screw it up like the other guy.
- charcircuit 3y agoThis does not seem legal. Stealing an access token to bypass access controls is illegal and I suspect these people didn't get permission to just scrape anything they want.
- beepbooptheory 3y agoI am not a lawyer but take your word for it. Is it illegal specifically in US, or elsewhere too? Either way, you'd surely agree its a noble pursuit? Just considering the wider context here!
- charcircuit 3y agoIt in illegal in most jurisdictions to access data which you are unauthorized to. >Either way, you'd surely agree its a noble pursuit? Considering it hurts Twitter's profitability by not showing ads, hurts Twitter's metrics by not having people sign in or sign up, hurts users who were accidently signed out from signing in to twitter and having a better user experience, and hurts content creators because nitter doesn't allow you to like or retweet posts. I do not see it as a noble pursuit.
- immibis 3y agoMore people need to remember that freedom works both ways. Twitter has the freedom to put up walls on its own property, and other people have the freedom to put up ladders on theirs so they can see over the walls. Although law enforcement always seems to end up biased towards the billionaires, it's not an automatic process and they have to go through some contortions to find an excuse, because the law says you and Elon Musk are equal. There's the CFAA, which is a blatantly unfair law targeting any computer activity billionaires don't like. I hear it hasn't been used in this way for a while, and not many times ever, but if it does get used on you you'll wish you were dead, but some people seem to be okay with low-probability high-impact risks. You might even be found innocent if you convince the judge you are authorized to access public tweets. It's not like you're running an SQL injection. There's the DMCA's anti-circumvention clause, but that's written by Hollywood billionaires for Hollywood-bought judges to abuse. Elon Musk can have a fun time trying to convince them his platform is equivalent to Disney to get a favourably corrupt judgement. Other than that, what's stopping you from sending any request you like to access public data? You can say your user agent is Snoopy the Dog, you can say you would prefer to accept MIME type ascii/emojipoo, you can pass the server 1000 IDs at once even though it won't give you that many, and you can tell it you're the Twitter app on Android. Remember: I am not a lawyer and this is not legal advice.
- gadders 3y agoCan someone do Fritter next please so I have a decent android Twitter client?
- jonjomckay 3y agoThe latest CI/GitHub Actions builds have a fix for this implemented already! I'll be publishing another beta once I have enough time to QA all the latest changes on the Fritter and Twitter side.
- gadders 3y agoThanks!
- flotzam 3y agoRight now the Squawker fork of Fritter is working fine. You can export from Fritter and import into Squawker (and presumably vice versa if Fritter catches up). It's in the IzzyOnDroid F-Droid repository: https://apt.izzysoft.de/fdroid/index/apk/org.ca.squawker https://apt.izzysoft.de/fdroid/index/apk/org.ca.squawker
- al1r4d 3y ago[flagged]
- linusg789 3y agoNot sure why you're being downvoted (Alhamdulillah - "thank god" in Arabic)
- ec109685 3y agoThis is like the analog loophole. If data is transferred from Server to Client, it's always possible to scrape. Though Apple is trying to make this harder: https://developer.apple.com/documentation/devicecheck/preparing_to_use_the_app_attest_service https://developer.apple.com/documentation/devicecheck/prepar...
- Tokumei-no-hito 3y agoI am not an iOS dev. What is the significance of App Attest wrt the loophole used here?
- ec109685 3y agoWith attest, Twitter can tell if there's an actual mobile device hitting Twitter with that bearer token versus today, you can spoof the client and unless they fingerprint the IP address or other subtleties, it's hard to distinguish fake traffic from real traffic.