5 ms·
Ask HN: What is nitter and why does it still work?
You can't access X without signing in, then there are the rate limits if you are signed in, but these things are not an issue with nitter. I went from checking Twitter as a logged out user every few days, to not using it at all after the access changes, to casually using nitter to check in on a few accounts I like.
https://nitter.net/search
- gadders 3y agoNice try Elon.
- linusg789 3y agohttps://github.com/zedeus/nitter/ https://github.com/zedeus/nitter/ I think it uses the same functionality as the guest account feature in the Android app as of now.
- loveparade 3y agoIf that's true, isn't it just a matter of time before it gets shut off? I don't really see a way to keep this working without somehow crowdsourcing accessing X. Even with an automated way to create fake accounts with phone verification, it would eventually be detected or guarded against.
- toomuchtodo 3y agoThat’s how archive.today works with news publication site accounts, will be fine if you have to make round robin requests with a bucket of X accounts.
- loveparade 3y agoI would disagree here, I think it's different for several reasons: - The volume and velocity of data and API requests needed for X is much larger than scraping the occasional news article. You'd need a huge number of accounts to make it work. - X has the technology to reliably detect such accounts unless they are incredibly well "hidden" by e.g. crowsourcing requests through regular accounts with normal browsing behavior - X has the incentive to reliably detect such accounts as it has shown in the past by shutting off data access. It seems to care much more about this than most news websites I believe the only reason things like Nitter still work is that it's not yet big enough for X to care about and invest resources into guarding against it, especially with all the other stuff going on at X right now. But it's only a matter of time before that changes.
- toomuchtodo 3y agoI can tell you that a project is already doing this today for archiving with no signs of mitigations (request denials, poisoning the responses, etc). It’ll likely eventually end up in the usual places when you want to see something from way back in the day.
- nitwit005 3y ago> But it's only a matter of time before that changes. If anyone at the helm is paying attention, which seems iffy these days.
- midoridensha 3y ago>- X has the technology to reliably detect such accounts unless they are incredibly well "hidden" Has, or had? With massive layoffs, they might not have the staff or know-how left to do this, or the time to implement it. >- X has the incentive to reliably detect such accounts as it has shown in the past by shutting off data access. It seems to care much more about this than most news websites OK, but still wanting to do something, and actually doing it, are two different things. I'm not convinced this hollowed-out shell of a company still has such ability. >especially with all the other stuff going on at X right now. But it's only a matter of time before that changes. This seems to assume X is going to continue to survive and in fact thrive; I don't think that's a good assumption.
- da9 3y agoThey arent scrapping the whole site. The entire site is 99% spam. Its scrap on request, if its not already cached.
- loveparade 3y agoSure, but that doesn't make a difference. Even the top 1% of the most requested timelines and tweets are huge amounts of requests. That's many many orders of magnitude more data then new articles and requires a large number of accounts.
- pimlottc 3y agoThat should be “scrape”/“scraping”, unless you mean they should be throwing the site away :)
- aaron695 3y ago[dead]
- rsynnott 3y agoI mean, possibly? The loginwall thing is a fairly obviously silly strategy which they may not be excited about doubling down on, though (I would expect it's fairly likely that the loginwall gets reversed at some point; they've already stepped it down so that non-logged-in users can see tweets). But honestly, at this point, _everything_ about twitter is up in the air. Who knows what state it'll be in in a year.
- brucethemoose2 3y agoHistorically it worked by using Twitter's undocumented internal API, but I have no idea what it does these days. Its not even getting rate limited anymore.
- 32gbsd 3y agoNo clue how it works nowadays
- JCharante 3y agoThe author made a post bragging about how easy it was to circumvent rate limits since X doesn’t even limit by IP address, so they had 1 address making requests for like 1000 accounts without any problems.
- supriyo-biswas 3y agoWhen I checked this a few days ago, Nitter wasn’t returning the latest data, so aggressive caching would be one of the tricks they’re using.
- nyanpasu64 3y agoI self-host a LAN-only Nitter instance for personal use. In the last few months of Elon trying to lock out third-party clients, it's gone from automatically acquiring tokens, to tweet access breaking and coming back and eventually breaking again, then only showing "top tweets" for users rather than full post history, to having to log into Twitter in a browser and extract some cookies/headers using Developer Tools (and failing to show user post history because they tried using RSS feeds that got pulled). The latest development is the use of "guest tokens" (https://github.com/zedeus/nitter/pull/985 https://github.com/zedeus/nitter/pull/985) that you have to create and install by hand (running a Node script) because Nitter hasn't automated the process yet. This works well for viewing all tweets, but hasn't been merged to the master branch yet (but nitter.net and the AUR nitter-git package are already using it).
- deng 3y agoNitter currently uses so called "guest accounts" and a proxy network. Start here if you want more details: https://github.com/zedeus/nitter/issues/983#issuecomment-1683134811 https://github.com/zedeus/nitter/issues/983#issuecomment-168... The required proxies are the reason currently only nitter.net is working, but not the other hosted instances.
- untitaker_ 3y agoother instances are working. they are denoted at https://github.com/zedeus/nitter/wiki/Instances https://github.com/zedeus/nitter/wiki/Instances and more readably at https://status.d420.de/ https://status.d420.de/
- runjake 3y agoPreviously, found using HN search: https://news.ycombinator.com/item?id=37397715 https://news.ycombinator.com/item?id=37397715 Spoiler: because apparently, it's API access somehow hasn't been pulled yet.