5 ms·
I created a tool that uses Twitter's public facing (but private) API. https://github.com/pauldotknopf/twitter-dump/ https://github.com/pauldotknopf/twitter-dum
by pknopf 6y ago
I created a tool that uses Twitter's public facing (but private) API.
https://github.com/pauldotknopf/twitter-dump/ https://github.com/pauldotknopf/twitter-dump/
You can use it to download every tweet from a user, not just the last 3000 that their API supports. It uses the same query syntax that the web search uses.
- arkanciscan 6y agoDoes it work on suspended accounts? Mine got reported by a bunch of Trump supporters so I lost 17 years of tweets and Twitter won't listen. I can still see my own tweets, but only a page at a time. I'm thinking I might need to use a scraper like Twint instead of the API.
- jchook 6y agoIf you can you see your own (suspended account) tweets when logged in, you could use your auth cookie to scrape them. Not sure how to do that with Twint off the top of my head, though. We tried to use Twint in production but it didn’t work for us. I ended up writing one that works very well. Let me know if I can help you get your tweets.
- arkanciscan 6y ago> you could use your auth cookie to scrape them Seems that Twint does all it's work unauthenticated so it may not be possible. I suppose I could try using the API but I don't think I care that much.
- jchook 6y agoLooks like you could just paste your auth header right here: https://github.com/twintproject/twint/blob/a5df26d988109838d3fddb77d9c20e8a1b5addf7/twint/run.py#L49 https://github.com/twintproject/twint/blob/a5df26d988109838d...
- arkanciscan 6y agoGood looking out, but I don't think that's working. > PS C:\Users\Jesse Hattabaugh\Documents\GitHub\twint> twint -u jessehattabaugh > CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable
- pknopf 6y agoIf you can see it while you are logged in, then yes. Check ```twitter-dump auth``` for instructions on how to use your web cookies with the command. When the command queries Twitter, it will look as if it is coming from you.
- arkanciscan 6y agoNice! I followed the instructions but after I paste the curl command I get this error: Unhandled exception. System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length') at System.String.Substring(Int32 startIndex, Int32 length) at TwitterDump.Program.ParseCurlCommand(String curlCommand, Dictionary`2& headers) in /home/pknopf/git/twitter-dump/src/TwitterDump/Program.cs:line 150 at TwitterDump.Program.Auth(AuthOptions options) in /home/pknopf/git/twitter-dump/src/TwitterDump/Program.cs:line 113 at TwitterDump.Program.<>c.<Main>b__0_1(AuthOptions opts) in /home/pknopf/git/twitter-dump/src/TwitterDump/Program.cs:line 33 at CommandLine.ParserResultExtensions.MapResult[T1,T2,TResult](ParserResult`1 result, Func`2 parsedFunc1, Func`2 parsedFunc2, Func`2 notParsedFunc) at TwitterDump.Program.Main(String[] args) in /home/pknopf/git/twitter-dump/src/TwitterDump/Program.cs:line 30 What'm I doing wrong?
- arkanciscan 6y agoNM, I figured it out. Chrome has several "copy as curl..." commands now. I was using "Copy as Curl(cmd)" but "Copy as Curl (bash)" is what worked.
- arkanciscan 6y agoDude! Thanks so much!!! If you're ever in Portland, I owe you a drink!
- arkanciscan 6y ago::reads old tweets:: Oh god... no wonder they suspended me
- arkanciscan 6y agolol @ the Trump supporters downvoting this. Y'all tired of all the winning?
- nvr219 6y agosnap