5 ms·
They currently have 150B posts (https://www.tumblr.com/about https://www.tumblr.com/about) so I expect that to be quite challenging. If you seriously make any p
by Veratyr 9y ago
They currently have 150B posts (https://www.tumblr.com/about https://www.tumblr.com/about) so I expect that to be quite challenging. If you seriously make any progress though, it'd be cool to see.
- nickpsecurity 9y agoStep 1: Hack into a supercomputing center with at least 1Gbps line. Step 2: Massively-parallel downloading of all the sites using clustered nodes, compression of it, and resulting data stored into high-performance, clustered filesystem. Step 3: Move it off of there when traffic is low or overnight if system doesn't go offline overnight.
- Veratyr 9y agoStep 0: Acquire at least ~1PB of storage to store all the data.
- toomuchtodo 9y agoI have over 1PB of storage at my disposal.
- Veratyr 9y agoBusiness or personal? It's doable but it's a lot of money to buy all those drives.
- boulos 9y ago$6k for 600T fully assembled backblaze storage pod: https://www.backuppods.com https://www.backuppods.com (no affiliation, just pointing out that ~$12k isn't a massive amount).
- noisem4ker 9y agoNote that hard drives are not included. 1000TB / 8TB = 125 HDDs 125 * $200 = $25k
- boulos 9y agoAww! Damn my quick posting (and too good to be true!). Thanks.
- toomuchtodo 9y agoPersonal.
- nickpsecurity 9y agoIt's in the supercomputing center. How aboug I modify it where a filterimg step is run deleting everything that doesn't match on desired image festures?
- Veratyr 9y ago> Step 3: Move it off of there when traffic is low or overnight if system doesn't go offline overnight. Where are you moving it to? You think that even if you manage to hack into a "supercomputing center" that nobody's going to notice 1PB of storage filled with GIFs?
- toomuchtodo 9y agoHaving worked in a supercomputing center for a detector at the LHC, yeah, you could probably get away with storing a few hundred TBs for a short period of time without anyone noticing or caring. A whole petabyte might be pushing it.
- nickpsecurity 9y agoAnd I learned about it from you people. All of them said security was lax. Most of them said they personally were using the supercomputer for their own stuff at some point.
- toomuchtodo 9y agoWhat's this "you people" bub? :) in this case, it's not a security issue, it's a resource accounting issue.
- nickpsecurity 9y agoPeople working in and around HPC centers. Obviously. :) And no, it's both accounting and security issue. One guy I know who does security in ASIC's that stole HPC time in the past did it by modifying the accounting system to not show his jobs. It was easy as it wasnt designed to stop accounting fraud by hackers.
- awalton 9y agoMaybe 1:100 of those (being incredibly generous) are actually unique posts and not someone else reposting something from someone else, so maybe it's not such a hard task...
- tangent128 9y agoIt's fairly common for commentary to be added to a reblog via tags rather than after the quoted post, so even reblogs without commentary may need saving.
- jandrese 9y agoThis assumes there is commentary on Tumblr worth saving.
- PhasmaFelis 9y agoI'm guessing you've bought into the meme that Tumblr is literally nothing but left-wing politics.
- amiga-workbench 9y agoThat and porn, and Stephen Universe fans.
- lmm 9y agoAre you saying that meme is inaccurate?
- breakingcups 9y agoAre you implying it's not?
- lmm 9y agoYes. The meme aligns with my own experience of tumblr, so I'm inclined to believe it over contrary anecdotes (of course a more rigorous study would be a different story)