25 ms·
Twitpic blocking archive team from backing up pictures
- ChuckMcM 12y agoThis feels like a hostage situation again. The last time they said they were going away they also prevented people from downloading their images. I'm having a hard time understanding that (granted S3 'out' bandwidth costs money but still.)
- craigmccaskill 12y agoMaybe they're having issues fulfilling individual users requests, so blocked the archive team to help reduce stress? I can't imagine they'll want to spin up new machines, so it makes sense to throttle/limit/block. Not sure I'd immediately jump to malice here.
- duskwuff 12y agoNo, like, right now images hosted on Twitpic are completely inaccessible. You can't view them at all.
- craigmccaskill 12y agoThat doesn't mean this isn't an effort to reduce stress on the user collection download system. I'm not saying that it isn't malicious. I just don't think it's wise to jump to that conclusion straight away.
- dthunt 12y agoName the next plausible explanation and how likely you think it is relative to 'save money, screw your data'. I think you'll have takers on a bet at those odds, given what you just said.
- deleted 12y ago[deleted]
- influx 12y agoIf they are on S3, would they be willing to take money to do a data export?
- IvyMike 12y agoThus far the Twitpic guys haven't been very open or cooperative with the archive team guys. https://twitter.com/textfiles/status/523162680400830464 https://twitter.com/textfiles/status/523162680400830464
- toomuchtodo 12y agoI sent a tweet to Noah offering to covert all costs of an export. We'll see what happens.
- saalweachter 12y agoErr, if they're on S3, isn't data transfer to EC2 free? [1] If I'm reading the pricing chart right, why would it cost Twitpic anything for someone else to spin up some EC2 instances and clone their data thataway? [1] http://aws.amazon.com/s3/pricing/ http://aws.amazon.com/s3/pricing/
- Mithaldu 12y agoImportant to the situation is also that, as of ~24hours ago, the users themselves couldn't download their own pictures: https://twitter.com/textfiles/status/522837349676236801 https://twitter.com/textfiles/status/522837349676236801 Edit: While at the same time twitpic had no announcement on the frontpage and users were still uploading: https://twitter.com/textfiles/status/522835537543974913 https://twitter.com/textfiles/status/522835537543974913
- SrslyJosh 12y agoAnd that's still the case. Twitpic's "download" link (from current the front page) is handing out 503s, and has been all day: http://twitpic.com/account/settings http://twitpic.com/account/settings I wish I could say this will be a good lesson in not trusting startups with personal data, but it won't be.
- eropple 12y agoPart of it is that almost every service you interact with on the internet is a "startup". It's not reasonable to expect everybody and their (literal) grandmother to know who's got an investor-friendly P/E.
- Retric 12y agoYou are really in a bubble. I can't think of any start-ups's I regularly interact with. Now older companies that might or might not be profitable sure, but 'start up's' almost by definition don't support large chunks of the worlds population.
- serve_yay 12y agoI dunno what the hell these people's problem is. Nothing about their actions makes any sense to me.
- brm 12y agoThis whole thing reads weird. From shutting down, to acquisition, to shutting down again and seemingly blocking exports. However, as this is the internet, until news breaks otherwise I will continue to believe that Hanlon's razor applies here... Never attribute to malice that which is adequately explained by stupidity.
- IvyMike 12y agoEither way, I wonder if the twitpic guys fully understand the damage they may be doing to their personal reputations. Do they expect to be able to run a web service in the future? Because if the current situation persists, no one will ever trust them again.
- gtaylor 12y agoIf we're being honest, the vast majority of their users don't know who was behind Twitpic. The HN crowd is probably a lot more likely to know/care than your typical Twitpic user. They'll probably be able to start new things without most people noticing. And they probably know this.
- javajosh 12y agoWell, yeah, but it's the HN crowd that they will likely want to work with/hire from, so reputation is actually important.
- jamesaguilar 12y agoWell, a few things. First off, a lot of people on here probably understand that when you don't have money to pay for bandwidth, banning people who use a lot of it might be reasonable. It's not like you have a whole lot of other options. Passing around the hat isn't likely to work, and if they could get additional funding, they already would have. Second, even if people didn't understand that, the HN crowd is a tiny slice of the good hackers out there. Most of the best people I know do not read it, or even actively disdain it. "Cesspit" and the like are a terms I have heard in person or on newsgroups used to describe the comments section here. Third, I'll bet most HN people don't even know the names of Twitpic's founders. I don't. I wouldn't be able to identify whether another company, down the line, was founded by them. Lastly, I doubt most HN people are so principled that a year or two from now they would turn down an otherwise compelling job offer because the founders of the company were once associated with banning Archive Team's IPs. It's mildly annoying at worst.
- sp332 12y ago"you let the visual component of arguably the greatest communication shift/revolution in the 21st century be under the whim of one idiot". 800 million pictures, just gone. For example https://twitter.com/textfiles/status/523160725158903808 https://twitter.com/textfiles/status/523160725158903808
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]
- toomuchtodo 12y agoAssuming ~120TB [1], that would cost ~$1300 in AWS Export charges, plus whatever the drives cost. Just sent a tweet to Noah offering to cover all AWS export/drive costs to export the archive. [1] https://twitter.com/textfiles/status/523162867789758464 https://twitter.com/textfiles/status/523162867789758464
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]
- sethish 12y agoYes. This was discussed at length with the Twitpic founder. He said something to the effect of: "don't worry, I'm working on a deal, trust me. But stop downloading the pics"
- toomuchtodo 12y agoWas this in the last 48 hours? I'm hanging out in #quitpic as rumbles if that's a better place to discuss.
- mariusz79 12y agoCome on, wasn't twitpic free? What why should they keep paying for hosting if the business went under?
- joshstrange 12y agoLet's not pretend that they didn't make money off the ads they placed on their websites. Yes it was "free" but they weren't doing this out of the goodness of their hearts. Also any goodwill they may have gained in their heyday is all but used up now with this whole shutting down, not shutting down, shutting down, fuck you and your pictures situation.
- Houshalter 12y agoIf it was profitable they wouldn't be shutting down.
- bdcravens 12y agoNot necessarily true. It stems from Twitter demanding they abandon their trademark. http://blog.twitpic.com/2014/09/twitpic-is-shutting-down/ http://blog.twitpic.com/2014/09/twitpic-is-shutting-down/
- sp332 12y agoPaying for hosting? They were making money from ads they put next to our pictures. They were using our content to drive traffic to their site. Without the user's photos, the site is worthless. They should have been paying users a cut of advertising revenue like YouTube does. "Free" means they're pocketing the profit for themselves.
- pistle 12y agoIf they had profits, maybe they wouldn't be closing down. Those ads were to pay bills and they are broke. Allowing massive bandwidth usage now is just throwing bad debt after bad.
- idlewords 12y agoFor context, before archive team was blocked Jason Scott commented on Facebook: "Archive Team has 900 IPs aimed at Twitpic. We're grabbing 100 pictures a second." He subsequently corrected this figure to 42,000 photos per hour, which works out to about 11/sec. Later he adds: "In some cases, twitpic (before removing all the image access) was banning entire ISPs to stop archive team backing it up."
- rgbrenner 12y agoIn fairness, the archive team is attempting to download 120TB of data.. which is going to cost Twitpic about $15K in outgoing data charges from Amazon. Maybe they just don't have $15K to pay for this. (and that's not counting all of the users who are also downloading their data)
- tedivm 12y agoI have absolutely no doubt that people would be willing to donate to cover the archival costs, especially if that donation was overseen by the archive team. Options like this can only be explored if there is a conversation.
- pjc50 12y agoRemember, any cloud service you use may shut down and delete all your data tomorrow, whether through bankruptcy, acqui-shutdown, accident, or vandalism. If it's a free service you will have no comeback. If it's a paid service you probably won't have a comeback either.
- iamtew 12y agoFree service yes, but if you're paying for SaaS and rely on it for your business you should really get a source code escrow agreement in place. That will allow you to get the necessary components from the provider in case they go belly-up or such. Of course hosting the application is a whole different thing, but hey, at least you got all your data available.
- pjc50 12y agoHas anyone successfully done the escrow thing? That would also only be available to the very largest customers.
- Istof 12y agoalso data corruption... which happened to me on Skydrive (now named Onedrive)
- cperciva 12y agoIn their defense, it's entirely possible that they simply don't have the capacity to handle the large volume of requests...
- djloche 12y agoSeeing as they're shutting down the company/service in a week, they're probably out of cash. While it sucks for your customers, that's what happens when you run out of money. It sucks for everyone. I'm guessing they were floating the company on personal funds in hopes the acquisition would go through, but it did not - so now they're doing everything they can to stop this turning into a personal financial bloodbath from the bandwidth costs spiral due to everyone rushing to get their photos before the shutdown. Just an ugly situation for everyone.
- eropple 12y agoThey're straight-up refusing to talk to Jason about giving Archive Team a data dump. There's an ugly situation, but only one side is precipitating it.
- billmalarky 12y agoMaybe they are holding out that someone will purchase still? They lose all their value the second the archive team gets all their data right? Edit: On second thought, I agree that it's probably related to the bandwidth cost.
- click170 12y agoSneakernet could be a cost effective option especially for large volumes of data, and its often overlooked. I wonder if the Archive Team has considered sending twitpic a hard drive and asking them to just rsync the data over? Edit: This assumes that they keep backups of said data outside of EC2. Walking a hard drive over to your local Amazon DC.. won't quite work.
- toomuchtodo 12y agoAmazon will provide drives for their export feature. People have already offered to foot the bill to have it done.
- stevencorona 12y ago
- wmeredith 12y agoIt was a free service right? Whatever happened to easy come, easy go? The indignation around this is alien to me.
- jamesbrownuhh 12y agoIt's more about having a moral obligation to history. Twitpic has been a home for hundreds of millions of images for a good chunk of a decade during which the most monumental shifts in communication and expression have occurred. Unique and irreplaceable moments have been captured in this place. To turn around and delete it all just because you didn't get the trademark application you wanted (and apparently can't be bothered to just think of a new name instead) is nothing other than an unprecedented act of wanton, petulant vandalism against history. It would be like the world's biggest art gallery turning around and saying 'Oh, sorry, they won't let us call our gallery McDonalds, so we're shutting down and burning all your pictures. After all, easy come, easy go.'
- idlewords 12y agoOkay, but we keep getting burned by these irreplaceable droves of stuff going offline (see GeoCities), and all our indignation doesn't bring them back. At some point we have to learn this lesson for good, and stop trusting random entities to be good stewards of things we find meaningful.
- ObviousScience 12y agoAs I understand it, GeoCities actually cooperated to some extent with the backup team that was trying to preserve the content, and didn't take steps to keep people from backing up content.
- db48x 12y agoGeoCities was entirely unmaintained; there was apparently only one person at Yahoo who was paid to do anything with it at all, and who couldn't even answer questions such as how many users there actually were. The primary pain point when spidering it was that each GeoCities user had a tiny hourly bandwidth cap; to do the job right the spiders had to keep track of the error responses that they got so that they could go back later in the hope of getting the real content rather than the error message.
- jtchang 12y agoThis whole thing is a fiasco. But maybe if it wasn't Twitpic wouldn't get as much exposure. My bet is that this is just a giant media whoring play. If you want to save the company data then you have to be transparent and ASK for help. If it is going to cost $15K someone might very well step up. But if you are tightlipped then people will assume the worst.
- inzax 12y agoAs shown on https://www.facebook.com/pages/Hacker-News/1558365107716690 https://www.facebook.com/pages/Hacker-News/1558365107716690
- ddorian43 12y agoI just opened the archiveWarrior and it's downloading from twitpic. Help them: http://archiveteam.org/index.php?title=Warrior http://archiveteam.org/index.php?title=Warrior
- asiekierka 12y agoIt's only downloading comments and metadata, not the images.
- CanSpice 12y agoI saw this and thought "eh, that doesn't bother me, I already used the 'export your photos' thing." So I went to my TwitPic settings page, clicked the link, and downloaded a 14MB zip file. The only thing is, it's empty. I don't know how you get a 14MB zip file that's empty, but Windows is telling me it's empty. That's not cool.
- unimpressive 12y agoAdvice on your issue from #quitpic: 14:23 <REDACTED> you have to enable the "show hidden folders" in windows, under their view options 14:24 <REDACTED> each version of windows is slightly different but it is usually press alt, select tools, folder options, view, and the radio next to show hidden files and folders and then ok 14:24 <REDACTED> then you close and reopen the zip file 14:30 <REDACTED> oh, and sometimes the first download is corrupt. 14:30 <REDACTED> I've had to help too many people at work with this problem plus a person or two on this channel
- ertdfgcb 12y agoWhat the hell? I wonder why that's the case. I can see why they wouldn't have time or money to fix it, but why is it that way in the first place?
- CanSpice 12y agoThanks, but that doesn't appear to be the problem with my file. I tried to extract the files, and then Windows tells me that the archive is invalid.
- selectodude 12y ago>14:30 <REDACTED> oh, and sometimes the first download is corrupt.
- differentView 12y agoWhat's the big deal about these pictures? It's not the first time a major service has shut down. I am assuming historically important images have been saved and hosted elsewhere.
- AznHisoka 12y agoi agree. what's so important about foodie pics, selfies, and baby pics you should have backed up already. Trust me, nobody will notice if Twitpic disappears overnight. Stop putting so much importance in such a trivial company.
- Argorak 12y agoThe pictures, yes, but not the conversation about them.
- differentView 12y agoWhy are those conversations important?
- dumbfounder 12y agoMajor headache for Twicsy! We now have to delete about a billion pictures from our index. Which isn't trivial... such a shame. We reached out to him several times over the past few weeks to see if we could arrange something, but he never responded.
- sp332 12y agoDo you have a cache of comments or photos from Twitpic? Could you stop by #quitpic on efnet and say hi? :)
- stevencorona 12y agoI used to be the CTO of Twitpic (I left about 2 years ago). It sucks to see your work crash into the ground like this. Just so it's clear, there's just Noah (and his parents) that are left running the company.
- deleted 12y ago[deleted]
- yuhong 12y agoDid you notice the people trying to contact Noah Everett?
- stevencorona 12y agoYes and he won't reply to me either.
- wasd 12y agoCan you put him in touch with Dustin Curtis? Dustin Curtis: "Can anyone put me in touch with the founder of Twitpic?(I'll pay to backup all of the photos, and even host them. Ridiculous situation.)" hi@dustincurtis.com
- nkozyra 12y agoGiven these points: - The whole shutdown / acquisition / shutdown saga - Tepid response wrt getting compensated to export data Someone's still holding out hope for an injection of capital. Because once the export happens and the data is gone, the value is entirely lost. I wouldn't bank on getting any traction on paid exports until the last possible second.
- thesis 12y agoI might be unfamiliar with how the archive team operates but shouldn't this have been ongoing over the years rather than all at once now?
- asiekierka 12y agoThe Archive Team only backs things up when there is a risk of them being gone. They cannot do it any other way - the Internet Archive does not have infinite resources to host things, and they need to prioritize. 120TB is not an easy thing to host.
- kalleboo 12y agoThat's what the Internet Archive tries to do. But they have finite (or rather, extremely limited) resources. The Internet is vast things fall through the cracks. The Archive Team is the "oh shit this site is shutting down and it's not all in the Internet Archive already, we gotta save it before the deadline when it's gone forever" squad.
- jszymborski 12y agoI don't understand why they wouldn't just put the pictures up on BitTorrent?
- Xorlev 12y agoThat's a LOT of data. 100s of TBs. Good luck.
- jszymborski 12y agoDoesn't the torrent protocol lend itself to huge amounts of data? Plus, you can select which files to download in the protocol.
- malandrew 12y agoI reckon the feature that is missing is not the ability to pick and choose, but instead to automatically assign people trying to help preserve something to one chunk, with the idea of getting full coverage of the entire data set. If there are 100 pieces, and 500 people, you want each piece to be held by 5 people for 5x redundancy. And, if nodes leave the swarm, the system automatically rebalances which nodes are maintaining a particular piece of the dataset. The only solution being worked on out there that does this is IPFS https://github.com/jbenet/ipfs https://github.com/jbenet/ipfs https://github.com/jbenet/node-ipfs https://github.com/jbenet/node-ipfs https://github.com/jbenet/go-ipfs https://github.com/jbenet/go-ipfs
- bond 12y agoEvery time I read about Twitpic closing I remember this... http://mixergy.com/interviews/twitpic-noah-everett/ http://mixergy.com/interviews/twitpic-noah-everett/
- paulhauggis 12y agoShould we now have this expectation for any free github project that's popular?
- paulhauggis 12y agoI guess the hypocrisy and entitlements will continue....