13 ms·

Storage for Photographers

263 points by PStamatiou 13y ago

ukd1 13y ago
Seems like http://www.loom.com/ http://www.loom.com/ would be great?
- PStamatiou 13y ago
  Not much info on it. What does it do? Email me if you're the founder. :)
  - ScottBurson 13y ago
    The blog has a pretty good intro.
- antr 13y ago
  I've been on the waiting list of Loom for ages. Thought it was in the deadpool...
  - ukd1 13y ago
    Nope, think they're just perfecting things!
- kevingibbon 13y ago
  +1 for loom. I've been using it for a while now. It's pretty awesome!
jeffasinger 13y ago
I believe that https://www.mediacloud.cc/ https://www.mediacloud.cc/ is trying to solve that right now. However, their website is sparse on details.
- PStamatiou 13y ago
  Definitely quite a few companies are attempting something in this space. I think the one that will win will be the one that actually has photographers as founders and understand the need. Unfortunately it is quite a niche and most photographers are gadget hounds and don't mind having 2-3 drobos (including an off-site one).
  - StacyC 13y ago
    It appears that Loom is founded by photographers. http://www.loom.com/about http://www.loom.com/about I'd love to hear more about this service from some users. I am bumping up against Dropbox limits with my photos as well.
petercooper 13y ago
If you want to avoid having any backup devices yourself, would you need to upload to two or more Glacier-like services? I also trust Amazon to stay in business but mistakes can always happen (not just hardware but ID theft, card expiry while you're on a 6 month hike, etc) and unless you're prepared to say "losing these is OK" using one service would be risky.
- PStamatiou 13y ago
  This is true, but what other Glacier-like services are there that have $0.01/GB/mon storage + retrieval?
  - dublinben 13y ago
    Offsite tape backup can approach that price, but I don't know of any exact alternatives. It would certainly fulfill the requirement of having two different media types and providers. http://www.dpbestflow.org/backup/backup-overview http://www.dpbestflow.org/backup/backup-overview
joebo 13y ago
How about git-annex assistant with the glacier remote? http://git-annex.branchable.com/special_remotes/glacier/ http://git-annex.branchable.com/special_remotes/glacier/. It will move files off your local once they have been transfered to the remote.
- zellyn 13y ago
  +1 to git-annex assistant.
- PStamatiou 13y ago
  The large part of this post for me is about the local thumbnail exports of the RAWs so I actually know what files are what without having to go just by album names, dates et cetera.
  - joebo 13y ago
    You could keep the thumbnails local and annex the raws. Based on the thumbnails, you could selectively git annex get from the remote glacier
    - PStamatiou 13y ago
      good point
azov 13y ago
Backup is about having several copies. If you delete files after you uploaded them to the cloud, you don't have a backup - with all the problems that come with it (say, if you delete a file by accident there will probably be no way to restore it). If your files are important to you - you need a back up. If not - just get rid of them in the name of simplicity :) PS. Also, uploading many gigabytes over a typical ADSL connection is just painfully slow... even local NAS is too slow over WiFi for my taste, and if I have to have a wire - I can just as well connect an external drive.
- PStamatiou 13y ago
  I only used Glacier in this example but I did mention how it could be tied to additional cloud services. The thing is that not many like cheap long-term Glacier alternatives exist yet. Rest assured when they do, this ideal app would have them in the settings. :)
- apaprocki 13y ago
  I wanted the benefit of Glacier plus backup functionality so I went with Arq. It encrypts everything and supports Glacier. Works really, really nicely.
  - tankbot 13y ago
    I use Arq as well. Easily one of the best backup options for Macs.
- dtauzell 13y ago
  I backup offsite via CrashPlan which keeps files even after I delete them. I sent them a hard drive for my initial upload. Now, I've got about 800M backed up. I haven't yet had to get it all back :)
famousactress 13y ago
Yes, PLEASE. We run a wedding photography business, and our numbers are something like this: - 40-60gigs shot in a weekend, need to offsite the RAWs as soon as possible. In six or seven years we've never needed offsite retrieval. Fine for it to be slow. - Only need around 100-200g of raws locally for jobs in progress - Once processed into final jpgs files are approx 8-10x smaller. Offsite these as well, and need access to last 1-2 yrs of jpgs randomly/sparsely w/in 1-2 days of a print order coming in I definitely think some well thought out service that uses maybe a hybrid of glacier and S3 could be really awesome. Especially if the pain is taken out of getting the bolus 40-50g of photos "uploaded" in the first place.. We'd happily pay for a service that just sends us an external drive and a postage-paid box.. we drop it in the mail after a job and get another drive sent to us right away. [EDIT: Occurs to me you might wonder what this looks like today. Today we get home, and immediately back up onto an 8-bay Drobo Pro (http://bit.ly/1diZUHD http://bit.ly/1diZUHD). It's a fancy proprietary RAID-like system. I think ours has 7 1TB Drives in it currently, and it's at about 70% capacity. Then we back up the same jobs to an external hard drive which gets driven offsite (used to be my office, but now that I work at home it's the in-laws place) and copied to a hard drive for permanent offsite storage. Once that happens, the cards from the camera can be cleared. After processing, the final JPGs get copied to the drobo and make it into the sneakernet offsite process as well.]
- PStamatiou 13y ago
  This thread is becoming great customer research for people to build this app. :)
  - rfnslyr 13y ago
    You can all go home. I've already started.
    - Veratyr 13y ago
      Seriously? Got a repo we can watch? Or a blog we can follow?
- alinspired 13y ago
danso 13y ago
I'm in a similar situation as the OP...an amateur but avid photographer with terabytes of RAW image files. Currently, I have several multi-terabyte drives that I erratically back up to...I just got a 4TB drive that I'm going to try to backup everything I've ever shot in the last 3 years... But I do want to move it to online storage...and I think this requires triage. For nearly every photoset, I've quickly gone through them with Lightroom and starred the ones that I kind of like, and then for maybe 1% of them, have taken the time to fully process, label, and upload them (in JPG form) to Flickr. So for online storage, I think what I'll do is write a batch script that dumps all the unstarred RAW files as JPGs, because they chances that I'll ever need these photos in RAW is very slim, and then upload them. For photos that I've given at least one star and/or taken the time to properly edit and categorize, I'll send up the RAW file along with the processed JPG. If I have about 3TB of RAW files...and maybe 10% of those are RAW images that weren't disposable...that's 300GB right there. And then the remaining RAW "keep em just in case" files would end up being compressed from 24MB to about 2-3 MB (let's say 15% as an average)...so 400GB. 700GB is still quite a big footprint. However, the biggest problem will be...let's say I give up local storage all together and rely on the cloud...what's the best way to browse any photoset at any arbitrary time? When it's all local, it's trivial to pop open Lightroom and do some quick browsing and filtering. For online storage, I'll have to put a ton of work into proper folder-naming, at the very least
- PStamatiou 13y ago
  Your last few sentences are exactly why I wrote this post. I want everything stored in the cloud but ALSO have a decent way of browsing them locally. The idea is to just store tiny jpegs (you can set how big they are, but something like 500-1000px wide likely, just for you to get a sense of which photo this is and allow you to remotely delete, retreive, etc)
  - stevesearer 13y ago
    This is basically what I need for Office Snapshots. I keep all of the original photos submitted so that I can use the full images in the future (if need be) and would love to store them somewhere other than my laptop. If I could browse locally with smaller images (as you describe) like I currently do with Picasa (galleries, tags, etc) and then have the ability to download the original images when and if I need them, I'd be in heaven.
uptown 13y ago
A service would be great ... but you might be able to "roll-your-own" with some basic scripts to handle the conversion (down-sizing) and backup sync - and something like Koken to handle the sharing with friends & family. http://koken.me/ http://koken.me/ For back-ups - just buy a huge disk or set of disks, and keep 'em off-site wherever you have a trustable location (friend, family, work - if allowed and bandwidth isn't an issue) and use CrashPlan's free software to sync a copy to the secondary location. It's free after the initial expense of the drives and gives you encrypted, but physical access to the content in case of an emergency. You could find a friend with the same problem, and just agree to dedicate a certain amount of disk-space to each-other for this purpose.
sreitshamer 13y ago
I think Filosync (www.filosync.com, currently in beta, I'm the founder) would work for this. You could have many projects in your cloud and only sync the one(s) you need to a given computer. We'd just need to add a feature that stores "previews" of the items in the folder so you could browse without downloading. But it uses S3 in your AWS account, so would be cost-prohibitive for a consumer storing terabytes.
healthenclave 13y ago
This is exactly what I was thinking about... I want cloud based storage which I am not locked into with a decent UI , way to browse and basic image management (something like ThisLife for the fronted) I think it's possible to have a either S3 or GDrive based system that has a Angular.js (or some other JS MVC) that can operate without a backend for very simple browsing features and connecting to the S3 via apis. I think it's doable ...
rdl 13y ago
I still prefer local storage and self-hosted storage, but what I'd like is a good plugin for Lightroom to manage cloud storage and multiple, versioned USB drives (or network locations) for images, from within the Lightroom interface. (for local storage, once you go beyond individual USB drives, I'd go with a Synology 1813+ NAS ($999 chassis, 8-bay) or an Areca ARC-8050 thunderbolt raid ($1499, 8-bay).)
mahyarm 13y ago
I've always wanted to work on this, but didn't want to work on the stereotypical nth photo sharing/management startup, especially with other projects taking up my time.
rb2k_ 13y ago
Wasn't that the idea behind "bitcasa" ? -> https://www.bitcasa.com/ https://www.bitcasa.com/
mrushton14 13y ago
Highly recommend http://www.mosaicarchive.com/ http://www.mosaicarchive.com/
geuis 13y ago
"I open the folder where Lightroom imports are stored and drag these shots to the RAWbox (fake name for this ideal photo app) desktop application." You shouldn't be touching the directory structure. This tool should be either a plugin for Lightroom/Aperture/Photoshop or an entirely separate application. I would lean towards separate application. It could be a background agent that detects when new files are added, would work for any known editor or user-defined directory.
Hovertruck 13y ago
I'm actually working on this idea right now! https://plover.co/ https://plover.co/ We want to take backing up, sharing, and monetizing and make them very simple and enjoyable to do. We've started rolling out slowly already, our goal is to be open to the public within a month or so.
- hamburglar 13y ago
  I hope your business model doesn't rely too heavily on the "monetizing" aspect. I know there are lots of photo hosting services with a "sell prints" option, but I get the impression not very many people actually use those in enough volume to cover the hosting costs. I'm imagining a non-pro track that's targeted at the enthusiast who just wants backup and sharing, but won't actually sell any prints. I'll be watching.
  - Hovertruck 13y ago
    We don't make any money off of prints, only the photographers do. The business model is subscriptions.
    - hamburglar 13y ago
      This is good to hear. BTW, it might help if you had a little more info on your landing page about the service. For example, I can't tell if this is going to be aimed at true pros who are willing to shell out a decent amount of money, or if I'm going to be able to afford it for my own pro-sumer needs. Having to sign up for a mailing list and wait in order to potentially find out that I'm nowhere near the target audience is frustrating.
      - Hovertruck 13y ago
        Heh, it is unfortunate that there's very little info there. We put it up a while ago and have been so focused on actually building the app that updating the website seemed unimportant. :)
tghw 13y ago
We applied to YC with exactly this idea. We made it to the interview, but were turned down. I wrote about that experience: http://tghw.com/blog/pull-hard http://tghw.com/blog/pull-hard We pulled our bootstraps hard and tried to get it going, but soon realized that there's a pretty big flaw with the idea: most photographers have no idea that they should backup their photos. They know hard drives crash, but they never expect it to happen to them. This meant that we'd essentially have to become insurance salesmen, convincing people that bad things will happen to them. Eventually, we decided to shut it down, which I also wrote about: http://tghw.com/blog/well-that-sucks-what-else-you-got http://tghw.com/blog/well-that-sucks-what-else-you-got The old homepage is still up at http://www.snaposit.com/ http://www.snaposit.com/.
- famousactress 13y ago
  Yeah, I'd expect the market for this is more niche but could be high revenue-per customer. As a wedding photography studio we'd pay 100$/mo in a heartbeat if this was done well. [Edit] Also note that OP wasn't really just pitching "backup". He wants you to reduce the footprint his images take up locally. I think the messaging to casual and hobbyist users probably ought to play on this. Note Loom's taglines "more room to play". Backup is probably boring for people not under the pressure of liability like pros are.
- tsycho 13y ago
  Would you be willing to open source your code? I am running out of hard disk space as well for my photos and have been thinking of writing coding up something very similar to this idea _when I get time_. But I would be willing to contribute to an open source project that does this for self-hosting my photos etc. Contact me if interested, email's in my profile.
  - tghw 13y ago
    Let me consult with my cofounder. I've considered it in passing, but haven't put any time into it.
  - dublinben 13y ago
    If you're looking for a more immediate solution, I've been pretty satisfied with FastGlacier, as an Amazon Glacier client. http://fastglacier.com/ http://fastglacier.com/
martin_kivi 13y ago
I believe Photry (www.photry.com, currently in beta, I'm the founder) could be a possible solution in the long run. Since we started, a lot has changed on online photo storage market. With that in mind we've started to look more on how photographers with larger photo volumes could benefit from our service. From the feedback from few photographers we've already included some features (RAW photos with smaller JPG thumbnails, workspaces and personal sharing for example) that will make the day to day workflow a bit easier. Currently our goal is to build something that would fit better for professional photographers (public for clients, ordering prints and possibly integrated payment flows, win/mac client + plugins for your favorite photo software). Pricing wise we are currently not the cheapest but we have thoughts on how to make this better for high volume photographers. If you're one of them then please send me an email (martin at photry.com) and I would love to talk a bit further what we plan to do and if we would fit your storage needs.
cenhyperion 13y ago
Great post. I appreciate the focus on minimalism and hobbyist photographers. I think it'd be an interesting project to take on. The technical side doesn't sound too difficult, plug into Amazon, lightroom/aperture. While the post mentions avoiding externals I think a handy feature would be to push to both an external drive, glacier, and then remove the RAWs locally. That way you have two copies, so the data is actually backed up. Trusting all of your photography to amazon not losing data doesn't sound ideal.
_delirium 13y ago
I've been looking into Glacier, but the pricing in the case where I might want to retrieve a substantial portion of my data is quite complex. As far as I can tell, the headline $0.01/GB rate for overages they quote is misleading. They don't, as you might expect from that language, calculate ([GB retrieved] - [5% of GB stored]) x $0.01 and bill you that. Rather, they charge you on a monthly basis, according to your highest hourly retrieval rate from any single hour applied to the whole month (standardized at 720 hours). So if your peak hourly retrieval in a month was 10 GB/hr above your pro-rated free quota for that hour, you're charged for 7200 GB of retrieval overage, or $72, even if you retrieved nowhere near 7 TB of data. For companies with relatively constant retrieval this doesn't matter, but for small users, someone who has a large overage one day will be charged as if they were incurring the same overage continually all month. To put it concretely, if you store 200GB of photos and then do a bulk retrieval, you aren't charged $1.90 ((200-10) x 0.01). Rather, the xfer is counted as 4 hours, so you have a peak transfer of 50 GB/hr. Your free daily retrieval is allocated to the hours in the day you retrieved data, so in this case 10/4 = 2.5 GB/hr free. So your overage is 47.5 GB/hr. Multiply by 720 and $0.01, and the fee for retrieving 200GB of photos is $342. Plus $22.88 in outgoing AWS bandwidth. Or at least, I think that's what their pricing page says. Maybe someone else can better interpret this explanation: http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_when_retrieving_large_amounts_of_data_from_Amazon_Glacier http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_wh...
- jychang 13y ago
  This should be mitigated if you're running a service, though. It averages out.
  - wmf 13y ago
    Glacier retrieval pricing has a strange and un-Amazony economy of scale. If you're just backing up your own data you'll be charged a ZOMG huge amount to ever download it. But if you're backing up 100 people you pay nothing to get data back.
    - _delirium 13y ago
short_circut 13y ago
This idea is great, but it ignores on very real problem. Namely that you still have not addressed the issue of bandwidth. Take for example your 4th of july example. If I were to do that upload the way you describe, I would already be over my monthly bandwidth allotment with my ISP as would many hobbyists.
- roc 13y ago
  The US bandwidth situation is the elephant in the room. There really is no good solution to automated offsite backup when it takes several days of uploading to handle a single day's worth of data capture. To make any internet-backup service work, you'd first have to pitch the photog on switching from their residential broadband package to something more expensive -- if that's even available short of their leasing office space somewhere. I guess you could try to build out a reverse-CDN sort of network, with local relay nodes scattered across the various ISP networks, that might achieve faster uploads from the user (and maybe not get counted against bandwidth caps) which then use a fat pipe to send that data to larger regional storage nodes. Neither seems particularly plausible at scale.
callmeed 13y ago
My company serves thousands of professional photographers (wedding, commercial, fashion, etc.). What you're describing is definitely an issue for photographers but I think getting them to use a tool like you've described would be an uphill battle. Right now, most of them use Drobos or similar devices. You'd be asking them to add steps to their workflow which is always tough to do. Also, I think your estimate of 18 mins to upload 6+GB is an order of magnitude off. We have tools that allow people to upload JPEGs and most customers claim it takes longer than that just for 100s of MBs. My (wild) guess is that Adobe is going to get in this space too. It only makes sense to have your Lightroom catalog in the cloud. Maybe they wont offer an affordable solution for terabytes of data, but I'm not sure photogs want their images in two different "clouds". I think the key is to do customer development from full-time photographers who aren't hackers. I'd be happy to send out a survey to some clients if you want do more research.
- beat 13y ago
  As a photographer with a couple of terabytes of old raws (I back up to Time Machine locally, and offsite at a friend's house occasionally), I'm very curious as to what Adobe will offer in this regard with LR5.
  - callmeed 13y ago
    They may not offer anything at the TB-level, who knows. But something is coming: http://www.adobe.com/products/creativecloud/faq.html#storage-services http://www.adobe.com/products/creativecloud/faq.html#storage... and I still think it will be hard to convince people to "put these files on Adobe's cloud and these files on this other cloud using X software"
- jwarren 13y ago
  Spot on. In a few years, cloud storage will be more acceptable, working photographers will be ever more tech literate and bandwidth will increase. I think you're totally right about Adobe too. Those Smart Previews they added for LR5 - those are not just for space-starved SSD drives...
- bambax 13y ago
Brandon0 13y ago
I'll jump in here and share our current problem and how we are addressing it today. Hopefully it proves useful for whomever tackles this project! So we are an ecommerce company and we do our own photography for products -- and we do a lot of it. We average about 2-3 photoshoots per month, and each photoshoot can be upwards of 40GB before any processing. We shoot raw images and are starting to experiment more and more with video. After we do the post-processing work and start the video editing we can push 70GB a shoot easily. Our current workflow is like this: Shoot tethered to a laptop each day (typically each shoot lasts two days). At the end of the day, we upload the files up to our local NAS (Netgear ReadyNAS, just shy of 3TB) which is redundant. At the end of the shoot, we load a copy of the entire shoot on the local computers that will be editing so that they don't have to work directly from the NAS. Over the next few weeks while they edit the files, they will push the new files up to the NAS into specific folders. (This is where it gets hairy) We also have an offsite NAS that is located in the business owner's house (about 6TB, but this one is not redundant). We used to attempt to upload files each night from the local NAS to an offsite NAS via rsync, but the pipe was just way too small. It would take days to upload a photoshoot, and everyone in the office suffered from slow internet while that happened. Now what we do is bring the offsite NAS onsite every few weeks and do a manual sync! Locally the sync only takes a few hours and then that night it goes back home with the business owner. Now since the local NAS is less than 3TB, we can't store all of our shoots on there forever. When we get close to full on the local NAS, we grab an external hard drive and archive old shoots (typically 2+ years old) to it, and then store that in our massive safe (on-site though). If we need access to just one or two files, we can retrieve them from the offsite NAS, but if we need access to quite a few of them, we can dig them out of the safe. So that's our workflow. Probably not bullet proof, but it has been serving us well for the last few years. If someone came up with a better solution, we would be all over it!
- josh2600 13y ago
  Out of curiosity, how big is the pipe at the office? The one holding the local NAS?
  - Brandon0 13y ago
hfanson1 13y ago
It is important to be careful with how you try to use Glacier. I have been looking its pricing; It costs $0.01 per GB per month only if you want to upload your data. Downloads add significant cost if you want to retrieve more than 0.17% of your data in a day. Downloading 5% of your data increases your monthly rate to $0.097 per GB. 10% increases to $0.187 per GB 50% increases to $0.907 per GB 100% increases to $1.807 per GB Downloads are also delayed by 4 hours after you make a request.
- PStamatiou 13y ago
  if I have 2TB stored and only want to retrieve a single 10GB photo shoot one month (or even just a single photo), it would not be anywhere close to 5% (.5% in fact) of my total amount stored. The key point here is that we are dealing with TBs of photos that are there for safe-keeping and they have already been tinkered with and all photo JPGs needed have been exported and put elsewhere. This is just for RAWs and you likely won't need to access them again, definitely not 5%+ of them like this.
- deleted 13y ago
  [deleted]
martingordon 13y ago
Has anyone used Lightroom 5's new Smart Preview feature? It generates lossy DNGs from your full-sized files that you can still work with while your original sits on an external drive. Once you plug your external drive back in it, it syncs the non-destructive changes back to the RAW file.
hnriot 13y ago
"I almost always shoot in small (5.5MP) or medium (11MP) on my 22.3MP Canon 5D Mark III" Kind of hard to take anything seriously after reading that. He buys a seriously expensive camera and shoots it in point and shoot size. That's just insane. What's so hard with a simple home NAS. I replace the drive in my laptop every year for whatever is then biggest (sweet spot) and ue a NAS for backups. RAID5
- rfnslyr 13y ago
  Because cloud this and cloud that! I feel like I'm the only person firmly against cloud storage. I'd constantly feel paranoid about my data randomly disappearing at the hands of a company I don't know run by people I don't know on hardware I don't trust. I want everything local.
  - joevandyk 13y ago
    What happens when there's an earthquake? Or theft? Or fire?
    - rfnslyr 13y ago
      My own cloud? It's not really hard to get your own service up and running. All my shits auto backed up to my dedicated which has terabytes and terabytes of storage.
      - icebraining 13y ago
        Where do you store it?
        rfnslyr 13y ago
        Family network. We all run servers in our houses that are connected and each keep backups of eachothers backups. Backups at my parents, grandparents, cousins in colorado, family in Poland. All encrypted, all silently running in the basement of each house. All updated every week.
tambourine_man 13y ago
Backblaze?
sreitshamer 13y ago
I'm not convinced that retrieval time doesn't matter. If you decide you want several photos, wait 4 hours to retrieve them from Glacier, then realize you wanted a few more and have to wait an additional 4 hours, I think that would get old pretty quickly.
- hadem 13y ago
  I was wondering something similar. Does Glacier have any way to see a preview of the images you are downloading? If I upload 100 images with serialized file names (001.jpg, 002.jpg) how do I know which images to download?
  - Pyrodogg 13y ago
    This is most of what the proposed software is. A local, lower-res index of the huge files you have deep archived.
  - sreitshamer 13y ago
    There's no preview. Actually there's no way to download or even list your archives through the AWS Console. You need to write (or find) an app that uses the Glacier API.
bigbento 13y ago
Over the past few years, I worked on a few versions of this as a side project. The last iteration didn't do the backup itself--it connected with your Dropbox, sync'd the raw images, and processed them at a preset +-2 EV, and 3 color temperatures. The image results were good, but honestly I never found it particularly compelling. In any case, building this sort of thing is a pain. The most obvious problem is the size of the images. Glacier is an interesting backup solution, and had just come out around the time I decided to stop working on my app. A less obvious problem is raw conversion. All the best converters are proprietary, meaning that I had to piggy back off the Mac OS converter engines. Again, I never got to the point where I felt the product was worth the trouble of buying racking up a Mac Mini to run a little market test. Maybe I'm wrong. I had a (frankly somewhat scattered) draft of a post mortem written, and since it seems like there's some interest in this subject, I posted it: http://scraps.bigbento.com/2013/05/30/hiatus.html http://scraps.bigbento.com/2013/05/30/hiatus.html
stevewilhelm 13y ago
Smugmug's SmugVault "SmugVault is an added service to your SmugMug account that lets you store almost anything for next to nothing. Including files not normally supported by SmugMug." http://help.smugmug.com/customer/portal/articles/84562-more-about-smugvault http://help.smugmug.com/customer/portal/articles/84562-more-...
- sreitshamer 13y ago
  Although Smugmug is awesome, the issue I'd have with them is the lack of control. If my objects are in my AWS account, that feels more permanent long-term. It's just a gut feeling.
throwaway420 13y ago
I wish I could trust the cloud because it would be a lot more convenient to not have to worry about storing photos, which involves a lot of effort once you advance beyond a casual hobby and you get into storing Terrabytes of large RAW files and maintaining adequate backups. But the idea of storing any files in the cloud is very distasteful to me ever since the NSA Snowden story broke. Even something as innocent as a photographer taking pictures of a skyline or some beautiful architecture could land him in hot water if the government is looking at metadata and figuring out that he took lots of photos of a government building or a bridge or a power plant and randomly throwing him on some watch lists or no-fly lists. People have been hassled by the feds for a lot less. My preference would be for all of these cloud storage providers like Google, Facebook, Apple, Dropbox, Amazon, etc to unambiguously and fiercely challenge the government on this and somehow unambiguously prove to people that they're respecting peoples' privacy and not complying with this stuff. Their denials so far have been obvious non-denials that are very carefully worded around "no direct access" when that's enough wiggle room to do just about anything. Using some kind of cloud photo storage is a bad idea IMO until this gets sorted out.
- chadk 13y ago
  FINALLY. I was going through every comment on this thread and I was wondering why no one was talking about security. This is one of the key features of the data warehousing business we have been contemplating.
- prirun 13y ago
  Everything must be encrypted on the client before sending to any cloud service. Then it doesn't matter if the encrypted backup is obtained.
post_break 13y ago
"I almost always shoot in small (5.5MP) or medium (11MP) on my 22.3MP Canon 5D Mark III3." Sorry but what? I actually had to read that a couple of times.
mark_l_watson 13y ago
I only have a 128GB SSD on my MacBook Air, which is my primary computer. I used to run out of space until I read somewhere that you can use DropBox advanced settings to not keep specific directories on specific devices. So, I am not likely to have directories like, for example, videos2012, pictures2012 synced on my Air but they are on my old Linux and MacBook Pro laptops which have lots of disk space. With high network speeds, I can quickly re sync directories, or just power on one of my old laptops.
bcl 13y ago
My solution is to copy them all to Glacier, I also have local copies, Glacier is only for worst case scenario retrieval. I wrote about the little script I use here: http://www.brianlane.com/automatic-backup-of-files-to-s3-and-glacier.html http://www.brianlane.com/automatic-backup-of-files-to-s3-and...
fsckin 13y ago
I'm the SO of a professional newborn photographer who generated somewhere in the neighborhood of 20-30TB of RAW photos, backups, LR xmp sidecars, etc in the last year. JPEGs don't even register. High end DSLR RAWs are gigantic! I'm the technical support for overall workflow, storage & backup engineer, etc, etc. I wear many hats. Right now I've got a backup on import to a FreeNAS (BSD) filer running on a HP Micro Server. Lightroom backups go to the same filer and are synced as well. My local server rsyncs the delta every night to an identical machine at a friends' house. This isn't perfect. The one place that isn't 100% backed up is current projects in post-production. That's an acceptable trade off for now. The most important thing to note is that there are two kinds of photographers: Those who never lost data, and those who care about backups. As mentioned elsewhere in this thread, converting members of Group A into Group B is a tough sell. Video testimonials might be a good option. A storage service for photographers MUST be invisible to their workflow. Make it as easy to use as an egg timer. At the MOST, an addition of a single screen with sensible defaults (just click next) on photo import. Ignoring error messages that get in their way is to be expected. Care more about their backups than they do. Send me (tech support guy) an email to notify if there haven't been backups added in X days, along with weekly reports of data usage per day -- how many photos, how large, what root folders, usage trends, etc. An ideal installation would be automatic detection of what program they're using, what places files are stored (hint: It's not just on a local drive). Detect new file locations and back those up too. An inexpensive (<$25) option of "only backup current projects I'm working on and anything I shot in the last month", that actually works... I'd sign up in a heartbeat.
FollowSteph3 13y ago
And what happens if one glacier decides to close? Do you think you'll get any notice ahead of time? I wrote about these types of risks at http://www.followsteph.com/2012/04/19/what-are-the-risks-of-cloud-services/ http://www.followsteph.com/2012/04/19/what-are-the-risks-of-...
foxhop 13y ago
http://www.snaposit.com/ http://www.snaposit.com/ tried but the market didn't pay.
foxhop 13y ago
snaposit : http://www.snaposit.com/ http://www.snaposit.com/ tried but the market wasn't there. I'm going to send this post to the founder to see his take on it.
josephlord 13y ago
Covering video too would be great but maybe a second phase handling them separately is a hassle for users.
gcb0 13y ago
What about flickr and the insane free space they provide? you it's free now right? searching is also a breeze there. not sure how you'd go about downloading them all though...
- philfrasty 13y ago
  no RAW support...
gerardmurphy 13y ago
I am the CEO/Co-Founder of Mosaic. We are a company that helps serious photographers backup their photographers. I am happy to share what we have learned in the past 2 years. www.mosaicarchive.com As a background, we now have thousands of users and customers. We also finished raising a series A in February. (Not that raising money proves anything) http://www.boston.com/business/technology/innoeco/2013/02/mosaic_storage_systems_collect.html http://www.boston.com/business/technology/innoeco/2013/02/mo... We are now growing at about 5%-7% a week. We recently peaked in the top 100 grossing US iPad Photo/Video Apps in iOS - although most of our revenue comes outside of iOS. (As they take a 30% cut.) We target serious photographers (pro's and prosumers) who use Adobe Lightroom. For pure backup customers our sweet spot are those who have outgrown BackBlaze and Crashplan. (Both great services but geared at consumers.) On the backup side, we help photographers manage their backups by using Adobe Lightroom metadata. Customers can choose to backup everything in Lightroom automatically. However, many photographers use metadata like stars or flags to designate their best photos. We allow users to automatically backup these "best" photos. This provides the automation of the best backup tools with more precision for larger customers. We use Amazon Glacier behind the scenes to store the original photos. (Yes, WMF the restoration costs even out for us over all of our users.) (Sidebar - we actually ran our data center for months before switching to Glacier...very glad to have made the switch.) However... we are not an online backup company. Online backup is a feature is our service... but not the entirety of it. Once we have have photographers photos, we also want to help photographers better use, manage, share, and enjoy those photos. At one point I would have said we are a backup company, now I call us a workflow company. We also offer a web and iOS App to privately view and share your Lightroom photos. (We take JPG snapshots of the RAW files so they load quickly and have the Lightroom edits. This part of the service runs in Amazon S3 not Glacier) This gives you access to your photos in the same way you are used to experiencing them in Lightroom. (Lightroom users try this for free - https://itunes.apple.com/us/app/mosaic-archive/id627973694?ls=1&mt=8 https://itunes.apple.com/us/app/mosaic-archive/id627973694?l...) The next step which we are working on now is adding two-way sync where you can manage your photos in the App and it populates back to your desktop files and Lightroom. We are also working on more collaborative features so photographers can share this culling and editing process with their clients or family members. The bigger problem we are solving is that photographers have loads of large files... backing them up is painful yes... but managing them all is more painful. Sifting through 000s of images to find your best stuff frankly stinks. This problem is only getting bigger as we shoot more and more. For those interested in this idea, please consider joining us. We are hiring a couple more software developers this year. http://www.mosaicarchive.com/about-mosaic-photo-storage/careers/software-engineer/ http://www.mosaicarchive.com/about-mosaic-photo-storage/care... Thanks for the discussion and happy to answer all questions. Best, Gerard www.mosaicarchive.com
esonderegger 13y ago
I really wish we as a developer community would better differentiate between "backup" and "preservation". They are both related to storage, but are fundamentally different problems. The article touches on the desire to have photos safe for decades, but doesn't really get into the strategy necessary to make that happen. If we care about decades, storing the raw NEF or CR2 file is almost certainly the wrong approach. Those files could easily be as difficult to open in fifteen years as a ClarisWorks document is today (just to choose one example - proprietary formats for professional software from the late 90's are all pretty hard to open today). Also, important metadata is kept in the lrcat file. Unless we expect to always be able to open our copies of Lightroom, we need to migrate that data too. We used to think of "archiving" as putting something on a shelf and forgetting about it, sometimes taking environmental factors like humidity into account. With our digital data, we need a plan for periodically checking it, both for fixity and to make sure at risk file formats are migrated to current best practices. This will require greater awareness for the complexities involved (and the need for open file formats) from everyone from creatives to storage and service providers. By the way, every creative industry is struggling with this right now. Recording studios have session files created in ProTools 4. Designers have really old Quark files, even though they've been working in InDesign for the past five years. Composers have Finale files. A lot of these file formats depend on plugins to be able to display properly. I don't mean to sound discouraging, I think there's plenty of opportunities for viable businesses here. We just need to start using the right terms.
- graue 13y ago
  In a word, VMs. If you make sure you can open that NEF or CR2 file in a virtual machine, and the virtual machine image is in a standard, open format, you're set. You just have to worry about the image format becoming obsolete and not 10 or 20 different proprietary formats. That said, I used to be a pretty prolific hobbyist musician and I haven't pulled this off myself. I have tons of old songs I can't open because they rely on a fiddly collection of old freeware plugins for freeware music apps from 11 years ago. I also have songs I can't open because they only play in a music tracker I wrote, which is open-source, but only runs on OpenBSD. It is a tricky problem and it seems hard to tackle because the market for professional software inherently favors closed-source, which has no incentive to adopt open, non-siloed formats.
saidajigumi 13y ago
I have a serious problem with the proposal offered in TFA and most variants in this thread: it encourages bad backup policy. Critical data needs three (or more) backups. In my experience, one cloud provider can only ever be considered as being one backup no matter their technical architecture. For example, you can't (and in the broad "you", aren't even qualified to) audit the provider for data-loss or SLA-impacting SPOFs. You just have to assume that they're there. You also take on non-technical failure modes: the provider can go out of business, get bought by an uninterested owner (think Delicious), change focus (Google Reader), or experience myriad other problems. FWIW, the best success I've had with helping others switch to good backup policy happened once it became economically feasible to buy a new laptop and two bus-powered external drives as big as the laptop's internal storage. The externals become bootable backups, one of which can be preiodically rotated to an offsite location. This has prevented severe dataloss events for myself and others far more times than I care to count now.
- famousactress 13y ago
  FWIW, I'm more technical than a typical user might be but I'd be very intrigued by a provider that used multiple cloud service and gave me the keys to the underlying S3 buckets (or whatever) so I could independently verify. That's probly a more valuable idea to a company selling backup to nerds though, I guess.
peteforde 13y ago
I'd use this. 100% for sure. In fact, "a plugin for Aperture/Lightroom that backs up to Amazon Glacier" is on my idea list. Great minds think alike, fools seldom differ. One thing I'd like to point out is that photographers are unlikely to be as cost conscious as other consumers. We put a lot of effort into taking these damn photos, and it'd be nice if I could be confident that they'd outlive me. I'm willing to pay a premium for that.
Paul12345534 13y ago
Currently using Crashplan to backup my photos. I copied 200+ DVDs of photos to my hard drive in numbered directories... then deleted them after I had them uploaded (Crashplan keeps deleted files if you choose that option).
jrochkind1 13y ago
> I just need to know they are somewhere safe for decades to come. The thing about Amazon Glacier... unless they are themselves making more than one copy, and taking hash fingerprints, and storing a couple copies of the hash fingerprints, and using it all to check for bit rot on a regular basis... ...and I don't think they're doing all (any?) of these things... ...then people are sadly going to find that their stuff _isn't_ neccesarily safe for decades to come. One copy on a single piece of media in an Amazon data center somewhere does not actually make for 'safe for decades to come'.
- eurleif 13y ago
  https://aws.amazon.com/glacier/#highlights https://aws.amazon.com/glacier/#highlights > Amazon Glacier is designed to provide average annual durability of 99.999999999% for an archive. The service redundantly stores data in multiple facilities and on multiple devices within each facility. To increase durability, Amazon Glacier synchronously stores your data across multiple facilities before returning SUCCESS on uploading archives. Unlike traditional systems which can require laborious data verification and manual repair, Glacier performs regular, systematic data integrity checks and is built to be automatically self-healing.
adelmand 13y ago
We (cintrexav.com - a small video conversion service provider) are building something very much like this right now. Will post here when we get to MVP in a month or so!
madao 13y ago
I did some on the side admin work for a Photographer once, a real big name. He was in the middle of making up exactly what this was, however he came up against the same issues everyone has. Firstly space costs money, Hardware costs alot too, not to mention bandwidth, this guy would easily do 100-200gb of photos in a couple shoots. all very high res shots (talking about for big company ads) his idea is to sell a server with a software suit that people can dump into a datacenter or at home and upload photos to it, personally I liked the idea but local storage and hardware failure + the ability to grow the storage was a real killer for his idea.. (not to mention that your playing with peoples businesses and lives if you lose those photos)
espinchi 13y ago
I'm helping build Photofeed, a platform that stores and organizes your private photos from your different devices in the cloud. Photographers with the problem described in the article are within the target audience, even if we didn't implement some of the key features for them yet (e.g. storage of images in RAW format). We'd love having some early feedback from you guys! (Didn't officially launch yet.)
VikingCoder 13y ago
I just came across the Plug Kickstarter, which I think might be the basis of a viable back-up storage for professional photographers. http://www.kickstarter.com/projects/cloud-guys/plug-the-brain-of-your-devices http://www.kickstarter.com/projects/cloud-guys/plug-the-brai... Specifically, the $129 level, where you get two plugs, which they claim, "to replicate your data in 2 different places." Talk two of your friends in remote parts of the country into letting you Plug in to their home networks, and this might be viable.
qq66 13y ago
Draft feature set: - Local daemon running that manages this process, on PC or Mac. It can't be done solely from the browser. - The software manages the import from the SD/CF cards, you put them all in, it sorts everything out and wipes the SD cards on the way out. - You then use Lightroom to do whatever you're going to do. The software would do well to have a Lightroom plugin that lets you select particular images to mark as "high priority" and "low priority." - Now the daemon uploads all of your RAWs, Lightroom metadata, and corrected JPGs to cloud storage. - Based on your settings, these RAWs get discarded after some set amount of time (maybe immediately, maybe never), and the highest-quality JPGs are also discarded after a longer amount of time for lower-quality JPGs just in case you need a emergency backup. After 6 months you lose the raws, 1 year the highest quality JPG, 2 years the medium quality JPG, etc. You are charged per gigabyte per month so you can decide how much you care about backup. Basically offload the cognitive effort of discarding old data that isn't worth its upkeep costs anymore. - The files are available via an API that you can use to build your own gallery/shopping cart site, with the ability to call a certain quality, PROOF watermarking, etc. on-the-fly. But the service offers a default site right there for you to do the same, that's stylable and can be implemented on your own domain. - You can get your entire catalog shipped to you on a hard drive at any time for a few hundred dollars.
resistore 13y ago
Relying on only one storage provider is a little bit risky to me. I started a small desktop app for backing up my files with my family photos in mind, leveraging the properties of erasure coding and multiple dropbox/google drive services. Also adding a bit of crypto to follow the post-prism trend... Please, take a look at http://yleprovost.bitbucket.org/ http://yleprovost.bitbucket.org/
prirun 13y ago
I've been developing a backup program for a few years now, HashBackup, that encrypts your files locally, with your own private key, and sends them to offsite storage you control: your own Glacier or S3 account, your offsite rsync server (rsync.net for example), your Gmail or other IMAP account, an ssh server, or ftp server. www.hashbackup.com

263 points by PStamatiou 13y ago

ukd1 13y ago

Seems like http://www.loom.com/ http://www.loom.com/ would be great?

PStamatiou 13y ago
Not much info on it. What does it do? Email me if you're the founder. :)
- ScottBurson 13y ago
  The blog has a pretty good intro.
antr 13y ago
I've been on the waiting list of Loom for ages. Thought it was in the deadpool...
- ukd1 13y ago
  Nope, think they're just perfecting things!
kevingibbon 13y ago
+1 for loom. I've been using it for a while now. It's pretty awesome!

jeffasinger 13y ago

I believe that https://www.mediacloud.cc/ https://www.mediacloud.cc/ is trying to solve that right now. However, their website is sparse on details.

PStamatiou 13y ago
Definitely quite a few companies are attempting something in this space. I think the one that will win will be the one that actually has photographers as founders and understand the need. Unfortunately it is quite a niche and most photographers are gadget hounds and don't mind having 2-3 drobos (including an off-site one).
- StacyC 13y ago
  It appears that Loom is founded by photographers. http://www.loom.com/about http://www.loom.com/about I'd love to hear more about this service from some users. I am bumping up against Dropbox limits with my photos as well.

petercooper 13y ago

If you want to avoid having any backup devices yourself, would you need to upload to two or more Glacier-like services? I also trust Amazon to stay in business but mistakes can always happen (not just hardware but ID theft, card expiry while you're on a 6 month hike, etc) and unless you're prepared to say "losing these is OK" using one service would be risky.

PStamatiou 13y ago
This is true, but what other Glacier-like services are there that have $0.01/GB/mon storage + retrieval?
- dublinben 13y ago
  Offsite tape backup can approach that price, but I don't know of any exact alternatives. It would certainly fulfill the requirement of having two different media types and providers. http://www.dpbestflow.org/backup/backup-overview http://www.dpbestflow.org/backup/backup-overview

joebo 13y ago

How about git-annex assistant with the glacier remote? http://git-annex.branchable.com/special_remotes/glacier/ http://git-annex.branchable.com/special_remotes/glacier/. It will move files off your local once they have been transfered to the remote.

zellyn 13y ago
+1 to git-annex assistant.
PStamatiou 13y ago
The large part of this post for me is about the local thumbnail exports of the RAWs so I actually know what files are what without having to go just by album names, dates et cetera.
- joebo 13y ago
  You could keep the thumbnails local and annex the raws. Based on the thumbnails, you could selectively git annex get from the remote glacier
  - PStamatiou 13y ago
    good point

azov 13y ago

Backup is about having several copies. If you delete files after you uploaded them to the cloud, you don't have a backup - with all the problems that come with it (say, if you delete a file by accident there will probably be no way to restore it). If your files are important to you - you need a back up. If not - just get rid of them in the name of simplicity :) PS. Also, uploading many gigabytes over a typical ADSL connection is just painfully slow... even local NAS is too slow over WiFi for my taste, and if I have to have a wire - I can just as well connect an external drive.

PStamatiou 13y ago
I only used Glacier in this example but I did mention how it could be tied to additional cloud services. The thing is that not many like cheap long-term Glacier alternatives exist yet. Rest assured when they do, this ideal app would have them in the settings. :)
apaprocki 13y ago
I wanted the benefit of Glacier plus backup functionality so I went with Arq. It encrypts everything and supports Glacier. Works really, really nicely.
- tankbot 13y ago
  I use Arq as well. Easily one of the best backup options for Macs.
dtauzell 13y ago
I backup offsite via CrashPlan which keeps files even after I delete them. I sent them a hard drive for my initial upload. Now, I've got about 800M backed up. I haven't yet had to get it all back :)

famousactress 13y ago

Yes, PLEASE. We run a wedding photography business, and our numbers are something like this: - 40-60gigs shot in a weekend, need to offsite the RAWs as soon as possible. In six or seven years we've never needed offsite retrieval. Fine for it to be slow. - Only need around 100-200g of raws locally for jobs in progress - Once processed into final jpgs files are approx 8-10x smaller. Offsite these as well, and need access to last 1-2 yrs of jpgs randomly/sparsely w/in 1-2 days of a print order coming in I definitely think some well thought out service that uses maybe a hybrid of glacier and S3 could be really awesome. Especially if the pain is taken out of getting the bolus 40-50g of photos "uploaded" in the first place.. We'd happily pay for a service that just sends us an external drive and a postage-paid box.. we drop it in the mail after a job and get another drive sent to us right away. [EDIT: Occurs to me you might wonder what this looks like today. Today we get home, and immediately back up onto an 8-bay Drobo Pro (http://bit.ly/1diZUHD http://bit.ly/1diZUHD). It's a fancy proprietary RAID-like system. I think ours has 7 1TB Drives in it currently, and it's at about 70% capacity. Then we back up the same jobs to an external hard drive which gets driven offsite (used to be my office, but now that I work at home it's the in-laws place) and copied to a hard drive for permanent offsite storage. Once that happens, the cards from the camera can be cleared. After processing, the final JPGs get copied to the drobo and make it into the sneakernet offsite process as well.]

PStamatiou 13y ago
This thread is becoming great customer research for people to build this app. :)
- rfnslyr 13y ago
  You can all go home. I've already started.
  - Veratyr 13y ago
    Seriously? Got a repo we can watch? Or a blog we can follow?
alinspired 13y ago

danso 13y ago

I'm in a similar situation as the OP...an amateur but avid photographer with terabytes of RAW image files. Currently, I have several multi-terabyte drives that I erratically back up to...I just got a 4TB drive that I'm going to try to backup everything I've ever shot in the last 3 years... But I do want to move it to online storage...and I think this requires triage. For nearly every photoset, I've quickly gone through them with Lightroom and starred the ones that I kind of like, and then for maybe 1% of them, have taken the time to fully process, label, and upload them (in JPG form) to Flickr. So for online storage, I think what I'll do is write a batch script that dumps all the unstarred RAW files as JPGs, because they chances that I'll ever need these photos in RAW is very slim, and then upload them. For photos that I've given at least one star and/or taken the time to properly edit and categorize, I'll send up the RAW file along with the processed JPG. If I have about 3TB of RAW files...and maybe 10% of those are RAW images that weren't disposable...that's 300GB right there. And then the remaining RAW "keep em just in case" files would end up being compressed from 24MB to about 2-3 MB (let's say 15% as an average)...so 400GB. 700GB is still quite a big footprint. However, the biggest problem will be...let's say I give up local storage all together and rely on the cloud...what's the best way to browse any photoset at any arbitrary time? When it's all local, it's trivial to pop open Lightroom and do some quick browsing and filtering. For online storage, I'll have to put a ton of work into proper folder-naming, at the very least

PStamatiou 13y ago
Your last few sentences are exactly why I wrote this post. I want everything stored in the cloud but ALSO have a decent way of browsing them locally. The idea is to just store tiny jpegs (you can set how big they are, but something like 500-1000px wide likely, just for you to get a sense of which photo this is and allow you to remotely delete, retreive, etc)
- stevesearer 13y ago
  This is basically what I need for Office Snapshots. I keep all of the original photos submitted so that I can use the full images in the future (if need be) and would love to store them somewhere other than my laptop. If I could browse locally with smaller images (as you describe) like I currently do with Picasa (galleries, tags, etc) and then have the ability to download the original images when and if I need them, I'd be in heaven.

uptown 13y ago

A service would be great ... but you might be able to "roll-your-own" with some basic scripts to handle the conversion (down-sizing) and backup sync - and something like Koken to handle the sharing with friends & family. http://koken.me/ http://koken.me/ For back-ups - just buy a huge disk or set of disks, and keep 'em off-site wherever you have a trustable location (friend, family, work - if allowed and bandwidth isn't an issue) and use CrashPlan's free software to sync a copy to the secondary location. It's free after the initial expense of the drives and gives you encrypted, but physical access to the content in case of an emergency. You could find a friend with the same problem, and just agree to dedicate a certain amount of disk-space to each-other for this purpose.

sreitshamer 13y ago

I think Filosync (www.filosync.com, currently in beta, I'm the founder) would work for this. You could have many projects in your cloud and only sync the one(s) you need to a given computer. We'd just need to add a feature that stores "previews" of the items in the folder so you could browse without downloading. But it uses S3 in your AWS account, so would be cost-prohibitive for a consumer storing terabytes.

healthenclave 13y ago

This is exactly what I was thinking about... I want cloud based storage which I am not locked into with a decent UI , way to browse and basic image management (something like ThisLife for the fronted) I think it's possible to have a either S3 or GDrive based system that has a Angular.js (or some other JS MVC) that can operate without a backend for very simple browsing features and connecting to the S3 via apis. I think it's doable ...

rdl 13y ago

I still prefer local storage and self-hosted storage, but what I'd like is a good plugin for Lightroom to manage cloud storage and multiple, versioned USB drives (or network locations) for images, from within the Lightroom interface. (for local storage, once you go beyond individual USB drives, I'd go with a Synology 1813+ NAS ($999 chassis, 8-bay) or an Areca ARC-8050 thunderbolt raid ($1499, 8-bay).)

mahyarm 13y ago

I've always wanted to work on this, but didn't want to work on the stereotypical nth photo sharing/management startup, especially with other projects taking up my time.

rb2k_ 13y ago

Wasn't that the idea behind "bitcasa" ? -> https://www.bitcasa.com/ https://www.bitcasa.com/

mrushton14 13y ago

Highly recommend http://www.mosaicarchive.com/ http://www.mosaicarchive.com/

geuis 13y ago

"I open the folder where Lightroom imports are stored and drag these shots to the RAWbox (fake name for this ideal photo app) desktop application." You shouldn't be touching the directory structure. This tool should be either a plugin for Lightroom/Aperture/Photoshop or an entirely separate application. I would lean towards separate application. It could be a background agent that detects when new files are added, would work for any known editor or user-defined directory.

Hovertruck 13y ago

I'm actually working on this idea right now! https://plover.co/ https://plover.co/ We want to take backing up, sharing, and monetizing and make them very simple and enjoyable to do. We've started rolling out slowly already, our goal is to be open to the public within a month or so.

hamburglar 13y ago
I hope your business model doesn't rely too heavily on the "monetizing" aspect. I know there are lots of photo hosting services with a "sell prints" option, but I get the impression not very many people actually use those in enough volume to cover the hosting costs. I'm imagining a non-pro track that's targeted at the enthusiast who just wants backup and sharing, but won't actually sell any prints. I'll be watching.
- Hovertruck 13y ago
  We don't make any money off of prints, only the photographers do. The business model is subscriptions.
  - hamburglar 13y ago
    This is good to hear. BTW, it might help if you had a little more info on your landing page about the service. For example, I can't tell if this is going to be aimed at true pros who are willing to shell out a decent amount of money, or if I'm going to be able to afford it for my own pro-sumer needs. Having to sign up for a mailing list and wait in order to potentially find out that I'm nowhere near the target audience is frustrating.
    - Hovertruck 13y ago
      Heh, it is unfortunate that there's very little info there. We put it up a while ago and have been so focused on actually building the app that updating the website seemed unimportant. :)

tghw 13y ago

We applied to YC with exactly this idea. We made it to the interview, but were turned down. I wrote about that experience: http://tghw.com/blog/pull-hard http://tghw.com/blog/pull-hard We pulled our bootstraps hard and tried to get it going, but soon realized that there's a pretty big flaw with the idea: most photographers have no idea that they should backup their photos. They know hard drives crash, but they never expect it to happen to them. This meant that we'd essentially have to become insurance salesmen, convincing people that bad things will happen to them. Eventually, we decided to shut it down, which I also wrote about: http://tghw.com/blog/well-that-sucks-what-else-you-got http://tghw.com/blog/well-that-sucks-what-else-you-got The old homepage is still up at http://www.snaposit.com/ http://www.snaposit.com/.

famousactress 13y ago
Yeah, I'd expect the market for this is more niche but could be high revenue-per customer. As a wedding photography studio we'd pay 100$/mo in a heartbeat if this was done well. [Edit] Also note that OP wasn't really just pitching "backup". He wants you to reduce the footprint his images take up locally. I think the messaging to casual and hobbyist users probably ought to play on this. Note Loom's taglines "more room to play". Backup is probably boring for people not under the pressure of liability like pros are.
tsycho 13y ago
Would you be willing to open source your code? I am running out of hard disk space as well for my photos and have been thinking of writing coding up something very similar to this idea _when I get time_. But I would be willing to contribute to an open source project that does this for self-hosting my photos etc. Contact me if interested, email's in my profile.
- tghw 13y ago
  Let me consult with my cofounder. I've considered it in passing, but haven't put any time into it.
- dublinben 13y ago
  If you're looking for a more immediate solution, I've been pretty satisfied with FastGlacier, as an Amazon Glacier client. http://fastglacier.com/ http://fastglacier.com/

martin_kivi 13y ago

I believe Photry (www.photry.com, currently in beta, I'm the founder) could be a possible solution in the long run. Since we started, a lot has changed on online photo storage market. With that in mind we've started to look more on how photographers with larger photo volumes could benefit from our service. From the feedback from few photographers we've already included some features (RAW photos with smaller JPG thumbnails, workspaces and personal sharing for example) that will make the day to day workflow a bit easier. Currently our goal is to build something that would fit better for professional photographers (public for clients, ordering prints and possibly integrated payment flows, win/mac client + plugins for your favorite photo software). Pricing wise we are currently not the cheapest but we have thoughts on how to make this better for high volume photographers. If you're one of them then please send me an email (martin at photry.com) and I would love to talk a bit further what we plan to do and if we would fit your storage needs.

cenhyperion 13y ago

Great post. I appreciate the focus on minimalism and hobbyist photographers. I think it'd be an interesting project to take on. The technical side doesn't sound too difficult, plug into Amazon, lightroom/aperture. While the post mentions avoiding externals I think a handy feature would be to push to both an external drive, glacier, and then remove the RAWs locally. That way you have two copies, so the data is actually backed up. Trusting all of your photography to amazon not losing data doesn't sound ideal.

_delirium 13y ago

I've been looking into Glacier, but the pricing in the case where I might want to retrieve a substantial portion of my data is quite complex. As far as I can tell, the headline $0.01/GB rate for overages they quote is misleading. They don't, as you might expect from that language, calculate ([GB retrieved] - [5% of GB stored]) x $0.01 and bill you that. Rather, they charge you on a monthly basis, according to your highest hourly retrieval rate from any single hour applied to the whole month (standardized at 720 hours). So if your peak hourly retrieval in a month was 10 GB/hr above your pro-rated free quota for that hour, you're charged for 7200 GB of retrieval overage, or $72, even if you retrieved nowhere near 7 TB of data. For companies with relatively constant retrieval this doesn't matter, but for small users, someone who has a large overage one day will be charged as if they were incurring the same overage continually all month. To put it concretely, if you store 200GB of photos and then do a bulk retrieval, you aren't charged $1.90 ((200-10) x 0.01). Rather, the xfer is counted as 4 hours, so you have a peak transfer of 50 GB/hr. Your free daily retrieval is allocated to the hours in the day you retrieved data, so in this case 10/4 = 2.5 GB/hr free. So your overage is 47.5 GB/hr. Multiply by 720 and $0.01, and the fee for retrieving 200GB of photos is $342. Plus $22.88 in outgoing AWS bandwidth. Or at least, I think that's what their pricing page says. Maybe someone else can better interpret this explanation: http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_when_retrieving_large_amounts_of_data_from_Amazon_Glacier http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_wh...

jychang 13y ago
This should be mitigated if you're running a service, though. It averages out.
- wmf 13y ago
  Glacier retrieval pricing has a strange and un-Amazony economy of scale. If you're just backing up your own data you'll be charged a ZOMG huge amount to ever download it. But if you're backing up 100 people you pay nothing to get data back.
  - _delirium 13y ago

short_circut 13y ago

This idea is great, but it ignores on very real problem. Namely that you still have not addressed the issue of bandwidth. Take for example your 4th of july example. If I were to do that upload the way you describe, I would already be over my monthly bandwidth allotment with my ISP as would many hobbyists.

roc 13y ago
The US bandwidth situation is the elephant in the room. There really is no good solution to automated offsite backup when it takes several days of uploading to handle a single day's worth of data capture. To make any internet-backup service work, you'd first have to pitch the photog on switching from their residential broadband package to something more expensive -- if that's even available short of their leasing office space somewhere. I guess you could try to build out a reverse-CDN sort of network, with local relay nodes scattered across the various ISP networks, that might achieve faster uploads from the user (and maybe not get counted against bandwidth caps) which then use a fat pipe to send that data to larger regional storage nodes. Neither seems particularly plausible at scale.

callmeed 13y ago

My company serves thousands of professional photographers (wedding, commercial, fashion, etc.). What you're describing is definitely an issue for photographers but I think getting them to use a tool like you've described would be an uphill battle. Right now, most of them use Drobos or similar devices. You'd be asking them to add steps to their workflow which is always tough to do. Also, I think your estimate of 18 mins to upload 6+GB is an order of magnitude off. We have tools that allow people to upload JPEGs and most customers claim it takes longer than that just for 100s of MBs. My (wild) guess is that Adobe is going to get in this space too. It only makes sense to have your Lightroom catalog in the cloud. Maybe they wont offer an affordable solution for terabytes of data, but I'm not sure photogs want their images in two different "clouds". I think the key is to do customer development from full-time photographers who aren't hackers. I'd be happy to send out a survey to some clients if you want do more research.

beat 13y ago
As a photographer with a couple of terabytes of old raws (I back up to Time Machine locally, and offsite at a friend's house occasionally), I'm very curious as to what Adobe will offer in this regard with LR5.
- callmeed 13y ago
  They may not offer anything at the TB-level, who knows. But something is coming: http://www.adobe.com/products/creativecloud/faq.html#storage-services http://www.adobe.com/products/creativecloud/faq.html#storage... and I still think it will be hard to convince people to "put these files on Adobe's cloud and these files on this other cloud using X software"
jwarren 13y ago
Spot on. In a few years, cloud storage will be more acceptable, working photographers will be ever more tech literate and bandwidth will increase. I think you're totally right about Adobe too. Those Smart Previews they added for LR5 - those are not just for space-starved SSD drives...
bambax 13y ago

Brandon0 13y ago

I'll jump in here and share our current problem and how we are addressing it today. Hopefully it proves useful for whomever tackles this project! So we are an ecommerce company and we do our own photography for products -- and we do a lot of it. We average about 2-3 photoshoots per month, and each photoshoot can be upwards of 40GB before any processing. We shoot raw images and are starting to experiment more and more with video. After we do the post-processing work and start the video editing we can push 70GB a shoot easily. Our current workflow is like this: Shoot tethered to a laptop each day (typically each shoot lasts two days). At the end of the day, we upload the files up to our local NAS (Netgear ReadyNAS, just shy of 3TB) which is redundant. At the end of the shoot, we load a copy of the entire shoot on the local computers that will be editing so that they don't have to work directly from the NAS. Over the next few weeks while they edit the files, they will push the new files up to the NAS into specific folders. (This is where it gets hairy) We also have an offsite NAS that is located in the business owner's house (about 6TB, but this one is not redundant). We used to attempt to upload files each night from the local NAS to an offsite NAS via rsync, but the pipe was just way too small. It would take days to upload a photoshoot, and everyone in the office suffered from slow internet while that happened. Now what we do is bring the offsite NAS onsite every few weeks and do a manual sync! Locally the sync only takes a few hours and then that night it goes back home with the business owner. Now since the local NAS is less than 3TB, we can't store all of our shoots on there forever. When we get close to full on the local NAS, we grab an external hard drive and archive old shoots (typically 2+ years old) to it, and then store that in our massive safe (on-site though). If we need access to just one or two files, we can retrieve them from the offsite NAS, but if we need access to quite a few of them, we can dig them out of the safe. So that's our workflow. Probably not bullet proof, but it has been serving us well for the last few years. If someone came up with a better solution, we would be all over it!

josh2600 13y ago
Out of curiosity, how big is the pipe at the office? The one holding the local NAS?
- Brandon0 13y ago

hfanson1 13y ago

It is important to be careful with how you try to use Glacier. I have been looking its pricing; It costs $0.01 per GB per month only if you want to upload your data. Downloads add significant cost if you want to retrieve more than 0.17% of your data in a day. Downloading 5% of your data increases your monthly rate to $0.097 per GB. 10% increases to $0.187 per GB 50% increases to $0.907 per GB 100% increases to $1.807 per GB Downloads are also delayed by 4 hours after you make a request.

PStamatiou 13y ago
if I have 2TB stored and only want to retrieve a single 10GB photo shoot one month (or even just a single photo), it would not be anywhere close to 5% (.5% in fact) of my total amount stored. The key point here is that we are dealing with TBs of photos that are there for safe-keeping and they have already been tinkered with and all photo JPGs needed have been exported and put elsewhere. This is just for RAWs and you likely won't need to access them again, definitely not 5%+ of them like this.
deleted 13y ago
[deleted]

martingordon 13y ago

Has anyone used Lightroom 5's new Smart Preview feature? It generates lossy DNGs from your full-sized files that you can still work with while your original sits on an external drive. Once you plug your external drive back in it, it syncs the non-destructive changes back to the RAW file.

hnriot 13y ago

"I almost always shoot in small (5.5MP) or medium (11MP) on my 22.3MP Canon 5D Mark III" Kind of hard to take anything seriously after reading that. He buys a seriously expensive camera and shoots it in point and shoot size. That's just insane. What's so hard with a simple home NAS. I replace the drive in my laptop every year for whatever is then biggest (sweet spot) and ue a NAS for backups. RAID5

rfnslyr 13y ago
Because cloud this and cloud that! I feel like I'm the only person firmly against cloud storage. I'd constantly feel paranoid about my data randomly disappearing at the hands of a company I don't know run by people I don't know on hardware I don't trust. I want everything local.
- joevandyk 13y ago
  What happens when there's an earthquake? Or theft? Or fire?
  - rfnslyr 13y ago
    My own cloud? It's not really hard to get your own service up and running. All my shits auto backed up to my dedicated which has terabytes and terabytes of storage.
    - icebraining 13y ago
      Where do you store it?
      - rfnslyr 13y ago
        Family network. We all run servers in our houses that are connected and each keep backups of eachothers backups. Backups at my parents, grandparents, cousins in colorado, family in Poland. All encrypted, all silently running in the basement of each house. All updated every week.

tambourine_man 13y ago

Backblaze?

sreitshamer 13y ago

I'm not convinced that retrieval time doesn't matter. If you decide you want several photos, wait 4 hours to retrieve them from Glacier, then realize you wanted a few more and have to wait an additional 4 hours, I think that would get old pretty quickly.

hadem 13y ago
I was wondering something similar. Does Glacier have any way to see a preview of the images you are downloading? If I upload 100 images with serialized file names (001.jpg, 002.jpg) how do I know which images to download?
- Pyrodogg 13y ago
  This is most of what the proposed software is. A local, lower-res index of the huge files you have deep archived.
- sreitshamer 13y ago
  There's no preview. Actually there's no way to download or even list your archives through the AWS Console. You need to write (or find) an app that uses the Glacier API.

bigbento 13y ago

Over the past few years, I worked on a few versions of this as a side project. The last iteration didn't do the backup itself--it connected with your Dropbox, sync'd the raw images, and processed them at a preset +-2 EV, and 3 color temperatures. The image results were good, but honestly I never found it particularly compelling. In any case, building this sort of thing is a pain. The most obvious problem is the size of the images. Glacier is an interesting backup solution, and had just come out around the time I decided to stop working on my app. A less obvious problem is raw conversion. All the best converters are proprietary, meaning that I had to piggy back off the Mac OS converter engines. Again, I never got to the point where I felt the product was worth the trouble of buying racking up a Mac Mini to run a little market test. Maybe I'm wrong. I had a (frankly somewhat scattered) draft of a post mortem written, and since it seems like there's some interest in this subject, I posted it: http://scraps.bigbento.com/2013/05/30/hiatus.html http://scraps.bigbento.com/2013/05/30/hiatus.html

stevewilhelm 13y ago

Smugmug's SmugVault "SmugVault is an added service to your SmugMug account that lets you store almost anything for next to nothing. Including files not normally supported by SmugMug." http://help.smugmug.com/customer/portal/articles/84562-more-about-smugvault http://help.smugmug.com/customer/portal/articles/84562-more-...

sreitshamer 13y ago
Although Smugmug is awesome, the issue I'd have with them is the lack of control. If my objects are in my AWS account, that feels more permanent long-term. It's just a gut feeling.

throwaway420 13y ago

I wish I could trust the cloud because it would be a lot more convenient to not have to worry about storing photos, which involves a lot of effort once you advance beyond a casual hobby and you get into storing Terrabytes of large RAW files and maintaining adequate backups. But the idea of storing any files in the cloud is very distasteful to me ever since the NSA Snowden story broke. Even something as innocent as a photographer taking pictures of a skyline or some beautiful architecture could land him in hot water if the government is looking at metadata and figuring out that he took lots of photos of a government building or a bridge or a power plant and randomly throwing him on some watch lists or no-fly lists. People have been hassled by the feds for a lot less. My preference would be for all of these cloud storage providers like Google, Facebook, Apple, Dropbox, Amazon, etc to unambiguously and fiercely challenge the government on this and somehow unambiguously prove to people that they're respecting peoples' privacy and not complying with this stuff. Their denials so far have been obvious non-denials that are very carefully worded around "no direct access" when that's enough wiggle room to do just about anything. Using some kind of cloud photo storage is a bad idea IMO until this gets sorted out.

chadk 13y ago
FINALLY. I was going through every comment on this thread and I was wondering why no one was talking about security. This is one of the key features of the data warehousing business we have been contemplating.
prirun 13y ago
Everything must be encrypted on the client before sending to any cloud service. Then it doesn't matter if the encrypted backup is obtained.

post_break 13y ago

"I almost always shoot in small (5.5MP) or medium (11MP) on my 22.3MP Canon 5D Mark III3." Sorry but what? I actually had to read that a couple of times.

mark_l_watson 13y ago

I only have a 128GB SSD on my MacBook Air, which is my primary computer. I used to run out of space until I read somewhere that you can use DropBox advanced settings to not keep specific directories on specific devices. So, I am not likely to have directories like, for example, videos2012, pictures2012 synced on my Air but they are on my old Linux and MacBook Pro laptops which have lots of disk space. With high network speeds, I can quickly re sync directories, or just power on one of my old laptops.

bcl 13y ago

My solution is to copy them all to Glacier, I also have local copies, Glacier is only for worst case scenario retrieval. I wrote about the little script I use here: http://www.brianlane.com/automatic-backup-of-files-to-s3-and-glacier.html http://www.brianlane.com/automatic-backup-of-files-to-s3-and...

fsckin 13y ago

I'm the SO of a professional newborn photographer who generated somewhere in the neighborhood of 20-30TB of RAW photos, backups, LR xmp sidecars, etc in the last year. JPEGs don't even register. High end DSLR RAWs are gigantic! I'm the technical support for overall workflow, storage & backup engineer, etc, etc. I wear many hats. Right now I've got a backup on import to a FreeNAS (BSD) filer running on a HP Micro Server. Lightroom backups go to the same filer and are synced as well. My local server rsyncs the delta every night to an identical machine at a friends' house. This isn't perfect. The one place that isn't 100% backed up is current projects in post-production. That's an acceptable trade off for now. The most important thing to note is that there are two kinds of photographers: Those who never lost data, and those who care about backups. As mentioned elsewhere in this thread, converting members of Group A into Group B is a tough sell. Video testimonials might be a good option. A storage service for photographers MUST be invisible to their workflow. Make it as easy to use as an egg timer. At the MOST, an addition of a single screen with sensible defaults (just click next) on photo import. Ignoring error messages that get in their way is to be expected. Care more about their backups than they do. Send me (tech support guy) an email to notify if there haven't been backups added in X days, along with weekly reports of data usage per day -- how many photos, how large, what root folders, usage trends, etc. An ideal installation would be automatic detection of what program they're using, what places files are stored (hint: It's not just on a local drive). Detect new file locations and back those up too. An inexpensive (<$25) option of "only backup current projects I'm working on and anything I shot in the last month", that actually works... I'd sign up in a heartbeat.

FollowSteph3 13y ago

And what happens if one glacier decides to close? Do you think you'll get any notice ahead of time? I wrote about these types of risks at http://www.followsteph.com/2012/04/19/what-are-the-risks-of-cloud-services/ http://www.followsteph.com/2012/04/19/what-are-the-risks-of-...

foxhop 13y ago

http://www.snaposit.com/ http://www.snaposit.com/ tried but the market didn't pay.

foxhop 13y ago

snaposit : http://www.snaposit.com/ http://www.snaposit.com/ tried but the market wasn't there. I'm going to send this post to the founder to see his take on it.

josephlord 13y ago

Covering video too would be great but maybe a second phase handling them separately is a hassle for users.

gcb0 13y ago

What about flickr and the insane free space they provide? you it's free now right? searching is also a breeze there. not sure how you'd go about downloading them all though...

philfrasty 13y ago
no RAW support...

gerardmurphy 13y ago

I am the CEO/Co-Founder of Mosaic. We are a company that helps serious photographers backup their photographers. I am happy to share what we have learned in the past 2 years. www.mosaicarchive.com As a background, we now have thousands of users and customers. We also finished raising a series A in February. (Not that raising money proves anything) http://www.boston.com/business/technology/innoeco/2013/02/mosaic_storage_systems_collect.html http://www.boston.com/business/technology/innoeco/2013/02/mo... We are now growing at about 5%-7% a week. We recently peaked in the top 100 grossing US iPad Photo/Video Apps in iOS - although most of our revenue comes outside of iOS. (As they take a 30% cut.) We target serious photographers (pro's and prosumers) who use Adobe Lightroom. For pure backup customers our sweet spot are those who have outgrown BackBlaze and Crashplan. (Both great services but geared at consumers.) On the backup side, we help photographers manage their backups by using Adobe Lightroom metadata. Customers can choose to backup everything in Lightroom automatically. However, many photographers use metadata like stars or flags to designate their best photos. We allow users to automatically backup these "best" photos. This provides the automation of the best backup tools with more precision for larger customers. We use Amazon Glacier behind the scenes to store the original photos. (Yes, WMF the restoration costs even out for us over all of our users.) (Sidebar - we actually ran our data center for months before switching to Glacier...very glad to have made the switch.) However... we are not an online backup company. Online backup is a feature is our service... but not the entirety of it. Once we have have photographers photos, we also want to help photographers better use, manage, share, and enjoy those photos. At one point I would have said we are a backup company, now I call us a workflow company. We also offer a web and iOS App to privately view and share your Lightroom photos. (We take JPG snapshots of the RAW files so they load quickly and have the Lightroom edits. This part of the service runs in Amazon S3 not Glacier) This gives you access to your photos in the same way you are used to experiencing them in Lightroom. (Lightroom users try this for free - https://itunes.apple.com/us/app/mosaic-archive/id627973694?ls=1&mt=8 https://itunes.apple.com/us/app/mosaic-archive/id627973694?l...) The next step which we are working on now is adding two-way sync where you can manage your photos in the App and it populates back to your desktop files and Lightroom. We are also working on more collaborative features so photographers can share this culling and editing process with their clients or family members. The bigger problem we are solving is that photographers have loads of large files... backing them up is painful yes... but managing them all is more painful. Sifting through 000s of images to find your best stuff frankly stinks. This problem is only getting bigger as we shoot more and more. For those interested in this idea, please consider joining us. We are hiring a couple more software developers this year. http://www.mosaicarchive.com/about-mosaic-photo-storage/careers/software-engineer/ http://www.mosaicarchive.com/about-mosaic-photo-storage/care... Thanks for the discussion and happy to answer all questions. Best, Gerard www.mosaicarchive.com

esonderegger 13y ago

I really wish we as a developer community would better differentiate between "backup" and "preservation". They are both related to storage, but are fundamentally different problems. The article touches on the desire to have photos safe for decades, but doesn't really get into the strategy necessary to make that happen. If we care about decades, storing the raw NEF or CR2 file is almost certainly the wrong approach. Those files could easily be as difficult to open in fifteen years as a ClarisWorks document is today (just to choose one example - proprietary formats for professional software from the late 90's are all pretty hard to open today). Also, important metadata is kept in the lrcat file. Unless we expect to always be able to open our copies of Lightroom, we need to migrate that data too. We used to think of "archiving" as putting something on a shelf and forgetting about it, sometimes taking environmental factors like humidity into account. With our digital data, we need a plan for periodically checking it, both for fixity and to make sure at risk file formats are migrated to current best practices. This will require greater awareness for the complexities involved (and the need for open file formats) from everyone from creatives to storage and service providers. By the way, every creative industry is struggling with this right now. Recording studios have session files created in ProTools 4. Designers have really old Quark files, even though they've been working in InDesign for the past five years. Composers have Finale files. A lot of these file formats depend on plugins to be able to display properly. I don't mean to sound discouraging, I think there's plenty of opportunities for viable businesses here. We just need to start using the right terms.

graue 13y ago
In a word, VMs. If you make sure you can open that NEF or CR2 file in a virtual machine, and the virtual machine image is in a standard, open format, you're set. You just have to worry about the image format becoming obsolete and not 10 or 20 different proprietary formats. That said, I used to be a pretty prolific hobbyist musician and I haven't pulled this off myself. I have tons of old songs I can't open because they rely on a fiddly collection of old freeware plugins for freeware music apps from 11 years ago. I also have songs I can't open because they only play in a music tracker I wrote, which is open-source, but only runs on OpenBSD. It is a tricky problem and it seems hard to tackle because the market for professional software inherently favors closed-source, which has no incentive to adopt open, non-siloed formats.

saidajigumi 13y ago

I have a serious problem with the proposal offered in TFA and most variants in this thread: it encourages bad backup policy. Critical data needs three (or more) backups. In my experience, one cloud provider can only ever be considered as being one backup no matter their technical architecture. For example, you can't (and in the broad "you", aren't even qualified to) audit the provider for data-loss or SLA-impacting SPOFs. You just have to assume that they're there. You also take on non-technical failure modes: the provider can go out of business, get bought by an uninterested owner (think Delicious), change focus (Google Reader), or experience myriad other problems. FWIW, the best success I've had with helping others switch to good backup policy happened once it became economically feasible to buy a new laptop and two bus-powered external drives as big as the laptop's internal storage. The externals become bootable backups, one of which can be preiodically rotated to an offsite location. This has prevented severe dataloss events for myself and others far more times than I care to count now.

famousactress 13y ago
FWIW, I'm more technical than a typical user might be but I'd be very intrigued by a provider that used multiple cloud service and gave me the keys to the underlying S3 buckets (or whatever) so I could independently verify. That's probly a more valuable idea to a company selling backup to nerds though, I guess.

peteforde 13y ago

I'd use this. 100% for sure. In fact, "a plugin for Aperture/Lightroom that backs up to Amazon Glacier" is on my idea list. Great minds think alike, fools seldom differ. One thing I'd like to point out is that photographers are unlikely to be as cost conscious as other consumers. We put a lot of effort into taking these damn photos, and it'd be nice if I could be confident that they'd outlive me. I'm willing to pay a premium for that.

Paul12345534 13y ago

Currently using Crashplan to backup my photos. I copied 200+ DVDs of photos to my hard drive in numbered directories... then deleted them after I had them uploaded (Crashplan keeps deleted files if you choose that option).

jrochkind1 13y ago

> I just need to know they are somewhere safe for decades to come. The thing about Amazon Glacier... unless they are themselves making more than one copy, and taking hash fingerprints, and storing a couple copies of the hash fingerprints, and using it all to check for bit rot on a regular basis... ...and I don't think they're doing all (any?) of these things... ...then people are sadly going to find that their stuff _isn't_ neccesarily safe for decades to come. One copy on a single piece of media in an Amazon data center somewhere does not actually make for 'safe for decades to come'.

eurleif 13y ago
https://aws.amazon.com/glacier/#highlights https://aws.amazon.com/glacier/#highlights > Amazon Glacier is designed to provide average annual durability of 99.999999999% for an archive. The service redundantly stores data in multiple facilities and on multiple devices within each facility. To increase durability, Amazon Glacier synchronously stores your data across multiple facilities before returning SUCCESS on uploading archives. Unlike traditional systems which can require laborious data verification and manual repair, Glacier performs regular, systematic data integrity checks and is built to be automatically self-healing.

adelmand 13y ago

We (cintrexav.com - a small video conversion service provider) are building something very much like this right now. Will post here when we get to MVP in a month or so!

madao 13y ago

I did some on the side admin work for a Photographer once, a real big name. He was in the middle of making up exactly what this was, however he came up against the same issues everyone has. Firstly space costs money, Hardware costs alot too, not to mention bandwidth, this guy would easily do 100-200gb of photos in a couple shoots. all very high res shots (talking about for big company ads) his idea is to sell a server with a software suit that people can dump into a datacenter or at home and upload photos to it, personally I liked the idea but local storage and hardware failure + the ability to grow the storage was a real killer for his idea.. (not to mention that your playing with peoples businesses and lives if you lose those photos)

espinchi 13y ago

I'm helping build Photofeed, a platform that stores and organizes your private photos from your different devices in the cloud. Photographers with the problem described in the article are within the target audience, even if we didn't implement some of the key features for them yet (e.g. storage of images in RAW format). We'd love having some early feedback from you guys! (Didn't officially launch yet.)

VikingCoder 13y ago

I just came across the Plug Kickstarter, which I think might be the basis of a viable back-up storage for professional photographers. http://www.kickstarter.com/projects/cloud-guys/plug-the-brain-of-your-devices http://www.kickstarter.com/projects/cloud-guys/plug-the-brai... Specifically, the $129 level, where you get two plugs, which they claim, "to replicate your data in 2 different places." Talk two of your friends in remote parts of the country into letting you Plug in to their home networks, and this might be viable.

qq66 13y ago

Draft feature set: - Local daemon running that manages this process, on PC or Mac. It can't be done solely from the browser. - The software manages the import from the SD/CF cards, you put them all in, it sorts everything out and wipes the SD cards on the way out. - You then use Lightroom to do whatever you're going to do. The software would do well to have a Lightroom plugin that lets you select particular images to mark as "high priority" and "low priority." - Now the daemon uploads all of your RAWs, Lightroom metadata, and corrected JPGs to cloud storage. - Based on your settings, these RAWs get discarded after some set amount of time (maybe immediately, maybe never), and the highest-quality JPGs are also discarded after a longer amount of time for lower-quality JPGs just in case you need a emergency backup. After 6 months you lose the raws, 1 year the highest quality JPG, 2 years the medium quality JPG, etc. You are charged per gigabyte per month so you can decide how much you care about backup. Basically offload the cognitive effort of discarding old data that isn't worth its upkeep costs anymore. - The files are available via an API that you can use to build your own gallery/shopping cart site, with the ability to call a certain quality, PROOF watermarking, etc. on-the-fly. But the service offers a default site right there for you to do the same, that's stylable and can be implemented on your own domain. - You can get your entire catalog shipped to you on a hard drive at any time for a few hundred dollars.

resistore 13y ago

Relying on only one storage provider is a little bit risky to me. I started a small desktop app for backing up my files with my family photos in mind, leveraging the properties of erasure coding and multiple dropbox/google drive services. Also adding a bit of crypto to follow the post-prism trend... Please, take a look at http://yleprovost.bitbucket.org/ http://yleprovost.bitbucket.org/

prirun 13y ago

I've been developing a backup program for a few years now, HashBackup, that encrypts your files locally, with your own private key, and sends them to offsite storage you control: your own Glacier or S3 account, your offsite rsync server (rsync.net for example), your Gmail or other IMAP account, an ssh server, or ftp server. www.hashbackup.com