6 ms·

Lol. I knew it was you as soon as I saw "I use tarsnap" as the only one checked. I love Tarsnap. I use you for a lot of very-critical-yet-small files. My pe

Lol. I knew it was you as soon as I saw "I use tarsnap" as the only one checked. I love Tarsnap. I use you for a lot of very-critical-yet-small files. My personal pain point is that I'd like to back up a few TB of data but can't afford to--it would be nice to back up an entire workstation. I'm interested in seeing if that's a sentiment that others share, or if it's just me.

cperciva 16y ago
My personal pain point is that I'd like to back up a few TB of data but can't afford to--it would be nice to back up an entire workstation. Why do you have a few TB of data on a workstation? What sort of work are you doing?
- drewcrawford 16y ago
  Right now about a quarter of respondents have 1TB or more, so while I'm not in the majority, it seems I'm in good company. 47% of respondents seem to be in the 100GB-1TB bracket as of right now. Here's the breakdown of my data (largish categories only) ==BUSINESS CRITICAL== * 5GB of codebases I've committed to in the last 3 years. * 50GB worth of software. Mostly development tools. Five different versions of XCode, I actually use most of them every day (scary, I know…) * 50GB of VM images I use for development. I could rebuild them, but I really don't want to. * 80GB on my boot volume. Stuff I would backup if and only if I want a live, bootable backup. Save me a lot of time during a restore. * 5GB in ebooks; primarily used for reference / searching * 20GB in music * 20GB in scanned documents, mostly financial and legal paperwork, contracts, etc. * 15GB of e-mail ==OTHER THINGS AVAILABLE NOWHERE ELSE== * 250GB of samples/patches/instruments that I've built over the years and songs I've written (I play keyboards for fun). Not business critical, but I'd be sad if I lost it. * 20GB of backups from computers I used to own growing up. The nostalgia of that game I was coding in middle-school, etc. * 100GB in learning resources I've collected over the years that are out of print or otherwise unavailable anywhere else. Not useful to me (anymore), but sharing them with others is valuable to both me and them. * 20GB of home video * 13GB in personal photos ==THINGS I DON'T WANT TO DOWNLOAD AGAIN== * 375GB of Apple Development videos. I refer to these fairly regularly. I typically watch a few each day to keep up-to-date. * 20GB of Steam games Looking at this list, sure, I could live without some of it. But a lot of it is pretty important, and it would be nice to back up most of it.
  - aw3c2 16y ago
    Backup the whole thing on harddisks and store one at your bank (if they do have that capability where you live) and another one at some relatives place. Encrypt and seal of course. Update that with "slow" data every month or couple of months. Decide what needs to be backupped daily/weekly and pay some money for that to do it online. How much would it be worth to you if you lost it in a second?
- pedrocr 16y ago
  I'm not the OP but I'll add my datapoint here. I have that just because of digital photography. My current 12MP camera fills up a 16GB card easily if I am covering a 2-3 hour event for some friend. That adds up fast. Right now I do a versioned rsync to a server I host myself that has a (just upgraded from 1TB) 4.5TB RAID5 array. That was ~400$ in disks when I upgraded. Even at the current ~1TB usage that would be 3600$ per year in tarsnap storage alone, without the bandwidth cost. What I'm still missing is a way to backup that server. I could just collocate a second server for a fraction of the tarsnap cost, and I don't really want to be backing up into physical media (tapes or drives) and shuttling that to a deposit box. What I need is tarsnap only 10x cheaper. I'll accept a lower redundancy than S3 for it and especially a lower turnaround time for restores. S3 seems to be built for on-line transactional stuff not backups so is a poor fit for these services. Even the reduced redundancy version only seems a marginal difference.
  - sreitshamer 16y ago
    Comparing S3 to an offsite RAID volume seems like apples to oranges. The biggest difference I think is the data are stored in multiple data centers and Amazon takes care of all the issues with that. There were a lot of interesting comments about S3 cost here: http://news.ycombinator.com/item?id=422225 http://news.ycombinator.com/item?id=422225 It would be awesome if Amazon offered something analogous to a single offsite automagically-expanding RAID volume for a lot less money than S3.
    - pedrocr 16y ago
      > Comparing S3 to an offsite RAID volume seems like apples to oranges. And that was exactly my point. Amazon is selling apples and I want an orange. Tarsnap is an apple pie and is the perfect kind of pie I want. But I want an orange pie. I would be ok with a service that didn't give me online access to my data. Run a very large stack of removable SATA drive slots. Copy incoming files into two of them in two locations. Store them securely when they are full. Fetch them back when I need to restore. That should cut way down on the CapEx of all the NAS/SAN hardware S3 has to run.
      - 16y ago
- bigsassy 16y ago
  FWIW, I also have the same problem (I'm a tarsnap user). I current have just shy of 2TB of GIS data (1.6TB is elevation data alone), so right now I backup everything except my GIS data. I loves me some tarsnap for the other stuff, but right now it's not a solution for this much data. This is definately a pain point of mine. Luckily, the data doesn't change often, and if I needed to restore from data loss it doesn't need to happen ASAP. So I'm thinking just saving the data to seperate hard disks and storing those in a safety deposit box will fulfill my needs. Does that sound right? Or am I missing some large dangers in this solution? (I'm pretty new to this stuff)
uggedal 16y ago
Personally I use Puppet to handle configuration of all my systems. All code is on git/hg on codebasehq.com. This means that I don't need to backup entire systems, only databases and other user created content (which I use tarsnap for).

cperciva 16y ago

My personal pain point is that I'd like to back up a few TB of data but can't afford to--it would be nice to back up an entire workstation. Why do you have a few TB of data on a workstation? What sort of work are you doing?

drewcrawford 16y ago
Right now about a quarter of respondents have 1TB or more, so while I'm not in the majority, it seems I'm in good company. 47% of respondents seem to be in the 100GB-1TB bracket as of right now. Here's the breakdown of my data (largish categories only) ==BUSINESS CRITICAL== * 5GB of codebases I've committed to in the last 3 years. * 50GB worth of software. Mostly development tools. Five different versions of XCode, I actually use most of them every day (scary, I know…) * 50GB of VM images I use for development. I could rebuild them, but I really don't want to. * 80GB on my boot volume. Stuff I would backup if and only if I want a live, bootable backup. Save me a lot of time during a restore. * 5GB in ebooks; primarily used for reference / searching * 20GB in music * 20GB in scanned documents, mostly financial and legal paperwork, contracts, etc. * 15GB of e-mail ==OTHER THINGS AVAILABLE NOWHERE ELSE== * 250GB of samples/patches/instruments that I've built over the years and songs I've written (I play keyboards for fun). Not business critical, but I'd be sad if I lost it. * 20GB of backups from computers I used to own growing up. The nostalgia of that game I was coding in middle-school, etc. * 100GB in learning resources I've collected over the years that are out of print or otherwise unavailable anywhere else. Not useful to me (anymore), but sharing them with others is valuable to both me and them. * 20GB of home video * 13GB in personal photos ==THINGS I DON'T WANT TO DOWNLOAD AGAIN== * 375GB of Apple Development videos. I refer to these fairly regularly. I typically watch a few each day to keep up-to-date. * 20GB of Steam games Looking at this list, sure, I could live without some of it. But a lot of it is pretty important, and it would be nice to back up most of it.
- aw3c2 16y ago
  Backup the whole thing on harddisks and store one at your bank (if they do have that capability where you live) and another one at some relatives place. Encrypt and seal of course. Update that with "slow" data every month or couple of months. Decide what needs to be backupped daily/weekly and pay some money for that to do it online. How much would it be worth to you if you lost it in a second?
pedrocr 16y ago
I'm not the OP but I'll add my datapoint here. I have that just because of digital photography. My current 12MP camera fills up a 16GB card easily if I am covering a 2-3 hour event for some friend. That adds up fast. Right now I do a versioned rsync to a server I host myself that has a (just upgraded from 1TB) 4.5TB RAID5 array. That was ~400$ in disks when I upgraded. Even at the current ~1TB usage that would be 3600$ per year in tarsnap storage alone, without the bandwidth cost. What I'm still missing is a way to backup that server. I could just collocate a second server for a fraction of the tarsnap cost, and I don't really want to be backing up into physical media (tapes or drives) and shuttling that to a deposit box. What I need is tarsnap only 10x cheaper. I'll accept a lower redundancy than S3 for it and especially a lower turnaround time for restores. S3 seems to be built for on-line transactional stuff not backups so is a poor fit for these services. Even the reduced redundancy version only seems a marginal difference.
- sreitshamer 16y ago
  Comparing S3 to an offsite RAID volume seems like apples to oranges. The biggest difference I think is the data are stored in multiple data centers and Amazon takes care of all the issues with that. There were a lot of interesting comments about S3 cost here: http://news.ycombinator.com/item?id=422225 http://news.ycombinator.com/item?id=422225 It would be awesome if Amazon offered something analogous to a single offsite automagically-expanding RAID volume for a lot less money than S3.
  - pedrocr 16y ago
    > Comparing S3 to an offsite RAID volume seems like apples to oranges. And that was exactly my point. Amazon is selling apples and I want an orange. Tarsnap is an apple pie and is the perfect kind of pie I want. But I want an orange pie. I would be ok with a service that didn't give me online access to my data. Run a very large stack of removable SATA drive slots. Copy incoming files into two of them in two locations. Store them securely when they are full. Fetch them back when I need to restore. That should cut way down on the CapEx of all the NAS/SAN hardware S3 has to run.
    - 16y ago
bigsassy 16y ago
FWIW, I also have the same problem (I'm a tarsnap user). I current have just shy of 2TB of GIS data (1.6TB is elevation data alone), so right now I backup everything except my GIS data. I loves me some tarsnap for the other stuff, but right now it's not a solution for this much data. This is definately a pain point of mine. Luckily, the data doesn't change often, and if I needed to restore from data loss it doesn't need to happen ASAP. So I'm thinking just saving the data to seperate hard disks and storing those in a safety deposit box will fulfill my needs. Does that sound right? Or am I missing some large dangers in this solution? (I'm pretty new to this stuff)

uggedal 16y ago

Personally I use Puppet to handle configuration of all my systems. All code is on git/hg on codebasehq.com. This means that I don't need to backup entire systems, only databases and other user created content (which I use tarsnap for).