5 ms·

Linux-Kernel Archive: RIP - dead harddisk

41 points by bsg75 13y ago

bsg75 13y ago
Additional: http://www.phoronix.com/scan.php?page=news_item&px=MTQ1ODY http://www.phoronix.com/scan.php?page=news_item&px=MTQ1ODY
hannibal5 13y ago
Relevant Linus quote: "Only wimps use tape backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" source: https://groups.google.com/forum/#!msg/linux.dev.kernel/2OEgUvDbNbo/bTk-VE1zrnYJ https://groups.google.com/forum/#!msg/linux.dev.kernel/2OEgU...
- LukeShu 13y ago
  That still mostly worked (s/ftp/git/) > I had pushed out _most_ of my pulls today, so > realistically I didn't lose a lot of work. You're always going to lose the things newer than your last backup.
  - astrodust 13y ago
    That's why the hourly backups of a system like Time Machine is great. I've had several drives fail, and I've only lost about half an hour of work, excluding that which was pushed, picking up almost exactly where I left off.
cjensen 13y ago
On the Mac with Time Machine, I can lose at most an hour of work. Not saying that to praise the Mac; I'm saying that because that's how all computers ought to be. This would be a swell time for Dropbox to offer Linus a free large business account so he can store everything.
- primelens 13y ago
  I've never used it, but do you mean a Desktop with a permanently attached external drive to backup to? How does that work on a laptop?
  - msoad 13y ago
    Apple Time Capsule. Look it up
    - primelens 13y ago
      Ah! But then I'd have another Apple specific gizmo in my life :-). But it looks interesting nevertheless. Thanks.
      - jws 13y ago
        Then you are probably the sort that wants to use rsync and the --link-dest option, which is about the same. (Except for the nice retrieval interface.) You can make a new backup and it will only consume space and copy bandwidth for the files which have changed. It is cheap to keep N backups for large N.
      - pfranz 13y ago
        I have a raspberry pi running http://netatalk.sourceforge.net/ http://netatalk.sourceforge.net/ However, it seems to get in a corrupted if I close my laptop lid mid-backup. I don't lose anything, I just need to run disk util on the backup image. Prior to that I was using time machine on USB and never had an issue. If you work plugged in you'll never lose more than 5-10min of work. It'll remind you every 7 days you haven't backed up. I was looking into a solution for Linux the other week and http://www.rsnapshot.org/ http://www.rsnapshot.org/ works very similarly and is what fits my needs there. The benefit of time machine is that excludes unnecessary OS folders/files by default and the OS installer has a simple interface for recovering the whole hdd from a time machine backup. I recently found time machien now has tmutil, a commandline tool for accessing time machine backups which really helps make managing them more transparent.
- 13y ago
thex86 13y ago
I have a Intel 520 SSD. Fantastic thing I must say and I don't think I can ever use a HDD again, but somehow -- and I realize this may just be baseless -- I don't trust it as much as I trusted my old Seagate HDD, which lasted 7 years. SSDs are getting better now, but are they as reliable as HDD? Can anyone shares his/her expertise on this?
- ori_b 13y ago
  You don't have a reliable option. It's that simple. If you value your data, keep backups. And if you want to avoid hiccups in productivity, use RAID. Data that you don't have backed up is data that you don't have. And keep in mind that RAID, while useful, is not a backup. It doesn't protect against theft, whole-computer destruction, human error, or many other things.
- regularfry 13y ago
  You can certainly buy SSDs now that are rated to more read/write cycles than your 7-year-old Seagate will have gone through. The ones we buy are rated for 3+ years of constant I/O at full bus speed. I don't know how much of that tech is in the sort of SSD you're likely to buy for a desktop, but they're certainly available.
- DanBC 13y ago
  > SSDs are getting better now, but are they as reliable as HDD? Can anyone shares his/her expertise on this? I'm especially interested to hear from people who use many discs, and if they can share some information about the kind of use those discs get. From what I've heard SSDs are more reliable that traditional spinning platter drives, unless you have a bad batch. This is especially true if there is more than one platter in the drive. But I also recognise that it doesn't really matter. All drives fail, and no drive is reliable, so you assume the life is going to be 3 years and you have robust mirroring and backups and replacement policies.
  - Samuel_Michon 13y ago
    > No drive is reliable, so you assume the life is going to be 3 years and you have robust mirroring and backups and replacement policies. A thousand times this. Over the last 2,5 decades I lost far more data than I’d like to admit. Floppy disks and CD-Rs that went bad, hard disks that failed, FTP services that were discontinued or corrupted data, etc. My backup strategy right now: * I have SSDs for use as system disks on all my computers. I keep track of their use and how long I’ve had them. Once one of them turns 3 years old, I’ll swap it out for a new one. * Those computers are all connected to a Time Capsule (Time Machine) for incremental changes. * Some files are also continually saved on iCloud (not all apps support it.) * At night, a real backup gets run from the computers to a Drobo RAID setup. * Everything of value (files I created) gets sent to Backblaze daily (an online backup service). * For web projects, I also use a hosted subversion service. * Most media I buy is cataloged by iTunes Match, where I can download it again whenever I need it. * I use Dropbox, but not for backup, I use it to view files on the go and to share them with others. I know my setup is not perfect, and I could use advice to improve it. I haven’t lost data in the last 3 years, but I’m keeping my fingers crossed. I do count myself lucky to still have all my emails dating back to 1998 (but not the emails from 1994-1997). In large part, I have iCloud (née iTools, .mac, MobileMe) to thank for that.
consider_this 13y ago
Maybe buy two hard disks, and back up one to the other. It is only the linux kernel afterall.
- jamesaguilar 13y ago
  *Linus' copy of the Linux kernel.
flexd 13y ago
If you are doing something as important as this, why aren't you at least running some sort of RAID setup that would have been fine with losing a disk? Or at the very minimum had a decent backup setup so that you would not lose data. For someone who created Linux in the first place, this must/should be very embarrassing.
- primelens 13y ago
  OTOH, it must be nice to know that even if your house blows up (hopefully when you're not in it), your life's work is mirrored and backed up on millions of computers. All he lost were some pull requests which have to be re-sent. The submitters know who they are and if the changes don't show up in a day they just send another email - no harm, no foul.
hakcermani 13y ago
Yes Linus. Why don't you use RAID ?
- astrodust 13y ago
  In a laptop?
  - vacri 13y ago
    There are some laptops with two hard drives, but whether or not these models are nice to use, I don't know. Or maybe they have other hardware with poor linux support....
EvanAnderson 13y ago
I find if vaguely disturbing that John Stoffel suggests to Linus that he mirror two discs in the future (presumably meaning a RAID-1 arrangement) and Linus responds with "I long ago gave up on doing backups." Linus conflating RAID with backup?!? Surely not. My world feels turned upside down... >smile<
- Dylan16807 13y ago
  RAID is a lesser class of backup, and he's given up on the entire realm.
masnick 13y ago
I have Arq (http://www.haystacksoftware.com/arq/ http://www.haystacksoftware.com/arq/) set to back up my ~/git folder every hour. At most I can lose an hour of work. This came in handy once when I accidentally did the equivalent of rm -rf ~/git. Lost 54 minutes of work. Arq is Mac-only, but you could accomplish the same thing with https://www.tarsnap.com/ https://www.tarsnap.com/ and a cron job. Both are changes-only backups so there shouldn't be a huge performance hit. I also use Time Machine when I'm at home but only sort of trust it.
- stingraycharles 13y ago
  Has anyone got any experience with using Dropbox as a backup service? They provide 30 days of file recovery for free, and for a few dollars a month you can get packrat, which provides eternal point-in-time recovery. Seems like a great use case of a git folder backup?
  - houk 13y ago
    I use it like that. iTunes, development, documents, all sorts of things. Easily synced to a new machine if needed. On top of that I pay for Backblaze, just for a little bit more protection.
  - yapcguy 13y ago
    Dropbox is not a backup service, it's for synchronizing local folders.
  - hannibalhorn 13y ago
    I've had weird issues with my whole Dropbox getting deleted - in the middle of the night, with my machine off, have never figured out why - and while your Dropbox can be restored through the web interface, it's just one file at a time. In the case of a "mass delete" you actually need to e-mail support and generally wait a couple of days before they restore everything, which is less than ideal. The last time it happened (the third time!) I wrote some scripts myself using their API to do the restore rather than wait on support. It really doesn't make for an ideal backup solution, though!
- 13y ago
deleted 13y ago
[deleted]
kunai 13y ago
So... does this mean that Apple's flash storage is low-quality? I use the same MacBook Air that Linus does (except it's a 13" model), never turn it off, have reinstalled the OS multiple times, archive my data, do around ~3GB worth of transfers every day, and run 3 different VMs more often than not. Maybe I should just upgrade to a newer SSD. Anyone had luck with OWC?
- hnha 13y ago
  All devices can fail. No use to overstate an anecdote.
lutusp 13y ago
This is especially ironic, as Linus has lately been extolling the virtues of SSDs: http://www.wired.com/wiredenterprise/2012/10/linus-torvalds-hard-disks/ http://www.wired.com/wiredenterprise/2012/10/linus-torvalds-... And when Linus speaks, a lot of people listen.

41 points by bsg75 13y ago

bsg75 13y ago

Additional: http://www.phoronix.com/scan.php?page=news_item&px=MTQ1ODY http://www.phoronix.com/scan.php?page=news_item&px=MTQ1ODY

hannibal5 13y ago

Relevant Linus quote: "Only wimps use tape backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" source: https://groups.google.com/forum/#!msg/linux.dev.kernel/2OEgUvDbNbo/bTk-VE1zrnYJ https://groups.google.com/forum/#!msg/linux.dev.kernel/2OEgU...

LukeShu 13y ago
That still mostly worked (s/ftp/git/) > I had pushed out _most_ of my pulls today, so > realistically I didn't lose a lot of work. You're always going to lose the things newer than your last backup.
- astrodust 13y ago
  That's why the hourly backups of a system like Time Machine is great. I've had several drives fail, and I've only lost about half an hour of work, excluding that which was pushed, picking up almost exactly where I left off.

cjensen 13y ago

On the Mac with Time Machine, I can lose at most an hour of work. Not saying that to praise the Mac; I'm saying that because that's how all computers ought to be. This would be a swell time for Dropbox to offer Linus a free large business account so he can store everything.

primelens 13y ago
I've never used it, but do you mean a Desktop with a permanently attached external drive to backup to? How does that work on a laptop?
- msoad 13y ago
  Apple Time Capsule. Look it up
  - primelens 13y ago
    Ah! But then I'd have another Apple specific gizmo in my life :-). But it looks interesting nevertheless. Thanks.
    - jws 13y ago
      Then you are probably the sort that wants to use rsync and the --link-dest option, which is about the same. (Except for the nice retrieval interface.) You can make a new backup and it will only consume space and copy bandwidth for the files which have changed. It is cheap to keep N backups for large N.
    - pfranz 13y ago
      I have a raspberry pi running http://netatalk.sourceforge.net/ http://netatalk.sourceforge.net/ However, it seems to get in a corrupted if I close my laptop lid mid-backup. I don't lose anything, I just need to run disk util on the backup image. Prior to that I was using time machine on USB and never had an issue. If you work plugged in you'll never lose more than 5-10min of work. It'll remind you every 7 days you haven't backed up. I was looking into a solution for Linux the other week and http://www.rsnapshot.org/ http://www.rsnapshot.org/ works very similarly and is what fits my needs there. The benefit of time machine is that excludes unnecessary OS folders/files by default and the OS installer has a simple interface for recovering the whole hdd from a time machine backup. I recently found time machien now has tmutil, a commandline tool for accessing time machine backups which really helps make managing them more transparent.
13y ago

thex86 13y ago

I have a Intel 520 SSD. Fantastic thing I must say and I don't think I can ever use a HDD again, but somehow -- and I realize this may just be baseless -- I don't trust it as much as I trusted my old Seagate HDD, which lasted 7 years. SSDs are getting better now, but are they as reliable as HDD? Can anyone shares his/her expertise on this?

ori_b 13y ago
You don't have a reliable option. It's that simple. If you value your data, keep backups. And if you want to avoid hiccups in productivity, use RAID. Data that you don't have backed up is data that you don't have. And keep in mind that RAID, while useful, is not a backup. It doesn't protect against theft, whole-computer destruction, human error, or many other things.
regularfry 13y ago
You can certainly buy SSDs now that are rated to more read/write cycles than your 7-year-old Seagate will have gone through. The ones we buy are rated for 3+ years of constant I/O at full bus speed. I don't know how much of that tech is in the sort of SSD you're likely to buy for a desktop, but they're certainly available.
DanBC 13y ago
> SSDs are getting better now, but are they as reliable as HDD? Can anyone shares his/her expertise on this? I'm especially interested to hear from people who use many discs, and if they can share some information about the kind of use those discs get. From what I've heard SSDs are more reliable that traditional spinning platter drives, unless you have a bad batch. This is especially true if there is more than one platter in the drive. But I also recognise that it doesn't really matter. All drives fail, and no drive is reliable, so you assume the life is going to be 3 years and you have robust mirroring and backups and replacement policies.
- Samuel_Michon 13y ago
  > No drive is reliable, so you assume the life is going to be 3 years and you have robust mirroring and backups and replacement policies. A thousand times this. Over the last 2,5 decades I lost far more data than I’d like to admit. Floppy disks and CD-Rs that went bad, hard disks that failed, FTP services that were discontinued or corrupted data, etc. My backup strategy right now: * I have SSDs for use as system disks on all my computers. I keep track of their use and how long I’ve had them. Once one of them turns 3 years old, I’ll swap it out for a new one. * Those computers are all connected to a Time Capsule (Time Machine) for incremental changes. * Some files are also continually saved on iCloud (not all apps support it.) * At night, a real backup gets run from the computers to a Drobo RAID setup. * Everything of value (files I created) gets sent to Backblaze daily (an online backup service). * For web projects, I also use a hosted subversion service. * Most media I buy is cataloged by iTunes Match, where I can download it again whenever I need it. * I use Dropbox, but not for backup, I use it to view files on the go and to share them with others. I know my setup is not perfect, and I could use advice to improve it. I haven’t lost data in the last 3 years, but I’m keeping my fingers crossed. I do count myself lucky to still have all my emails dating back to 1998 (but not the emails from 1994-1997). In large part, I have iCloud (née iTools, .mac, MobileMe) to thank for that.

consider_this 13y ago

Maybe buy two hard disks, and back up one to the other. It is only the linux kernel afterall.

jamesaguilar 13y ago
*Linus' copy of the Linux kernel.

flexd 13y ago

If you are doing something as important as this, why aren't you at least running some sort of RAID setup that would have been fine with losing a disk? Or at the very minimum had a decent backup setup so that you would not lose data. For someone who created Linux in the first place, this must/should be very embarrassing.

primelens 13y ago
OTOH, it must be nice to know that even if your house blows up (hopefully when you're not in it), your life's work is mirrored and backed up on millions of computers. All he lost were some pull requests which have to be re-sent. The submitters know who they are and if the changes don't show up in a day they just send another email - no harm, no foul.

hakcermani 13y ago

Yes Linus. Why don't you use RAID ?

astrodust 13y ago
In a laptop?
- vacri 13y ago
  There are some laptops with two hard drives, but whether or not these models are nice to use, I don't know. Or maybe they have other hardware with poor linux support....

EvanAnderson 13y ago

I find if vaguely disturbing that John Stoffel suggests to Linus that he mirror two discs in the future (presumably meaning a RAID-1 arrangement) and Linus responds with "I long ago gave up on doing backups." Linus conflating RAID with backup?!? Surely not. My world feels turned upside down... >smile<

Dylan16807 13y ago
RAID is a lesser class of backup, and he's given up on the entire realm.

masnick 13y ago

I have Arq (http://www.haystacksoftware.com/arq/ http://www.haystacksoftware.com/arq/) set to back up my ~/git folder every hour. At most I can lose an hour of work. This came in handy once when I accidentally did the equivalent of rm -rf ~/git. Lost 54 minutes of work. Arq is Mac-only, but you could accomplish the same thing with https://www.tarsnap.com/ https://www.tarsnap.com/ and a cron job. Both are changes-only backups so there shouldn't be a huge performance hit. I also use Time Machine when I'm at home but only sort of trust it.

stingraycharles 13y ago
Has anyone got any experience with using Dropbox as a backup service? They provide 30 days of file recovery for free, and for a few dollars a month you can get packrat, which provides eternal point-in-time recovery. Seems like a great use case of a git folder backup?
- houk 13y ago
  I use it like that. iTunes, development, documents, all sorts of things. Easily synced to a new machine if needed. On top of that I pay for Backblaze, just for a little bit more protection.
- yapcguy 13y ago
  Dropbox is not a backup service, it's for synchronizing local folders.
- hannibalhorn 13y ago
  I've had weird issues with my whole Dropbox getting deleted - in the middle of the night, with my machine off, have never figured out why - and while your Dropbox can be restored through the web interface, it's just one file at a time. In the case of a "mass delete" you actually need to e-mail support and generally wait a couple of days before they restore everything, which is less than ideal. The last time it happened (the third time!) I wrote some scripts myself using their API to do the restore rather than wait on support. It really doesn't make for an ideal backup solution, though!
13y ago

deleted 13y ago

[deleted]

kunai 13y ago

So... does this mean that Apple's flash storage is low-quality? I use the same MacBook Air that Linus does (except it's a 13" model), never turn it off, have reinstalled the OS multiple times, archive my data, do around ~3GB worth of transfers every day, and run 3 different VMs more often than not. Maybe I should just upgrade to a newer SSD. Anyone had luck with OWC?

hnha 13y ago
All devices can fail. No use to overstate an anecdote.

lutusp 13y ago

This is especially ironic, as Linus has lately been extolling the virtues of SSDs: http://www.wired.com/wiredenterprise/2012/10/linus-torvalds-hard-disks/ http://www.wired.com/wiredenterprise/2012/10/linus-torvalds-... And when Linus speaks, a lot of people listen.