2 ms·

Tahoe: A Secure Distributed Filesystem

28 points by bootload 19y ago

bootload 19y ago
Found this via a twisted article & remembered that http://news.ycombinator.com/user?id=secorp http://news.ycombinator.com/user?id=secorp is working on this as well ~ http://news.ycombinator.com/item?id=108139 http://news.ycombinator.com/item?id=108139
secorp 19y ago
Yes, Brian presented at PyCon a couple weeks ago. Tahoe is the open-source project that we are working on together along with a few other guys. We also are involved in the commercial entity Allmydata.com which uses Tahoe as a back-end. We sell a managed storage service on top of a grid of servers that we maintain. Oh, I should add that we are having a HackFest at our office tonight! I know it's short notice, but every month or so we order pizza and beer and have a couple loose presentations, usually about distributed systems. Anybody is free to come, consume, present, or just sit in a corner and program. Logistics are below: http://allmydata.org/pipermail/tahoe-dev/2008-March/000462.html http://allmydata.org/pipermail/tahoe-dev/2008-March/000462.h...
Herring 19y ago
What's the use, when you can get terabyte USB drives these days? It's impressive engineering don't get me wrong, but..
- inovica 19y ago
  I can think of several reasons: 1) How often do you remember to plug your USB drive into your laptop to do your backups 2) What if your machine and your USB drive are caught in a fire, burglary etc I could go on, but whats the point :)
  - cstejerean 19y ago
    From my perspective the ideal solution involves a network attached 1TB+ of storage which all my machines can back up to (similar to Time Capsule). This drive in turn should rsync to one or more off-site locations. You have some cool technology, don't get me wrong. But from a user perspective I don't care how my data is stored as long as I know I have n copies, where n depends on my paranoia level (one local and one remote copies probably works for most people).
    - inovica 19y ago
      Agreed. I have a time capsule here (works very well) but we're also looking at this technology to provide that added backup level. As we all know, its only after a disaster that many people put solutions in place
- secorp 19y ago
  Tahoe is aimed at the issues that arise after you buy your 1TB drive which provides local storage quite well. We try to provide data reliability and availability that can survive colocation outages, computer and hard disk crashes, complete local data loss due to natural disasters, etc. Because the encrypted data is distributed over a wide number of servers, it is very difficult to lose data once it is stored in a grid. Also, it allows you to remotely access your data from the web or other authenticated clients. For our production servers, we can lose up to 66% of all of our servers simultaneously before any data loss will occur. This parameter is tunable such that you can make it even more robust at the expense of more data being stored and pushed around the network.
dhouston 19y ago
what's the latency like? there's no way these things can feel like local filesystems.
- wmf 19y ago
  It's for backup; the latency is mostly irrelevant. Tahoe sounds cool, but I think it's a stretch to call it a filesystem.

28 points by bootload 19y ago

Found this via a twisted article & remembered that http://news.ycombinator.com/user?id=secorp http://news.ycombinator.com/user?id=secorp is working on this as well ~ http://news.ycombinator.com/item?id=108139 http://news.ycombinator.com/item?id=108139

secorp 19y ago

Yes, Brian presented at PyCon a couple weeks ago. Tahoe is the open-source project that we are working on together along with a few other guys. We also are involved in the commercial entity Allmydata.com which uses Tahoe as a back-end. We sell a managed storage service on top of a grid of servers that we maintain. Oh, I should add that we are having a HackFest at our office tonight! I know it's short notice, but every month or so we order pizza and beer and have a couple loose presentations, usually about distributed systems. Anybody is free to come, consume, present, or just sit in a corner and program. Logistics are below: http://allmydata.org/pipermail/tahoe-dev/2008-March/000462.html http://allmydata.org/pipermail/tahoe-dev/2008-March/000462.h...

Herring 19y ago

What's the use, when you can get terabyte USB drives these days? It's impressive engineering don't get me wrong, but..

inovica 19y ago
I can think of several reasons: 1) How often do you remember to plug your USB drive into your laptop to do your backups 2) What if your machine and your USB drive are caught in a fire, burglary etc I could go on, but whats the point :)
- cstejerean 19y ago
  From my perspective the ideal solution involves a network attached 1TB+ of storage which all my machines can back up to (similar to Time Capsule). This drive in turn should rsync to one or more off-site locations. You have some cool technology, don't get me wrong. But from a user perspective I don't care how my data is stored as long as I know I have n copies, where n depends on my paranoia level (one local and one remote copies probably works for most people).
  - inovica 19y ago
    Agreed. I have a time capsule here (works very well) but we're also looking at this technology to provide that added backup level. As we all know, its only after a disaster that many people put solutions in place
secorp 19y ago
Tahoe is aimed at the issues that arise after you buy your 1TB drive which provides local storage quite well. We try to provide data reliability and availability that can survive colocation outages, computer and hard disk crashes, complete local data loss due to natural disasters, etc. Because the encrypted data is distributed over a wide number of servers, it is very difficult to lose data once it is stored in a grid. Also, it allows you to remotely access your data from the web or other authenticated clients. For our production servers, we can lose up to 66% of all of our servers simultaneously before any data loss will occur. This parameter is tunable such that you can make it even more robust at the expense of more data being stored and pushed around the network.

dhouston 19y ago

what's the latency like? there's no way these things can feel like local filesystems.

wmf 19y ago
It's for backup; the latency is mostly irrelevant. Tahoe sounds cool, but I think it's a stretch to call it a filesystem.