4 ms·

I could see this as a Kickstarter project of sorts. Sell a device - a Raspberry Pi with an SD card preloaded with "plug and go" software - that people can buy

I could see this as a Kickstarter project of sorts. Sell a device - a Raspberry Pi with an SD card preloaded with "plug and go" software - that people can buy for $50. Sell alongside it an inexpensive SATA and/or PATA enclosure into which people can put their old drives. They basically plug in their old hard drives and forget about it, perhaps writing off the tops ~$25/mo in electricity cost as a donation†. When they plug it in, the RPi gets an IP and announces its availability via DNS-SD (Bonjour/Avahi/etc.). The user downloads a tool or just knows to visit `myarchiver.local` to pull up the control panel. The control panel lets the user set up IP options, time, etc. as well as control bandwidth usage, scheduling, etc. They can also see the amount of space used and set up notifications of some kind (reboot for updates, disk space, etc.). This little device just sits in a corner and is effectively a multi-TB shard of this 20 PB and growing cloud storage system preserving the Internet. Box it up with some sexy industrial design and some attractive packaging, and it becomes a conversation piece, just like your Apple Time Capsule, Amazon Echo, Butterfly Labs Jalapeño, or other newfangled life device, except it's actually storing more than porn, cat pictures, or magic internet money. † I don't know how feasible it is, but go with it.

greggyb 12y ago
You're actually grossly overestimating the electricity for most of the US. Raspberry Pi and similar run for <$10/year
- brianshaler 12y ago
  > plug in their old hard drives I honestly have no idea how inefficient 'old' spinning disk hard drives might be, but I wouldn't be surprised if an older drive consumes more electricity than a modern, power-sipping ARM-based RPi. Maybe GP took that into consideration?
  - Crito 12y ago
    Compared to the power requirements of an RPi, the power for several spindle drives would be pretty significant.. except you have to figure they won't be spun up most of the time.
  - lunixbochs 12y ago
    11W is about the highest active draw I've seen on a drive datasheet. As far as I know, the power usage hasn't really changed in a while because it's mostly attributed to the 5400-7200RPM motor. The control board is very specialized and draws effectively no power in comparison. The Raspberry Pi can run on a 5V / 500mAh bus, which is a 2.5W max load. This gives us a theoretical 13.5W max constant draw (which is likely much higher than reality, as we're talking about rated power instead of measured). 13.5W * 24h * 365 = 118260Wh = 118.260kWh max in a year At 16 cents per kWh (a lowish power tier for PG&E in Silicon Valley), this comes out to $18.92 a year. These numbers assume a 100% efficient power supply running the unit, but are also talking about theoretical maximum power draw and not observed power. This is also a fraction of the average power draw of any of these you might use: electric oven/range, microwave, fridge, incandescent lights, electric dryer, electric water heater. As far as observed power: I've seen a fairly efficient Intel motherboard (CPU TDP of 17W) with 8 modern spinning disks drawing around 75W (measured at the wall using a Kill-A-Watt with all drives active) so I'd guess the real-world number is <8W per active drive.
    - e12e 12y ago
      There's really no reason for the hd to be powered most of the time, for this use-case. Download the backup (couple of days to a week for 500gb?) -- spin down the drive. Only needs to spin up if anyone wants to read the data. Now if we move away from the idea of "backup" to just: lets make a set of torrents, and try for many copies of each chunk/DHT -- then that's a different story. It'd be interesting to see how efficient one could make such a "massively distributed content dispersion network". Imagine: suddenly the www is distributed, not centralized again!
      - lunixbochs 12y ago
        "Active idle" on a drive is still rated at around 5W. Maybe they don't spin down completely in the idle state? BackBlaze says it's better for the longevity of a drive to leave it running than to power it on and off regularly.
        e12e 12y ago
ris 12y ago
The Archive Team have something called the Archive Team Warrior (http://archiveteam.org/index.php?title=ArchiveTeam_Warrior http://archiveteam.org/index.php?title=ArchiveTeam_Warrior) which seems quite similar to this except it's a virtual rather than physical appliance. Worth a look. (edit: although looking further this is more for distributed crawling, while the results will be uploaded to their archive)
- sp332 12y ago
  The warriors download items assigned by a central tracker, then upload each item to a holding area. The warriors don't keep items around when they're finished. Also, the current tracker keeps everything in RAM, and can't keep up with more than ~2 million items at once. It would take a major rewrite just to keep track of all the pieces. P.S. Please run a warrior instance. It comes in a small VM image that you can run in the background all the time.
jpalomaki 12y ago
One option to fund this would be to team up with some storage provider like Bittorrent Sync. They would sponsor the device, I would provide hard disk, network and electricity. Sponsor would get 20-30% of my disk space for their purposes and the rest would go for Internet Archive. A bit similar to what Fon was doing. http://en.wikipedia.org/wiki/FON http://en.wikipedia.org/wiki/FON

greggyb 12y ago

You're actually grossly overestimating the electricity for most of the US. Raspberry Pi and similar run for <$10/year

brianshaler 12y ago
> plug in their old hard drives I honestly have no idea how inefficient 'old' spinning disk hard drives might be, but I wouldn't be surprised if an older drive consumes more electricity than a modern, power-sipping ARM-based RPi. Maybe GP took that into consideration?
- Crito 12y ago
  Compared to the power requirements of an RPi, the power for several spindle drives would be pretty significant.. except you have to figure they won't be spun up most of the time.
- lunixbochs 12y ago
  11W is about the highest active draw I've seen on a drive datasheet. As far as I know, the power usage hasn't really changed in a while because it's mostly attributed to the 5400-7200RPM motor. The control board is very specialized and draws effectively no power in comparison. The Raspberry Pi can run on a 5V / 500mAh bus, which is a 2.5W max load. This gives us a theoretical 13.5W max constant draw (which is likely much higher than reality, as we're talking about rated power instead of measured). 13.5W * 24h * 365 = 118260Wh = 118.260kWh max in a year At 16 cents per kWh (a lowish power tier for PG&E in Silicon Valley), this comes out to $18.92 a year. These numbers assume a 100% efficient power supply running the unit, but are also talking about theoretical maximum power draw and not observed power. This is also a fraction of the average power draw of any of these you might use: electric oven/range, microwave, fridge, incandescent lights, electric dryer, electric water heater. As far as observed power: I've seen a fairly efficient Intel motherboard (CPU TDP of 17W) with 8 modern spinning disks drawing around 75W (measured at the wall using a Kill-A-Watt with all drives active) so I'd guess the real-world number is <8W per active drive.
  - e12e 12y ago
    There's really no reason for the hd to be powered most of the time, for this use-case. Download the backup (couple of days to a week for 500gb?) -- spin down the drive. Only needs to spin up if anyone wants to read the data. Now if we move away from the idea of "backup" to just: lets make a set of torrents, and try for many copies of each chunk/DHT -- then that's a different story. It'd be interesting to see how efficient one could make such a "massively distributed content dispersion network". Imagine: suddenly the www is distributed, not centralized again!
    - lunixbochs 12y ago
      "Active idle" on a drive is still rated at around 5W. Maybe they don't spin down completely in the idle state? BackBlaze says it's better for the longevity of a drive to leave it running than to power it on and off regularly.
      - e12e 12y ago

ris 12y ago

The Archive Team have something called the Archive Team Warrior (http://archiveteam.org/index.php?title=ArchiveTeam_Warrior http://archiveteam.org/index.php?title=ArchiveTeam_Warrior) which seems quite similar to this except it's a virtual rather than physical appliance. Worth a look. (edit: although looking further this is more for distributed crawling, while the results will be uploaded to their archive)

sp332 12y ago
The warriors download items assigned by a central tracker, then upload each item to a holding area. The warriors don't keep items around when they're finished. Also, the current tracker keeps everything in RAM, and can't keep up with more than ~2 million items at once. It would take a major rewrite just to keep track of all the pieces. P.S. Please run a warrior instance. It comes in a small VM image that you can run in the background all the time.

jpalomaki 12y ago

One option to fund this would be to team up with some storage provider like Bittorrent Sync. They would sponsor the device, I would provide hard disk, network and electricity. Sponsor would get 20-30% of my disk space for their purposes and the rest would go for Internet Archive. A bit similar to what Fon was doing. http://en.wikipedia.org/wiki/FON http://en.wikipedia.org/wiki/FON