6 ms·

Yev from Backblaze here -> Yes, unless the storage pod is in maintenance, it remains on so that backup and B2 data can be uploaded and accessed.

by atYevP 9y ago

amelius 9y ago
Interesting. Have you ever computed how much power you would save if you turned them off when not needed? Or is every pod constantly in use? Would it help if smart algorithms repartition the data on storage pods, to optimize power use?
- NikolaeVarius 9y ago
  Why wouldn't that be handled already by the drives themselves?
  - amelius 9y ago
    Perhaps partly. But a drive can't repartition data according to frequency of use.
- x4d66a5ce 9y ago
  I wonder about this, too. Since access latency is the primary concern, a neat solution would be to create some dynamic "fastup" storage scheme. For large files, loosen the always-on requirement; to fulfill a request the first N bytes are served from the always-on pod, and the tail is stored and served from pods that dynamically power up drives only at access time. The size of N is computed such that the IO time it takes to make it through the fragment stored on the always-on pod is > the power-on time for the pods where the tail is stored.
  - amelius 9y ago
    I don't think latency is an issue for a backup solution.
    - x4d66a5ce 9y ago
      Except currently it is a concern: > it remains on so that backup and B2 data can be uploaded and accessed This is also why Amazon has tiered pricing between S3 and Glacier.
- brianwski 9y ago
  Brian from Backblaze here. > Have you ever computed how much power you would save if you turned them off when not needed? Electrical power is a gigantic part of our datacenter bill (like 60%). However, we cannot really ever turn off the hard drives. If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back. > is every pod constantly in use? The way we built it, yes. If one user deletes a backup or unsubscribes from the Backblaze service, it frees a little bit of space on pods all over the datacenter. So we allow some other customer to use that space up as soon as we can. We also use ALL the pods in the datacenter to give us "breadth" to accept data faster. This is particularly helpful on surges of new customers due to a press event, or if a competitor like CrashPlan decides to exit the market.
  - amelius 9y ago
    > If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back. Well, speaking for myself, I don't mind if I have to restore a backup, and have to wait 1 minute extra for a system to come online (that's about the typical boot time of a Linux system, so I suppose for a storage pod it would be about the same). By the way, it seems possible to power down an individual SATA drive if it is not in use: [1]. [1] https://unix.stackexchange.com/questions/112117/shutdown-my-backup-hard-disk-on-linux-when-i-dont-use-it https://unix.stackexchange.com/questions/112117/shutdown-my-...
    - infogulch 9y ago
      This is what I thought amelius' comment was originally about. You could save power on drives plus it would only take a few seconds to come back online. I imagine it could save a lot of electricity if the additional complication is worth it. Complications like a write cache drive in case of power failure, or grouping expected-cold data together. Also I wonder if powering the drive up and down occasionally will have a positive or negative impact on drive reliability compared to having it on 100% of the time.
    - sml156 9y ago

Yev from Backblaze here -> Yes, unless the storage pod is in maintenance, it remains on so that backup and B2 data can be uploaded and accessed.

by atYevP 9y ago

Yev from Backblaze here -> Yes, unless the storage pod is in maintenance, it remains on so that backup and B2 data can be uploaded and accessed.

amelius 9y ago

Interesting. Have you ever computed how much power you would save if you turned them off when not needed? Or is every pod constantly in use? Would it help if smart algorithms repartition the data on storage pods, to optimize power use?

NikolaeVarius 9y ago
Why wouldn't that be handled already by the drives themselves?
- amelius 9y ago
  Perhaps partly. But a drive can't repartition data according to frequency of use.
x4d66a5ce 9y ago
I wonder about this, too. Since access latency is the primary concern, a neat solution would be to create some dynamic "fastup" storage scheme. For large files, loosen the always-on requirement; to fulfill a request the first N bytes are served from the always-on pod, and the tail is stored and served from pods that dynamically power up drives only at access time. The size of N is computed such that the IO time it takes to make it through the fragment stored on the always-on pod is > the power-on time for the pods where the tail is stored.
- amelius 9y ago
  I don't think latency is an issue for a backup solution.
  - x4d66a5ce 9y ago
    Except currently it is a concern: > it remains on so that backup and B2 data can be uploaded and accessed This is also why Amazon has tiered pricing between S3 and Glacier.
brianwski 9y ago
Brian from Backblaze here. > Have you ever computed how much power you would save if you turned them off when not needed? Electrical power is a gigantic part of our datacenter bill (like 60%). However, we cannot really ever turn off the hard drives. If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back. > is every pod constantly in use? The way we built it, yes. If one user deletes a backup or unsubscribes from the Backblaze service, it frees a little bit of space on pods all over the datacenter. So we allow some other customer to use that space up as soon as we can. We also use ALL the pods in the datacenter to give us "breadth" to accept data faster. This is particularly helpful on surges of new customers due to a press event, or if a competitor like CrashPlan decides to exit the market.
- amelius 9y ago
  > If they were powered down, it would take too long to retrieve data. It would be like Amazon Glacier where it can take 15 minutes to get a single image back. Well, speaking for myself, I don't mind if I have to restore a backup, and have to wait 1 minute extra for a system to come online (that's about the typical boot time of a Linux system, so I suppose for a storage pod it would be about the same). By the way, it seems possible to power down an individual SATA drive if it is not in use: [1]. [1] https://unix.stackexchange.com/questions/112117/shutdown-my-backup-hard-disk-on-linux-when-i-dont-use-it https://unix.stackexchange.com/questions/112117/shutdown-my-...
  - infogulch 9y ago
    This is what I thought amelius' comment was originally about. You could save power on drives plus it would only take a few seconds to come back online. I imagine it could save a lot of electricity if the additional complication is worth it. Complications like a write cache drive in case of power failure, or grouping expected-cold data together. Also I wonder if powering the drive up and down occasionally will have a positive or negative impact on drive reliability compared to having it on 100% of the time.
  - sml156 9y ago