4 ms·

The difference is that you typically have lots of more tapes than readers. Gmail had an event with data loss in their early days and restoring from tapes took

by tjoff 7y ago

The difference is that you typically have lots of more tapes than readers. Gmail had an event with data loss in their early days and restoring from tapes took (from memory) many weeks, if not months.

adrian_b 7y ago
The post above was about the latency of random access to a file, not about the duration of a restoring operation. The tapes have the fastest sequential transfer speed of any alternative, so the restoring duration should be better than from anything else, if the restoring software plans correctly the sequence of retrievals. It is true that if the backup software is not clever, you could waste a lot of time to determine what you must restore and where is it. However, this has nothing to do with the tapes, the same can happen with backup on HDDs if the backup software is equally bad. I am not sure what you mean by having lots of more tapes than reader, as that is the point of tape storage and the cause of it low cost per terabyte. The only way that I could understand that phrase as a disadvantage for tapes is that when you would need to read as fast as possible e.g. 300 TB with one reader, that would need 10 days, but if the 300 TB were stored on 25 HDDs and you would also have 25 spare computers and some technicians that could open the computers and connect the HDDs inside them and you would also have an 100 Gb/s Ethernet network from those computers to the destination of those 300 TB, then yes, you could read the 300 TB 12 times faster that with a single tape reader. However, such circumstances seem too unbelievable. For such a huge organization it would be more realistic to buy immediately e.g. 3 more readers to reduce the reading time from 10 days to 2.5 days. On the other hand, if the 25 HDDs would have been located from the beginning inside a backup system, then that system would have had a price with at least 5 digits, far higher than the cost of a tape backup system, and unless it would have been connected by 100 Gb/s or at least 25 Gb/s Ethernet, its speed could not have been much higher than that of a single tape drive.
- tjoff 7y ago
  Well, they are tightly coupled. If you only have one tape to consider then fine - whatever works and the discussion would be pointless. Also, why on earth would you need 25 spare computers for 25 drives??? 25 drives fit into a single 4U chassi. A single operation and everything is done. Seeking a file on a harddrive is about 7ms. Seeking 25 drives is also about 7ms. But will be quite the operation for a tape system. You don't just "buy three more readers", you are likely making use of tape robots and fitting them into your infrastructure probably isn't an afternoon of work. And the whole cost benefit of tapes greatly rely on the fact that you don't have to. Which is also why the latency skyrocket. You still want to perform your backups of course, maybe doubly so because of any unforeseen secondary effects that whatever caused the restore process might lead to - which will compete with precious time. I'd love to use tape, but the cost is prohibitive and you really want to verify your backups so for home use it is quite the hassle. In my opinion. But if I stumble on a great deal, sure! It'll be fun.
  - adrian_b 7y ago
    > why on earth would you need 25 spare computers for 25 drives??? 25 drives fit into a single 4U chass I have already mentioned that with other words at the end of my reply and in that case you will not have a speed 25 times higher, but maybe at most 4 times higher than a single HDD, i.e. twice higher than a single tape drive. Maybe I misunderstood, but the initial posting mentioned a restoring operation from tapes that was slow and my point was that using HDDs is unlikely to make it faster. For a restoring operation the duration will be dominated by sequential transfers, not by seeking, so tape drives would be faster than HDDs, unless you use ridiculous resources, like in my example with a computer per HDD and with a network fast enough to aggregate the data from all computers. If we are talking about general-purpose data storage on tapes, then obviously, what is frequently accessed should be on SSDs, because the average seek time for a file stored on tape will be between 1 minute and 2 minutes. I suppose that there might exist applications where there exists a quantity of data so large that it would be too expensive to be stored on SSDs and with worst case latency requirements so severe that it cannot be stored on tapes, so you have to use HDDs. However, in some 40 years of using all kinds of computer systems in many companies, I have never encountered such an application. Everything I have seen could be better handled by a combination of SSDs and tapes, without any HDDs. The only problem with tapes is the prohibitive cost of drives, which limits their use to data sizes of at least 50 to 100 TB, unless long term storage reliability is more important than the cost.

by tjoff 7y ago

The difference is that you typically have lots of more tapes than readers. Gmail had an event with data loss in their early days and restoring from tapes took (from memory) many weeks, if not months.

adrian_b 7y ago

The post above was about the latency of random access to a file, not about the duration of a restoring operation. The tapes have the fastest sequential transfer speed of any alternative, so the restoring duration should be better than from anything else, if the restoring software plans correctly the sequence of retrievals. It is true that if the backup software is not clever, you could waste a lot of time to determine what you must restore and where is it. However, this has nothing to do with the tapes, the same can happen with backup on HDDs if the backup software is equally bad. I am not sure what you mean by having lots of more tapes than reader, as that is the point of tape storage and the cause of it low cost per terabyte. The only way that I could understand that phrase as a disadvantage for tapes is that when you would need to read as fast as possible e.g. 300 TB with one reader, that would need 10 days, but if the 300 TB were stored on 25 HDDs and you would also have 25 spare computers and some technicians that could open the computers and connect the HDDs inside them and you would also have an 100 Gb/s Ethernet network from those computers to the destination of those 300 TB, then yes, you could read the 300 TB 12 times faster that with a single tape reader. However, such circumstances seem too unbelievable. For such a huge organization it would be more realistic to buy immediately e.g. 3 more readers to reduce the reading time from 10 days to 2.5 days. On the other hand, if the 25 HDDs would have been located from the beginning inside a backup system, then that system would have had a price with at least 5 digits, far higher than the cost of a tape backup system, and unless it would have been connected by 100 Gb/s or at least 25 Gb/s Ethernet, its speed could not have been much higher than that of a single tape drive.

tjoff 7y ago
Well, they are tightly coupled. If you only have one tape to consider then fine - whatever works and the discussion would be pointless. Also, why on earth would you need 25 spare computers for 25 drives??? 25 drives fit into a single 4U chassi. A single operation and everything is done. Seeking a file on a harddrive is about 7ms. Seeking 25 drives is also about 7ms. But will be quite the operation for a tape system. You don't just "buy three more readers", you are likely making use of tape robots and fitting them into your infrastructure probably isn't an afternoon of work. And the whole cost benefit of tapes greatly rely on the fact that you don't have to. Which is also why the latency skyrocket. You still want to perform your backups of course, maybe doubly so because of any unforeseen secondary effects that whatever caused the restore process might lead to - which will compete with precious time. I'd love to use tape, but the cost is prohibitive and you really want to verify your backups so for home use it is quite the hassle. In my opinion. But if I stumble on a great deal, sure! It'll be fun.
- adrian_b 7y ago
  > why on earth would you need 25 spare computers for 25 drives??? 25 drives fit into a single 4U chass I have already mentioned that with other words at the end of my reply and in that case you will not have a speed 25 times higher, but maybe at most 4 times higher than a single HDD, i.e. twice higher than a single tape drive. Maybe I misunderstood, but the initial posting mentioned a restoring operation from tapes that was slow and my point was that using HDDs is unlikely to make it faster. For a restoring operation the duration will be dominated by sequential transfers, not by seeking, so tape drives would be faster than HDDs, unless you use ridiculous resources, like in my example with a computer per HDD and with a network fast enough to aggregate the data from all computers. If we are talking about general-purpose data storage on tapes, then obviously, what is frequently accessed should be on SSDs, because the average seek time for a file stored on tape will be between 1 minute and 2 minutes. I suppose that there might exist applications where there exists a quantity of data so large that it would be too expensive to be stored on SSDs and with worst case latency requirements so severe that it cannot be stored on tapes, so you have to use HDDs. However, in some 40 years of using all kinds of computer systems in many companies, I have never encountered such an application. Everything I have seen could be better handled by a combination of SSDs and tapes, without any HDDs. The only problem with tapes is the prohibitive cost of drives, which limits their use to data sizes of at least 50 to 100 TB, unless long term storage reliability is more important than the cost.