6 ms·
> “In a typical small business/home NAS environment, workloads tend to be bursty in nature, leaving sufficient idle time for garbage collection and other mainte
by justinclift 7y ago
> “In a typical small business/home NAS environment, workloads tend to be bursty in nature, leaving sufficient idle time for garbage collection and other maintenance operations.”
Only up until the array needs to be rebuilt (eg replacing a drive). At which point the workload is the literal opposite of "bursty with idle time". :(
- reitzensteinm 7y agoDoes SMR impose a penalty on reading? My understanding is that it removes the ability to do random writes, since you'll disturb the nearby tracks. Like drawing pen thickness rings with a sharpie by overlapping them. Instead you read data in, modify it, then write it out, like an extreme version of updating a byte in a cache line. Your ability to read is more precise, so you have no trouble reading data laid out in this way. Large contiguous writes are also pretty quick. I've owned one, and if you stick to large files it's fine. Pathological workloads like creating millions of small files can take days.
- klodolph 7y agoSMR doesn’t impose a penalty on reading, but DM-SMR must impose a penalty, by its nature.
- londons_explore 7y agoI'm not sure if drives do this, but it's theoretically possible to linearize a random write workload, with the disadvantage that when you come to read the data, a sequential read becomes a random read. Typically stuff you write randomly (small files, databases) one tends to read randomly, so it's usually worth linearizing writes. The only case where you are at a performance disadvantage is when you did random writes followed by more than one linear read. Things like LogFS do this, as do most SSD's. I'm pretty surprised modern HD's don't do it internally too.
- mikepurvis 7y ago> theoretically possible to linearize a random write workload Are there filesystems or databases which are specially designed to optimize for this constraint? It would basically boil down to structuring your whole system as a set of interlinked mostly-append journals.
- barrkel 7y agoYes, they're called log-structured file systems. Been around for decades, lots of research.
- londons_explore 7y agoBigtable is exactly this. An entire database is saved in a small number of files which are are append-only (and therefore can be written fully linearly). Reads of a single record usually only read a single disk block. In-memory indexes to locate the block are fairly small, even for 1B+ records. Reads of a range of records do N range reads of blocks, where N is typically < 5, and the number of bytes read compared to the number of bytes returned to the user is typically <10% overhead.
- deleted 7y ago[deleted]
- pwg 7y agoYes, see this at least as one reference: https://en.wikipedia.org/wiki/Log_structured_filesystem https://en.wikipedia.org/wiki/Log_structured_filesystem
- jandrese 7y ago> it's theoretically possible to linearize a random write workload In fact OSes have been doing this for years now. Over a decade. There is a drawback however. To make this work you need to buffer your writes in a cache, typically in volatile memory, so they can be sorted before being written to the drive. The larger the cache the more improvement you see (although there are diminishing returns on this), but also the more data you lose in the event of power loss or kernel crash. So it is a tradeoff between speed and resilience. Interesting thought about how this changes for SSDs. While you don't have seek times to consider anymore, if you have two writes that happen to fall on the same block in the SSD it is a big win to combine them, so SSDs still want to buffer data and have to make the speed/resilience tradeoff.
- londons_explore 7y agoYou're talking about something else here. Linearized writes means write the data immediately, but rather than write the data to the 'correct' location on disk, just write the data immediately after the last bit of data you wrote. Then keep some accounting information elsewhere to map the real location of data to the place it would have been written had linearization not been done.
- jandrese 7y agoThat sounds awful from a fragmentation standpoint, but I guess it would be ok for write-mostly data. That metadata map would get pretty gnarly if you have a lot of interleaved writes though.
- vetinari 7y agoRebuilding is not random i/o though, it goes sequentially and that's what SMRs are good at. Still, I did recently reshape from 4x6TB raid5 to 5x6TB raid6. The new drive was SMR WD Red, it took 12 days and I was quite pissed when I found out the SMR bit. It wasn't in the datasheet, if I knew upfront, I would never bought it.
- syshum 7y ago>> I knew upfront, I would never bought it. SMR has a terrible rep for a reason, and no one with any knowlege of what SMR is would buy which is why they have to be unethical by removing this spec from the spec sheets SMR is cheaper for them to manufacture, and is TERRIBLE technology that really has very little actual use case, except Archive storage. The manufacturers want to push SMR for more than that because it is more profitable for them
- zozbot234 7y ago> SMR is cheaper for them to manufacture, and is TERRIBLE technology It's not that terrible if the customer knows what they're getting. It's just that SMR competes with tape more so than with conventional (CMR) hard drives. Many workloads are good enough for SMR.
- syshum 7y ago>>Many workloads are good enough for SMR. One workload is, which is what you said.. Archiving, what one would normally do to Tape. Everything else is is substandard and should not be used
- chmod775 7y agoBut what if you need random reads? SMR sounds like it's great (value) at providing random read access to large amounts of data if you only have sequential writes or infrequent writes.
- 7y ago