4 ms·

Files are always represented as lists of blocks or block spans within a file system. Individual blocks could in theory be partially shared between files at the

Files are always represented as lists of blocks or block spans within a file system. Individual blocks could in theory be partially shared between files at the complexity cost of a reference counter for each block. So changing a single byte in a copy on write file could take the same time regardless of file size because only the affected bock would have to be duplicated. I don't know at all how MacOS implements this copynon write scheme, though.

MBCook 2y ago
APFS is a copy on write filesystem if you use the right APIs, so it does what you describe but only for entire files. I believe as soon as you change a single bite you get a complete copy that’s your own. And that’s how this program works. It finds perfect duplicates and then effectively deletes and replaces them with a copy of the existing file so in the background there’s only one copy of the bits on the disk.
- mintplant 2y ago
  I suppose this means that you could find yourself unexpectedly out of disk space in unintuitive ways, if you're only trying to change one byte in a cloned file but there isn't enough space to copy its entire contents?
  - pansa777 2y ago
    It doesn't work like you think. If you change one byte of duplicated file - the only "byte" will be changed on disk (a "byte", because, technically is not a byte, but a block). As far as I understand, it works like a reflink feature in the modern linux FSs. If so, thats really cool, and thats also a bit better than the zfs's snapshots. Iam newbie on macos, but it looks amazing
    - jonhohle 2y ago
      That’s true as long as the writing application only writes to blocks that have changed. Is a VM tool going to write blocks, or write multi-MB segments of a sparse image that can be swapped atomically? Unfortunately, once a file changes there are no APIs to check which blocks are still shared (at least there weren’t as of macOS 13).
  - MBCook 2y ago
    I’m not sure if it works on a file or block level for CoW, but yes. However APFS gives you a number of space related foot-guns if you want. You can overcommit partitions, for example. It also means if you have 30 GB of files on disk that could take up anywhere from a few hundred K to 30 GB of actual data depending on how many dupes you have. It’s a crazy world, but it provides some nice features.
- tonyedgecombe 2y ago
  > I believe as soon as you change a single bite you get a complete copy that’s your own. I think it stores a delta: https://en.m.wikipedia.org/wiki/Apple_File_System#Clones https://en.m.wikipedia.org/wiki/Apple_File_System#Clones
- alwillis 2y ago
  That’s not how this works. Nothing is deleted. It creates zero-space clones of existing files. https://en.wikipedia.org/wiki/Apple_File_System?wprov=sfti1#Clones https://en.wikipedia.org/wiki/Apple_File_System?wprov=sfti1#...

MBCook 2y ago

APFS is a copy on write filesystem if you use the right APIs, so it does what you describe but only for entire files. I believe as soon as you change a single bite you get a complete copy that’s your own. And that’s how this program works. It finds perfect duplicates and then effectively deletes and replaces them with a copy of the existing file so in the background there’s only one copy of the bits on the disk.

mintplant 2y ago
I suppose this means that you could find yourself unexpectedly out of disk space in unintuitive ways, if you're only trying to change one byte in a cloned file but there isn't enough space to copy its entire contents?
- pansa777 2y ago
  It doesn't work like you think. If you change one byte of duplicated file - the only "byte" will be changed on disk (a "byte", because, technically is not a byte, but a block). As far as I understand, it works like a reflink feature in the modern linux FSs. If so, thats really cool, and thats also a bit better than the zfs's snapshots. Iam newbie on macos, but it looks amazing
  - jonhohle 2y ago
    That’s true as long as the writing application only writes to blocks that have changed. Is a VM tool going to write blocks, or write multi-MB segments of a sparse image that can be swapped atomically? Unfortunately, once a file changes there are no APIs to check which blocks are still shared (at least there weren’t as of macOS 13).
- MBCook 2y ago
  I’m not sure if it works on a file or block level for CoW, but yes. However APFS gives you a number of space related foot-guns if you want. You can overcommit partitions, for example. It also means if you have 30 GB of files on disk that could take up anywhere from a few hundred K to 30 GB of actual data depending on how many dupes you have. It’s a crazy world, but it provides some nice features.
tonyedgecombe 2y ago
> I believe as soon as you change a single bite you get a complete copy that’s your own. I think it stores a delta: https://en.m.wikipedia.org/wiki/Apple_File_System#Clones https://en.m.wikipedia.org/wiki/Apple_File_System#Clones
alwillis 2y ago
That’s not how this works. Nothing is deleted. It creates zero-space clones of existing files. https://en.wikipedia.org/wiki/Apple_File_System?wprov=sfti1#Clones https://en.wikipedia.org/wiki/Apple_File_System?wprov=sfti1#...