5 ms·

If you open the properties of a folder, you can see how long it can take to calculate the size of a folder. This is a feature I've never really needed - what i

If you open the properties of a folder, you can see how long it can take to calculate the size of a folder. This is a feature I've never really needed - what is your scenario for needing this at a glance? When I'm chasing large files that need deleting due to free space pressure I typically use something like the following https://windirstat.net/ https://windirstat.net/

Causality1 7y ago
I don't see why the size of the folder couldn't be written to the folder's metadata every time the folder contents are modified.
- ken 7y ago
  It certainly can. Apple does this in APFS. They call it "fast directory sizing": https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sierra-the-ars-technica-review/6/ https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sie...
  - kilburn 7y ago
    Apple said they would be doing this. For some reason it doesn't seem they actually did it [1]. [1] https://eclecticlight.co/2019/02/06/how-big-is-that-folder-what-happened-to-apfs-fast-directory-sizing/ https://eclecticlight.co/2019/02/06/how-big-is-that-folder-w...
- deleted 7y ago
  [deleted]
- ComputerGuru 7y ago
  Because it's recursive. It means a file nested a few hundred folders down needs to update each and every parent. And what about hard linked files? Directory junction points? Drives mounted as folders? It would be insane to think the performance would be acceptable.
  - hexomancer 7y ago
    First of all, most files are nowhere near 10 levels deep, let alone hundreds of levels. Second, we don't have to do a naive update every single time a file is changed. We can amortize the cost by updating the parent folder sizes only when the file size is changed by a significant amount since last update (say, 10%). And we can do this process recursively is the parent directories. This was it only takes O(1) time to update folder metadata for each file operation and all the metadata are accurate within 10%. Also I don't see how drives and hard linked files are any different than regular files in this context.
    - kilburn 7y ago
      > most files are nowhere near 10 levels deep, let alone hundreds of levels. Filesystem (and low-level in general) stuff must consider worst cases. There is a lot of software out there doing weird things. For instance, npm created a very deep folder hierarchy for a long time (so deep it messed up some path length restrictions in fact). One way or another the worst case is going to be hit. And then what? The entire computer grinds to a halt? How is the user supposed to discover why? > We can amortize the cost by updating the parent folder sizes only when the file size is changed by a significant amount since last update (say, 10%). So you have a log file inside a folder. Because it just keeps growing line by line then the folder's size is never updated. Now you have many Gb's of log file in that folder, and the folder says it is using "4Kbytes". Moreover, this propagates upwards the filesystem. In the end, your root drive has a "folder size" of X but its actual usage is Y >> X. How is that not going to confuse everyone? Put in another way, who is going to trust the X number ever? Why would you pay all that accounting penalty for every write to every file to end up with a half-asset broken-by-design feature? > Also I don't see how drives and hard linked files are any different than regular files in this context. They are different in that they exist in multiple folders at the same time (so they would trigger multiple size-updating branches).
      - hexomancer 7y ago
        > So you have a log file inside a folder. Because it just keeps growing line by line then the folder's size is never updated. Now you have many Gb's of log file in that folder, and the folder says it is using "4Kbytes". You misunderstood what I meant. I didn't mean we should only update if a single change is significant. I meant we only update when the cumulative changes since last update is significant. An example: Let's say we create a 1mb file. The next time the file is changed, we only update the parent if the change is more than 10% (the new size is greater than 1.1mb or less than 0.9mb). For example if it is a log file and each line is 100 bytes, we update the parent after 1000 new lines (even though the 1000th line is still only 100 bytes). (of course the number of lines before the update depends on the size of the file, so if we had a 1GB log file, we would update after 1000000 new lines). It is trivial to prove that the estimate is never off by a factor of more than 10% (even for the parent folders). So this is not a "half-assed broken-by-design feature" since it provides strong guarantees in bounds and at least in my personal day-to-day usage, I almost never care about the exact size of a folder but want to have a rough idea of how large it is.
- jmgao 7y ago
  Because if you do that, you have to walk up the entire directory tree on every file operation.
userbinator 7y ago
you can see how long it can take to calculate the size of a folder On older Windows (95, 98, XP) I've never had it take an unacceptably long time, and that was with a regular HDD. With an SSD and the large file caches which are possible with today's machines with lots of RAM, it should be even faster. Besides, it's not as if the operation needs to be synchronous; the sizes can be calculated and displayed when they're ready. Having an option to show them would be useful, and those who don't need it/don't like the extra disk activity could leave it off.
- Bjartr 7y ago
  Older versions of Windows used the FAT filesystem rather than the current NTFS filesystem. FAT happened to get you directory size "for free" due to how it works under the hood, whereas NTFS does not (but it's faster at other things as a result). For much more detail check out this[1] post on The Old New Thing blog by Raymond Chen. [1] http://blogs.msdn.com/b/oldnewthing/archive/2011/12/26/10251026.aspx http://blogs.msdn.com/b/oldnewthing/archive/2011/12/26/10251...
- Sharlin 7y ago
  It is absolutely hideously slow on NTFS to compute the size of a directory that (recursively or not) contains thousands of files. Indeed, it is absolutely hideously slow on NTFS to do just about anything that needs to touch thousands of files at a time.
kalleboo 7y ago
The Mac has been able to do this since what, System 7.5 in 1995? Yeah it can take a while to calculate but on a modern SSD it's not a problem. If you don't like the performance hit it's an option that defaults to off.
adzm 7y ago
Note that if you parse the MFT itself you can calculate this info for the entire drive in the time explorer can calculate it for a single folder. I know there is some overhead with the official api of course and it needs to support other filesystems but we really needed a way to iterate directories faster just to gather this kind of info. There are several disk space utilization / treesize apps and file search apps that take advantage of the speed you get by bypassing the API.
- Tempest1981 7y ago
  The Wiztree app is a great example of the benefit -- 3 seconds to view the entire drive. Although reading the MFT requires elevation. https://antibody-software.com/web/software/software/wiztree-finds-the-files-and-folders-using-the-most-disk-space-on-your-hard-drive/ https://antibody-software.com/web/software/software/wiztree-...
mqus 7y ago
Since Windows 98, we used Idoswin (https://www.idoswin.de/ https://www.idoswin.de/) which always calculated directory sizes in the background. It showed the sizes of directories it already calculated and made the field empty for every folder where the size was not ready yet. Usually the small directories are ready first and only the big (>1gb and many files) take a few seconds. This is done in the background and therefore doesn't disrupt your workflow at all. Another method would be to make it midnight-commander style: add a button which will calculate & display the sizes of the directories. So the time to calculate it is not really an issue. Use-cases are exactly cleaning up space and roughly calculating backup sizes/sizes of folders I want to copy at a single glance. You could also ask: why do we need to display the size of files in the explorer. The reasons are roughly the same.
NikkiA 7y ago
I use xyplorer, which has a toggle feature for always displaying the folder sizes, and a keyboard shortcut (shift-F5) to display them for the current folder until changed. This combination seems to work just fine, I don't usually need folder sizes, so I keep the toggle off, and just hit shift-F5 if I really need to see them.

Causality1 7y ago

I don't see why the size of the folder couldn't be written to the folder's metadata every time the folder contents are modified.

ken 7y ago
It certainly can. Apple does this in APFS. They call it "fast directory sizing": https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sierra-the-ars-technica-review/6/ https://arstechnica.com/gadgets/2017/09/macos-10-13-high-sie...
- kilburn 7y ago
  Apple said they would be doing this. For some reason it doesn't seem they actually did it [1]. [1] https://eclecticlight.co/2019/02/06/how-big-is-that-folder-what-happened-to-apfs-fast-directory-sizing/ https://eclecticlight.co/2019/02/06/how-big-is-that-folder-w...
deleted 7y ago
[deleted]
ComputerGuru 7y ago
Because it's recursive. It means a file nested a few hundred folders down needs to update each and every parent. And what about hard linked files? Directory junction points? Drives mounted as folders? It would be insane to think the performance would be acceptable.
- hexomancer 7y ago
  First of all, most files are nowhere near 10 levels deep, let alone hundreds of levels. Second, we don't have to do a naive update every single time a file is changed. We can amortize the cost by updating the parent folder sizes only when the file size is changed by a significant amount since last update (say, 10%). And we can do this process recursively is the parent directories. This was it only takes O(1) time to update folder metadata for each file operation and all the metadata are accurate within 10%. Also I don't see how drives and hard linked files are any different than regular files in this context.
  - kilburn 7y ago
    > most files are nowhere near 10 levels deep, let alone hundreds of levels. Filesystem (and low-level in general) stuff must consider worst cases. There is a lot of software out there doing weird things. For instance, npm created a very deep folder hierarchy for a long time (so deep it messed up some path length restrictions in fact). One way or another the worst case is going to be hit. And then what? The entire computer grinds to a halt? How is the user supposed to discover why? > We can amortize the cost by updating the parent folder sizes only when the file size is changed by a significant amount since last update (say, 10%). So you have a log file inside a folder. Because it just keeps growing line by line then the folder's size is never updated. Now you have many Gb's of log file in that folder, and the folder says it is using "4Kbytes". Moreover, this propagates upwards the filesystem. In the end, your root drive has a "folder size" of X but its actual usage is Y >> X. How is that not going to confuse everyone? Put in another way, who is going to trust the X number ever? Why would you pay all that accounting penalty for every write to every file to end up with a half-asset broken-by-design feature? > Also I don't see how drives and hard linked files are any different than regular files in this context. They are different in that they exist in multiple folders at the same time (so they would trigger multiple size-updating branches).
    - hexomancer 7y ago
      > So you have a log file inside a folder. Because it just keeps growing line by line then the folder's size is never updated. Now you have many Gb's of log file in that folder, and the folder says it is using "4Kbytes". You misunderstood what I meant. I didn't mean we should only update if a single change is significant. I meant we only update when the cumulative changes since last update is significant. An example: Let's say we create a 1mb file. The next time the file is changed, we only update the parent if the change is more than 10% (the new size is greater than 1.1mb or less than 0.9mb). For example if it is a log file and each line is 100 bytes, we update the parent after 1000 new lines (even though the 1000th line is still only 100 bytes). (of course the number of lines before the update depends on the size of the file, so if we had a 1GB log file, we would update after 1000000 new lines). It is trivial to prove that the estimate is never off by a factor of more than 10% (even for the parent folders). So this is not a "half-assed broken-by-design feature" since it provides strong guarantees in bounds and at least in my personal day-to-day usage, I almost never care about the exact size of a folder but want to have a rough idea of how large it is.
jmgao 7y ago
Because if you do that, you have to walk up the entire directory tree on every file operation.

userbinator 7y ago

you can see how long it can take to calculate the size of a folder On older Windows (95, 98, XP) I've never had it take an unacceptably long time, and that was with a regular HDD. With an SSD and the large file caches which are possible with today's machines with lots of RAM, it should be even faster. Besides, it's not as if the operation needs to be synchronous; the sizes can be calculated and displayed when they're ready. Having an option to show them would be useful, and those who don't need it/don't like the extra disk activity could leave it off.

Bjartr 7y ago
Older versions of Windows used the FAT filesystem rather than the current NTFS filesystem. FAT happened to get you directory size "for free" due to how it works under the hood, whereas NTFS does not (but it's faster at other things as a result). For much more detail check out this[1] post on The Old New Thing blog by Raymond Chen. [1] http://blogs.msdn.com/b/oldnewthing/archive/2011/12/26/10251026.aspx http://blogs.msdn.com/b/oldnewthing/archive/2011/12/26/10251...
Sharlin 7y ago
It is absolutely hideously slow on NTFS to compute the size of a directory that (recursively or not) contains thousands of files. Indeed, it is absolutely hideously slow on NTFS to do just about anything that needs to touch thousands of files at a time.

kalleboo 7y ago

The Mac has been able to do this since what, System 7.5 in 1995? Yeah it can take a while to calculate but on a modern SSD it's not a problem. If you don't like the performance hit it's an option that defaults to off.

adzm 7y ago

Note that if you parse the MFT itself you can calculate this info for the entire drive in the time explorer can calculate it for a single folder. I know there is some overhead with the official api of course and it needs to support other filesystems but we really needed a way to iterate directories faster just to gather this kind of info. There are several disk space utilization / treesize apps and file search apps that take advantage of the speed you get by bypassing the API.

Tempest1981 7y ago
The Wiztree app is a great example of the benefit -- 3 seconds to view the entire drive. Although reading the MFT requires elevation. https://antibody-software.com/web/software/software/wiztree-finds-the-files-and-folders-using-the-most-disk-space-on-your-hard-drive/ https://antibody-software.com/web/software/software/wiztree-...

mqus 7y ago

Since Windows 98, we used Idoswin (https://www.idoswin.de/ https://www.idoswin.de/) which always calculated directory sizes in the background. It showed the sizes of directories it already calculated and made the field empty for every folder where the size was not ready yet. Usually the small directories are ready first and only the big (>1gb and many files) take a few seconds. This is done in the background and therefore doesn't disrupt your workflow at all. Another method would be to make it midnight-commander style: add a button which will calculate & display the sizes of the directories. So the time to calculate it is not really an issue. Use-cases are exactly cleaning up space and roughly calculating backup sizes/sizes of folders I want to copy at a single glance. You could also ask: why do we need to display the size of files in the explorer. The reasons are roughly the same.

NikkiA 7y ago

I use xyplorer, which has a toggle feature for always displaying the folder sizes, and a keyboard shortcut (shift-F5) to display them for the current folder until changed. This combination seems to work just fine, I don't usually need folder sizes, so I keep the toggle off, and just hit shift-F5 if I really need to see them.