4 ms·
A Disk Usage Mystery
- NoZebra120vClip 3y agoA mystery is made extra-mysterious by the nonstandard abbreviations in the article. What's a "Tb"? It's not bits. Does he mean terabyte? tebibyte? Same for "Gb". It's mostly about disk space, so it's powers of 10 right? Except it's memory-mapped, and memory's in powers of 2. Here you go: https://en.wikipedia.org/wiki/Byte#Multiple-byte_units https://en.wikipedia.org/wiki/Byte#Multiple-byte_units
- two_handfuls 3y agoYeah that’s sloppy.
- Dylan16807 3y agoIf you're confident that it's not bits, then it must be bytes. So that's not a mystery. Not that it would affect the meaning of the article anyway. So that just leaves the nitpick about powers of ten and two. Why do you think it matters? The numbers are rounded anyway, so... it's both. Enjoy.
- NoZebra120vClip 3y agoPerhaps it works for e.e. cummings, but if the author professes to be a SWE, and can't/won't express standard units in standard format, especially in a rendering that actually means something else, author looks lazy and ignorant. Do you want hallucinating AIs? Because this is how we get hallucinating AIs.
- deleted 3y ago[deleted]
- Dylan16807 3y agoThis is way too strong of a complaint for just capitalization. And except for the capitalization, everything is a standard unit and nothing is incorrect. That wiki article is irrelevant. You may be unsure if they were intending base 10 or base 2, but that doesn't affect the correctness. It's both, because the numbers are very approximate.
- NoZebra120vClip 3y agoYou keep insisting that the numbers are approximate, yet there's one very real and concrete number included in this article: the 2GiB limit for Java memory-mapped files. https://bugs.openjdk.org/browse/JDK-6347833 https://bugs.openjdk.org/browse/JDK-6347833 There is/was a very real hard limit on memory-mapping files, because the index was held in a 32-bit pointer with "Integer.MAX_VALUE". There is no rounding or approximation there: it's 2^32-1 bytes, with no ifs, ands, or buts. Two gibibytes is 2,147,483,648 bytes, while two gigabytes is 2,000,000,000 bytes. That's an approximation of ±147,483,648! Wow!
- Dylan16807 3y agoAnd the total mapping is 200x as big. It doesn't matter.
- RetroTechie 3y agoMystery is solved: "In most cases, removing all filenames associated with an inode will reduce the reference counter to zero, but it turns out you can also have a file referred to by zero file names, but with a non-zero reference counter, which is what was happening here. The file had a non-zero reference count because it was memory mapped, even though no filename pointed to it. The reference counter doesn’t decrement until you un-map the memory." At 1st glance this may appear weird, but not when thinking about it. It means diskspace that isn't referred to by a filename, but which is allocated / used in some way ("reserved" comes to mind). On the other hand, it does seem weird one could create memory mapped diskspace without somehow assigning a filename to that? Or delete memory mapped file, but have the memory mapping + corresponding diskspace remain? More logical -imho- would be delete mm file -> corresponding RAM either freed, or non-file mapped. Appearently Java version used (specifically: its GC) isn't good at handling / cleaning up in situations like this.
- marginalia_nu 3y agoYeah I'd call the behavior unexpected rather than wrong. As a design decision it makes a lot of sense, especially given the ideas that went into UNIX filesystems that didn't quite pan out (hard links are a very neat idea in theory, but I'm not sure when I'd ever actually use them).
- somat 3y agoHard links are tags. A file is just a big lump of data with an address(the inode) A name is then linked to the data(If you wanted you could call it a tag, it is almost exactly the same thing). If the data needs more metadata go ahead and link additional names to it. Note that no name is the "real name" with hard links they are all equal. I built a whole music database once where all the metadata and playlists were hardlinks... The main problem encountered was you had to be very careful archiving the structure, it tended to explode into a million duplicate files(this is probably why people don't use hardlinks)
- Szpadel 3y agoit's actually fairly near way to handling doing temporary disk space, you create file, open it and delete while still having open handle to it. in that way you can still use it but without having files in the filesystem. in that way you have less corner cases to carry to cleanup ( like crashes, power interruptions, etc ) mpv actually can utilize this mechanism to cache played media from network locally