3 ms·
> With a billion empty files, the image uses 276 GiB disk space. I wonder why one needs to use as much as 296 bytes per empty file.
by tromp 3y ago
> With a billion empty files, the image uses 276 GiB disk space.
I wonder why one needs to use as much as 296 bytes per empty file.
- berkes 3y agoI would suspect it's a combination of the file-pointers, the file-names and other metadata, and the directories. Directories with lots of files grow very big too. Indexes cost space.
- jasode 3y ago>I wonder why one needs to use as much as 296 bytes per empty file. File systems like ext4/NTFS/APFS/etc require extra disk space for "housekeeping" of data blocks and allocations and metadata (lastwritetime, permission flags, etc) . This reserved space for each file has a minimum fixed amount even if the file's size is zero. E.g. ">By default, ext4 inode records are 256 bytes," -- from https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#:~:text=By%20default%2C%20ext4%20inode%20records%20are%20256%20bytes%2C https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#:~:t... (Scroll up on that page to see all the fields of metadata in each inode record.)
- mnw21cam 3y agoAnd you can override that default by giving the -I <size> option to mkfs.ext4, so you could make a filesystem with 128-byte inodes, and it would require about half the amount of disc space.
- cesarb 3y ago> mkfs.ext4, so you could make a filesystem with 128-byte inodes At the cost of introducing a Y2038 issue. The timestamp fields in the first 128 bytes of an ext2/ext3/ext4 inode are only 32 bits each, they are extended with an extra 32 bits each in the next 128 bytes of the inode.
- deleted 3y ago[deleted]
- kyledrake 3y agoIf you want/need to reduce the metadata, there's some nice solutions for that https://github.com/seaweedfs/seaweedfs https://github.com/seaweedfs/seaweedfs Seaweed uses 40 bytes of metadata per file.