3 ms·

I don't think the 8KB calculation is quite right - there are allocations in the kernel for every thread as well, and I don't think that'll be accounted for by /

I don't think the 8KB calculation is quite right - there are allocations in the kernel for every thread as well, and I don't think that'll be accounted for by /use/bin/time.

anarazel 3y ago
Indeed. Making the program wait (using a barrier) after all the threads have started, I see a a substantially higher memory usage than reported by time. I only used 100k threads. For me time reports 1018376kB, but looking at /proc/meminfo I see a difference of 5231092kB - so about 52kB per thread. That's a far cry from 8MB, but also not 8kB. /proc/slabinfo shows some drastic differences: slabtop -o|head -n 15 with 100k threads running: Active / Total Objects (% used) : 10134567 / 10280497 (98.6%) Active / Total Slabs (% used) : 337756 / 337756 (100.0%) Active / Total Caches (% used) : 132 / 175 (75.4%) Active / Total Size (% used) : 3598333.95K / 3674904.09K (97.9%) Minimum / Average / Maximum Object : 0.01K / 0.36K / 11.81K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 3419013 3418579 99% 0.10K 87667 39 350668K buffer_head 1485372 1484947 99% 0.19K 35366 42 282928K dentry 635299 635038 99% 0.68K 13517 47 432544K proc_inode_cache 575792 575708 99% 1.14K 20564 28 658048K ext4_inode_cache 514794 514703 99% 0.04K 5047 102 20188K extent_status 503140 502198 99% 0.18K 11435 44 91480K vm_area_struct 501738 500514 99% 0.04K 4919 102 19676K vma_lock 381304 378068 99% 0.57K 13618 28 217888K radix_tree_node without: Active / Total Objects (% used) : 7235070 / 7678039 (94.2%) Active / Total Slabs (% used) : 187813 / 187813 (100.0%) Active / Total Caches (% used) : 132 / 175 (75.4%) Active / Total Size (% used) : 1751469.45K / 1851083.31K (94.6%) Minimum / Average / Maximum Object : 0.01K / 0.24K / 11.81K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 3419013 3418579 99% 0.10K 87667 39 350668K buffer_head 898296 887716 98% 0.19K 21388 42 171104K dentry 575792 575708 99% 1.14K 20564 28 658048K ext4_inode_cache 514794 514703 99% 0.04K 5047 102 20188K extent_status 381304 376987 98% 0.57K 13618 28 217888K radix_tree_node 164352 164352 100% 0.06K 2568 64 10272K kmalloc-rcl-64 157360 53925 34% 0.07K 2810 56 11240K vmap_area 139638 109093 78% 0.04K 1369 102 5476K vma_lock
gpderetta 3y ago
Incidentally the minimum per-thread kernel stack (which is distinct from the user stack) size used to be 8Kb. Not sure if it has changed.

I don't think the 8KB calculation is quite right - there are allocations in the kernel for every thread as well, and I don't think that'll be accounted for by /use/bin/time.

anarazel 3y ago

Indeed. Making the program wait (using a barrier) after all the threads have started, I see a a substantially higher memory usage than reported by time. I only used 100k threads. For me time reports 1018376kB, but looking at /proc/meminfo I see a difference of 5231092kB - so about 52kB per thread. That's a far cry from 8MB, but also not 8kB. /proc/slabinfo shows some drastic differences: slabtop -o|head -n 15 with 100k threads running: Active / Total Objects (% used) : 10134567 / 10280497 (98.6%) Active / Total Slabs (% used) : 337756 / 337756 (100.0%) Active / Total Caches (% used) : 132 / 175 (75.4%) Active / Total Size (% used) : 3598333.95K / 3674904.09K (97.9%) Minimum / Average / Maximum Object : 0.01K / 0.36K / 11.81K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 3419013 3418579 99% 0.10K 87667 39 350668K buffer_head 1485372 1484947 99% 0.19K 35366 42 282928K dentry 635299 635038 99% 0.68K 13517 47 432544K proc_inode_cache 575792 575708 99% 1.14K 20564 28 658048K ext4_inode_cache 514794 514703 99% 0.04K 5047 102 20188K extent_status 503140 502198 99% 0.18K 11435 44 91480K vm_area_struct 501738 500514 99% 0.04K 4919 102 19676K vma_lock 381304 378068 99% 0.57K 13618 28 217888K radix_tree_node without: Active / Total Objects (% used) : 7235070 / 7678039 (94.2%) Active / Total Slabs (% used) : 187813 / 187813 (100.0%) Active / Total Caches (% used) : 132 / 175 (75.4%) Active / Total Size (% used) : 1751469.45K / 1851083.31K (94.6%) Minimum / Average / Maximum Object : 0.01K / 0.24K / 11.81K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 3419013 3418579 99% 0.10K 87667 39 350668K buffer_head 898296 887716 98% 0.19K 21388 42 171104K dentry 575792 575708 99% 1.14K 20564 28 658048K ext4_inode_cache 514794 514703 99% 0.04K 5047 102 20188K extent_status 381304 376987 98% 0.57K 13618 28 217888K radix_tree_node 164352 164352 100% 0.06K 2568 64 10272K kmalloc-rcl-64 157360 53925 34% 0.07K 2810 56 11240K vmap_area 139638 109093 78% 0.04K 1369 102 5476K vma_lock

gpderetta 3y ago

Incidentally the minimum per-thread kernel stack (which is distinct from the user stack) size used to be 8Kb. Not sure if it has changed.