Actually I just checked, and the bottom 1.91M files out of 5M are less than 1024 bytes, and sums to only 1 GB ! The other 3.1 M files sum to 34 G (largest file

Actually I just checked, and the bottom 1.91M files out of 5M are less than 1024 bytes, and sums to only 1 GB ! The other 3.1 M files sum to 34 G (largest file being 2GB or so). The distribution was more skewed than I thought. So I think the hybrid solution with internal sqlite for small files may be a good idea. Thanks!