3 ms·
> to store small files in RAM (for some well-chosen value of “small”, of course... The Backblaze client code currently puts that dividing line at 100 MBytes.
by brianwski 6y ago
> to store small files in RAM (for some well-chosen value of “small”, of course...
The Backblaze client code currently puts that dividing line at 100 MBytes. Any file less than 100 MBytes we call a "small file" and those don't get subdivided into 10 MByte chunks on disk, they get slurped up into RAM and just held there for the entire duration of being sent. For that code path, Backblaze can be configured to send up to 30 of these 100 MByte files in 30 separate threads which means (if the customer configures it this way) it can eat up 3 GBytes of RAM at peak. If you have that much RAM, it can really rip. If you don't, we recommend using fewer threads. :-)
The sub-division into 10 MByte chunks taking a one time "Snapshot" of the large file is for all files larger than 100 MBytes.