4 ms·
Compactions happen on the tablet server. They only write data. By definition, they increase disk usage, right away. Only the garbage collector, which needs glob
by puzzle 8y ago
Compactions happen on the tablet server. They only write data. By definition, they increase disk usage, right away. Only the garbage collector, which needs global state and thus runs on the master, can actually delete files in Colossus and bring down disk usage. It doesn't always run as frequently as one would think or hope. It definitely doesn't run immediately after one tablet has been compacted.
I remember one incident where a team got an alert that they were at 90% of quota or so. They compacted data too aggressively and they actually hit 100% before the GC could do its job. That's why I mentioned that you should compact a fraction of the table, then let GC run. Google being Google, I'm sure such a tool has been written, in several variants, by N different teams.
- shereadsthenews 8y agoAs long as we're splitting hairs, only the Colossus curator can really delete the file and if the quota manager is behind you can still run out of space.
- londons_explore 8y agoThe quota manager can be behind in the opposite direction and let you go way over quota too. Yay - free unlimited disk space for all! (for ~5 minutes!)