4 ms·

If you build an image for testing on every commit and don't have a retention policy set up, you could be using a massive amount of space without realizing it. I

If you build an image for testing on every commit and don't have a retention policy set up, you could be using a massive amount of space without realizing it. I can see why they did this.

snoopy_telex 4y ago
Are retention policies easy? I recall fighting on GitHub to setup a sane retention policy awhile back, but haven't revisited in years.
- sofixa 4y ago
  I wouldn't say easy because regexes are needed for complex ones (e.g. retain -prod images for 6 months, -staging for 24 days) but they are powerful and not that complicated.
- dnsmichi 4y ago
  GitLab offers ways to manage retention policies for your container registry. Please see the docs for guidance: https://docs.gitlab.com/ee/user/packages/container_registry/reduce_container_registry_storage.html#create-a-cleanup-policy https://docs.gitlab.com/ee/user/packages/container_registry/...
khuey 4y ago
Gitlab didn't even have retention policies until sometime in 2020. I have a six year old project that's consuming something like 4TB of space in their container registry.
- akudha 4y ago
  Woah. 4 TB? What kind of project is it?
  - georgyo 4y ago
    Not op, but a simple node application can very easily make container images that are well over a gigabyte. Build and store that container images on every push and boom, you can explode in usage.
- Gigachad 4y ago
  Yeah tbh we are leaving the age of abundant cheap money where everything is unlimited and unpriced. Having people pay for their usage will get them to actually start cleaning up all their junk build artifacts rather than just eating the loss on storing petabytes of old docker images which have no value.
pm215 4y ago
Also, for some things there is no retention policy configurable at all -- eg pipelines and their associated stdout logs appear to not be auto-cleaned-up at any expiry date (no default, nothing configurable). If you want to get rid of those you need to script it via the API, it seems: eg https://gitlab.com/eskultety/gitlab_cleaner https://gitlab.com/eskultety/gitlab_cleaner (I just ran that on the QEMU project and reduced the usage from 295GB to 165GB by deleting pipelines older than 1 Jan... so that's a lot of low-hanging logfile fruit gitlab could be auto-deleting.)

If you build an image for testing on every commit and don't have a retention policy set up, you could be using a massive amount of space without realizing it. I can see why they did this.

snoopy_telex 4y ago

Are retention policies easy? I recall fighting on GitHub to setup a sane retention policy awhile back, but haven't revisited in years.

sofixa 4y ago
I wouldn't say easy because regexes are needed for complex ones (e.g. retain -prod images for 6 months, -staging for 24 days) but they are powerful and not that complicated.
dnsmichi 4y ago
GitLab offers ways to manage retention policies for your container registry. Please see the docs for guidance: https://docs.gitlab.com/ee/user/packages/container_registry/reduce_container_registry_storage.html#create-a-cleanup-policy https://docs.gitlab.com/ee/user/packages/container_registry/...

khuey 4y ago

Gitlab didn't even have retention policies until sometime in 2020. I have a six year old project that's consuming something like 4TB of space in their container registry.

akudha 4y ago
Woah. 4 TB? What kind of project is it?
- georgyo 4y ago
  Not op, but a simple node application can very easily make container images that are well over a gigabyte. Build and store that container images on every push and boom, you can explode in usage.
Gigachad 4y ago
Yeah tbh we are leaving the age of abundant cheap money where everything is unlimited and unpriced. Having people pay for their usage will get them to actually start cleaning up all their junk build artifacts rather than just eating the loss on storing petabytes of old docker images which have no value.

pm215 4y ago

Also, for some things there is no retention policy configurable at all -- eg pipelines and their associated stdout logs appear to not be auto-cleaned-up at any expiry date (no default, nothing configurable). If you want to get rid of those you need to script it via the API, it seems: eg https://gitlab.com/eskultety/gitlab_cleaner https://gitlab.com/eskultety/gitlab_cleaner (I just ran that on the QEMU project and reduced the usage from 295GB to 165GB by deleting pipelines older than 1 Jan... so that's a lot of low-hanging logfile fruit gitlab could be auto-deleting.)