3 ms·
Great write up. We use S3 for our Apt & Yum repos at work, too, only with website hosting turned on for the buckets (losing some of the security OP's solution h
by leetrout 10y ago
Great write up. We use S3 for our Apt & Yum repos at work, too, only with website hosting turned on for the buckets (losing some of the security OP's solution has but adding simplicity).
I'd like to point out a tool I'm using now for building and managing our Apt repos- Aptly https://www.aptly.info/ https://www.aptly.info/
It's really nice and Just Works™. It even has S3 publishing built in. Oh, and you can build Apt repos on Debian, CentOS, MacOS X & FreeBSD (thanks Go!).
To solve for Yum & Apt I wrote a tool in Go that keeps all the metadata in a JSON file in S3. It's just a map of S3 buckets that are repos to a list S3 URLs for each RPM / Deb. Whenever that file gets updated, just like OP, I send an event from S3 to Lambda. Unlike OP, tho, I just have Lambda launch a task in ECS and then my repo builds (& S3 syncing) takes place in containers (1 Ubuntu container for Apt repos & 1 CentOS container for Yum).
- daenney 10y agoSo, this way be dragons but... You could just have a small Javascript or Python wrapper that simply calls out to Aptly. As long as you compile Aptly for the right target you can bundle it up in the zip that your ship to Lambda. Since it's Go it'll just run. Then your wrapper just takes care of calling Aptly with the right arguments depending on what event got sent from S3.
- leetrout 10y agoI actually prefer using the containers. It's easily repeatable anywhere I can run docker (doesn't have to be AWS). I can also avoid any race conditions by checking to see if the task is already running. (And in general I'm not a fan of trying to do a lot in lambda. I like it a lot more as a glue layer / event handler).