9 ms·

LambCI – A continuous integration system built on AWS Lambda

160 points by hharnisch 10y ago

illumin8 10y ago
This looks very cool, but the 5 minute build time limit (an inherent limitation of the Lambda service) makes this less than ideal for a build system. The author does address this by recommending that you use Docker containers on ECS as an alternative for long running builds.
nulltype 10y ago
Is a Google App Engine application also serverless?
- buckbova 10y ago
  Perhaps that's PaaS? Lambda and others are now described as FaaS (function as a service). This might help: http://martinfowler.com/bliki/Serverless.html http://martinfowler.com/bliki/Serverless.html Google has a seperate cloud functions offering: https://cloud.google.com/functions/ https://cloud.google.com/functions/
  - hactually 10y ago
    Kinda surprised Google don't have Golang as their language. AWS offers nodejs, Java and Python. I wonder if it's due to being able to charge more for slow to start VM based languages?
oneplane 10y ago
How is it serverless if it runs on an Amazon server? Also, how is it serverless if you need to consume a service? (AWS in this case)... Every time I see something nice, there is this increasing chance that I'm gonna end up sad because it requires some sort of external provider like AWS, DO, Heroku, GCE... I don't have any of those and I don't want any of them.
- btown 10y ago
  "Serverless" here refers to not needing a pre-requisitioned server to exist in advance of the request. Sure, the physical hardware exists in AWS, but previously you'd need to have some sort of CI system running on a server that needs to be running and accepting web requests at any time that you might push to Github. With AWS Lambda, which LambCI is run on, a lightweight server boots up in realtime in response to that webhook, runs code, and shuts down. So you have a CI server that can respond any time of the day or night, but only consumes resources when it's actually running.
  - deleted 10y ago
    [deleted]
  - dragonwriter 10y ago
    Serverless is a fairly new and content-free buzzword specific to lambda. To the extent it is sometimes rationalized as communicating something meaningful, it's not something distinct from what almost every PaaS ever has always provided. (Lambda is distinct from earlier PaaS's, and other solutions which mimic Lambda have emerged, but the way in which they are distinct from most earlier PaaS offerings isn't part of how people try to rationalize "serverless" and wouldn't really fit into such a rationalization.
  - inopinatus 10y ago
    Serverless also describes the application invocation model. The design intention for serverless applications uses pub-sub events rather than client-server calls. That's a major enabler of async processing and containers-on-demand, and it looks like LambCI has followed the pattern to a tee. If you have to accept requests from a non-event-driven world, AWS offer their API Gateway to provide a listening server endpoint, but I think it's telling that this was not available when Lambda was released, and that LambCI does not need it.
  - oneplane 10y ago
    The point I was trying to make is that it seems like people don't want to make actual services or servers, but rather depend on some sort of closed loop or ecosystem elsewhere, and if you happen to not use that, you can't use the nice stuff they put up on GitHub.
empath75 10y ago
You win buzzword bingo for today.
somesaba 10y ago
Cool! I had an idea for something like this but instead of having each build be it's own lambda event, I wanted to make each individual test it's own lambda event. The goal is to have the build time for a complex project boil down to the time it takes to setup + run the longest test.
- adamb 10y ago
  Have you written any code to that effect? I've been trying to think through how booting up dependencies might work (for things like integration tests). I have aspirations for QA (https://github.com/ajbouh/qa https://github.com/ajbouh/qa) to learn this trick, but it needs some lambda-specific smarts before it gets there.
- rapind 10y ago
  I love that idea. Map -> Reduce of unit tests.
- falsedan 10y ago
  We're doing something similar, but using mesos. We bundle tests up, so we don't have to eat the setup costs for every test, and try tracking test pollution by splitting bundles when they fail & running them separately, again.
  - dominotw 10y ago
    where do you store build caches? npm/gems ect?
    - falsedan 10y ago
      Dependencies? On the host; they're ephemeral but they'll run multiple builds between coming into existence and disappearing.
      - dominotw 10y ago
        but first build always take a hit making the build times unpredictable making build time regressions hard to track.
dkarapetyan 10y ago
Cute. > No root access > 5 min max build time > Bring-your-own-binaries – Lambda has a limited selection of installed software > 1.5GB max memory > Linux only There's a reason Jenkins is still used so widely. It's not because of utilization or all the other things pointed out. When your project gets big enough, managing the CI pipeline turns into a distributed systems problem with distributed queues, locks, error/failure recovery, and all the other headaches that such systems bring. Heck, reporting alone on a test suite with 12k tests is a problem in and of itself.
- empath75 10y ago
  Yeah, there is a place for this kind of thing though. For a lot of small projects, Jenkins is overkill. Also, a lot of those issues can be worked around by using ecs for the builds.
  - toomuchtodo 10y ago
    Or just spend the time setting up the right tool in the beginning, so you don't have to waste time later moving to the right tool?
- Rezo 10y ago
  A Lambda / "serverless" approach makes a ton of sense for CI system just on the premise alone. Why have x number of build slaves sitting doing nothing 95% of the time? The ideal build system would paralellize on-demand to as many workers as possible within a few seconds of a build triggering (I don't want a build slave, I want 50), and then immediately terminate them, which is pretty much the whole point of Lambda. Jenkins is a huge pain the behind, fragile, the machines always inevitably turns into special snowflakes, configs in git are poorly supported. It's also very expensive in any non-trivial setting and requires full-time babysitters. Cloud CIs still charge quite a lot for any meaningful parallel builds, and there's the security aspect of uploading your code to third-parties, I trust my S3 bucket permissions more than a random CI SaaS. This seems like a great start for a sweet spot between on-prem and SaaS CI.
  - 10y ago
adamb 10y ago
There's a lot of harsh commentary, but I think people here are missing the point. The fact that so much software needs root access to a (often mutable) global environment in order to properly build is a bug. There are an increasing number of build systems that encourage squashing these bugs. The resulting build outputs are simpler and are often more portable. They're also easier to reason about. That translates to simpler deployment, simpler operations, and fewer edge cases to debug. IMHO, the most promising answer to the 5 minute limit is finer granularity and better caching of dependency inputs.
dllthomas 10y ago
"LambCI - A continuous integration system built on surface dwellers"
nzoschke 10y ago
Very nice! This looks very close to the ideal CI infrastructure. I'm used to waiting on queues and long VM or container boots and configuration on other services. We can almost certainly count on Lambda getting longer execution times and higher memory limits. We can also count on containerization solving the root problem. We should also be building software with the goal of tests that run within reasonable limits like this. `time make test` takes 39 seconds on my businesses Go projects. I'd consider a 5m test suite serious tech debt. The time that developers wait for feedback on tests and deployment is becoming a business bottleneck in the continuous delivery age.
- rgbrgb 10y ago
  Wow, what's your secret to such fast builds? We're at ~35 minutes to build and test our rails monolith on Wercker. I'm guessing you're not hitting a db too much or loading phantomjs for end-to-end tests of a web ui?
  - Intermernet 10y ago
    At a guess, it's because they're using Go. The build and test processes are refreshingly (ludicrously) fast in that language. Note that doesn't mean you should switch to Go, as the ridiculously fast compile times could be outweighed by other factors (retraining, lack of specific features, etc.) I would however investigate it as an option as it seems to have found a place in the hearts of many former rails shops!
nzoschke 10y ago
Very nice! This looks very close to the ideal CI infrastructure. I'm used to waiting on queues and long VM or container boots and configuration on other services. We can almost certainly count on Lambda getting longer execution times and higher memory limits. We can also count on containerization solving the root problem. We should also be building software with the goal of tests that run within reasonable limits like this. `time make test` takes 39 seconds on my businesses Go projects. I'd consider a 5m test suite serious tech debt. The time that developers wait for feedback on tests and deployment is becoming a business bottleneck in the continuous delivery age.
a_imho 10y ago
Hardware is usually the cheapest component when it comes to software manufacturing, but we found out we wanted CI/CD to spin 24x365 as much as it can, increasing the resolution to ~single commits with the shortest possible cycles. With a sizeable codebase and a thorough testsuite AWS bills went up so quickly even the proponents decided it was not worth it. We restored our old CI infra and were able to add a couple of new servers too. The throughput increased considerably with money to spare. Still an interesting experiment, but it showed burning money on Amazon not defaults to moving faster.

160 points by hharnisch 10y ago

This looks very cool, but the 5 minute build time limit (an inherent limitation of the Lambda service) makes this less than ideal for a build system. The author does address this by recommending that you use Docker containers on ECS as an alternative for long running builds.

nulltype 10y ago

Is a Google App Engine application also serverless?

buckbova 10y ago
Perhaps that's PaaS? Lambda and others are now described as FaaS (function as a service). This might help: http://martinfowler.com/bliki/Serverless.html http://martinfowler.com/bliki/Serverless.html Google has a seperate cloud functions offering: https://cloud.google.com/functions/ https://cloud.google.com/functions/
- hactually 10y ago
  Kinda surprised Google don't have Golang as their language. AWS offers nodejs, Java and Python. I wonder if it's due to being able to charge more for slow to start VM based languages?

oneplane 10y ago

How is it serverless if it runs on an Amazon server? Also, how is it serverless if you need to consume a service? (AWS in this case)... Every time I see something nice, there is this increasing chance that I'm gonna end up sad because it requires some sort of external provider like AWS, DO, Heroku, GCE... I don't have any of those and I don't want any of them.

btown 10y ago
"Serverless" here refers to not needing a pre-requisitioned server to exist in advance of the request. Sure, the physical hardware exists in AWS, but previously you'd need to have some sort of CI system running on a server that needs to be running and accepting web requests at any time that you might push to Github. With AWS Lambda, which LambCI is run on, a lightweight server boots up in realtime in response to that webhook, runs code, and shuts down. So you have a CI server that can respond any time of the day or night, but only consumes resources when it's actually running.
- deleted 10y ago
  [deleted]
- dragonwriter 10y ago
  Serverless is a fairly new and content-free buzzword specific to lambda. To the extent it is sometimes rationalized as communicating something meaningful, it's not something distinct from what almost every PaaS ever has always provided. (Lambda is distinct from earlier PaaS's, and other solutions which mimic Lambda have emerged, but the way in which they are distinct from most earlier PaaS offerings isn't part of how people try to rationalize "serverless" and wouldn't really fit into such a rationalization.
- inopinatus 10y ago
  Serverless also describes the application invocation model. The design intention for serverless applications uses pub-sub events rather than client-server calls. That's a major enabler of async processing and containers-on-demand, and it looks like LambCI has followed the pattern to a tee. If you have to accept requests from a non-event-driven world, AWS offer their API Gateway to provide a listening server endpoint, but I think it's telling that this was not available when Lambda was released, and that LambCI does not need it.
- oneplane 10y ago
  The point I was trying to make is that it seems like people don't want to make actual services or servers, but rather depend on some sort of closed loop or ecosystem elsewhere, and if you happen to not use that, you can't use the nice stuff they put up on GitHub.

empath75 10y ago

You win buzzword bingo for today.

somesaba 10y ago

Cool! I had an idea for something like this but instead of having each build be it's own lambda event, I wanted to make each individual test it's own lambda event. The goal is to have the build time for a complex project boil down to the time it takes to setup + run the longest test.

adamb 10y ago
Have you written any code to that effect? I've been trying to think through how booting up dependencies might work (for things like integration tests). I have aspirations for QA (https://github.com/ajbouh/qa https://github.com/ajbouh/qa) to learn this trick, but it needs some lambda-specific smarts before it gets there.
rapind 10y ago
I love that idea. Map -> Reduce of unit tests.
falsedan 10y ago
We're doing something similar, but using mesos. We bundle tests up, so we don't have to eat the setup costs for every test, and try tracking test pollution by splitting bundles when they fail & running them separately, again.
- dominotw 10y ago
  where do you store build caches? npm/gems ect?
  - falsedan 10y ago
    Dependencies? On the host; they're ephemeral but they'll run multiple builds between coming into existence and disappearing.
    - dominotw 10y ago
      but first build always take a hit making the build times unpredictable making build time regressions hard to track.

dkarapetyan 10y ago

Cute. > No root access > 5 min max build time > Bring-your-own-binaries – Lambda has a limited selection of installed software > 1.5GB max memory > Linux only There's a reason Jenkins is still used so widely. It's not because of utilization or all the other things pointed out. When your project gets big enough, managing the CI pipeline turns into a distributed systems problem with distributed queues, locks, error/failure recovery, and all the other headaches that such systems bring. Heck, reporting alone on a test suite with 12k tests is a problem in and of itself.

empath75 10y ago
Yeah, there is a place for this kind of thing though. For a lot of small projects, Jenkins is overkill. Also, a lot of those issues can be worked around by using ecs for the builds.
- toomuchtodo 10y ago
  Or just spend the time setting up the right tool in the beginning, so you don't have to waste time later moving to the right tool?
Rezo 10y ago
A Lambda / "serverless" approach makes a ton of sense for CI system just on the premise alone. Why have x number of build slaves sitting doing nothing 95% of the time? The ideal build system would paralellize on-demand to as many workers as possible within a few seconds of a build triggering (I don't want a build slave, I want 50), and then immediately terminate them, which is pretty much the whole point of Lambda. Jenkins is a huge pain the behind, fragile, the machines always inevitably turns into special snowflakes, configs in git are poorly supported. It's also very expensive in any non-trivial setting and requires full-time babysitters. Cloud CIs still charge quite a lot for any meaningful parallel builds, and there's the security aspect of uploading your code to third-parties, I trust my S3 bucket permissions more than a random CI SaaS. This seems like a great start for a sweet spot between on-prem and SaaS CI.
- 10y ago

adamb 10y ago

There's a lot of harsh commentary, but I think people here are missing the point. The fact that so much software needs root access to a (often mutable) global environment in order to properly build is a bug. There are an increasing number of build systems that encourage squashing these bugs. The resulting build outputs are simpler and are often more portable. They're also easier to reason about. That translates to simpler deployment, simpler operations, and fewer edge cases to debug. IMHO, the most promising answer to the 5 minute limit is finer granularity and better caching of dependency inputs.

dllthomas 10y ago

"LambCI - A continuous integration system built on surface dwellers"

nzoschke 10y ago

Very nice! This looks very close to the ideal CI infrastructure. I'm used to waiting on queues and long VM or container boots and configuration on other services. We can almost certainly count on Lambda getting longer execution times and higher memory limits. We can also count on containerization solving the root problem. We should also be building software with the goal of tests that run within reasonable limits like this. `time make test` takes 39 seconds on my businesses Go projects. I'd consider a 5m test suite serious tech debt. The time that developers wait for feedback on tests and deployment is becoming a business bottleneck in the continuous delivery age.

rgbrgb 10y ago
Wow, what's your secret to such fast builds? We're at ~35 minutes to build and test our rails monolith on Wercker. I'm guessing you're not hitting a db too much or loading phantomjs for end-to-end tests of a web ui?
- Intermernet 10y ago
  At a guess, it's because they're using Go. The build and test processes are refreshingly (ludicrously) fast in that language. Note that doesn't mean you should switch to Go, as the ridiculously fast compile times could be outweighed by other factors (retraining, lack of specific features, etc.) I would however investigate it as an option as it seems to have found a place in the hearts of many former rails shops!

nzoschke 10y ago

a_imho 10y ago

Hardware is usually the cheapest component when it comes to software manufacturing, but we found out we wanted CI/CD to spin 24x365 as much as it can, increasing the resolution to ~single commits with the shortest possible cycles. With a sizeable codebase and a thorough testsuite AWS bills went up so quickly even the proponents decided it was not worth it. We restored our old CI infra and were able to add a couple of new servers too. The throughput increased considerably with money to spare. Still an interesting experiment, but it showed burning money on Amazon not defaults to moving faster.