5 ms·

Lambas are not simple functions because your environment is different in local compared to production. If I run a Node.js function in AWS Lambda, my Node.js ve

Lambas are not simple functions because your environment is different in local compared to production. If I run a Node.js function in AWS Lambda, my Node.js version might be different, my dependencies might be different, the OS is different, the filesystem is different, so I or one of my node_modules might be able to write to /tmp but not elsewhere, etc. It's the reason people started using Docker really. If you don't have the same environment, you can't call it reproducible or testable for that matter.

TazeTSchnitzel 7y ago
It is far too easy to have all the tests pass locally and be completely broken in production for this reason.
- jtdev 7y ago
  As with any compute environment...
- erikerikson 7y ago
  See https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery Note performance, acceptance, and monitoring modes
philliphaydon 7y ago
Nothing you mentioned has anything to do with the ability to test a Lambda. You’re trying to use limitations and restrictions as friction to backup your inability to test. There’s a lot of annoying things about lambda. And a lot of stuff I wish was easier to find in documentation. But that doesn’t change the fact that Lambda is more or less passing an event object to your function and executing it. Writing a function in node 12 and then running it on node 4 and throwing your hands in the air cos it didn’t work isn’t the fault of Lambda.
- LunaSea 7y ago
  It's great to see that factual evidence is answered with ad-hominem by the Lambda hype crowd. In any case, if you have a Node.js module or code with a native C/C++ build, that runs shell commands, that writes to disk (not allowed besides /tmp in Lambda) or makes assumptions about the OS, your "simple" function will absolutely return different results. e.g: My lambda is called when somebody uploads an image and returns a resized and compressed version of it. This is done using Node.js and the mozjpeg module which is dependent on cjpeg which is built natively on install. If I test my function on my machine and in Lambda it's very possible that I get different results. Also, certain OSs like Alpine which are heavily used for Docker don't event use glibc as compiler, so again, another difference.
  - rustyboy 7y ago
    Why not then have lambda run the same container you can run and test locally? I don't use lambda but we have our jenkins spin up the same ec2 to run tests that we would spin up to do development so that we never run into this problem.
    - LunaSea 7y ago
      I'm not sure I understood your question correctly. If you mean running a Docker container in Lambda, that is to may knowledge to possible. You could schedule Docker tasks in AWS ECS (their managed container service) but it's not meant for anything realtime and more for cron job type tasks. If you mean emulating the Lambda environment in Docker, then I wrote an answer with the difficulties of doing that below to another user.
  - k__ 7y ago
    I think you are right with your assumption that Docker images that don't resemble the production environment aren't sufficient to test. But isn't the idea of Docker that you can recreate the production environment? If you can't why use Docker in the first place?
    - LunaSea 7y ago
      You are absolutely right that you could recreate a similar environment to Lambda in Docker. But you would first need to reverse engineer Lambda's environment to discover how it is actually configured and the limits that are set. Even if you did find a way, you would still need to keep it up to date in case AWS decides to update that environment.
nailer 7y ago
> Node.js version might be different Yes, in major only. Your lambda has node 10, you might be running 10.x or 10.y > my dependencies might be different No, the lockfile does that. > the OS is different, the filesystem is different Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS? > It's the reason people started using Docker really. VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
- LunaSea 7y ago
  > No, the lockfile does that. No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed. > Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS? Plenty. For example filesystem change events are known to have filesystem and OS dependent behaviours and quirks / bugs. When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool, or maybe a different version altogether. The Linux user with which you are running the function might also have different rights which could become an issue when accessing the filesystem in any way. > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless. Maybe, but serverless doesn't answer those questions at all. It just hand waves testing and vendor independent infrastructure.
  - philliphaydon 7y ago
    > No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed. You shouldn’t be uploading node_modules folder to your deployed lambda so this is an issue of your development environment, not lambda. > Maybe, but serverless doesn't answer those questions at all. “Serverless” or lambda/Azure functions, etc, are not a silver bullet that solve every single scenario. Just like docker doesn’t solve every single scenario, not does cloud servers or bare metal. It’s just another tool for us to do our job.
  - nailer 7y ago
    > > > my dependencies might be different > > No, the lockfile does that. > your lockfile doesn't care about build steps Then you're not talking about dependency versioning are you? you're talking about install order. In practice it hasn't been an issue, I should find out how deterministic install order is but I'd only be doing this to win a silly argument rather than anything that has come up in nearly a decade of making serverless apps. > For example filesystem change events are known to have filesystem and OS dependent behaviours > When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem? All your issues seem to come from edge cases. Like if you decide to run FreeB or ReiserFS locally and run a sandbox it it, fine, but know that's going to differ from a Linux / systemd / GNU / extX environment. > > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless. > Maybe, but serverless doesn't answer those questions at all. Serverless exists precisely to answer the question. I can throw all my MicroVMs in the ocean with no knowledge of dockerfiles, no VM snapshots, no knowledge of cloudinit, no environment knowledge other than 'node 10 on Linux' and get my entire environment back immediately.

TazeTSchnitzel 7y ago

It is far too easy to have all the tests pass locally and be completely broken in production for this reason.

jtdev 7y ago
As with any compute environment...
erikerikson 7y ago
See https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery Note performance, acceptance, and monitoring modes

philliphaydon 7y ago

Nothing you mentioned has anything to do with the ability to test a Lambda. You’re trying to use limitations and restrictions as friction to backup your inability to test. There’s a lot of annoying things about lambda. And a lot of stuff I wish was easier to find in documentation. But that doesn’t change the fact that Lambda is more or less passing an event object to your function and executing it. Writing a function in node 12 and then running it on node 4 and throwing your hands in the air cos it didn’t work isn’t the fault of Lambda.

LunaSea 7y ago
It's great to see that factual evidence is answered with ad-hominem by the Lambda hype crowd. In any case, if you have a Node.js module or code with a native C/C++ build, that runs shell commands, that writes to disk (not allowed besides /tmp in Lambda) or makes assumptions about the OS, your "simple" function will absolutely return different results. e.g: My lambda is called when somebody uploads an image and returns a resized and compressed version of it. This is done using Node.js and the mozjpeg module which is dependent on cjpeg which is built natively on install. If I test my function on my machine and in Lambda it's very possible that I get different results. Also, certain OSs like Alpine which are heavily used for Docker don't event use glibc as compiler, so again, another difference.
- rustyboy 7y ago
  Why not then have lambda run the same container you can run and test locally? I don't use lambda but we have our jenkins spin up the same ec2 to run tests that we would spin up to do development so that we never run into this problem.
  - LunaSea 7y ago
    I'm not sure I understood your question correctly. If you mean running a Docker container in Lambda, that is to may knowledge to possible. You could schedule Docker tasks in AWS ECS (their managed container service) but it's not meant for anything realtime and more for cron job type tasks. If you mean emulating the Lambda environment in Docker, then I wrote an answer with the difficulties of doing that below to another user.
- k__ 7y ago
  I think you are right with your assumption that Docker images that don't resemble the production environment aren't sufficient to test. But isn't the idea of Docker that you can recreate the production environment? If you can't why use Docker in the first place?
  - LunaSea 7y ago
    You are absolutely right that you could recreate a similar environment to Lambda in Docker. But you would first need to reverse engineer Lambda's environment to discover how it is actually configured and the limits that are set. Even if you did find a way, you would still need to keep it up to date in case AWS decides to update that environment.

nailer 7y ago

> Node.js version might be different Yes, in major only. Your lambda has node 10, you might be running 10.x or 10.y > my dependencies might be different No, the lockfile does that. > the OS is different, the filesystem is different Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS? > It's the reason people started using Docker really. VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.

LunaSea 7y ago
> No, the lockfile does that. No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed. > Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS? Plenty. For example filesystem change events are known to have filesystem and OS dependent behaviours and quirks / bugs. When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool, or maybe a different version altogether. The Linux user with which you are running the function might also have different rights which could become an issue when accessing the filesystem in any way. > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless. Maybe, but serverless doesn't answer those questions at all. It just hand waves testing and vendor independent infrastructure.
- philliphaydon 7y ago
  > No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed. You shouldn’t be uploading node_modules folder to your deployed lambda so this is an issue of your development environment, not lambda. > Maybe, but serverless doesn't answer those questions at all. “Serverless” or lambda/Azure functions, etc, are not a silver bullet that solve every single scenario. Just like docker doesn’t solve every single scenario, not does cloud servers or bare metal. It’s just another tool for us to do our job.
- nailer 7y ago
  > > > my dependencies might be different > > No, the lockfile does that. > your lockfile doesn't care about build steps Then you're not talking about dependency versioning are you? you're talking about install order. In practice it hasn't been an issue, I should find out how deterministic install order is but I'd only be doing this to win a silly argument rather than anything that has come up in nearly a decade of making serverless apps. > For example filesystem change events are known to have filesystem and OS dependent behaviours > When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem? All your issues seem to come from edge cases. Like if you decide to run FreeB or ReiserFS locally and run a sandbox it it, fine, but know that's going to differ from a Linux / systemd / GNU / extX environment. > > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless. > Maybe, but serverless doesn't answer those questions at all. Serverless exists precisely to answer the question. I can throw all my MicroVMs in the ocean with no knowledge of dockerfiles, no VM snapshots, no knowledge of cloudinit, no environment knowledge other than 'node 10 on Linux' and get my entire environment back immediately.