4 ms·
I've hacked around lambda quite a bit (I think the compressed size of one function is a tad under the max allowed). My hacks I remember are: - Run strip all .s
by tcas 10y ago
I've hacked around lambda quite a bit (I think the compressed size of one function is a tad under the max allowed). My hacks I remember are:
- Run strip all .so libraries -- many aren't stripped fully
- In Python I manually deleted sub packages of numpy/scipy I didn't need
- If you're loading large models at initialize, numpy load routines are _much_ faster than cPickle. Have it load at module initialization, not during each invocation.
I should really write a blog post about my experience with it.
At a certain point I decided I was doing something that lambda really wasn't designed for -- I'm looking at migrating off, but the current implementation makes capacity planning super easy. Provisioning 1000 machines with 1GB of RAM for 15 minutes every day to read off a queue isn't a trivial problem.
(Also, if anyone from AWS is reading, being able to limit the max concurrency of a single function vs account level limits would be super useful).
- btown 10y agoWould love to see that blog post!
- niklasrde 10y agoThat sounds quite similar to what we're doing! Would love to see a blog post to compare experiences.. (Speaking of blogposts.. that was somewhere on the todo list..)
- jedberg 10y agoJust to throw this out there as another possible optimization, if you find that you're putting a big fat library into every function, one possibility is to run the library as its own lambda function. You'll be slowed down a bit by the network but it might be made up for by not having to constantly initialize the same thing.