4 ms·
If Lambda is able to expand the /tmp disk space from 500MB to a few GB I think that will further make the adopter better on my end. I have a requirement in whic
by yeukhon 11y ago
If Lambda is able to expand the /tmp disk space from 500MB to a few GB I think that will further make the adopter better on my end. I have a requirement in which Lambda is a perfect solution except the disk space is not enough. Of course I can further divide the work into smaller chunks. If you have use distributed task library like Celery, just think of that no longer tie to your application, and you can run on event based (S3, SNS, email notification via SNS). It's quite awesome. You can probably build a public API using Lambda and API Gateway.
However, once you start using Lambda, like distributed system, testing becomes a challenge if you are used to testing everything you own in one or two monolithic codebase. I wish someone could come up with a mock version of AWS :-)
- count 11y agoCan you mount an EFS on the lambda instance (I don't know, but that comes to mind as a possible solution), possibly via a java NFS daemon or something in userspace?
- yeukhon 11y agoI have never done that before but I imagine no, you are not supposed to have access to the operating system and know where the instance is. Similar to SNS and SQS, you don't get an IP of your SQS. I could set up an EC2 instance with some storage, but that goes back to my current design. If you do get that IP :-) contact AWS. That's probably a million dollar vulnerability. I'd suppose underneath they run container/lxc! Also, sadly, EFS is only available in us-west oregon :|
- bolchevich 11y agoIt's an EC2 instance running inside a VPC using a custom Amazon Linux that uses some LXC trickery to contain your process. In theory EFS is possible.
- twagner 11y agoAppreciate the feedback/suggestions! We'll take a look at expanding the disk space, probably as part of the "power level" adjustment that already affects memory, CPU power, etc.
- RA_Fisher 11y agoOne minor feature request would be to allow Scheduled Events every minute. My current solution is EC2 -> S3 file every minute, but it's a kludge.
- yeukhon 11y agoThere is : http://www.slideshare.net/AmazonWebServices/cmp407-lambda-as-cron-scheduling-invocations-in-aws-lambda http://www.slideshare.net/AmazonWebServices/cmp407-lambda-as... but I kinda dislike the whole complexity). This is a new feature.
- spullara 11y agoWhy can't I access EFS? That seems like the best solution to this.
- stephenitis 11y agoI used to work at Iron.io they have a product called IronWorker which gave 10GB temp disk. They allowed you to use your own docker containers also. they have a priority queueing system however and come at a premium over lamdba pricing due due to being more feature rich. custom docker image use came with the overhead of burning in your instances though which I don't think is publicly available yet on the service.
- eistrati 11y agoWe have used a combination of AWS Lambda + Amazon ECS, which is kind of similar what you're describing. We are working now to extend the abstracted library that empowers developers to choose programmatically what service to use: https://www.npmjs.com/package/deep-resource https://www.npmjs.com/package/deep-resource ;)