Step Functions have great potential for orchestrating complex executions of Lambdas or even fleets of them. My co-workers and I designed a threadpool pattern wi

Step Functions have great potential for orchestrating complex executions of Lambdas or even fleets of them. My co-workers and I designed a threadpool pattern with step functions that we use for orchestrating complex long-running tasks: https://github.com/kislyuk/domovoi/blob/master/domovoi/examples/state_machine_threadpool_app.py https://github.com/kislyuk/domovoi/blob/master/domovoi/examp... However, there are currently some big issues with step functions: compared to Lambdas, they are not very scalable, not very economical, and have low I/O size limits (because all I/O gets interpreted by the executor, so they limit the state size to 32KB). Running Lambdas, we're used to be able to burst to many thousands of concurrent executions. The Step Functions burst start limit is 500, with just 25 per second after that (and 400 for state transitions). This can be seriously limiting if you need to process more activities in your system.