Depends on a lot of things, - What is your input : a stream, files, a queue (rabbit, kafka,...)? - How often do you need to process your input? - Can you sca

Depends on a lot of things, - What is your input : a stream, files, a queue (rabbit, kafka,...)? - How often do you need to process your input? - Can you scale at will/need? - What is the max quantity of data per second you can recieve from your source? - What is the maximum rate of data your infrastructure can absorb in input? Usually you try not to update your software config every 5 minutes. You find one that suits best your infrastructure.