2 ms·
Don't forget the power of batch processing. Even today, you can dramatically speed up processing of work by putting things into batches. Speedups of 3-4 orders
by bob1029 6y ago
Don't forget the power of batch processing. Even today, you can dramatically speed up processing of work by putting things into batches. Speedups of 3-4 orders of magnitude can easily be realized, even if you hedge and do micro batches (10-1000 microseconds wide).
To a consumer of one of these systems, you can make the batched processing occur so quickly as to seem real-time in nature, while still realizing most of the low-level CPU architecture benefits of doing things in reasonably-sized chunks.
There are benefits to doing it the old-school way too (i.e. nightly). Giving the business ample time to respond to errors before EOD processing is a huge perk in many industries where the downstream consequences of errors cannot be reconciled. Real-time processing is wonderful, but there are still a lot of places where you don't really need it or where it causes more harm than good.