3 ms·
Can you elaborate on what the "job" is or does? Can things be batched?
by dmarlow 8y ago
Can you elaborate on what the "job" is or does? Can things be batched?
- bballer 8y agoI wont fully elaborate on what a job is but I'll give you a couple examples: Updating anywhere from 500-5M items over APIs that are rate limited. Dumping datasets that have to be normalized and massaged into files/ and dropped off at third party servers anywhere from every 15 minutes to once a month. These files could contain anywhere from 500 lines to 5M lines. Injesting datasets just as large as described above but massaged and saved into our caches and DB.