33 ms·
Maybe I'm missing something but as the other comments have pointed out, this is very low scale. I'm not sure it can even be called at-scale in any sense these d
by manigandham 11y ago
Maybe I'm missing something but as the other comments have pointed out, this is very low scale. I'm not sure it can even be called at-scale in any sense these days.
Also the architecture seems incredibly fragile and complicated for what it's doing. I get that they've built some integrations and backend processes to assemble this data but the API serving could all run on a single app and database server since it's all just reads.
- harlow 11y agoHi manigandham, Harlow here (author of the post). You are absolutely correct; serving 2M API requests from a local data store wouldn't be considered "at-scale" these days. In hindsight I should have added more information about where the actual work is being done -- I definitely missed the mark on parts of this post. The lions-share of our work is done in an async fashion. 2M API requests (lookups) turns into 40M+ background jobs. These jobs fetch, aggregate, and scrub data from a number of downstream providers.