4 ms·
3 xlarge EMR instances sounds like overkill assuming a volume of around 11gb every 10 mins. Using postgres COPY I've loaded larger files into tables in seconds.
by wefarrell 7y ago
3 xlarge EMR instances sounds like overkill assuming a volume of around 11gb every 10 mins. Using postgres COPY I've loaded larger files into tables in seconds. Semi complex queries will also take seconds depending on indexes. My understanding is that EMR doesn't make sense unless you're processing terabytes.
- TuringNYC 7y agoI read through the Readme but didn’t see any volume or velocity figures (I saw the entity count, but what does this translate into w/r/t bytes?) Anyone run this who could comment on the metrics and, consequently, server sizing?