4 ms·
I have 3 sizeable EC2 instances running an ElasticSearch cluster, plus a beefy box for data preprocessing and crawling. A big chunk actually goes to BigQuery.
by alangibson 5y ago
I have 3 sizeable EC2 instances running an ElasticSearch cluster, plus a beefy box for data preprocessing and crawling.
A big chunk actually goes to BigQuery. There are publicly available datasets for HN, Stackoverflow and a few others there. I've also loaded up the Common Crawl index. The query and storage fees really add up.
I'm hopefully done with huge BigQuery queries, so that $200 will probably drop for a while.
- rmbyrro 5y agoI'm probably wrong about my assumptions, but presume you are open to any kind of constructive feedback, so here it goes... Maybe you're overkilling with the infra stack. I would simplify until having a mature product, especially if I'm bootstrapping, which I think is your case. Right now, you're still a bit far from MVP, from my point of view. Those $200 can probably be reduced by 50%-75% if you compromise on stuff only important to non-alpha services (i.e. 99.99% availability). A single EC2 box should be enough. Maybe look into Postgres or another FOSS instead of BigQuery. These $100-$150 savings per week can go into promoting your service, getting as much attention as possible to maximize feedback. Good luck!