3 ms·

Ruby and Sidekiq as the messaging queue Postgres to store the data Elasticsearch as a search index. My ES cluster has around 10 nodes, 64 GB RAM, quad-core.

Ruby and Sidekiq as the messaging queue Postgres to store the data Elasticsearch as a search index. My ES cluster has around 10 nodes, 64 GB RAM, quad-core. Postgres database cluster is 4 nodes, 1 TB, 64 GB RAM, quad-core. 800 crawler threads distributed across 10 dedicated servers.

outpan 10y ago
Thanks a lot! This sounds reasonable. Did you guys look into professional services for this?
- AznHisoka 10y ago
  Nope. We have lots of custom needs.