3 ms·
Back then they had mentioned how their architecture was fairly low cost (they use a much more modern stack than Slack does, for one). That was before search tho
by shados 9y ago
Back then they had mentioned how their architecture was fairly low cost (they use a much more modern stack than Slack does, for one). That was before search though.
With search, I'm really not sure how they pull it off.
- jhgg 9y agoI'm the engineer who worked on the Search backend. Read our blog here: https://blog.discordapp.com/how-discord-indexes-billions-of-messages-e3d5e9be866f https://blog.discordapp.com/how-discord-indexes-billions-of-... Cost isn't actually too bad, given how we've designed our search schema - and how we lazily index.
- shados 9y agoMakes sense. I did read that blog before, but I just can't keep thinking about just the cost of storing that much text on disc, even if it was compressed, would quickly add up. Maybe my mental math is failing me (or I'm overestimating how many messages there are), but that still seemed expensive. In the end, as long as you don't go out of business, because I'd be very very sad :)
- jhgg 9y agoStorage is actually really cheap for this, we don't store the message data in ES, it just builds the inverted indices. I explain this more in-depth on the blog post. On GCP, 50TB of Persistent SSD is roughly $9000/month before any discounts.