3 ms·
I'm a little fuzzy on the numbers, but it was 2-3x (or there about), since we didn't have to run multiple gigabyte clusters anymore. Even with auto-scaling base
by no_wizard 4y ago
I'm a little fuzzy on the numbers, but it was 2-3x (or there about), since we didn't have to run multiple gigabyte clusters anymore. Even with auto-scaling based on demand, the clusters were more expensive simply due to the fact that if you wanted users to have a good experience you had to make sure you had a a reasonable amount of their data AOT cached, which itself was alot of complexity to manage.
With SQLite, we could just scale instances during peak demand (so you could read from different instances of the same data if there was a bottleneck) and scale back down again, without (!usually) losing the data.
It was a really complex - but fun - project all told, but its underpinnings were really simple.
How Segment approached the problem isn't dissimilar to how we did it, honestly.
The only thing that we (may) have done different is we had failover. If SQLite didn't respond our API layer could then talk directly to a database service to get the data. That was surprisingly complex to do.
Its entirely possible that even more robust setups than ours was when we did this would yield higher cost savings. We did this before we hit our next scale of customers, just to add a little more context