3 ms·

Note: I apologize if this is overstepping, its hard to tell! I think a strong - extremely strong - selling point is the point I made about "prebaked" data for

Note: I apologize if this is overstepping, its hard to tell! I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs, since the entire strength of these SQLite based systems reside in their fast read capacity (as mentioned elsewhere and in this article, its very fast for read heavy applications, which is most) you could take on an angle around that to get people "in the door" by showing a pathway of how this fits inside your existing data warehouse / storage model. We found we liked the SQLite durability to do this. It was a bit smarter than just a plain cache (such as Redis) with better durability and (for our needs) comparable enough performance (I think in absolute terms, a tuned Redis instance will always be faster, but up to a certain point, speed isn't everything, especially when factoring cost). We found it was cheaper - by a good margin - to do this over caching everything AOT in a redis cluster, and we could therefore much more cheaply go multi-region and have DB's sitting next to our customers that acted as a nearline cache. The complexity - which is an area where this might help in the future, and why I'm mentioning it - is shipping changes back. What we ended up doing is setting up a write Redis cluster that clients write to, and we take those writes and trigger a propogation job back to the database. This allowed us to run a much slimmer redis cluster and made us feel more comfortable doing cache eviction since we could verify writes pretty easily. You could do this with memcache or whatever too. Sounds convoluted, but it worked really well. It allowed us to keep our centralized database intact without having to spin up expensive instances to be multi-region or commit to ever growing Redis cluster(s). the SQLite flat file model + history of durability made things the perfect tradeoff for this use case. Of course, YMMV, however it was a novel solution that used "enterprise grade" parts all the way down, which made it an easy selling point. You might find it worth exploring this more. As far as the comparisons go, I think it'd be cool to see a deep dive, and run a test suite against each of the major SQLite as a distributed database model. For that, I don't think it has to be open source to do a reasonable comparison?

benbjohnson 4y ago
> I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs. I think caches are an excellent use case for LiteFS early on. Sorry I didn't make that point in my previous reply. It's a good way to get benefits out of LiteFS without committing to it as your source of truth. Also related, Segment built a custom SQLite-based solution[1] for distributing out cached data that worked well for them. [1]: https://segment.com/blog/separating-our-data-and-control-planes-with-ctlstore/ https://segment.com/blog/separating-our-data-and-control-pla... > We found it was cheaper - by a good margin - to do this over caching everything AOT in a redis cluster Do you remember specifics of the cost difference? I can imagine it'd be pretty significant since you don't need to spin up servers with a bunch of RAM. > Note: I apologize if this is overstepping, its hard to tell! Not overstepping at all! It's great to hear folks' feedback.
- no_wizard 4y ago
  I'm a little fuzzy on the numbers, but it was 2-3x (or there about), since we didn't have to run multiple gigabyte clusters anymore. Even with auto-scaling based on demand, the clusters were more expensive simply due to the fact that if you wanted users to have a good experience you had to make sure you had a a reasonable amount of their data AOT cached, which itself was alot of complexity to manage. With SQLite, we could just scale instances during peak demand (so you could read from different instances of the same data if there was a bottleneck) and scale back down again, without (!usually) losing the data. It was a really complex - but fun - project all told, but its underpinnings were really simple. How Segment approached the problem isn't dissimilar to how we did it, honestly. The only thing that we (may) have done different is we had failover. If SQLite didn't respond our API layer could then talk directly to a database service to get the data. That was surprisingly complex to do. Its entirely possible that even more robust setups than ours was when we did this would yield higher cost savings. We did this before we hit our next scale of customers, just to add a little more context
- ignoramous 4y ago
  > I think caches are an excellent use case for LiteFS early on. I mean, folks can do stuff like this on Fly with Redis backed by disk, too: https://fly.io/blog/last-mile-redis/ https://fly.io/blog/last-mile-redis/

benbjohnson 4y ago

> I think a strong - extremely strong - selling point is the point I made about "prebaked" data for your APIs. I think caches are an excellent use case for LiteFS early on. Sorry I didn't make that point in my previous reply. It's a good way to get benefits out of LiteFS without committing to it as your source of truth. Also related, Segment built a custom SQLite-based solution[1] for distributing out cached data that worked well for them. [1]: https://segment.com/blog/separating-our-data-and-control-planes-with-ctlstore/ https://segment.com/blog/separating-our-data-and-control-pla... > We found it was cheaper - by a good margin - to do this over caching everything AOT in a redis cluster Do you remember specifics of the cost difference? I can imagine it'd be pretty significant since you don't need to spin up servers with a bunch of RAM. > Note: I apologize if this is overstepping, its hard to tell! Not overstepping at all! It's great to hear folks' feedback.

no_wizard 4y ago
I'm a little fuzzy on the numbers, but it was 2-3x (or there about), since we didn't have to run multiple gigabyte clusters anymore. Even with auto-scaling based on demand, the clusters were more expensive simply due to the fact that if you wanted users to have a good experience you had to make sure you had a a reasonable amount of their data AOT cached, which itself was alot of complexity to manage. With SQLite, we could just scale instances during peak demand (so you could read from different instances of the same data if there was a bottleneck) and scale back down again, without (!usually) losing the data. It was a really complex - but fun - project all told, but its underpinnings were really simple. How Segment approached the problem isn't dissimilar to how we did it, honestly. The only thing that we (may) have done different is we had failover. If SQLite didn't respond our API layer could then talk directly to a database service to get the data. That was surprisingly complex to do. Its entirely possible that even more robust setups than ours was when we did this would yield higher cost savings. We did this before we hit our next scale of customers, just to add a little more context
ignoramous 4y ago
> I think caches are an excellent use case for LiteFS early on. I mean, folks can do stuff like this on Fly with Redis backed by disk, too: https://fly.io/blog/last-mile-redis/ https://fly.io/blog/last-mile-redis/