welder

40 ms·

481.
by welder 5y ago
Looks slightly better for our use case than MinIO with optimization for many small files, I'll look more into it. Thanks!
482.
by welder 5y ago
Yep, WakaTime uses DigitalOcean for almost everything, except S3 is just too good and the price is worth it.
483.
by welder 5y ago
No, the SSDB cache can go offline and the app just reads from S3 directly. I appreciate everyone trying to suggest better ways to solve this, but know that I've gone through many solutions dual-writing in production, comparing latency
484.
by welder 5y ago
LightSail didn't exist yet when we switched to DigitalOcean.
485.
by welder 5y ago
No, pengaru's right. They're not mutually exclusive. For ex: making the site faster and more stable means I spend less time fixing and keeping things running and more time building new features.
486.
by welder 5y ago
It's just me nobody else is working on WakaTime, so it's ok. Also, S3 scales beautifully and has been the best database decision ever. Built-in reliability and redundancy. SSDB has also been wonderful. It's just as easy and p
487.
by welder 5y ago
Oh, in that case I still prefer using S3 because it scales and has built-in redundancy and reliability.
488.
by welder 5y ago
No, I'm not using S3 as a CDN I'm using it like a database. S3 works wonderfully as a database.
489.
by welder 5y ago
SSDB doesn't support every single Redis command, it's only a subset. Most people don't even know about SSDB. There's power behind the Redis snowball. We've switched to SSDB, but with that we lost the Redis community
490.
by welder 5y ago
The main difference is you can use existing code/libraries without changing anything if you're already using Redis. That was a big win for us, since we could just swap out the Redis instances that suffered from low hit rates (beca
491.
by welder 5y ago
Yep, it's true SSDB is a hidden gem that deserves more exposure. LevelDB/RocksDB has been around for a while with benchmarks. Here's a related reply about it from earlier https://news.ycombinator.com/item?id=2
492.
by welder 5y ago
The economics don't work out on that... we migrated away from our $3k/mo AWS bill back in the day to save $2k/mo with DO. Going back to EC2 would save $200/mo but cost $2,000/mo in compute.
493.
by welder 5y ago
Totally get that. The journey here wasn't straightforward. I've tried dual-writing in production to many databases including Cassandra, RethinkDB, CockroachDB, TimescaleDB, and more. Haven't tried ClickHouse/VictoriaMetr
494.
by welder 5y ago
How much experience bootstrapping a product do you have? It's not always a choice between spending time optimizing vs. new features. Sometimes you have time for both. Sometimes optimization means a more stable product that frees up you
495.
by welder 5y ago
Yep, someone suggested MinIO in the comments https://news.ycombinator.com/item?id=26956330
496.
by welder 5y ago
There's a post about another Redis-clone on the front page today called SSDB. Seems to be a common occurrence: https://news.ycombinator.com/item?id=19213261
497.
by welder 5y ago
Because files are hard, a lot of projects use a file system library called LevelDB/RocksDB [1] They interact with the library instead of directly with files. CockroachDB and SSDB are examples of projects using LevelDB. [1] https:&#x2F
498.
by welder 5y ago
They're talking about the Redis Virtual Memory feature not Linux Swap: https://redis.io/topics/virtual-memory
499.
by welder 5y ago
The magic of Google's LevelDB? Seriously though, I don't know and I'm not sure if it's an accurate benchmark. I just know our statsd metrics show it performing just as fast as official Redis on much larger data sets, so
500.
by welder 5y ago
Hadn't heard of MinIO. We looked into another S3 caching solution that someone linked in comments here, but it came down to we already used SSDB in production. Also, a personal preference of mine is to keep all logic in the same layer:
501.
by welder 5y ago
Cache invalidation is much easier with SSDB than Nginx or Varnish. Http proxies are good for public facing uses but not a good use case here.
502.
by welder 5y ago
Back in the day I did use EC2, but to get the same performance (especially SSD IOPs) it cost a lot more than DigitalOcean. Back then, the monthly bill for EC2 was over $3,000/mo. Switching to DigitalOcean we got better performance for
503.
by welder 5y ago
Yep, that's why we only use Spaces for backups. https://wakatime.com/blog/46-latency-of-digitalocean-spaces-...
504.
by welder 5y ago
We're a one-man shop, so the cost savings made sense in the long run. I already use SSDB and Redis in production so I'm very familiar with supporting it.
505.
by welder 5y ago
That's kind of what we're doing, until we start using SSDB Clustering. Filesystem as cache would be good for a local same-server cache, but it doesn't scale well when used over the network.
506.
by welder 5y ago
Because we invalidate some SSDB keys in various cases, for ex: like old code stats data incoming, renaming a project, etc. Http proxies with etag seem like a good public-facing cache option, but our caching is internal to our DigitalOcean s
507.
by welder 5y ago
Very cool! Thanks for sharing, I hadn't seen this before. Edit: Actually I looked at this before using SSDB and it was overly complicated for my use case. We went with SSDB because we already used Redis and SSDB, were very familiar wit
508.
by welder 5y ago
Sure, I meant a mirror is just a cache for data that doesn't change. I edited the original comment, because it doesn't help explain why we used SSDB here.
509.
by welder 5y ago
Our S3 costs were between $400/mo. - $800/mo. After SSDB it's more stable around $200/mo, so a conservative number is $200/mo savings. A related comment about our costs: https://news.ycombinator.com/
510.
by welder 5y ago
The majority of code stats (the data being stored) is received for the current day. Older data still comes in, but in that case we just update the SSDB cache after writing the new data to S3. The reason we don't use SSDB as the main so

More ›