5 ms·

You should really just foot the bill for AWS. This is going to be technical debt for you later on that will need to be paid. As a software/systems engineer I'll

by kaydub 5y ago

You should really just foot the bill for AWS. This is going to be technical debt for you later on that will need to be paid. As a software/systems engineer I'll walk away from anywhere doing stuff like this to save a few dollars. I've worked a lot of janky places and now I work somewhere that goes the extra mile and puts that little bit extra in to do things right the first time. I know the on-going costs of doing stuff like this as well as the fact that you will have to clean it up at some point. EDIT: Not necessarily YOU will have to clean it up at some point, but the next guy most likely will.

welder 5y ago
It's just me nobody else is working on WakaTime, so it's ok. Also, S3 scales beautifully and has been the best database decision ever. Built-in reliability and redundancy. SSDB has also been wonderful. It's just as easy and powerful as Redis, without the RAM limitation. I've had zero issues with SSDB regarding maintenance and reliability, as long as you increase your ulimit [1] and run ssdb-cli compact periodically [2]. I've tried dual-writing in production to many databases including Cassandra, RethinkDB, CockroachDB, TimescaleDB, and more. So far, this setup IS the right solution for this problem. [1] https://wakatime.com/blog/47-maximize-your-concurrent-web-server-connections https://wakatime.com/blog/47-maximize-your-concurrent-web-se... [2] ssdb-cli compact runs garbage collection and needs to be run every day/week/month depending on your writes load, or you'll eventually run out of disk space. Check the blog post for my crontab automating ssdb-cli compact.
- philihp 5y ago
  I don't think there's anything wrong with this. The correct answer is not always "outsource to AWS"; dependence on AWS (and any external dependency, especially those you have to pay for) is also tech debt. Everything you do is tech debt, and since this is just you, building it yourself is not only respectable (here, have an upvote!) but could also perhaps be something you end up doing 2-3 years down the line, and you've just gotten ahead of yourself.
heliodor 5y ago
For many startups, that code that you mentioned as technical debt will be deleted en masse as the business pivots and adapts. For startups, moving fast and getting 80% done in 20% of the time is the correct choice. It's frustrating to look at the result as a software developer, but the engineer is there to serve the business, which pays the salary. As a software engineer, I very much disliked working in engineering teams that were engineering for the sake of engineering. It quickly devolves into meaningless arguments and drama.
PixyMisa 5y ago
I would run a mile from anything that would lock me into AWS. S3 is fine though because there's any number of compatible alternatives, including self-hosted ones like MinIO.
- throwaway823882 5y ago
  When was the last time you desperately needed to get off of a single AWS service and couldn't?
  - sintaxi 5y ago
    Im in the process of purging data off s3. I suspect it will be about two weeks of work and I calculate it will save me roughly 250k over the "lifetime" of my product. What I think too few people understand about s3 is that costs on S3 compound. Every month you not only pay for data you stored in the current month but you also pay for the data stored in every month previously.
    - throwaway823882 5y ago
      S3 is a de facto standard supported by multiple vendors and open source technology. You can pick lots of other cloud vendors and use S3 there, or you can self-host S3. It's one of many AWS technologies that is not vendor-locked.
      - welder 5y ago
        Yep, for ex: we use the exact same code to get/put files to DigitalOcean Spaces... the only thing that changes is a config value for the S3 endpoint. S3 is basically an open standard. The only vendor lock-in would be the time/money it takes to copy your data out of S3, but that's the same for anywhere you store your data... even when self-hosting.
simtel20 5y ago
At around, say, 10k requests/sec, by far the dominant cost is the s3 api cost for GET requests at $.0004 per 1000 calls. Having a cache saves millions of dollars per month in this scenario, and if scaling is expected, then planning ahead like this is a good idea.

by kaydub 5y ago

welder 5y ago

It's just me nobody else is working on WakaTime, so it's ok. Also, S3 scales beautifully and has been the best database decision ever. Built-in reliability and redundancy. SSDB has also been wonderful. It's just as easy and powerful as Redis, without the RAM limitation. I've had zero issues with SSDB regarding maintenance and reliability, as long as you increase your ulimit [1] and run ssdb-cli compact periodically [2]. I've tried dual-writing in production to many databases including Cassandra, RethinkDB, CockroachDB, TimescaleDB, and more. So far, this setup IS the right solution for this problem. [1] https://wakatime.com/blog/47-maximize-your-concurrent-web-server-connections https://wakatime.com/blog/47-maximize-your-concurrent-web-se... [2] ssdb-cli compact runs garbage collection and needs to be run every day/week/month depending on your writes load, or you'll eventually run out of disk space. Check the blog post for my crontab automating ssdb-cli compact.

philihp 5y ago
I don't think there's anything wrong with this. The correct answer is not always "outsource to AWS"; dependence on AWS (and any external dependency, especially those you have to pay for) is also tech debt. Everything you do is tech debt, and since this is just you, building it yourself is not only respectable (here, have an upvote!) but could also perhaps be something you end up doing 2-3 years down the line, and you've just gotten ahead of yourself.

heliodor 5y ago

For many startups, that code that you mentioned as technical debt will be deleted en masse as the business pivots and adapts. For startups, moving fast and getting 80% done in 20% of the time is the correct choice. It's frustrating to look at the result as a software developer, but the engineer is there to serve the business, which pays the salary. As a software engineer, I very much disliked working in engineering teams that were engineering for the sake of engineering. It quickly devolves into meaningless arguments and drama.

PixyMisa 5y ago

I would run a mile from anything that would lock me into AWS. S3 is fine though because there's any number of compatible alternatives, including self-hosted ones like MinIO.

throwaway823882 5y ago
When was the last time you desperately needed to get off of a single AWS service and couldn't?
- sintaxi 5y ago
  Im in the process of purging data off s3. I suspect it will be about two weeks of work and I calculate it will save me roughly 250k over the "lifetime" of my product. What I think too few people understand about s3 is that costs on S3 compound. Every month you not only pay for data you stored in the current month but you also pay for the data stored in every month previously.
  - throwaway823882 5y ago
    S3 is a de facto standard supported by multiple vendors and open source technology. You can pick lots of other cloud vendors and use S3 there, or you can self-host S3. It's one of many AWS technologies that is not vendor-locked.
    - welder 5y ago
      Yep, for ex: we use the exact same code to get/put files to DigitalOcean Spaces... the only thing that changes is a config value for the S3 endpoint. S3 is basically an open standard. The only vendor lock-in would be the time/money it takes to copy your data out of S3, but that's the same for anywhere you store your data... even when self-hosting.

simtel20 5y ago

At around, say, 10k requests/sec, by far the dominant cost is the s3 api cost for GET requests at $.0004 per 1000 calls. Having a cache saves millions of dollars per month in this scenario, and if scaling is expected, then planning ahead like this is a good idea.