13 ms·

If I see premature scaling it's usually at the database side of things. Inexperienced devs can be quick to dismiss a SQL database and go with something based on

If I see premature scaling it's usually at the database side of things. Inexperienced devs can be quick to dismiss a SQL database and go with something based on key value pair caching. So I agree with the article keep it simple and easy to implement, worry about scaling later.

mkarliner 9y ago
Yep. Not long ago I asked a dev why he had suddenly introduced Riak into the system. 'Because it's cool' Oy.
- mgkimsal 9y ago
  Similar experiences. Worked on a project, and a 'lead' guy left with .. a few weeks of a node/koa codebase in place that only he really knew. Between the rest of us we had loads of C#, PHP and Java experience (multiples years each over 3 people). "Let's rebuild something based on what the bulk of us already know," I suggested. (Keep in mind what we had was a few weeks of prototype code that was not in production.) "No, I really want to learn more node," was the only other input, and that 'won' the decision. 18 months later... "yeah, koa isn't really a goof fit for our use cases".
  - collyw 9y ago
    We have one of those guys on our team. He is friends with one of the founders so nothing he says seems to be wrong.
  - softawre 9y ago
    This is where as a tech manager you step in and push people the right way. Fun tech is fun, business is business. C# is a great language, JVM is great w/ Kotlin/Clojure, both of those have tons of web servers.
    - cutler 9y ago
      Whilst the JVM may be established, ie. boring technology Kotlin and Clojure certainly aren't. It's now 8 years since Clojure 1.0 and there is a grand total of 15 Clojure jobs listed on Indeed.co.uk, 12 of which are in London. Kotlin boasts 2.
  - mgkimsal 9y ago
    edit: ack - should be "good" fit, not "goof" fit, obviously. And yes, per another point in the replies, someone else should have stepped in to decide differently when there was a tie. I could possibly have pushed harder, but the 'bozo bit' basically was flipped on that project, and much of what I lobbied for was ignored, at least in part because I was the one saying it.
    - didip 9y ago
      If the LoC is low for that Koa project, you should rewrite it quickly using something that the team familiar with. Otherwise, it will grow into a monster that nobody want to deal with.
      - mgkimsal 9y ago
        You'd think that, right? That was another part of my argument. And yes, it's ... not quite grown in to a monster, but few on the team really know how to deal with it. That actually came up a couple months ago - I heard a comment along the lines of "well, no one's really fully understood how to deal with that koa code since the beginning". Even the koa stuff, I think was done by someone who had used it a couple of times previously and liked it, but... no long term track record of using it in production. So... about 2 years later, there's some traction from folks to rebuild it in something else. The latest shiny object is AWS lambda.
- mkosmo 9y ago
  I've asked many devs why they chose Mongo over a RDBMS. They can never give me a technical answer. It's always 'it's cool' or because they just read something about it on some non-technical blog. It usually only takes me a minute or two to ask them a future function question that's not reasonably attainable (and performant) with a NoSQL solution... so they wind up going back to whatever RDBMS is their standard.
  - zzzcpan 9y ago
    I don't think I ever heard a good technical reason to use a traditional RDBMS. And as a default choice it is not a very good one, when something distributed can deliver an easy to maintain resilient solution pretty much out of the box.
    - SOLAR_FIELDS 9y ago
      I'm sure many can be produced, but the first off of the top of my head is accessibility for quasi-developers. If you ever want to hire some analyst roles you are going to find that it's a lot easier to get people that have at least some familiarity with SQL instead of having to train them on whatever custom NoSQL flavor you are using. Of course you could solve that by having an ETL process dump your stuff into SQL (we keep our stuff in Cassandra and dump it into SQL for analysis), but that's another ETL you need to keep track of and in our case we use a quite complex data model and the ETL only supports a subset of it. Another one just off the top of my head is joins. Some data is just inherently better suited in normal form.
    - milquetoastaf 9y ago
      Relational databases can be distributed lol. You are clueless.
      - sctb 9y ago
        Please stop violating the guidelines like this. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- michaelmior 9y ago
  For a side project, that's a fairly reasonable answer. For a business, rather scary.
sbarre 9y ago
Yup! MySQL + Redis (for query/object caching and queues) gets you the best of both worlds in a lot of scenarios, and is reasonably easy to set up and manage. And it will probably scale better than you expect at first!
- zzzcpan 9y ago
  "MySQL + Redis" I think this would be 10x more expensive to set up, manage, maintain and use than something like Riak.
  - libertymcateer 9y ago
    I am using postgres and redis - they are both trivial to set up and cost about a few lattes a month for a fairly scalable solution. If I had huge traffic, a few hundred bucks a month would be totally adequate - and this is running on the too-expensive heroku. So, unless Riak costs pennies and takes literal minutes to set up, the 10x claim won't hold. Riak looks like it would be overpowered by my needs - but then again, I learn something new every day.
  - Posibyte 9y ago
    Can you explain why you think that? I would imagine using two rather popular systems in a somewhat common combination would be a boon to maintainability and set up, if at least for the large pool of developers and online resources already available for it. But then again, I could be super mistaken. I'm interested in seeing another side of it.
    - sbarre 9y ago
      This was also my thought when I made the comment... I am sure that for certain use cases, there are more tailored solutions, but "SQL + keystore" seems like a fairly universal, affordable and versatile stack.
      - user5994461 9y ago
        They are different databases. Riak is a key value store, not unlike the core of redis. The SQL database is obviously relational. Ignoring the difference for a minute. If you can fit data in Redis, chances are that they can fit in riak and a RDBMS is not needed at all. Why Riak only then? It is one of the very few databases that can be clustered. It is sharded, multi-master, highly available, to talk in buzzwords. It is very reliable and very robust, you can add/remove nodes as you wish, replication is highly configurable, etc... Postgre/Mysql/MariaDB + Redis have no HA. They are both the disaster in disaster recovery.
awinder 9y ago
I always see this discussion cast as one between scalability (KV) and dev efficiency (SQL), but other reasons to adopt columnar databases include: 1. Difficulty in transitioning a system with SQL-Gone-Wild to a system that can be implemented with other dbs without complexity overdrive 2. Escalating costs to get any additional availability guarantees out of SQL (availability & cost are key business issues) In AWS & Google Cloud, you're going to need to pay outright for capacity as well as capacity devoted db recovery (a replica / set of replicas in another availability zone). On the other side with something like dynamo / cloud datastore, your payment model is based on requests / sec and the costs for replicated data. I'm just saying that there's nothing simple to deciding where price / availability / dev productivity balances will lead you to, and you can be an inexperienced developer who chooses either side of these technologies without really understanding the full story.
hashkb 9y ago
A potential client told me he was planning on having three SQL databases. I asked him why, what each would be for, and he didn't really know. Just a vague sense that his app was going to be too much for a single db.
balls187 9y ago
> Inexperienced devs can be quick to dismiss a SQL...So I agree with the article keep it simple and easy to implement If you're inexperienced with relational databases, perhaps a Key-value store with a simple API is much simpler and easier to implement.

mkarliner 9y ago

Yep. Not long ago I asked a dev why he had suddenly introduced Riak into the system. 'Because it's cool' Oy.

mgkimsal 9y ago
Similar experiences. Worked on a project, and a 'lead' guy left with .. a few weeks of a node/koa codebase in place that only he really knew. Between the rest of us we had loads of C#, PHP and Java experience (multiples years each over 3 people). "Let's rebuild something based on what the bulk of us already know," I suggested. (Keep in mind what we had was a few weeks of prototype code that was not in production.) "No, I really want to learn more node," was the only other input, and that 'won' the decision. 18 months later... "yeah, koa isn't really a goof fit for our use cases".
- collyw 9y ago
  We have one of those guys on our team. He is friends with one of the founders so nothing he says seems to be wrong.
- softawre 9y ago
  This is where as a tech manager you step in and push people the right way. Fun tech is fun, business is business. C# is a great language, JVM is great w/ Kotlin/Clojure, both of those have tons of web servers.
  - cutler 9y ago
    Whilst the JVM may be established, ie. boring technology Kotlin and Clojure certainly aren't. It's now 8 years since Clojure 1.0 and there is a grand total of 15 Clojure jobs listed on Indeed.co.uk, 12 of which are in London. Kotlin boasts 2.
- mgkimsal 9y ago
  edit: ack - should be "good" fit, not "goof" fit, obviously. And yes, per another point in the replies, someone else should have stepped in to decide differently when there was a tie. I could possibly have pushed harder, but the 'bozo bit' basically was flipped on that project, and much of what I lobbied for was ignored, at least in part because I was the one saying it.
  - didip 9y ago
    If the LoC is low for that Koa project, you should rewrite it quickly using something that the team familiar with. Otherwise, it will grow into a monster that nobody want to deal with.
    - mgkimsal 9y ago
      You'd think that, right? That was another part of my argument. And yes, it's ... not quite grown in to a monster, but few on the team really know how to deal with it. That actually came up a couple months ago - I heard a comment along the lines of "well, no one's really fully understood how to deal with that koa code since the beginning". Even the koa stuff, I think was done by someone who had used it a couple of times previously and liked it, but... no long term track record of using it in production. So... about 2 years later, there's some traction from folks to rebuild it in something else. The latest shiny object is AWS lambda.
mkosmo 9y ago
I've asked many devs why they chose Mongo over a RDBMS. They can never give me a technical answer. It's always 'it's cool' or because they just read something about it on some non-technical blog. It usually only takes me a minute or two to ask them a future function question that's not reasonably attainable (and performant) with a NoSQL solution... so they wind up going back to whatever RDBMS is their standard.
- zzzcpan 9y ago
  I don't think I ever heard a good technical reason to use a traditional RDBMS. And as a default choice it is not a very good one, when something distributed can deliver an easy to maintain resilient solution pretty much out of the box.
  - SOLAR_FIELDS 9y ago
    I'm sure many can be produced, but the first off of the top of my head is accessibility for quasi-developers. If you ever want to hire some analyst roles you are going to find that it's a lot easier to get people that have at least some familiarity with SQL instead of having to train them on whatever custom NoSQL flavor you are using. Of course you could solve that by having an ETL process dump your stuff into SQL (we keep our stuff in Cassandra and dump it into SQL for analysis), but that's another ETL you need to keep track of and in our case we use a quite complex data model and the ETL only supports a subset of it. Another one just off the top of my head is joins. Some data is just inherently better suited in normal form.
  - milquetoastaf 9y ago
    Relational databases can be distributed lol. You are clueless.
    - sctb 9y ago
      Please stop violating the guidelines like this. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
michaelmior 9y ago
For a side project, that's a fairly reasonable answer. For a business, rather scary.

sbarre 9y ago

Yup! MySQL + Redis (for query/object caching and queues) gets you the best of both worlds in a lot of scenarios, and is reasonably easy to set up and manage. And it will probably scale better than you expect at first!

zzzcpan 9y ago
"MySQL + Redis" I think this would be 10x more expensive to set up, manage, maintain and use than something like Riak.
- libertymcateer 9y ago
  I am using postgres and redis - they are both trivial to set up and cost about a few lattes a month for a fairly scalable solution. If I had huge traffic, a few hundred bucks a month would be totally adequate - and this is running on the too-expensive heroku. So, unless Riak costs pennies and takes literal minutes to set up, the 10x claim won't hold. Riak looks like it would be overpowered by my needs - but then again, I learn something new every day.
- Posibyte 9y ago
  Can you explain why you think that? I would imagine using two rather popular systems in a somewhat common combination would be a boon to maintainability and set up, if at least for the large pool of developers and online resources already available for it. But then again, I could be super mistaken. I'm interested in seeing another side of it.
  - sbarre 9y ago
    This was also my thought when I made the comment... I am sure that for certain use cases, there are more tailored solutions, but "SQL + keystore" seems like a fairly universal, affordable and versatile stack.
    - user5994461 9y ago
      They are different databases. Riak is a key value store, not unlike the core of redis. The SQL database is obviously relational. Ignoring the difference for a minute. If you can fit data in Redis, chances are that they can fit in riak and a RDBMS is not needed at all. Why Riak only then? It is one of the very few databases that can be clustered. It is sharded, multi-master, highly available, to talk in buzzwords. It is very reliable and very robust, you can add/remove nodes as you wish, replication is highly configurable, etc... Postgre/Mysql/MariaDB + Redis have no HA. They are both the disaster in disaster recovery.

awinder 9y ago

I always see this discussion cast as one between scalability (KV) and dev efficiency (SQL), but other reasons to adopt columnar databases include: 1. Difficulty in transitioning a system with SQL-Gone-Wild to a system that can be implemented with other dbs without complexity overdrive 2. Escalating costs to get any additional availability guarantees out of SQL (availability & cost are key business issues) In AWS & Google Cloud, you're going to need to pay outright for capacity as well as capacity devoted db recovery (a replica / set of replicas in another availability zone). On the other side with something like dynamo / cloud datastore, your payment model is based on requests / sec and the costs for replicated data. I'm just saying that there's nothing simple to deciding where price / availability / dev productivity balances will lead you to, and you can be an inexperienced developer who chooses either side of these technologies without really understanding the full story.

hashkb 9y ago

A potential client told me he was planning on having three SQL databases. I asked him why, what each would be for, and he didn't really know. Just a vague sense that his app was going to be too much for a single db.

balls187 9y ago

> Inexperienced devs can be quick to dismiss a SQL...So I agree with the article keep it simple and easy to implement If you're inexperienced with relational databases, perhaps a Key-value store with a simple API is much simpler and easier to implement.