4 ms·

I would prefer Python or Node over JVM for a light weight microservice. I would choose a NoSQL store for something that needs to scale.

pc86 6y ago
Both SQL and NoSQL will scale fine for 99% of apps (and let's be honest, ~90% of apps don't need any scale). Your data schema/format should be dictated by the data itself more than some handwavey "we might need to scale" requirement that isn't true the majority of the time.
- LordHumungous 6y ago
  K but what if you your service needs to scale.
jpgvm 6y ago
Python and Node both have highly fragmented ecosystems, low quality packages, poor tooling and neither are statically typed, capable of multi-threading in a meaningful way and other then their niches (data science for Python and client-side web for JS) are worse at everything than JVM or CLR. I understand their attraction, they are "simple" and "easy" languages. But they are not boring. If anything they create a ton of distinctly un-boring problems like build chains, packaging, framework of the week is no longer supported (or has a new incompatible version). Engineers may like these for whatever reasons, especially before they have tried the higher quality tooling provided by real boring tech but inevitably they lead to projects that either run behind time because of technical issues not related to business problems and/or rot after development is paused and are hard to resuscitate and maintain afterwards. Also NoSQL doesn't "scale" better than SQL. NoSQL stores can scale better in certain data access patterns but if your data model is inherently relational and you implement it on top of NoSQL all you have done is reinvented a relational store in your application model and likely crippled integrity, scalability and performance in one fell swoop.
- LordHumungous 6y ago
  I disagree with pretty much everything in your comment. > NoSQL stores can scale better in certain data access patterns Exactly.
donor20 6y ago
And that's where folks go wrong. NoSQL is not boring tech. As soon as you need to scale, it is MORE likely, not less, that you will end up in a weird state in your app, user enrollment flow etc. NoSQL makes scaling MUCH harder in my view. SQL tools give you a common interface many folks can engage with, and many tools. I really want to talk to people building piles of spaghetti on noSQL because "it scales". At some point you've just go to start tearing your hair out. Postgresql can do 1.5M queries (read/write) per second on OLTP loads on one box just to get started. If you really need more you can get extremely high with replicas. Then application design comes into play. I'm tired of folks picking "noSQL" so things can scale. Dealing with all the edges cases as these things scale is a nightmare (plug mongodb and friends seem to fall over MUCH more often, recovery is miserable with them etc).
- LordHumungous 6y ago
  How do you load balance across replicas? How do you shard across replicas?
  - jpgvm 6y ago
    Load balancing across read replicas is usually handled by your connection bouncer, say pgBouncer/pgPool/etc though you may also do some amount of more complex both L3 and L7 balancing if you get really big. Sharding is usually a matter of actually splitting the masters. There are many techniques for achieving this. If you want the database to do all the work you will probably want to use something like Citus for PostgreSQL or Vitess for MySQL. You can also build bespoke topologies using PostgreSQL logical or MySQL binlog replication. Failing that you can do application level sharding if you don't want the database doing anything fancy for you and manage each shard as an independent database cluster. By the time you actually need to do this you will be able to afford one of these options. :) In the meantime you will save a ton of CPU, storage and development time vs a "NoSQL" store as databases like PostgreSQL are inherently more efficient for all but the simplest of KV access patterns.
    - deleted 6y ago
      [deleted]

pc86 6y ago

Both SQL and NoSQL will scale fine for 99% of apps (and let's be honest, ~90% of apps don't need any scale). Your data schema/format should be dictated by the data itself more than some handwavey "we might need to scale" requirement that isn't true the majority of the time.

LordHumungous 6y ago
K but what if you your service needs to scale.

jpgvm 6y ago

Python and Node both have highly fragmented ecosystems, low quality packages, poor tooling and neither are statically typed, capable of multi-threading in a meaningful way and other then their niches (data science for Python and client-side web for JS) are worse at everything than JVM or CLR. I understand their attraction, they are "simple" and "easy" languages. But they are not boring. If anything they create a ton of distinctly un-boring problems like build chains, packaging, framework of the week is no longer supported (or has a new incompatible version). Engineers may like these for whatever reasons, especially before they have tried the higher quality tooling provided by real boring tech but inevitably they lead to projects that either run behind time because of technical issues not related to business problems and/or rot after development is paused and are hard to resuscitate and maintain afterwards. Also NoSQL doesn't "scale" better than SQL. NoSQL stores can scale better in certain data access patterns but if your data model is inherently relational and you implement it on top of NoSQL all you have done is reinvented a relational store in your application model and likely crippled integrity, scalability and performance in one fell swoop.

LordHumungous 6y ago
I disagree with pretty much everything in your comment. > NoSQL stores can scale better in certain data access patterns Exactly.

donor20 6y ago

And that's where folks go wrong. NoSQL is not boring tech. As soon as you need to scale, it is MORE likely, not less, that you will end up in a weird state in your app, user enrollment flow etc. NoSQL makes scaling MUCH harder in my view. SQL tools give you a common interface many folks can engage with, and many tools. I really want to talk to people building piles of spaghetti on noSQL because "it scales". At some point you've just go to start tearing your hair out. Postgresql can do 1.5M queries (read/write) per second on OLTP loads on one box just to get started. If you really need more you can get extremely high with replicas. Then application design comes into play. I'm tired of folks picking "noSQL" so things can scale. Dealing with all the edges cases as these things scale is a nightmare (plug mongodb and friends seem to fall over MUCH more often, recovery is miserable with them etc).

LordHumungous 6y ago
How do you load balance across replicas? How do you shard across replicas?
- jpgvm 6y ago
  Load balancing across read replicas is usually handled by your connection bouncer, say pgBouncer/pgPool/etc though you may also do some amount of more complex both L3 and L7 balancing if you get really big. Sharding is usually a matter of actually splitting the masters. There are many techniques for achieving this. If you want the database to do all the work you will probably want to use something like Citus for PostgreSQL or Vitess for MySQL. You can also build bespoke topologies using PostgreSQL logical or MySQL binlog replication. Failing that you can do application level sharding if you don't want the database doing anything fancy for you and manage each shard as an independent database cluster. By the time you actually need to do this you will be able to afford one of these options. :) In the meantime you will save a ton of CPU, storage and development time vs a "NoSQL" store as databases like PostgreSQL are inherently more efficient for all but the simplest of KV access patterns.
  - deleted 6y ago
    [deleted]