3 ms·

Postgres fills a niche that has a lot going for it. It’s standard SQL with transactions so it’s easy to code against compared to Nosql like Cassandra. There are

by Sevii 9y ago

Postgres fills a niche that has a lot going for it. It’s standard SQL with transactions so it’s easy to code against compared to Nosql like Cassandra. There are lots of powerful extensions like JSON and full text search. And you can scale it pretty far.

rantanplan 9y ago
A niche?
- nucleardog 9y ago
  If you get deep enough into the hipster development movement (or anything) you lose sight of everything that's going on around you.
- ForHackernews 9y ago
  That niche being "anyone who needs to store structured data and reliably retrieve it". It's a small market, but there's demand there.
sidlls 9y ago
Postgres is likely suitable for 90% (or more) of the crap "engineers" currently fall all over themselves to implement Kafka, Spark, and whatever other bizzword laden "big data" or "machine learning" resume driven development for. It's not a niche.
- 3pt14159 9y ago
  I'm still trying to find an engineer that can explain to me why they use Redis for job queuing over Postgres. Redis for HTML caching: Cool. Redis for queueing: Are you really sure you need the performance gain? Really? You can do 10k writes per second with Postgres (100k per second with COPY). If you need more than that fine, but for 99% of web apps out there it's just one more moving piece that probably doesn't get included in your backups.
  - stevenwoo 9y ago
    I was looking at that choice for my latest project and the gem recommended (resque?) only using Redis for the job queuing at the benefit of much higher performance over something like delayed_jobs using postgresql in jobs. It didn't seem worth it as job queueing was not the main feature of the server. I suppose if one were doing some sort of real time feature where you had to queue things off a Rails API response thread it would make sense but wasn't doing that for now. But just judging by the number of downloads, the Redis based job queue is possibly more popular for some reason.
  - rickbergfalk 9y ago
    Proper queuing in RDBMSs can be tricky. 2ndQuadrant has a good write up of what is required to make sure you don't fall into any traps: https://blog.2ndquadrant.com/what-is-select-skip-locked-for-in-postgresql-9-5/ https://blog.2ndquadrant.com/what-is-select-skip-locked-for-... Now if we're just talking about persisting data to somewhere - postgres COPY is pretty amazing.
- orangejewce 9y ago
  Spark is not a buzzword if you do analytics, it's the preferred platform, especially over native Hadoop. Honestly most developers have no idea what the heck Spark is or how to use it. NoSQL/Kafka yes, Analytics/Machine Learning is still far too complicated for most to stand up on their own.
  - jack9 9y ago
    Kafka was useful in a project because of the semantics for what amounts to writing to file from network sockets. Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. If you want a common format to trade with buddies, it's useful now. I expect something else will come along to replace that fad tech.
    - tabacof 9y ago
      >Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. Can you expand on that?
    - mdekkers 9y ago
      Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. You gave a typo there. Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs I have ever come across. There, fixed that for you.
      - jack9 9y ago
        I don't think that's fixed, because it's not what I assert. Any job that is predictable, can be done faster and cheaper with some C (or Go) and ad-hoc delegation to load balanced VMs. Spark is difficult to optimize for consistent processes, poor on resource usage, and requires specialized knowledge. Just pay for someone who has done a little embedded programming and stop creating buzzword jobs because a prototype went to production without comparison.
  - sidlls 9y ago
    Spark has a place: at large scale. For 100s of GB to a few TB of data PostgreSQL works very well. At least, it does for my team. I don't want Spark, Kafka, NoSQL or any other modern fad near my team's data. It's just not appropriate.

by Sevii 9y ago

rantanplan 9y ago

A niche?

nucleardog 9y ago
If you get deep enough into the hipster development movement (or anything) you lose sight of everything that's going on around you.
ForHackernews 9y ago
That niche being "anyone who needs to store structured data and reliably retrieve it". It's a small market, but there's demand there.

sidlls 9y ago

Postgres is likely suitable for 90% (or more) of the crap "engineers" currently fall all over themselves to implement Kafka, Spark, and whatever other bizzword laden "big data" or "machine learning" resume driven development for. It's not a niche.

3pt14159 9y ago
I'm still trying to find an engineer that can explain to me why they use Redis for job queuing over Postgres. Redis for HTML caching: Cool. Redis for queueing: Are you really sure you need the performance gain? Really? You can do 10k writes per second with Postgres (100k per second with COPY). If you need more than that fine, but for 99% of web apps out there it's just one more moving piece that probably doesn't get included in your backups.
- stevenwoo 9y ago
  I was looking at that choice for my latest project and the gem recommended (resque?) only using Redis for the job queuing at the benefit of much higher performance over something like delayed_jobs using postgresql in jobs. It didn't seem worth it as job queueing was not the main feature of the server. I suppose if one were doing some sort of real time feature where you had to queue things off a Rails API response thread it would make sense but wasn't doing that for now. But just judging by the number of downloads, the Redis based job queue is possibly more popular for some reason.
- rickbergfalk 9y ago
  Proper queuing in RDBMSs can be tricky. 2ndQuadrant has a good write up of what is required to make sure you don't fall into any traps: https://blog.2ndquadrant.com/what-is-select-skip-locked-for-in-postgresql-9-5/ https://blog.2ndquadrant.com/what-is-select-skip-locked-for-... Now if we're just talking about persisting data to somewhere - postgres COPY is pretty amazing.
orangejewce 9y ago
Spark is not a buzzword if you do analytics, it's the preferred platform, especially over native Hadoop. Honestly most developers have no idea what the heck Spark is or how to use it. NoSQL/Kafka yes, Analytics/Machine Learning is still far too complicated for most to stand up on their own.
- jack9 9y ago
  Kafka was useful in a project because of the semantics for what amounts to writing to file from network sockets. Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. If you want a common format to trade with buddies, it's useful now. I expect something else will come along to replace that fad tech.
  - tabacof 9y ago
    >Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. Can you expand on that?
  - mdekkers 9y ago
    Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs. You gave a typo there. Spark is a terribly inefficient solution to any known/stable data processing or analytics jobs I have ever come across. There, fixed that for you.
    - jack9 9y ago
      I don't think that's fixed, because it's not what I assert. Any job that is predictable, can be done faster and cheaper with some C (or Go) and ad-hoc delegation to load balanced VMs. Spark is difficult to optimize for consistent processes, poor on resource usage, and requires specialized knowledge. Just pay for someone who has done a little embedded programming and stop creating buzzword jobs because a prototype went to production without comparison.
- sidlls 9y ago
  Spark has a place: at large scale. For 100s of GB to a few TB of data PostgreSQL works very well. At least, it does for my team. I don't want Spark, Kafka, NoSQL or any other modern fad near my team's data. It's just not appropriate.