10 ms·

I get where you're coming from, but I think that folks often underestimate the maintenance burden of additional components in their system. Sure, if your use c

I get where you're coming from, but I think that folks often underestimate the maintenance burden of additional components in their system. Sure, if your use case requires it, use Redis for sessions or RabbitMQ for queues. But you can also use a library with postgres, or even write the 40 lines of code yourself. Each component has its own debugging requirements and tooling. Each component adds a bunch of complexity. Sometimes it's worth it, sometimes it's not. It's not as clear-cut as you're making it be. There are pros and cons to both alternatives.

xwolfi 4y ago
Those 40 lines of codes give me nightmare. What if the dev disappear and we know only the C binary, source disappeared ? Little tiny custom stop gaps are the worst in a bit system.
serverholic 4y ago
I actually had this debate at work a few years ago. We needed a queue system and I wrote a postgres prototype in about 100 lines of code that would have worked fine for our use-case and would have likely scaled with us for years of growth. My boss said he just wanted to install the best thing and be done with it forever. So we ended up spending 10x the time and wrote like 10x the code to integrate a third-party solution and we only ended up using the most basic features. Not to mention additional infrastructure.
- jbverschoor 4y ago
  In other words: boss has no idea, boss doesn’t trust your technical nor business skills Conclusion: abandon this company.. you’re probably just a cost center and nagging headache in his eyes. If you’re not valued or respected, there’s only way way up, and that’s out
- grenoire 4y ago
  I think there exists a midway, where you rely on the heavily-tested open source solution that implements what you want, and a sprinkle more. RabbitMQ for example is that layer that your 40 lines of code would do, but pays off dividends in good nights' sleep.
  - stolsvik 4y ago
    RabbitMQ is not free. You need to learn it, configure it, understand it, interface towards it, install it, upgrade it. Lots and lots of things.
- WJW 4y ago
  So those 1000 lines of code (your prototype was 100 lines and this was 10x that) would still have been a week tops, right? That doesn't seem too bad a price to pay for scaling potential, even if in hindsight it wasn't needed. You can't always know in advance which features you are going to need and which ones are superfluous.
  - shakow 4y ago
    If what you need is a simple work queue, you really need to scale very far away (in the magnitude of several thousands of requests per second) to overgrow a solution built on top of PostgreSQL.
- antonvs 4y ago
  People underestimate how bad databases are at implementing queuing systems. I’ve helped replace such systems multiple times. But it doesn’t need to require 10x the time or code. That just sounds like overengineering. Database-as-queue vs. 10x ultimate solution is a false dichotomy.
  - dalyons 4y ago
    Yeah they suck at it. There are so many good queuing systems, and if you’re on any cloud there’s gunna be a cheap managed queue that is super reliable and near zero effort. If you get beyond trivial scale just use a managed queue.
  - shakow 4y ago
    > how bad databases are at implementing queuing systems. What difficulties did you meet?
  - jbverschoor 4y ago
    People overestimate their requirements, because almost nobody had ANY idea what computers are capable of, and that big tech is HUUUUGE compared to the rest. You are not Google and you are not Steve Jobs
deleted 4y ago
[deleted]
SpeedilyDamage 4y ago
It takes effectively zero effort to maintain my redis instance in AWS, so I’m not sure you’ve got a real argument here…
- efxhoy 4y ago
  Every added service also gets multiplied by the number of environments you need. Sure, prod is just one. But you need a staging env too. And CI needs the service to run the tests. And you probably want a feature-branch environment. And local development of course. It adds up and every service you add gets multiplied by the number of environments.
  - koliber 4y ago
    And each developer has their own dev env. QA’s may also run things locally. And then there is monitoring, dealing with security issues, and upgrading. It all multiplies. Elsewhere people are debating whether Redis adds that much more overhead. I think they are missing the wider point. It’s about how many pieces you will have. 1 is simpler than 2. 2 is simpler than 3. 3 is simpler than 12. Nowadays there are so many great specialized tools that do certain things really well. And for most use cases, you don’t need them. And when a time comes that you do start needed them, you add them then. And people will grumble about the idiot who implemented full text search inside of Postgres, while being completely blind about the 7 years of saved time NOT managing elasticsearch across 38 environments and quarterly upgrades.
- mikl 4y ago
  SaaS is nice, but you still have to worry about Redis drivers, local development set-ups, etc. And unless you can make do with AWS’ free tier, that Redis instance isn’t free.
- roflyear 4y ago
  This doesn't apply to everything. But yeah redis does what it does really well. I almost feel like I want to spin redis up before I spin a DB up. It is excellent tech. Can use it for caching. Sessions. Distributed locks (niche but when you need it it's fantastic). But yeah I think this is more applied to the folks who decide to spin up a large service when they just need a messaging layer that their DB can solve for them. It has non insignificant cost. Redis is pretty low maintenance tho in many configs (k8s, on a vm, or a service like on aws have all basically been zero maintenance for me).

xwolfi 4y ago

Those 40 lines of codes give me nightmare. What if the dev disappear and we know only the C binary, source disappeared ? Little tiny custom stop gaps are the worst in a bit system.

serverholic 4y ago

I actually had this debate at work a few years ago. We needed a queue system and I wrote a postgres prototype in about 100 lines of code that would have worked fine for our use-case and would have likely scaled with us for years of growth. My boss said he just wanted to install the best thing and be done with it forever. So we ended up spending 10x the time and wrote like 10x the code to integrate a third-party solution and we only ended up using the most basic features. Not to mention additional infrastructure.

jbverschoor 4y ago
In other words: boss has no idea, boss doesn’t trust your technical nor business skills Conclusion: abandon this company.. you’re probably just a cost center and nagging headache in his eyes. If you’re not valued or respected, there’s only way way up, and that’s out
grenoire 4y ago
I think there exists a midway, where you rely on the heavily-tested open source solution that implements what you want, and a sprinkle more. RabbitMQ for example is that layer that your 40 lines of code would do, but pays off dividends in good nights' sleep.
- stolsvik 4y ago
  RabbitMQ is not free. You need to learn it, configure it, understand it, interface towards it, install it, upgrade it. Lots and lots of things.
WJW 4y ago
So those 1000 lines of code (your prototype was 100 lines and this was 10x that) would still have been a week tops, right? That doesn't seem too bad a price to pay for scaling potential, even if in hindsight it wasn't needed. You can't always know in advance which features you are going to need and which ones are superfluous.
- shakow 4y ago
  If what you need is a simple work queue, you really need to scale very far away (in the magnitude of several thousands of requests per second) to overgrow a solution built on top of PostgreSQL.
antonvs 4y ago
People underestimate how bad databases are at implementing queuing systems. I’ve helped replace such systems multiple times. But it doesn’t need to require 10x the time or code. That just sounds like overengineering. Database-as-queue vs. 10x ultimate solution is a false dichotomy.
- dalyons 4y ago
  Yeah they suck at it. There are so many good queuing systems, and if you’re on any cloud there’s gunna be a cheap managed queue that is super reliable and near zero effort. If you get beyond trivial scale just use a managed queue.
- shakow 4y ago
  > how bad databases are at implementing queuing systems. What difficulties did you meet?
- jbverschoor 4y ago
  People overestimate their requirements, because almost nobody had ANY idea what computers are capable of, and that big tech is HUUUUGE compared to the rest. You are not Google and you are not Steve Jobs

deleted 4y ago

[deleted]

SpeedilyDamage 4y ago

It takes effectively zero effort to maintain my redis instance in AWS, so I’m not sure you’ve got a real argument here…

efxhoy 4y ago
Every added service also gets multiplied by the number of environments you need. Sure, prod is just one. But you need a staging env too. And CI needs the service to run the tests. And you probably want a feature-branch environment. And local development of course. It adds up and every service you add gets multiplied by the number of environments.
- koliber 4y ago
  And each developer has their own dev env. QA’s may also run things locally. And then there is monitoring, dealing with security issues, and upgrading. It all multiplies. Elsewhere people are debating whether Redis adds that much more overhead. I think they are missing the wider point. It’s about how many pieces you will have. 1 is simpler than 2. 2 is simpler than 3. 3 is simpler than 12. Nowadays there are so many great specialized tools that do certain things really well. And for most use cases, you don’t need them. And when a time comes that you do start needed them, you add them then. And people will grumble about the idiot who implemented full text search inside of Postgres, while being completely blind about the 7 years of saved time NOT managing elasticsearch across 38 environments and quarterly upgrades.
mikl 4y ago
SaaS is nice, but you still have to worry about Redis drivers, local development set-ups, etc. And unless you can make do with AWS’ free tier, that Redis instance isn’t free.
roflyear 4y ago
This doesn't apply to everything. But yeah redis does what it does really well. I almost feel like I want to spin redis up before I spin a DB up. It is excellent tech. Can use it for caching. Sessions. Distributed locks (niche but when you need it it's fantastic). But yeah I think this is more applied to the folks who decide to spin up a large service when they just need a messaging layer that their DB can solve for them. It has non insignificant cost. Redis is pretty low maintenance tho in many configs (k8s, on a vm, or a service like on aws have all basically been zero maintenance for me).