3 ms·
I actually had this debate at work a few years ago. We needed a queue system and I wrote a postgres prototype in about 100 lines of code that would have worked
by serverholic 4y ago
I actually had this debate at work a few years ago. We needed a queue system and I wrote a postgres prototype in about 100 lines of code that would have worked fine for our use-case and would have likely scaled with us for years of growth.
My boss said he just wanted to install the best thing and be done with it forever. So we ended up spending 10x the time and wrote like 10x the code to integrate a third-party solution and we only ended up using the most basic features. Not to mention additional infrastructure.
- jbverschoor 4y agoIn other words: boss has no idea, boss doesn’t trust your technical nor business skills Conclusion: abandon this company.. you’re probably just a cost center and nagging headache in his eyes. If you’re not valued or respected, there’s only way way up, and that’s out
- grenoire 4y agoI think there exists a midway, where you rely on the heavily-tested open source solution that implements what you want, and a sprinkle more. RabbitMQ for example is that layer that your 40 lines of code would do, but pays off dividends in good nights' sleep.
- stolsvik 4y agoRabbitMQ is not free. You need to learn it, configure it, understand it, interface towards it, install it, upgrade it. Lots and lots of things.
- WJW 4y agoSo those 1000 lines of code (your prototype was 100 lines and this was 10x that) would still have been a week tops, right? That doesn't seem too bad a price to pay for scaling potential, even if in hindsight it wasn't needed. You can't always know in advance which features you are going to need and which ones are superfluous.
- shakow 4y agoIf what you need is a simple work queue, you really need to scale very far away (in the magnitude of several thousands of requests per second) to overgrow a solution built on top of PostgreSQL.
- antonvs 4y agoPeople underestimate how bad databases are at implementing queuing systems. I’ve helped replace such systems multiple times. But it doesn’t need to require 10x the time or code. That just sounds like overengineering. Database-as-queue vs. 10x ultimate solution is a false dichotomy.
- dalyons 4y agoYeah they suck at it. There are so many good queuing systems, and if you’re on any cloud there’s gunna be a cheap managed queue that is super reliable and near zero effort. If you get beyond trivial scale just use a managed queue.
- shakow 4y ago> how bad databases are at implementing queuing systems. What difficulties did you meet?
- jbverschoor 4y agoPeople overestimate their requirements, because almost nobody had ANY idea what computers are capable of, and that big tech is HUUUUGE compared to the rest. You are not Google and you are not Steve Jobs