sudhirj

50 ms·

541.
by sudhirj 6y ago
Added NSQ notes.
542.
by sudhirj 6y ago
The example of factory QA is really an OLTP / near real time system, for which queues are definitely a bad idea. Any system where you "would much rather an exception be raised immediately and crater the caller" is by definiti
543.
by sudhirj 6y ago
Yeah... let me think about this. Not sure how to boil this down to generalized notes or advice.
544.
by sudhirj 6y ago
Thanks for the points, yeah, definitely need to cover poison pills and dead letter queues. Re. size the main examples I'm looking at are SQS, Google's Pub/Sub etc which are pretty hard to fill up or overload. Will think about
545.
by sudhirj 6y ago
I learnt a ton form this talk by Tim Bray about how Amazon does event driven architecture. The talk isn't directly about queues, but they play a massive part in the narrative, including notes about idempotency, duplication, FIFO (or la
546.
by sudhirj 6y ago
Might add a note, but I don't want to muddy the waters. Technically, neither is the Redis Streams implementation. But they both quack like a queue, and are meant to be used like a queue, so will probably talk about implementation detai
547.
by sudhirj 6y ago
Ah, got it. I’m not entirely sure I want to put that in, though. The message I’m “preaching” is that you have full decoupling with queues - not sure that should be diluted. Wouldn’t this kind of checking defeat the purpose of using the queu
548.
by sudhirj 6y ago
Yeah, forgot to talk about dead letters, will add a section. And notes for Pulsar, once I read up on it.
549.
by sudhirj 6y ago
Thought it was already implicit in the capacity decoupling, could you elaborate a bit more / add an example. I’ve always thought back pressure isn’t applicable to queues because there’s no pressure to begin with, front or otherwise.
550.
by sudhirj 6y ago
Yeah, a similar nightmare prompted a lot of this reasearch. A colleague would argue for what felt like hours against SQS because it was “random” ordering, and kept demanding we use something that had strict FIFO. Despite the clients who pla
551.
by sudhirj 6y ago
As a protocol? Not sure how it would deserve more mention than JSON or some other wire protocol. Almost every example I’ve given will use a client SDK, which may or many not use MQTT under the hood. Anything specific you think is worth poin
552.
by sudhirj 6y ago
Don’t quite get the gist of this... a consensus based system like Raft or a CRDT operation will converge on to exactly one state, yes. I haven’t seen these concepts applied to a message queue. One could build a consensus or CRDT system that
553.
by sudhirj 6y ago
Might not even need to use Redis. If the message has a proper idempotent ID a transactional database is more than enough. If the consumer is running MySQL/Postgres/DynamoDB etc nothing else is needed.
554.
by sudhirj 6y ago
Will add NSQ notes. Researched it once for distributed web socket pub sub, did find it pretty interesting.
555.
by sudhirj 6y ago
The way I’ve handled incoming webhooks is to run a Lambda+API Gateway to put them into an SQS queue, and the inverse, SQS to Lambda, for sending webhooks out. This was possible only because AWS provides these services, of course. If you’re
556.
by sudhirj 6y ago
If the poster is a point-haired-middle-manager archetype, maybe. More often though, it’s very useful for a manager or team leader to ensure that everyone on the team has a common base of knowledge, concepts and terminology. That way the tea
557.
by sudhirj 6y ago
Thanks, fixing.
558.
by sudhirj 6y ago
AWS makes the same claim with FIFO SQS, and maybe I’m getting it wrong, but these claims 1) have a lot of caveats and 2) only work inside the boundaries of the messaging system. There’s a note in the next paragraph about how systems manage
559.
by sudhirj 6y ago
It’s Ghost (Ghost.org). The theme is Dawn (it’s on the official theme marketplace). My pandemic project is to offer Ghost hosting ( https://ghosting.dev ) - let me know if you want a free personal setup.
560.
by sudhirj 6y ago
Interesting. This is actually sort of a distillation of all the new hire training / discussions I’ve done. It’s as much a “things I wish someone had told me 15 years ago” thing too.
561.
by sudhirj 6y ago
Author here. This was self-posted, so hope you find it useful. Please ask me anything.
562.
by sudhirj 6y ago
Will post a note.
563.
by sudhirj 6y ago
Thanks. Yeah, DIY was pretty much the norm. Jobs as table rows, in memory arrays, the whole deal. Spent quite a bit of my life writing glue and management code that should have just been queues.
564.
by sudhirj 6y ago
I assume you’re saying the theme doesn’t work for you? Thanks for the feedback, will customise it to something more readable, better colours, nicer fonts etc soon.
565.
The Big Little Guide to Message Queues(sudhir.io)
295 points by sudhirj 6y ago|108 comments
566.
by sudhirj 6y ago
If the fund raises are being done on favourable terms, which is quite likely given they’re likely doing billions in revenue, they probably still maintain voting control. They won’t be subject to much pressure if they are.
567.
by sudhirj 6y ago
I see this reacting depending on the interests, hobbies or identity of the person. The same people who decide to get a professional to paint their homes or install a heater can / will also rant about how you can save money by assemblin
568.
by sudhirj 6y ago
Just to clarify for those who might not get it, a new major-ish version of Ruby releases every Christmas.
569.
by sudhirj 6y ago
Preventing stagnation and maintaining backwards compatibility aren’t the same thing. Go has a very strong backwards compatibility guarantee, and since they made the guarantee the entire compiler has been rewritten and there have been plenty
570.
by sudhirj 6y ago
Isn’t this all the more reason to use it for those who need automated ACH? I’d assume they fixed the bug. It seems counterintuitive but like AWS, Stripe is anti fragile. Each problem makes the system better the next day, and so more worthy

More ›