4 ms·
I've seen Tibco Rendezvous used in manufacturing. ~300 megabytes per hour of raw log generated 24/7/365 by tools and control systems in a factory setting. Proba
by bob1029 3y ago
I've seen Tibco Rendezvous used in manufacturing. ~300 megabytes per hour of raw log generated 24/7/365 by tools and control systems in a factory setting. Probably on the order of 10k+ participants in the pub/sub network.
If you are running something like a factory where thousands of independent systems need to communicate in some way, this kind of tech starts to look like the only option.
If you are orchestrating the concerns of 5-10 services, I think you are making your life harder than it needs to be with ESB-style abstractions. Direct method invocation is much more reliable than whatever any one of these vendors could ever sell you. Put all your services into one exe. If you can't be bothered to use one language/repo, there are still ways to achieve this.
The real architect's dilemma is shoving one's edifice-constructing ego into a box long enough to produce a useful shack for the business.
- strangemonad 3y agoOr NATS
- relay23 3y agoor kafka, or any other publish/subscribe system
- kuchenbecker 3y agoOur prod cluster generates that about every minute at O(1M) qps. We JUST turned on remote Logs because until now Kafka didn't have capacity.
- inkyoto 3y agoTIBCO Rendezvous is tech from 1998/99, millions of messages per second didn't exist at the time. Only NYSE and NASDAQ were capable of producing millions of events back then (still not by minute nor by second). TIBCO Rendezvous was one of the first successful large scale, low latency and near real-time pub/sub implementations, and it had a very efficient, Avro like, wire level serialisation format that made messages very compact and efficient to deliver. It was very popular in finance, banking and manufacturing, and is all but legacy now.
- kuchenbecker 3y ago$$$ not capability. We have ~50 hosts that generate up to TB per day in just logs and 50k hosts that generate O(200mb/day). The large hosts ssh and grep works surprisingly well, but the smaller hosts is the real benefit. Hard to justify a team of 7 burning several million in just Logging infra costs.
- blindriver 3y agoMy previous company's Kafka cluster was handing 20 million messages per second 5 years ago, and dozens of petabytes of data per day. Maybe your particular cluster that didn't have the capacity to handle 1M qps, but Kafka easily had that capacity years ago.
- _a_a_a_ 3y agoI have to ask, what value is this adding business-wise to store so much?
- sharms 3y agoNot OP but I think it isn't always about storing, but having a log of events which get routed, processed, and aggregated in many cases.
- blindriver 3y agoKafka when used correctly is the like the nervous system for your entire company. You use it like a message bus and dump every single thing into it, and you can extract it out at your leisure, but mostly real-time. It completely transforms how you do services in a company, but it also means you have to invest a lot of money and manpower into maintaining it because it is mission critical.
- kuchenbecker 3y agoIt was quota and hardware, not ability. This is a single service onboarding and they need the hardware. And at that scale, we need to grep the logs so the downstream need the ability to process that volume, which it couldn't until about 2 years ago.
- gunapologist99 3y agoNATS was developed by an ex-TIBCO engineer.
- xupybd 3y agoI use NATS to create a secure pipe between buildings and the cloud. I don't need speed but I do need routing and I do need a security layer. Nats just works and took very little setup.
- opentokix 3y agoYour usecase is EASY - its EASY! This is a SOLVED usecase.
- fnxjdjc 3y ago[dead]
- continuitylimit 3y agoIt’s not apparent as a dilemma until the said architect has spent years -convinced- that the grand edifice is “good architecture”, and finally matured as a practitioner. Only after that phase passes is there an actual ego-driven dilemma, strictly speaking.