8 ms·
Lahja – A generic event bus implementation written in Python
- curzondax 7y agocannot make sense of this library. whats this for?
- StavrosK 7y agoSeems to be pubsub for asyncio.
- kitd 7y agoNeat. This feels quite a bit like Vert.x from the Java world, which flows data between components via an event bus. Noob python question: does Python support reactive extensions and chaining? That would fit well on top of this too.
- linsomniac 7y agoThere is RxPY: https://rxpy.readthedocs.io/en/latest/ https://rxpy.readthedocs.io/en/latest/
- erdewit 7y agoFor push-type events there is eventkit: https://github.com/erdewit/eventkit https://github.com/erdewit/eventkit (I'm the author).
- leetrout 7y agoThat’s pretty neat! I’ve used Redis in the past for lightweight pub/sub and zeromq for IPC before we had adopted Python 3 (and I left that job before then). I hated juggling threads and queues before that and then having to run Redis just to run your program was a pain. The syntax looks similar enough to both of those to get started quickly and easily. I’ve been writing Go so much I don’t know how much this is “needed” in Python but the examples using context managers ring the “it’s Pythonic” bell.
- kgraves 7y agoLove to try this out some day, a shame that this isn't in Python 2. EDIT: Thanks HN, try working in an industry where every new project is in Python 2, and have no intention to move on, before saying "moeve to 3", it doesn't help at all.
- solarkraft 7y agoPlease, please don't use Python 2.
- kgraves 7y agoTell that to the industry I work in (Games, Movies & VFX) where most of the tools & 3D software scripting are still in Python 2. Not all of us move fast and break things here, unlike the web dev world.
- ggm 7y agoOh please. The EOL on python2 has been massively pre announced. Games, Movies and VFX can afford the port costs and have a huge liability looming. There was no "move fast and break things" in this python 3
- kgraves 7y ago> Oh please. The EOL on python2 has been massively pre announced. Nope, not when new versions of software comes out with Python 2 still, (can't do anything about those licences bought) small studios cannot afford moving/rewriting tools in P2 to 3, not a priority, rather keep P2.
- joshuamorton 7y agoThen complain to the companies you license software from for continuing to build on a soft-deprecated platform. Don't complain to developers for ignoring a platform whose deprecation was announced over a decade ago. And certainly don't badmouth people who are using the not-deprecated tool as being ok with unstable infra.
- steventhedev 7y agoFrom glancing at the code, this is an observer pattern over asyncio and unix sockets. That means you're unable to communicate outside the current machine (short of hacks like ssh socket forwarding), but adding support for tcp endpoints shouldn't be too hard. I'd be interested to see how well this holds up under load, especially compared to other common event bus patterns.
- pipermerriam 7y agoWe've talked a bit about adding TCP support and really like some of the options it opens up. However, we have to move away from the use of pickle for event serialization before we can do that, otherwise we would have some security issues with remote code execution.
- sandGorgon 7y agowhat is the usecase for something like this versus celery/rq ?
- nurettin 7y agoAFAIK, celery doesn't provide a message passing bus between main process and sub-processes.
- zbentley 7y agoWell, it technically does (through its invocations of billiard, it's multiprocessing fork), but it's pretty ad hoc and not exposed for much user control. So you're basically right, I'm just being pedantic.
- nurettin 7y agoYes ok initial parameter passing and gathering results may be considered a message bus. I was going to mention that, but I got lazy.
- sandGorgon 7y agoDon't understand that clearly. Can't I put something in the queue in the main process and read it from subprocesses ?
- nurettin 7y agoIs that going to provide bi-directional communication between main and subprocesses while they are running? If it does, disregard what I wrote.
- sandGorgon 7y agoSure. You can put results in an "output" queue (which is backed by Redis) that is read by the main application. Incidentally this is how you do a notification on your UI .
- d__k 7y agoWhat is the closest (conceptual) analogue for this model: - https://github.com/jodal/pykka https://github.com/jodal/pykka (Python) - https://github.com/quantmind/pulsar https://github.com/quantmind/pulsar (Python) - https://github.com/akka/akka https://github.com/akka/akka (Java) - https://github.com/eclipse-vertx/vert.x https://github.com/eclipse-vertx/vert.x (Java) - etc.
- turbinerneiter 7y agoHow does this compare to D-Bus and when would you use which (apart from D-Bus obviously being Linux-only)?
- adamcharnock 7y agoI've been working on something similar for a while now called Lightbus: https://lightbus.org https://lightbus.org Like Lahja this too is rather much in the alpha stage, but I have been using it in production for over a year now. I mention it here in case it is interesting to anyone, interest/feedback always welcome!
- poorman 7y agoPair this with Arrow IPC and you're golden https://arrow.apache.org/docs/python/ipc.html https://arrow.apache.org/docs/python/ipc.html
- eerikkivistik 7y agoAny chance the name is a play on words from the Estonian word "Lahja" as in light-weight?
- cxcorp 7y agoGood question, could also be the Finnish word "lahja" that translates to "gift"
- xlog 7y agoCould also mean 'talent'.
- maouida 7y ago"lahja" in Arabic means dialect
- kyjan 7y agoIt's also a pretty popular girl's name ;)
- dcbadacd 7y agoAlso means "weak" (in concentration) in Estonian though
- shreyanshd 7y ago“Lah ja” in Hindi means “take away”
- noisy_boy 7y agoThat sounds more Punjabi. It is "le ja" in Hindi.
- cburgdorf 7y agoChristoph, founder of the project here. I recently came across the Estonian meaning and jokingly used it as an explanation for the project name when I gave a presentation. However, the real story is that Lahja is the name of my daughter and I selfishly named this project after her :)
- Tinfoilhat666 7y agoIs this somehow connected to processing cryptocurrencies? This is under ethereum repo.
- pipermerriam 7y agoIt's a library we use in the "Trinity" client for the ethereum network. The lahja library itself is however just an event bus and has no cryptocurrency ties other than being used in the Trinity codebase. https://trinity.ethereum.org/ https://trinity.ethereum.org/