bzurak

4 ms·

1.
Fear and Loathing in Python: Building a Distributed Context System for Wool(gist.github.com)
2 points by bzurak 4mo ago|1 comments
2.
by bzurak 4mo ago
For those of you unfamiliar with Wool (which I'm assuming is most of you reading this), it's a distributed execution runtime for Python I've been working on that adds parallelism to Python's existing async primitives, al
3.
Distributed Python with Wool: Hello World(medium.com)
1 points by bzurak 8mo ago|0 comments
4.
by bzurak 8mo ago
I’m not sure why people are assuming I’m pushing for distributed OOP here. Wool is completely unopinionated about how you decide to architect your application - that’s the entire point.
5.
by bzurak 8mo ago
So if you wanted to scale, say, a hyperparameter search across a cluster, what would you prefer?
6.
by bzurak 8mo ago
I have a feeling people will begin to purposely use slightly incorrect grammar to give the impression they are indeed human in their writing.
7.
by bzurak 8mo ago
Wasn’t previously aware of this opinion, but having read it, I can tell you I’m not actually committing this fallacy within the framework- I’m not forcing anyone into making their instance methods RPCs- I leave that up to my users. They are
8.
by bzurak 8mo ago
Fair enough
9.
by bzurak 8mo ago
I think LLMs are incredible tools that I will continue to use unapologetically, but I’m also very particular and not going to be putting my name to AI slop. Those are my genuine thoughts on the matter, they just happened to be cleaned up by
10.
by bzurak 8mo ago
Not on my watch
11.
by bzurak 8mo ago
You mean the cadence where everything is presented in threes? Yeah I hate it too, but beats paying some blogger to do it
12.
by bzurak 8mo ago
Sure, my argument is that those concerns should be composable with the distribution layer, not defined by it. There are plenty of different ways to pass data across a network or between machines, I'm not going to reinvent them all. The
13.
by bzurak 8mo ago
I agree
14.
by bzurak 8mo ago
Yeah but... who does that?
15.
by bzurak 8mo ago
To be clear, not all of your coroutines become distributed. You explicitly declare what you want executed remotely, and you'll probably want your workers running in a VPC - it's just a nifty way of scaling your app horizontally wi
16.
by bzurak 8mo ago
But seriously, I just wanted to have an easy way to parallelize my projects without a bunch of slow, bloated, boilerplate ridden frameworks, and I'm having fun building it. And I must not be the only to have had this desire considering
17.
by bzurak 8mo ago
Is it possible it's both?
18.
by bzurak 8mo ago
I'm running this in a trusted environment, I'm not so ambitious as to try to make this some sort of whacky trustless distributed Python runtime. Just a fun project that's been marinating for a while, and now I have an army of
19.
by bzurak 8mo ago
I guess I'm not old enough.
20.
by bzurak 8mo ago
For one, it's faster than prefect and ray so I can use it for more stuff without having to import a 10000000 line library.
21.
What if Python was natively distributable?(medium.com)
62 points by bzurak 8mo ago|64 comments
22.
by bzurak 8mo ago
I should add- Wool supports ephemeral worker pools, i.e., pools that are spawned by your application directly that live for the life of the WorkerPool context. The limitation right now is that there’s no remote worker factory - you would ne
23.
by bzurak 8mo ago
I wouldn't say it abstracts the locking mechanisms away - if you need synchronization in your app, it's probably best to leave how that's achieved up to the user - what it does is make it possible to contain your business log
24.
Show HN: I built Wool, a lightweight distributed Python runtime(github.com)
15 points by bzurak 8mo ago|3 comments
25.
by bzurak 8mo ago
Thanks for the show hn tip btw
26.
by bzurak 8mo ago
Out of the box, machine-local and LAN workers (via mDNS) are supported, and, yes, the local discovery protocol is effectively just multiprocessing with ser/des taken care of for you (with cloudpickle exclusively, for now). The nice thi
27.
Wool – A no-nonsense distributed Python runtime(github.com)
4 points by bzurak 8mo ago|3 comments