10 ms·
Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before i
by rdtsc 14y ago
Sigh, another Async framework.
Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle https://github.com/saucelabs/monocle
Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that.
I am surprised that gevent was dismissed. Ok, there is also eventlet, if someone doesn't like gevent. Monkey patching is scary? Is it really _that_ scary? Most sane and portable IO code probably runs on that today. Why? Because there is no need to create a parallel world of libraries. Write a test does it pass? Does it handle your use case? I'll take not knowing if my green threads switch tasks and add another green thread lock, instead of doubling my code size with yields, callbacks and futures.
Let's talk about Twisted (sorry couldn't resist WAT video reference). I remember for years searching for parallel libraries to parse exotic protocols. Regular Python library is there, but no, can't use that, sorry. Gotta go find or write one that returns Deferreds. You add a single Twisted module in your code, good luck! -- it ripples all the way to the top through your API and you are doomed being locked into the Twisted world forever.
When gevent and eventlet came around it was like a breath of fresh air. This is what sane concurrent IO looks like in Python:
http://eventlet.net/doc/examples.html http://eventlet.net/doc/examples.html
My fear is that many will just say fuck it, I'll just use Go/Rust/Erlang for IO bound concurrent problems.
It is nice having a benevolent dictator, except when he goes a little crazy, then dictatorship doesn't sounds so much fun anymore.
- pjscott 14y agoYES. Thank you for saying this. Some of the async code that I maintain uses Twisted and some of it uses Eventlet, and the difference between them is night and day. The code using Eventlet is so much cleaner, so much easier to maintain, and (oddly enough) so much less magical than the Twisted stuff. This was written by the same people, and they're all really good programmers, so the obvious confounding variables are not an issue here. Eventlet and Gevent are just so much better. Worried about monkey-patching? Then only monkey-patch the parts you need to be asynchronous. Worried about magic that you don't understand? Have a look at the code; the magic is actually pretty straightforward after you've paid a little attention to the man behind the curtain. If you're interested in async stuff for Python, I urge you to have a look at Eventlet or Gevent.
- hosay123 14y agoSane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran, etc) In the case of async I'm glad to see a from scratch implementation for the standard library. It's a weird area that necessitates some constructions that there is no really standard Python style for. You only need to look at Twisted and e.g. its method chaining to realize this stuff would need a thorough sanity rework before it ever became standard anyway. Also, most other implementations take the approach of building their own little world. This is definitely true of Twisted. You write code for Twisted, not for Python. Gevent at least doesn't suffer from this.
- Ixiaus 14y agoI used gevent at first for a project that needed Async I/O and it worked really well, but then I switched to Erlang and I realized how poor a choice Python is for such tasks. The language really needs to be designed from the start for it (like Go, Rust, Erlang &c... Haskell wasn't designed from start for it but because of it's functional purity, bolting it on was "natural" but - it isn't so for Python IMHO).
- rdtsc 14y agoAgreed, I am learning Erlang currently as well.
- craigyk 14y agoYeah, if you're going to have to do async to be performant than it better be pretty pervasive throughout all the libraries. Bonus points if the language supports syntax to make async easier as well. Node is beating out Python for server stuff not simply because it is "async", but because it is so much FASTER. The speed of V8 vs. CPython is a big part of that. In fact, vanilla JS doesn't have much to make async programming particularly easy: it has verbose function declaration and no yield mechanisms. Even library-level solutions like promises are merely 'OK'. Still, it is easier to build a fast server that can handle streams in Node than it is in Python. Async Python? I'll just stick to async JS in that case.
- simeonf 14y ago>Sigh, another Async framework. >Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yeah but - this will be in stdlib. And I think the hope is one-event-loop-to-rule-them-all will let the various frameworks play nice with each other. For instance - Glyph just mentioned to me that he doesn't use IPython any more to work on Twisted code because IPython now has a Tornado event loop which can conflict with the Twisted code he's playing with... I don't think the hope is that this will be better than twisted or gevent in terms of implementation (obviously the API will be nicer than twisted given Python 3) just that it will be the standard by virtue of being in stdlib.
- pekk 14y agoPython 3 won't make a Twisted-like API any better than Twisted's API is.
- njharman 14y agoDid you actually read the slides? Most of your comments and complaints are addressed and sound like non-nonsensical fud in context. I wasn't a fan of stdlib async until reading these slides.
- pekk 14y agoThis isn't FUD. The API itself (as mentioned in the slides, and the draft PEP) looks like Twisted. If you are a fan of this API, try Twisted
- stuaxo 14y agoEh, the whole point is that this is at a lower level to all of those. All the above frameworks will be ported to this and become interoperable - in a similar way to how wsgi works for the web.
- ekimekim 14y agoIt won't be useful if the standard interface is, basically, Twisted. Stack-based solutions like gevent, eventlet get screwed over by this.
- lucian1900 14y agoNo, they don't. Gevent and eventlet can simply use this instead of their own event loops. In addition, they could install their own event loops. Tulip separates API from event loop mechanism.
- deleted 14y ago[deleted]
- twism 14y agoTornado has a well documented (and least complicated regarding gevent & twisted are concerned IMHO) but not well known/used "inline callback" with yields to simplify async code. http://www.tornadoweb.org/en/branch2.3/gen.html http://www.tornadoweb.org/en/branch2.3/gen.html From there, I made a wrapper to wrap potentially blocking logic in threads so I could have code that isn't async/non-blocking behave like it was. https://gist.github.com/anonymous/5190528 https://gist.github.com/anonymous/5190528
- simonw 14y agoIf you're using monkey patching in the standard library for a language, something has gone very wrong.
- jessaustin 14y agoThis is really a matter of taste. You should at least be aware that you're monkeypatching, and code and test accordingly. Many people have good results from monkeypatching, and even more have good results from calling well-written-and-tested libraries that monkeypatch.
- codygman 14y agoBut monkey patching turns python from explicit to implicit. It just doesn't feel pythonic to me, and I don't think I'm alone in this. A big reason I use (and enjoy using) python is because it doesn't feel like a "bolted on" solution. All current concurrency options for python feel bolted on to me personally. The python internals weren't designed for this, which is the reason they have to use monkey patching. It doesn't mean that you can't make something work, but it means the language/interpreter sure aren't going to help you make it work. I just couldn't write something which depends on speed and concurrency in python right now, knowing there are solutions much better designed for the problem. Python holds a special place in my heart, but unicode and concurrency aren't so good right now. However, this future async support and unicode support in python 3 very much excites me!
- rdtsc 14y ago> It just doesn't feel pythonic to me, and I don't think I'm alone in this. But how does a tangled mess of callback1 callback2 callback3 feel when all you want to do is do a couple of db reads and writes while processing a simple shopping cart. Is that Pythonic? > The python internals weren't designed for this, which is the reason they have to use monkey patching. So fix the internals. Here is a practical way people use Python every day, make that the default, don't revert to some academic or callback mechanism. > I just couldn't write something which depends on speed and concurrency in python right now, knowing there are solutions much better designed for the problem. See that is what saddens me. gevent and eventlet do let you write reasonably good and concise IO concurrent code. Some have run large sites and deployments with it. I haven't found any major slowdowns or downsides to switch yet. Because I it is easy and simple to experiment, I'll always try Python first, even though later I might switch to Go or Erlang.
- codygman 14y agoUnicode support, concurrency/parallelism, and monkey patching were very big reasons for me using Go rather than python.
- visitorx 14y agoseconded, I doubt many a sane person would use Python for IO bound concurrency, they should definitely be looking at Rust (in the future slightly) and Erlang (currently the defacto standard for easy-of-use concurrency).
- eurleif 14y agoOne thing I love about gevent is that you can share code between async and non-async. Most of my project benefits from async IO, but there's one part that needs to use a lot of CPU within a single process. So that part uses multithreaded Jython, the rest uses gevent, the common code is shared, and it all just works.
- rdtsc 14y agoExactly, I was surprised how in the whole "ideas" mailing list discussion Guido had and in other forums that is dismissed as "meh" not even mentioned. Discussions quickly turn theoretical and academic. "But you don't know when your green threads will switch, man, so I'll add yields in there for you". Yes, and then also make sure there is a complete universe of libraries. Python is awesome not just because it is fun to write little hello world examples in (so is Logo), it is awesome because it is easy to GetShitDone(TM) quickly. The big part of GetShitDone(TM) quickly is reusing libraries not rewriting everything from scratch. Using an exotic database for some reason -- great. Found a Python library to interact with it -- great. Oh but my framework is based on Deferreds and this one was written without Deferreds or this one returns Futures. Sorry, go write your own from scratch. This has been the story of my life for 5+ years search or re-writing Twisted version of already existing libraries. Now at least just adopt Twisted and go with it if they are going this route. But now, they are 'standardizing' on something new. I think had they done this in 2007, yeah rock on, that would have made sense. They didn't. What saved and kept Python on the back-end during the past 5 or so years was greenlet (eventlet and gevent). Guido is kicking all those people in nuts and saying, "no", we'll do Twisted now (with some changes).
- aba_sababa 14y agoThe only reason I'm not doing that is because I'm anchored by Python's stats libraries.