6 ms·
Python Generator Tricks for Systems Programmers
- iseff 19y agoThe author of this, David Beazley, was a professor of mine for two courses as an undergrad at UChicago. He taught Networking and Operating Systems and was one of the top 3 professors in the CS department. He left teaching for a while, but it looks like he's back now, which is extremely encouraging for all CS students! If anyone has the chance to listen to him speak or teach, I highly recommend it. (Also, if anyone has the chance to see him perform jazz music, I highly recommend that, too!)
- kirubakaran 19y agoThis tutorial is awesome. It is not just the content. The sequence and the presentation is just plain awesome. You are a lucky basketball.
- jcl 19y agoI was a little surprised that the genlog.py program is said in the slides to be slightly faster than the nongenlog.py... That is, several chained generator expressions actually ran faster than an explicitly written "for" loop. I suppose it may have something to do with using the C-implemented sum() function rather than doing a running total in Python; nevertheless, I expected more overhead from the generators. Very cool.
- jmtulloss 19y agoStackless Python makes a lot of these possible without fancy generator tricks. Not that I don't love fancy generator tricks, but I think the stackless model is more intuitive.
- bayareaguy 19y agoCan you provide a more explicit example, perhaps a Stackless alternative to one of the ones from Beazley's page?
- jmtulloss 18y agoThe most obvious is coroutines: def ping(): while ping_channel.receive(): #blocks here print "PING" pong_channel.send("from ping") def pong(): while pong_channel.receive(): print "PONG" ping_channel.send("from pong") Most of the things that he has in there don't need to be done with generators, though generators make it more efficient and arguably cleaner. However, sometimes generators are just a twisted workaround for controlling program flow. Channels and tasklets in Stackless express program flow very well, in my opinion. You can see more of what stackless has to offer here: http://members.verizon.net/olsongt/stackless/why_stackless.html http://members.verizon.net/olsongt/stackless/why_stackless.h...
- mrtron 19y agoVery cool, just going through it now. It has the feel of my course slides in university, but much more to the point and with better examples.
- omakase 19y agonice examples. just a note, pickle shouldn't be used for serialization if the source is untrusted.
- pmorici 19y agoSource of what? Do you care to elaborate...?
- kirubakaran 19y agohttp://www.python.org/doc/2.2.3/lib/pickle-sec.html http://www.python.org/doc/2.2.3/lib/pickle-sec.html
- ntoshev 19y agoSo add another step: sign the pickled data and check that upon arrival.
- omakase 19y agosorry i didn't elaborate, it isn't a question of signing requests, kirubakaran posted a link below that explains the basics (unpickling can create objects and run potentially harmful methods) depending on what you are sending there are several alternatives for serializing data. there are a handful of json libraries to take a look at and some good benchmarks if you google for them.
- icky 19y ago> sorry i didn't elaborate, it isn't a question of signing requests, kirubakaran posted a link below that explains the basics (unpickling can create objects and run potentially harmful methods) Parent poster meant signing the pickled data as a text stream, not as a collection of unserialized python objects. If the signatures don't match, you just discard the data, and don't unpickle it. This guarantees that (as long as your keys aren't compromised, and your signing algorithm is unbroken) you will never unpickle tampered-with data.