5 ms·

This is great work. Stuff I've been thinking about for a number of years but never actually doing. I wish there were something like this for Python. I know it

This is great work. Stuff I've been thinking about for a number of years but never actually doing. I wish there were something like this for Python. I know it's not pure, but it's my drug of choice. Anyhow, kudos.

aria 14y ago
Graph would be possible, but not as clean in Python. Consider switching up drugs. Clojure is way more potent man.
- msandford 14y ago
  Yeah agreed. But I can ready Python (and many non-functional languages) effortlessly and Clojure (and most other functional languages) take WAY more effort for me to read. I mean huge amounts. Enough to where I've never been able to make a serious effort since getting anything done takes forever.
  - obviouslygreen 14y ago
    I'm currently going through Clojure Programming (O'Reilly) to try and get into it... so far it's pretty good. I think it's going to take a couple weeks to get to the "AHA" moment, but the language itself makes a lot of sense and is well-explained... it seems like something that could be phenomenally useful once you get over the initial "holy crap, what am I looking at."
  - babs474 14y ago
    It's interesting because I had trouble reading lisp until somebody told me that you don't read lisp by worrying about parentheses, but by reading it as an indentation level aware language. Just like Python. Here is a more in depth link: http://stackoverflow.com/questions/1894209/how-to-read-mentally-lisp-clojure-code http://stackoverflow.com/questions/1894209/how-to-read-menta... but the idea is, delete all the parentheses from you mind, the code should still make sense. The parentheses are to help the computer out, not you.
    - grayrest 14y ago
      To follow up on this, I configure my editor to de-emphasize parens: http://gr.ayre.st/s/images/parens.png http://gr.ayre.st/s/images/parens.png I can still see them if I need to but they have the same visual weight as tabs/end of line spaces. I like this because it makes it feel more like python to me.
  - dllthomas 14y ago
    Not necessarily the best strategy, but having ever implemented a parser for a compiler for any "ordinary" language makes lisps much clearer - you're just hand-writing the AST.
- msutherl 14y ago
  How do people get around the fact that so much of what you would want to do has a well-supported solution in Python while in Clojure you're to a large extent on your own?
  - prospero 14y ago
    There's the entire JVM ecosystem available. Batteries are very much included.
    - msutherl 14y ago
      Yeah, but do people actually use stuff from Java-land in their web-apps? Seems like it would be a bummer to deal with.
      - w01fe 14y ago
        Yes, we do! Using Java from Clojure is much more pleasant than using it from Java IMO :)
agentultra 14y ago
You can do push-based dataflow graphs in Python rather easily (especially in 3.3+ using `yield from`). Coroutines work especially well for this.
- aria 14y ago
  Definitely not the same thing. Take a look at our earlier graph post. http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation http://blog.getprismatic.com/blog/2013/2/1/graph-abstraction...
  - agentultra 14y ago
    I might have to read the source of your implementation, but in what way is it different? I'll grant that Python makes it difficult to wrap it up in a nice DSL. I'm building a stream-processing system much like Riemann in Python and the user configuration is built on co-routines. The stream is a graph of co-routines essentially (although typically with only one input and many possible buckets). I tend to think of it as data flowing through functions. It might look like: stream = when(lambda event: event['metric'] > 2.0, by(lambda event: (event['host'], event['metric']), rollup(5, email("example@foo.com")), every(7200, alert("555-555-5555")))) Which is a very simple graph of co-routines. And maybe that's what's different from you library (ie: it's not doing anything with intermediate values, but Python has a rich number of libraries providing various executor strategies). Anyway I wasn't suggesting Graph was somehow trivial or something. The OP claims to be a Python programmer and I was suggesting Python options.
tlrobinson 14y ago
Here's my attempt in JavaScript: https://gist.github.com/tlrobinson/b991855433de688bf4e5 https://gist.github.com/tlrobinson/b991855433de688bf4e5
phreeza 14y ago
There is the Modular Data Processing Toolkit[0] which allows for graph based computation. Not sure if it uses many performance enhancing tricks as in the OP though. [0] http://mdp-toolkit.sourceforge.net/ http://mdp-toolkit.sourceforge.net/

aria 14y ago

Graph would be possible, but not as clean in Python. Consider switching up drugs. Clojure is way more potent man.

msandford 14y ago
Yeah agreed. But I can ready Python (and many non-functional languages) effortlessly and Clojure (and most other functional languages) take WAY more effort for me to read. I mean huge amounts. Enough to where I've never been able to make a serious effort since getting anything done takes forever.
- obviouslygreen 14y ago
  I'm currently going through Clojure Programming (O'Reilly) to try and get into it... so far it's pretty good. I think it's going to take a couple weeks to get to the "AHA" moment, but the language itself makes a lot of sense and is well-explained... it seems like something that could be phenomenally useful once you get over the initial "holy crap, what am I looking at."
- babs474 14y ago
  It's interesting because I had trouble reading lisp until somebody told me that you don't read lisp by worrying about parentheses, but by reading it as an indentation level aware language. Just like Python. Here is a more in depth link: http://stackoverflow.com/questions/1894209/how-to-read-mentally-lisp-clojure-code http://stackoverflow.com/questions/1894209/how-to-read-menta... but the idea is, delete all the parentheses from you mind, the code should still make sense. The parentheses are to help the computer out, not you.
  - grayrest 14y ago
    To follow up on this, I configure my editor to de-emphasize parens: http://gr.ayre.st/s/images/parens.png http://gr.ayre.st/s/images/parens.png I can still see them if I need to but they have the same visual weight as tabs/end of line spaces. I like this because it makes it feel more like python to me.
- dllthomas 14y ago
  Not necessarily the best strategy, but having ever implemented a parser for a compiler for any "ordinary" language makes lisps much clearer - you're just hand-writing the AST.
msutherl 14y ago
How do people get around the fact that so much of what you would want to do has a well-supported solution in Python while in Clojure you're to a large extent on your own?
- prospero 14y ago
  There's the entire JVM ecosystem available. Batteries are very much included.
  - msutherl 14y ago
    Yeah, but do people actually use stuff from Java-land in their web-apps? Seems like it would be a bummer to deal with.
    - w01fe 14y ago
      Yes, we do! Using Java from Clojure is much more pleasant than using it from Java IMO :)

agentultra 14y ago

You can do push-based dataflow graphs in Python rather easily (especially in 3.3+ using `yield from`). Coroutines work especially well for this.

aria 14y ago
Definitely not the same thing. Take a look at our earlier graph post. http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation http://blog.getprismatic.com/blog/2013/2/1/graph-abstraction...
- agentultra 14y ago
  I might have to read the source of your implementation, but in what way is it different? I'll grant that Python makes it difficult to wrap it up in a nice DSL. I'm building a stream-processing system much like Riemann in Python and the user configuration is built on co-routines. The stream is a graph of co-routines essentially (although typically with only one input and many possible buckets). I tend to think of it as data flowing through functions. It might look like: stream = when(lambda event: event['metric'] > 2.0, by(lambda event: (event['host'], event['metric']), rollup(5, email("example@foo.com")), every(7200, alert("555-555-5555")))) Which is a very simple graph of co-routines. And maybe that's what's different from you library (ie: it's not doing anything with intermediate values, but Python has a rich number of libraries providing various executor strategies). Anyway I wasn't suggesting Graph was somehow trivial or something. The OP claims to be a Python programmer and I was suggesting Python options.

tlrobinson 14y ago

Here's my attempt in JavaScript: https://gist.github.com/tlrobinson/b991855433de688bf4e5 https://gist.github.com/tlrobinson/b991855433de688bf4e5

phreeza 14y ago

There is the Modular Data Processing Toolkit[0] which allows for graph based computation. Not sure if it uses many performance enhancing tricks as in the OP though. [0] http://mdp-toolkit.sourceforge.net/ http://mdp-toolkit.sourceforge.net/