6 ms·

Event Driven Programming in Clojure

68 points by Mitt 15y ago

prospero 15y ago
Hey, I'm the guy in the video. If anyone has questions, I'd be happy to answer them. Any questions regarding odd numbers will have to wait for my upcoming presentation.
- muyuu 15y ago
  Can you put the slides somewhere else? the link in infoQ is not working for me.
  - prospero 15y ago
    http://github.com/strangeloop/2011-slides/raw/master/Tellman-EventDrivenProgrammingInClojure.pdf http://github.com/strangeloop/2011-slides/raw/master/Tellman...
- andrewcooke 15y ago
  i just had time to watch this; it's very nice work. i guess this is either obvious or wrong, but your async macro reminded me of logic variables (eg in oz) (ie you appear to be rewriting the system so that, effectively, the variables in your dataflow are logic variables, and then adding concurrency to match). that suggests that the debugging issues you have are similar to ones that they have to deal with. i don't know what solutions they have, but if this makes sense, and you haven't talked to van roy et al, it might be worth doing so. (and i guess other parallel logic program(mer)s too - oz is just what i am most familiar with. mercury?).
  - prospero 15y ago
    From now on, I'm going to start all my questions with "this is either obvious or wrong, but ...". I hadn't made the connection between dataflow and logic programming. I'm not sure if the programming practices surrounding logic programming would be useful in this context, but I'll certainly look into it. Thanks for the suggestion.
    - andrewcooke 15y ago
      "logic programming" is a bit misleading. oz (at least) doesn't have built-in DFS like prolog. but it does keep the idea of logic variables, which it uses to make concurrent programming "easy" (in the sense that what you are doing is deadlock free), so it is more similar than you may think...
      - prospero 15y ago
        Okay, I've only ever skimmed CTM before, I'll give it a closer look.
drivebyacct2 15y ago
The odds don't get to drive?
- __abc 15y ago
  Have you ever seen an odd drive? Horrible! HORRIBLE!
grayrest 15y ago
Do you have an overview of how the async macro works? The presentation wasn't long enough to fit in but I'm hoping you have a written explanation somewhere. Cursory search didn't turn up anything. I've seen flows of roughly the same complexity from a variety of abstractions: monadic (F# async), generator/coroutine (Python gevent, node fibers), futures/promises (twisted, mochikit, commonjs q), and callbacks (a million node async flow helpers) but they always involve more code. This looks like auto-transformed futures with error bubbling instead of error callback but I'd like to know how it's actually working. The lack of ceremony/annotation is commendable but I'm concerned it's fragile magic.
- prospero 15y ago
  I haven't really written about this anywhere, and the code in its current state is far from self-documenting, so I'll give it a shot here. The code transformation is pretty much all local. Since everything in Clojure is an expression (this is not true of Python or JS, I'm not sure about F#), I don't need to perform much analysis, I just need to make sure that each expression doesn't care about the difference between a future and a realized value. Let's assume three primitives: * an asynchronous future [1] * something which merges a list of futures and realized values into a single future representing a list of values * something which takes a function and a future, and returns a future representing '(apply fn value-of-future)', that will be realized once the future is realized All we need to do is take every expression where a function is called, merge the arguments together, and apply the final primitive to the expression. The expression will return a future, which can then be picked up by the parent expression, and so on. This glosses over how special forms are handled, and how the 'force' functionality works. Also, as I mentioned in the presentation, how the code executes is pretty opaque, and some sort of higher-order analysis would be nice so that it could be visualized using graphviz or something. This is still an experimental feature, and will probably remain so until the 1.0.0 release of Lamina (I'm working on 0.5.0 right now). I'm not aware of any other macro that rips apart normal Clojure code quite as completely, which means that this is sort of uncharted territory. I want to make sure I've avoided all the pitfalls before calling it ready for production usage. [1] https://github.com/ztellman/lamina/wiki/Introduction https://github.com/ztellman/lamina/wiki/Introduction

68 points by Mitt 15y ago

Hey, I'm the guy in the video. If anyone has questions, I'd be happy to answer them. Any questions regarding odd numbers will have to wait for my upcoming presentation.

muyuu 15y ago
Can you put the slides somewhere else? the link in infoQ is not working for me.
- prospero 15y ago
  http://github.com/strangeloop/2011-slides/raw/master/Tellman-EventDrivenProgrammingInClojure.pdf http://github.com/strangeloop/2011-slides/raw/master/Tellman...
andrewcooke 15y ago
i just had time to watch this; it's very nice work. i guess this is either obvious or wrong, but your async macro reminded me of logic variables (eg in oz) (ie you appear to be rewriting the system so that, effectively, the variables in your dataflow are logic variables, and then adding concurrency to match). that suggests that the debugging issues you have are similar to ones that they have to deal with. i don't know what solutions they have, but if this makes sense, and you haven't talked to van roy et al, it might be worth doing so. (and i guess other parallel logic program(mer)s too - oz is just what i am most familiar with. mercury?).
- prospero 15y ago
  From now on, I'm going to start all my questions with "this is either obvious or wrong, but ...". I hadn't made the connection between dataflow and logic programming. I'm not sure if the programming practices surrounding logic programming would be useful in this context, but I'll certainly look into it. Thanks for the suggestion.
  - andrewcooke 15y ago
    "logic programming" is a bit misleading. oz (at least) doesn't have built-in DFS like prolog. but it does keep the idea of logic variables, which it uses to make concurrent programming "easy" (in the sense that what you are doing is deadlock free), so it is more similar than you may think...
    - prospero 15y ago
      Okay, I've only ever skimmed CTM before, I'll give it a closer look.

drivebyacct2 15y ago

The odds don't get to drive?

__abc 15y ago
Have you ever seen an odd drive? Horrible! HORRIBLE!

grayrest 15y ago

Do you have an overview of how the async macro works? The presentation wasn't long enough to fit in but I'm hoping you have a written explanation somewhere. Cursory search didn't turn up anything. I've seen flows of roughly the same complexity from a variety of abstractions: monadic (F# async), generator/coroutine (Python gevent, node fibers), futures/promises (twisted, mochikit, commonjs q), and callbacks (a million node async flow helpers) but they always involve more code. This looks like auto-transformed futures with error bubbling instead of error callback but I'd like to know how it's actually working. The lack of ceremony/annotation is commendable but I'm concerned it's fragile magic.

prospero 15y ago
I haven't really written about this anywhere, and the code in its current state is far from self-documenting, so I'll give it a shot here. The code transformation is pretty much all local. Since everything in Clojure is an expression (this is not true of Python or JS, I'm not sure about F#), I don't need to perform much analysis, I just need to make sure that each expression doesn't care about the difference between a future and a realized value. Let's assume three primitives: * an asynchronous future [1] * something which merges a list of futures and realized values into a single future representing a list of values * something which takes a function and a future, and returns a future representing '(apply fn value-of-future)', that will be realized once the future is realized All we need to do is take every expression where a function is called, merge the arguments together, and apply the final primitive to the expression. The expression will return a future, which can then be picked up by the parent expression, and so on. This glosses over how special forms are handled, and how the 'force' functionality works. Also, as I mentioned in the presentation, how the code executes is pretty opaque, and some sort of higher-order analysis would be nice so that it could be visualized using graphviz or something. This is still an experimental feature, and will probably remain so until the 1.0.0 release of Lamina (I'm working on 0.5.0 right now). I'm not aware of any other macro that rips apart normal Clojure code quite as completely, which means that this is sort of uncharted territory. I want to make sure I've avoided all the pitfalls before calling it ready for production usage. [1] https://github.com/ztellman/lamina/wiki/Introduction https://github.com/ztellman/lamina/wiki/Introduction