9 ms·

Three examples of this: 1. core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, sin

by drcode 6y ago

Three examples of this: 1. core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation. 2. The Expression Problem: One of the core challenges in language design is designing it so that (simplifying a bit) you can transparently add support for new function methods to objects designed by a third party. Clojure makes this easy https://www.infoq.com/presentations/Clojure-Expression-Problem/ https://www.infoq.com/presentations/Clojure-Expression-Probl... 3. Lots of languages have attempted to allow you to write a single library that you can use both on the client and the server, having your code transpiled to different languages in each case. However, this type of feature is rarely used in production, because there are usually lots of headaches involved, with many limitations. However, for clojure developers it is perfectly normal (and usually expected) that all code is written in cljc files & so that it can be used on both the client (transpiled to javascript) and the server (transpiled to java). It is easy to do this, even for cooperatively multithreaded code.

hy3rm0n10us 6y ago
2. requires static typing in my personal opinion cf. http://lambda-the-ultimate.org/node/4136#comment-62959 http://lambda-the-ultimate.org/node/4136#comment-62959
- mumblemumble 6y ago
  tl;dr on the above: The Expression Problem was originally expressly formulated with maintaining a Haskellian level of static type checking as one of its core requirements. Saying you've solved it, when your solution involves on dynamic typing, or even casts in an otherwise static language, is arguably akin to kicking the ball across the center field line and then claiming you've scored a goal. (See: http://homepages.inf.ed.ac.uk/wadler/papers/expression/expression.txt http://homepages.inf.ed.ac.uk/wadler/papers/expression/expre...)
  - heretoo 6y ago
    Actually, clojure compiles down to java classes, which are statically typed. Existing java types can be extended with clojure protocols. The link below talks about this, specifically about the expression problem. See https://www.ibm.com/developerworks/library/j-clojure-protocols/index.html https://www.ibm.com/developerworks/library/j-clojure-protoco... However, I learnt something. I didn't know that the definition of the expression problem required static typing. I prefer to think of it as adding static types to multiple dispatch.
    - mumblemumble 6y ago
      You missed a detail: or even casts in an otherwise static language. Clojure does compile down to Java classes, but Java is only a partially statically typed language. It also permits run-time casting, which, in a language that even tries to be strongly typed, means run-time (to wit: dynamic) type checking. And Clojure relies heavily on that. That's why I invoked Haskell. It's an example of a language that is truly statically typed, in that it doesn't (generally) permit any run-time type conversions. ALL type checks must be done statically.
      - heretoo 6y ago
        Then, the conclusion can be that dynamic languages don't suffer from the expression problem.
        mumblemumble 6y ago
        Pretty much. Not entirely unlike how one can't implement the Y combinator in a statically typed language, but it's trivial to do so in a dynamic language.
  - joppy 6y ago
    “I know of no widely-used language that solves The Expression Problem while satisfying the constraints of independent compilation and static typing” - kind of implies that the problem is different from whether or not the language is statically typed, doesn’t it? I don’t think you should be writing off Clojure’s multimethods too quickly - they are quite nice, and I haven’t seen anything like them in other dynamic languages like Python.
- slowmovintarget 6y ago
  > So, the expression problem is the problem of solving another, unnamed problem, while satisfying the constraints of static typing? It seems a not very useful term then, and implies that every problem will need two names - the name for the actual problem and the name for the problem of solving that problem while satisfying type system constraints. Bleh. -- Rich Hickey Solving the problem doesn't require static typing.
- roenxi 6y ago
  Spec is pretty close to being a library for static typing; Clojure core being dynamically typed isn't really a blocker. If you really want static typing Clojure can probably do it. The community support isn't going to be as solid as for something like Haskell. But I'm pretty sure spec offers stronger control of the useful parts of a type system than something like, eg, C. Except the control over RAM; that isn't a Clojure competence.
enitihas 6y ago
> and think it is better than golang's implementation. How is core.async better than the golang implementation. In golang, all network I/O you do is automatically asynchronous, so your goroutines won't block a thread on network I/O. That is simply something core.async can't do.
- patrickthebold 6y ago
  I don't know if it does, but why couldn't it? The Java has non blocking IO.
  - lenkite 6y ago
    It's not just a matter of non-blocking IO. core.async uses heavy-weight threads. Go-lang uses light-weight go-routines. The JVM also would need to incorporate a lightweight process scheduler that can wait on I/O and schedule j-routines when I/O is blocked.
    - gavinray 6y ago
      I don't know anything about the JVM and my experience with JVM-based languages is incredibly limited, but isn't that the essence of the Loom Project? https://developers.redhat.com/blog/2019/06/19/project-loom-lightweight-java-threads/ https://developers.redhat.com/blog/2019/06/19/project-loom-l... https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.ht...
      - lenkite 6y ago
        Yes, however, the Loom project is still some years away and the Java standard library will need to be extensively revamped to support this. I see this easily a decade in the future.
        Skinney 6y ago
        They have been rewamping the Java standard library slowly and silently over the last couple of years in anticipation for Loom. With Java 14 they re-wrote the tcp socket class. In Java 15 they're re-implementing the udp socket class. These re-implementations are being done to support loom seamlessly in a future update. They also collaborated with IntelliJ to find bugs related to debugging Java apps, to make sure Loom doesn't break anything there. Loom is well underway. I wouldn't be surprised if we saw Loom in Java 19 or 20. So in 2-3 years.
armitron 6y ago
core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels. Summarizing: core.async is macro sugar on top of java.util.concurrent. Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).
- puredanger 6y ago
  Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine). There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik. This will change once Project Loom lands in Java and provides m:n fibers.
  - fooster 6y ago
    Yes, yes in the future at some undefined point java will be better. Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.
    - ashes-of-sol 6y ago
      Could you expand on the problems you see with the JVM and ecosystem? I was skeptical, but it hasn't been a problem for us so far, and having java libraries and good java interop has helped us more than it has hurt.
    - deleted 6y ago
      [deleted]
ethagnawl 6y ago
> So the guys who built golang did it because they had this cool idea for creating coroutines using channels. Communicating sequential processes is an "old" idea. See: https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...
- drcode 6y ago
  That is true, but my impression is that the original syntax was cumbersome and that golang made major advancements on that front (though I'm no expert and haven't studied the original papers)
  - ethagnawl 6y ago
    I'm no expert myself and wasn't trying to call you out or anything. (I've only read that paper once or twice before a Papers We Love event I attended.) I think it's a common misconception, though, and thought people might find the history interesting.
  - didibus 6y ago
    This stack overflow answer has a very fascinating explanation of the history of the evolution of CSP from Hoare's original paper, to the work done on it with Occam up onto Go, I recommend reading it if you are interested in this stuff: https://stackoverflow.com/a/32696464/172272 https://stackoverflow.com/a/32696464/172272
scythe 6y ago
>core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation. It is very limited -- it was done by a single university student as his MS thesis -- but luaproc had added this functionality to Lua before Go even existed: http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf Other M.S. theses from PUC have extended the library (in particular, adding support for sending tables) but it is otherwise unmaintained: https://www.maxwell.vrac.puc-rio.br/30267/30267.PDF https://www.maxwell.vrac.puc-rio.br/30267/30267.PDF https://github.com/lmillanfdez/luaproc-master https://github.com/lmillanfdez/luaproc-master https://www.maxwell.vrac.puc-rio.br/35424/35424.PDF https://www.maxwell.vrac.puc-rio.br/35424/35424.PDF https://github.com/fernando-ala/luaproc---messaging-tables-async-read https://github.com/fernando-ala/luaproc---messaging-tables-a...
dkersten 6y ago
I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, for example, you cannot nest core.async functions inside other functions as they must be directly visible by the go macro, which cannot see jnside function calls. I’ve also had errors where the stack trace did not reference my code files at all, because it happened inside some core.async setup (IIRC it was a variable that was meant to be a channel but was nil, inside a sub or mix or something, ie I connected two core.async things together, it created go blocks internally, the exception happened inside that and therefore never referenced any if my source files, since the exceotion happened in an asynchronous context after my code ran).This was extremely painful to figure out. Neither of these issues can be solved as long as core.async is implemented as a macro. However, it is extremely cool how far it was able to be taken without changing the language!
- drcode 6y ago
  I don't think it makes logical sense for there to be a "core.async function". Instead, you have each function return a channel with it's own go macro, then have the parent function consume data from the channels.
  - deleted 6y ago
    [deleted]
  - dkersten 6y ago
    Sure, that’s a solution, but its necessary only because of a limitation that only exists because its a macro that needs to look into the code to rewrite it. It also adds extra (cognitive) overhead that such functions are always themselves asynchronous and the return value are channels - they can never just return a raw value - which limits what you can do with them or how you can call them or pass them elsewhere. I mean, yes, usually it isn’t a problem at all, but it IS a limitation of core.async that wouldn’t need to exist if it had tighter integration into the language.
- mschaef 6y ago
  > I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, Agreed. The core abstraction of a go block is essentially a user-space thread. Because macros are limited to locally analyzing/rewriting code (and limitations of the JVM), go blocks are super limited in scope. As you point out, one call to a function in a go block, and you can't switch out of that go block during the function's execution. And functions calls are very common in Clojure code and can easily be hidden behind macros that obscure the details of the control flow. There are 'core.async'y ways around all this, but the net effect is that 'core.async' imposes a very distinct style of writing (and one that tends to be contagious). Python's 'async/await' is contagious also, but because it's more tightly integrated into the runtime/compiler, it doesn't feel nearly as restrictive. (at least to me).
butterisgood 6y ago
Go was inspired by several systems that have come before it. You can write coroutine code with channels in C. http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread It’s available for unix systems as well in the Plan9Port, and that predates Go by a bit. I’m not sure what any of this has to do with Clojure though other than to say “look we have Go-like concurrency without starting over”. It’s been shown that you don’t have to start over to add features to other languages as well. See also libmill http://libmill.org/ http://libmill.org/ Go was not solely created to have goroutines and channels, but that was definitely one of the features desired.
blak3mill3r 6y ago
Yes! I must point out a little factual error: JVM Clojure is not transpiled to Java, it is compiled on-demand to JVM bytecode.

by drcode 6y ago

hy3rm0n10us 6y ago

2. requires static typing in my personal opinion cf. http://lambda-the-ultimate.org/node/4136#comment-62959 http://lambda-the-ultimate.org/node/4136#comment-62959

mumblemumble 6y ago
tl;dr on the above: The Expression Problem was originally expressly formulated with maintaining a Haskellian level of static type checking as one of its core requirements. Saying you've solved it, when your solution involves on dynamic typing, or even casts in an otherwise static language, is arguably akin to kicking the ball across the center field line and then claiming you've scored a goal. (See: http://homepages.inf.ed.ac.uk/wadler/papers/expression/expression.txt http://homepages.inf.ed.ac.uk/wadler/papers/expression/expre...)
- heretoo 6y ago
  Actually, clojure compiles down to java classes, which are statically typed. Existing java types can be extended with clojure protocols. The link below talks about this, specifically about the expression problem. See https://www.ibm.com/developerworks/library/j-clojure-protocols/index.html https://www.ibm.com/developerworks/library/j-clojure-protoco... However, I learnt something. I didn't know that the definition of the expression problem required static typing. I prefer to think of it as adding static types to multiple dispatch.
  - mumblemumble 6y ago
    You missed a detail: or even casts in an otherwise static language. Clojure does compile down to Java classes, but Java is only a partially statically typed language. It also permits run-time casting, which, in a language that even tries to be strongly typed, means run-time (to wit: dynamic) type checking. And Clojure relies heavily on that. That's why I invoked Haskell. It's an example of a language that is truly statically typed, in that it doesn't (generally) permit any run-time type conversions. ALL type checks must be done statically.
    - heretoo 6y ago
      Then, the conclusion can be that dynamic languages don't suffer from the expression problem.
      - mumblemumble 6y ago
        Pretty much. Not entirely unlike how one can't implement the Y combinator in a statically typed language, but it's trivial to do so in a dynamic language.
- joppy 6y ago
  “I know of no widely-used language that solves The Expression Problem while satisfying the constraints of independent compilation and static typing” - kind of implies that the problem is different from whether or not the language is statically typed, doesn’t it? I don’t think you should be writing off Clojure’s multimethods too quickly - they are quite nice, and I haven’t seen anything like them in other dynamic languages like Python.
slowmovintarget 6y ago
> So, the expression problem is the problem of solving another, unnamed problem, while satisfying the constraints of static typing? It seems a not very useful term then, and implies that every problem will need two names - the name for the actual problem and the name for the problem of solving that problem while satisfying type system constraints. Bleh. -- Rich Hickey Solving the problem doesn't require static typing.
roenxi 6y ago
Spec is pretty close to being a library for static typing; Clojure core being dynamically typed isn't really a blocker. If you really want static typing Clojure can probably do it. The community support isn't going to be as solid as for something like Haskell. But I'm pretty sure spec offers stronger control of the useful parts of a type system than something like, eg, C. Except the control over RAM; that isn't a Clojure competence.

enitihas 6y ago

> and think it is better than golang's implementation. How is core.async better than the golang implementation. In golang, all network I/O you do is automatically asynchronous, so your goroutines won't block a thread on network I/O. That is simply something core.async can't do.

patrickthebold 6y ago
I don't know if it does, but why couldn't it? The Java has non blocking IO.
- lenkite 6y ago
  It's not just a matter of non-blocking IO. core.async uses heavy-weight threads. Go-lang uses light-weight go-routines. The JVM also would need to incorporate a lightweight process scheduler that can wait on I/O and schedule j-routines when I/O is blocked.
  - gavinray 6y ago
    I don't know anything about the JVM and my experience with JVM-based languages is incredibly limited, but isn't that the essence of the Loom Project? https://developers.redhat.com/blog/2019/06/19/project-loom-lightweight-java-threads/ https://developers.redhat.com/blog/2019/06/19/project-loom-l... https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.ht...
    - lenkite 6y ago
      Yes, however, the Loom project is still some years away and the Java standard library will need to be extensively revamped to support this. I see this easily a decade in the future.
      - Skinney 6y ago
        They have been rewamping the Java standard library slowly and silently over the last couple of years in anticipation for Loom. With Java 14 they re-wrote the tcp socket class. In Java 15 they're re-implementing the udp socket class. These re-implementations are being done to support loom seamlessly in a future update. They also collaborated with IntelliJ to find bugs related to debugging Java apps, to make sure Loom doesn't break anything there. Loom is well underway. I wouldn't be surprised if we saw Loom in Java 19 or 20. So in 2-3 years.

armitron 6y ago

core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels. Summarizing: core.async is macro sugar on top of java.util.concurrent. Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).

puredanger 6y ago
Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine). There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik. This will change once Project Loom lands in Java and provides m:n fibers.
- fooster 6y ago
  Yes, yes in the future at some undefined point java will be better. Meanwhile go works very well today. And you don't need to be saddled with the JVM and the very bloated java ecosystem.
  - ashes-of-sol 6y ago
    Could you expand on the problems you see with the JVM and ecosystem? I was skeptical, but it hasn't been a problem for us so far, and having java libraries and good java interop has helped us more than it has hurt.
  - deleted 6y ago
    [deleted]

ethagnawl 6y ago

> So the guys who built golang did it because they had this cool idea for creating coroutines using channels. Communicating sequential processes is an "old" idea. See: https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...

drcode 6y ago
That is true, but my impression is that the original syntax was cumbersome and that golang made major advancements on that front (though I'm no expert and haven't studied the original papers)
- ethagnawl 6y ago
  I'm no expert myself and wasn't trying to call you out or anything. (I've only read that paper once or twice before a Papers We Love event I attended.) I think it's a common misconception, though, and thought people might find the history interesting.
- didibus 6y ago
  This stack overflow answer has a very fascinating explanation of the history of the evolution of CSP from Hoare's original paper, to the work done on it with Occam up onto Go, I recommend reading it if you are interested in this stuff: https://stackoverflow.com/a/32696464/172272 https://stackoverflow.com/a/32696464/172272

scythe 6y ago

>core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation. It is very limited -- it was done by a single university student as his MS thesis -- but luaproc had added this functionality to Lua before Go even existed: http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf Other M.S. theses from PUC have extended the library (in particular, adding support for sending tables) but it is otherwise unmaintained: https://www.maxwell.vrac.puc-rio.br/30267/30267.PDF https://www.maxwell.vrac.puc-rio.br/30267/30267.PDF https://github.com/lmillanfdez/luaproc-master https://github.com/lmillanfdez/luaproc-master https://www.maxwell.vrac.puc-rio.br/35424/35424.PDF https://www.maxwell.vrac.puc-rio.br/35424/35424.PDF https://github.com/fernando-ala/luaproc---messaging-tables-async-read https://github.com/fernando-ala/luaproc---messaging-tables-a...

dkersten 6y ago

I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, for example, you cannot nest core.async functions inside other functions as they must be directly visible by the go macro, which cannot see jnside function calls. I’ve also had errors where the stack trace did not reference my code files at all, because it happened inside some core.async setup (IIRC it was a variable that was meant to be a channel but was nil, inside a sub or mix or something, ie I connected two core.async things together, it created go blocks internally, the exception happened inside that and therefore never referenced any if my source files, since the exceotion happened in an asynchronous context after my code ran).This was extremely painful to figure out. Neither of these issues can be solved as long as core.async is implemented as a macro. However, it is extremely cool how far it was able to be taken without changing the language!

drcode 6y ago
I don't think it makes logical sense for there to be a "core.async function". Instead, you have each function return a channel with it's own go macro, then have the parent function consume data from the channels.
- deleted 6y ago
  [deleted]
- dkersten 6y ago
  Sure, that’s a solution, but its necessary only because of a limitation that only exists because its a macro that needs to look into the code to rewrite it. It also adds extra (cognitive) overhead that such functions are always themselves asynchronous and the return value are channels - they can never just return a raw value - which limits what you can do with them or how you can call them or pass them elsewhere. I mean, yes, usually it isn’t a problem at all, but it IS a limitation of core.async that wouldn’t need to exist if it had tighter integration into the language.
mschaef 6y ago
> I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, Agreed. The core abstraction of a go block is essentially a user-space thread. Because macros are limited to locally analyzing/rewriting code (and limitations of the JVM), go blocks are super limited in scope. As you point out, one call to a function in a go block, and you can't switch out of that go block during the function's execution. And functions calls are very common in Clojure code and can easily be hidden behind macros that obscure the details of the control flow. There are 'core.async'y ways around all this, but the net effect is that 'core.async' imposes a very distinct style of writing (and one that tends to be contagious). Python's 'async/await' is contagious also, but because it's more tightly integrated into the runtime/compiler, it doesn't feel nearly as restrictive. (at least to me).

butterisgood 6y ago

Go was inspired by several systems that have come before it. You can write coroutine code with channels in C. http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread It’s available for unix systems as well in the Plan9Port, and that predates Go by a bit. I’m not sure what any of this has to do with Clojure though other than to say “look we have Go-like concurrency without starting over”. It’s been shown that you don’t have to start over to add features to other languages as well. See also libmill http://libmill.org/ http://libmill.org/ Go was not solely created to have goroutines and channels, but that was definitely one of the features desired.

blak3mill3r 6y ago

Yes! I must point out a little factual error: JVM Clojure is not transpiled to Java, it is compiled on-demand to JVM bytecode.