4 ms·

So the actual challenge is to provide a means, in your language of choice, to write an arbitrary expression like: user_defined(x) + y + z Which may mix ma

So the actual challenge is to provide a means, in your language of choice, to write an arbitrary expression like: user_defined(x) + y + z Which may mix matrices (x), vectors (y), scalars (z), and user_defined or built-in (+) functions applied elementwise; has automatic extension of dimensions; and must operate in a single pass, performing only one traversal. This problem statement is rather tailored to Julia’s way of doing things—I don’t believe any other language can really “succeed” as a result, and not because Julia is any better (nor worse). In Haskell I might simply rely on laziness and rewrite rules for fusion, or reach for a package like repa¹ or folds², depending on the actual problem at hand. There wouldn’t need to be a library providing this “broadcast” functionality, because it can be expressed already with other composable tools. If you work with the language and type system, there are lots of tools available for giving the compiler enough information about the structure of your code to guarantee good performance, abstracted behind a reasonable API. You might end up writing something more explicit, like: zipWith3 (+) (userDefined <$> x) (extend y) (extend z) -- With MonadComprehensions + ParallelListComp [ userDefined a + b + c | a <- x | b <- extend y | c <- extend z ] But I’d be perfectly happy with that, since it uses abstractions like functors and monads that I already understand. I probably wouldn’t reach for the direct equivalent of this Julia solution, which would be to write some Template Haskell macro that rewrites an expression to insert dimensional conversions (“extend” in the pseudocode above) and fusion of operations—that approach feels too brittle and difficult to extend, and not because of any deficiency in the language. ¹ http://hackage.haskell.org/package/repa http://hackage.haskell.org/package/repa ² https://hackage.haskell.org/package/folds https://hackage.haskell.org/package/folds

simondanisch 8y ago
>There wouldn’t need to be a library providing this “broadcast” functionality Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation. So in Julia there is no need either, since it is already implemented much better in Base. I think you're overlooking the actual point of my challenge: it's not that we need a broadcast package, but that a lot of other scientific libraries will need to use similar patterns, to write new packages with new algorithms! The point is also, that I can go into the Julia repository, and without a lot of time & effort, can improve the current implementation of our broadcast - because it is not that much code and pretty easy to follow. So, an equivalent in your example would be, if you can quickly implement something like repa or folds in a similar minimal example like my Julia implementation. If you don't like the details of my challenge, feel free to make a minimal folds implementation from scratch, and show me that the result is fast and composable ;)
- simondanisch 8y ago
  >Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation. To add some more historical facts to this: before broadcast, we mostly used maps & zips in Julia. Which is not very elegant if you want to write down your math. Even with broadcast, some math nuts still complain that it isn't close enough to the actual math :D So consider, that those people are Julia's user base ;) And then, someone pretty much implemented broadcast in their freetime and added it as a PR to the Julia language. This is how the language grows, and a large part of that is possible, because it takes relatively little time to come up with a performant implementation of such a difficult problem. And now we have broadcasting in the language, and maps + zips are indeed less idiomatic... Because anyone who started using the new broadcast immediately thought it was an improvement to "the old ways".

simondanisch 8y ago

>There wouldn’t need to be a library providing this “broadcast” functionality Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation. So in Julia there is no need either, since it is already implemented much better in Base. I think you're overlooking the actual point of my challenge: it's not that we need a broadcast package, but that a lot of other scientific libraries will need to use similar patterns, to write new packages with new algorithms! The point is also, that I can go into the Julia repository, and without a lot of time & effort, can improve the current implementation of our broadcast - because it is not that much code and pretty easy to follow. So, an equivalent in your example would be, if you can quickly implement something like repa or folds in a similar minimal example like my Julia implementation. If you don't like the details of my challenge, feel free to make a minimal folds implementation from scratch, and show me that the result is fast and composable ;)

simondanisch 8y ago
>Actually, your sample "solution" is (was?) also a much used pattern in Julia - especially before we had such a powerful broadcast implementation. To add some more historical facts to this: before broadcast, we mostly used maps & zips in Julia. Which is not very elegant if you want to write down your math. Even with broadcast, some math nuts still complain that it isn't close enough to the actual math :D So consider, that those people are Julia's user base ;) And then, someone pretty much implemented broadcast in their freetime and added it as a PR to the Julia language. This is how the language grows, and a large part of that is possible, because it takes relatively little time to come up with a performant implementation of such a difficult problem. And now we have broadcasting in the language, and maps + zips are indeed less idiomatic... Because anyone who started using the new broadcast immediately thought it was an improvement to "the old ways".