4 ms·

Quick summary: Gilad Bracha introduces a new programming language called ShapeRank. It seems to be based on APL but introduces the concept of streams which are

Quick summary: Gilad Bracha introduces a new programming language called ShapeRank. It seems to be based on APL but introduces the concept of streams which are [vectors | tensors | arrays] of unbounded length.

Snoddas 6y ago
Little longer summary: The ideas of APL and its successors, the array programming languages, were two generations ahead of their time. These languages are based on the notion that everything is a tensor, and all operations are rank-polymorphic: they extend automatically to tensors of any rank. These ideas are perfectly suited to an era of machine learning, large scale data, GPUs and other accelerators. Building on recent academic research, we are building ShapeRank, a new statically typed, purely functional language for industrial use, that extends rank-polymorphism to streams. We’ll introduce the key ideas and show how they are realized in ShapeRank. https://2020.splashcon.org/details/splash-2020-rebase/26/A-Ray-of-Hope-Array-Programming-for-the-21st-Century https://2020.splashcon.org/details/splash-2020-rebase/26/A-R...
- justincormack 6y ago
  Have you got links to the research papers?
- Gravityloss 6y ago
  Can it be done in some easier syntax than APL? Easier means - less effort in learning coming from someone who knows mainstream languages like Java. The idea is not only limited to APL. I don't like crafting for loops or maintain indexes. Fortran has something similar. With Matlab many operators operate in an intuitive way on vectors and matrices. It breaks down quite quickly if you try to do something more complex though. This somewhat extends to Julia. In Ruby also you can have .map or .each. Julia: x=10 v=[1 2 3 4] x.*v #1×4 Array{Int64,2}: # 10 20 30 40
  - PeCaN 6y ago
    If you watch the video it looks like their proposed syntax is not APL-like but closer to mainstream languages. I'm honestly not sure if this is a good thing or not. You said "easier" syntax than APL but APL is honestly a very easy syntax for working with arrays. That's a significant part of the advantage of APL, it makes it very easy to come up with, talk about, and maintain array algorithms. Matlab and Julia and other languages aimed at scientific computing have some array language-like traits but lack a lot of the functions that make APL more generally applicable. And .map is all wrong; it's extra noise and it doesn't generalize down to scalars or up to matrices—the defining feature of array languages is that operations are implicitly polymorphic over the rank of the input.
    - Gravityloss 6y ago
      I understand that. I still don't want to spend the effort to learn APL. It's like digital cameras that came around. Many users knew how to use film cameras so you made the digital cameras to be mostly like film cameras even if the digital medium would have enabled a very different, much better camera straight out of the box. But the market had invested so much time in this learning how to work with film that you had to do it like that. Path dependency is not just about rigid thinking, it's about using what you have because that saves a lot of resources. Regarding, .map being all wrong, in Ruby it's not a property of an array, it's a method for enumerables. Array is one type of enumerable, but it works with hashmaps etc. https://ruby-doc.org/core-2.6.5/Enumerable.html https://ruby-doc.org/core-2.6.5/Enumerable.html So it's not that non-general. It is noisy (and weird with the pipes) because it's general.
      - PeCaN 6y ago
        To be honest I don't really see people who don't want to learn APL being that interested in putting in the effort to completely upend how they think about programming and algorithms in order to use other array languages, regardless of syntax. (After all this is by far the hardest part of learning APL, the symbols are easy enough and easy to look up anyway.) map is general in kind of the wrong way. You could after all add a #map method to Object for scalars and make a Matrix class that also implements it and then just call map everywhere. However you still run into the problem, mentioned in the video, that it doesn't easily generalize to x + y where both x and y are arrays; you have to use zip or map2 or something (and now you still have to figure out how to do vector + matrix) and yes you can kind of do explicit "array programming" in Ruby if for some reason you're really compelled to do that but it will look awful. And that's just what array languages do for you implicitly. As a paradigm there's a bit more too it than "just call map everywhere"—there's still all the functions for expressing algorithms as computations on arrays.
teleforce 6y ago
Personally I think array programming languages are the future and one of the most popular programming languages in science and engineering is Matlab, and it is to some extent an array based programming language [1]. I am surprised that the author (and reviewers of the paper) has missed to perform proper literature review, for example it missed other recent and promising works on functional array programming languages namely Single Assignment C (SAC) and Futhark [2],[3]. ShapeRank also seems to take vector algebra "tensor" concept to the extreme and to be honest it's better to based on "versor" since geometric algebra is probably the future of computer algebra [4]. Last but not least and probably the most controversial is that why create another standalone array language from scratch? It will be better to make a seamless DSL based on general purpose language like D language and you do not have to re-invent most of the libraries (and C library support in D is second to none). Arguably the most successful recent effort on array based scientific programming language is Julia and it is still very much dependent on some Fortran based libraries for speed. While with D you can go "turtle all the way down" and still meet the speed requirements that are needed in scientific computing [5]. [1]https://en.m.wikipedia.org/wiki/Array_programming https://en.m.wikipedia.org/wiki/Array_programming [2]http://www.sac-home.org/doku.php http://www.sac-home.org/doku.php [3]https://futhark-lang.org/ https://futhark-lang.org/ [4]https://en.m.wikipedia.org/wiki/Comparison_of_vector_algebra_and_geometric_algebra https://en.m.wikipedia.org/wiki/Comparison_of_vector_algebra... [5]http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...
- 7thaccount 6y ago
  I'd love to drop Python and go all in with a true array language (i.e. not Matlab), but you only really have 4 options: Dyalog APL, J, Kdb+, Shakti. All of those are closed source and expensive (Dyalog is fairly affordable, but still a paid product) with the exception of J. J is a cool language, but isn't quite my cup of tea. So if one of the new projects ever picked up steam and got a decent sized community with hooks into all the same numeric libraries as Numpy, and some decent charting libraries...then we would have something nice.
  - beagle3 6y ago
jb_s 6y ago
In my undergrad at UNSW I did a "baby's first interpreter" project on a language with a similar concept - function evaluation is defined in terms of unbounded-length arrays, being able to cache function evaluation for performance etc. Looking back it was pretty cool and helped form a more abstracted view of programming beyond the low-level edit:found some papers https://cartesianprogramming.files.wordpress.com/2020/07/semantics-20.pdf https://cartesianprogramming.files.wordpress.com/2020/07/sem... http://www.cse.unsw.edu.au/~plaice/archive/JAP/U-CSE-201306.pdf http://www.cse.unsw.edu.au/~plaice/archive/JAP/U-CSE-201306....

Snoddas 6y ago

Little longer summary: The ideas of APL and its successors, the array programming languages, were two generations ahead of their time. These languages are based on the notion that everything is a tensor, and all operations are rank-polymorphic: they extend automatically to tensors of any rank. These ideas are perfectly suited to an era of machine learning, large scale data, GPUs and other accelerators. Building on recent academic research, we are building ShapeRank, a new statically typed, purely functional language for industrial use, that extends rank-polymorphism to streams. We’ll introduce the key ideas and show how they are realized in ShapeRank. https://2020.splashcon.org/details/splash-2020-rebase/26/A-Ray-of-Hope-Array-Programming-for-the-21st-Century https://2020.splashcon.org/details/splash-2020-rebase/26/A-R...

justincormack 6y ago
Have you got links to the research papers?
Gravityloss 6y ago
Can it be done in some easier syntax than APL? Easier means - less effort in learning coming from someone who knows mainstream languages like Java. The idea is not only limited to APL. I don't like crafting for loops or maintain indexes. Fortran has something similar. With Matlab many operators operate in an intuitive way on vectors and matrices. It breaks down quite quickly if you try to do something more complex though. This somewhat extends to Julia. In Ruby also you can have .map or .each. Julia: x=10 v=[1 2 3 4] x.*v #1×4 Array{Int64,2}: # 10 20 30 40
- PeCaN 6y ago
  If you watch the video it looks like their proposed syntax is not APL-like but closer to mainstream languages. I'm honestly not sure if this is a good thing or not. You said "easier" syntax than APL but APL is honestly a very easy syntax for working with arrays. That's a significant part of the advantage of APL, it makes it very easy to come up with, talk about, and maintain array algorithms. Matlab and Julia and other languages aimed at scientific computing have some array language-like traits but lack a lot of the functions that make APL more generally applicable. And .map is all wrong; it's extra noise and it doesn't generalize down to scalars or up to matrices—the defining feature of array languages is that operations are implicitly polymorphic over the rank of the input.
  - Gravityloss 6y ago
    I understand that. I still don't want to spend the effort to learn APL. It's like digital cameras that came around. Many users knew how to use film cameras so you made the digital cameras to be mostly like film cameras even if the digital medium would have enabled a very different, much better camera straight out of the box. But the market had invested so much time in this learning how to work with film that you had to do it like that. Path dependency is not just about rigid thinking, it's about using what you have because that saves a lot of resources. Regarding, .map being all wrong, in Ruby it's not a property of an array, it's a method for enumerables. Array is one type of enumerable, but it works with hashmaps etc. https://ruby-doc.org/core-2.6.5/Enumerable.html https://ruby-doc.org/core-2.6.5/Enumerable.html So it's not that non-general. It is noisy (and weird with the pipes) because it's general.
    - PeCaN 6y ago
      To be honest I don't really see people who don't want to learn APL being that interested in putting in the effort to completely upend how they think about programming and algorithms in order to use other array languages, regardless of syntax. (After all this is by far the hardest part of learning APL, the symbols are easy enough and easy to look up anyway.) map is general in kind of the wrong way. You could after all add a #map method to Object for scalars and make a Matrix class that also implements it and then just call map everywhere. However you still run into the problem, mentioned in the video, that it doesn't easily generalize to x + y where both x and y are arrays; you have to use zip or map2 or something (and now you still have to figure out how to do vector + matrix) and yes you can kind of do explicit "array programming" in Ruby if for some reason you're really compelled to do that but it will look awful. And that's just what array languages do for you implicitly. As a paradigm there's a bit more too it than "just call map everywhere"—there's still all the functions for expressing algorithms as computations on arrays.

teleforce 6y ago

Personally I think array programming languages are the future and one of the most popular programming languages in science and engineering is Matlab, and it is to some extent an array based programming language [1]. I am surprised that the author (and reviewers of the paper) has missed to perform proper literature review, for example it missed other recent and promising works on functional array programming languages namely Single Assignment C (SAC) and Futhark [2],[3]. ShapeRank also seems to take vector algebra "tensor" concept to the extreme and to be honest it's better to based on "versor" since geometric algebra is probably the future of computer algebra [4]. Last but not least and probably the most controversial is that why create another standalone array language from scratch? It will be better to make a seamless DSL based on general purpose language like D language and you do not have to re-invent most of the libraries (and C library support in D is second to none). Arguably the most successful recent effort on array based scientific programming language is Julia and it is still very much dependent on some Fortran based libraries for speed. While with D you can go "turtle all the way down" and still meet the speed requirements that are needed in scientific computing [5]. [1]https://en.m.wikipedia.org/wiki/Array_programming https://en.m.wikipedia.org/wiki/Array_programming [2]http://www.sac-home.org/doku.php http://www.sac-home.org/doku.php [3]https://futhark-lang.org/ https://futhark-lang.org/ [4]https://en.m.wikipedia.org/wiki/Comparison_of_vector_algebra_and_geometric_algebra https://en.m.wikipedia.org/wiki/Comparison_of_vector_algebra... [5]http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

7thaccount 6y ago
I'd love to drop Python and go all in with a true array language (i.e. not Matlab), but you only really have 4 options: Dyalog APL, J, Kdb+, Shakti. All of those are closed source and expensive (Dyalog is fairly affordable, but still a paid product) with the exception of J. J is a cool language, but isn't quite my cup of tea. So if one of the new projects ever picked up steam and got a decent sized community with hooks into all the same numeric libraries as Numpy, and some decent charting libraries...then we would have something nice.
- beagle3 6y ago

jb_s 6y ago

In my undergrad at UNSW I did a "baby's first interpreter" project on a language with a similar concept - function evaluation is defined in terms of unbounded-length arrays, being able to cache function evaluation for performance etc. Looking back it was pretty cool and helped form a more abstracted view of programming beyond the low-level edit:found some papers https://cartesianprogramming.files.wordpress.com/2020/07/semantics-20.pdf https://cartesianprogramming.files.wordpress.com/2020/07/sem... http://www.cse.unsw.edu.au/~plaice/archive/JAP/U-CSE-201306.pdf http://www.cse.unsw.edu.au/~plaice/archive/JAP/U-CSE-201306....