9 ms·

There are many many tomes of discussion that have led to this point (I'm mbauman). In short, there is an inherent disconnect between arrays-as-containers and m

by 3JPLW 11y ago

There are many many tomes of discussion that have led to this point (I'm mbauman). In short, there is an inherent disconnect between arrays-as-containers and matrices-as-linear-algebra-operators. There are many ways that you can define array operations such that the universe is closed and self-consistent, but there always seems to be some awkwardness. Right now, Julia uses some Matlab-ish semantics (sometimes ignoring trailing singleton dimensions), but there is an interaction with Julia's types that makes the status quo a little strange. So there are "just" two major breaking changes proposed here: changing our array semantics universe to be more APL-like (instead of Matlabish), and making indexing return a view instead of a copy.

ViralBShah 11y ago
A little bit of history here. Back when we started Julia 6 years ago, I personally was in a much more of a Matlab/Octave frame of mind from my own experience. That is why the early array implementation followed that style. We had no idea where Julia would go. In some sense, it has been a personal journey of maturity in my own thinking, and if I can take the liberty to say so, the same is true for the community as well.
- exDM69 11y ago
  Has your stance on one-based array indexing changed in those 6 years? How many off-by-one issues have you encountered? I'm not from a Matlab/Octave background, so one-based indexing feels really foreign to me.
  - jules 11y ago
    For what it's worth, I've ran into this a couple of times. Maybe it's just because I'm more familiar with 0-based indexing, but I think it's not just that. In every case the 0-based indexing had fewer (+1)'s and (-1)'s than the 1-based code. For instance k%n returns a number in the range [0,n), or in Julia notation [0:n-1]. When you want to do array indexing with that, you need (k%n)+1 in Julia. Btw, Matlab also uses 1-based indexing, no? I thought that Julia chose 1-based indexing to get brain backwards compatibility with Matlab.
    - ViralBShah 11y ago
      Not just Matlab, but many technical computing languages use 1-based indexing. Julia was never intended to have any compatibility with Matlab.
      - jjaredsimpson 11y ago
        It's just bikeshedding IMO. When I write C# code, array indices are like memory operation in my mind. When I go to Octave indices are like subscripts on sequences to me. Anyone who think it really makes a difference is being ridiculous. I think zero based is "prettier" because of half open intervals, but it really doesn't matter.
        eggy 11y ago
        I don't think it is bikeshedding IMO from a pure mathematics point of view. In set theory you begin with the empty set as zero, and the next integer is N+1. I find it interesting that the Julia devs have gone with APLs style of slicing, and did not move to 0-based indexing. Ken Iverson, who created APL and later the J programming language, chose 0-based indexing for J. APL has a system variable QUAD-IO that allows you to choose your indexing value of 0 or 1, which leads to many issues. Iverson's books on math (Arithmetic, Calculus, and the Concrete Math Companion (Knuth's book done in J)) all to demonstrate mathematics in J, all begin at Chapter 0. I think the choice of index in APL is a bad idea. I have run other people's code, and realized it doesn't work because they set their indexing to 1 vs. 0 for me. People and programmers can work around most things, but I think if you are a functional programmer, and mathematics is your thing, you are more comfortable with 0-based indexing. This of course does not address scientists, for whom pure mathematics is not at the forefront of whatever they are working on. I do think it leads to problems down the road for 1-based indexing.
    - onnoonno 11y ago
      Julia looks very interesting but honestly, exactly the 1-based indexing makes it feel really awkward and turns me off every time I think about using it somewhere. Wouldn't it be possible to create array classes in Julia that all indexed zero-based? I would welcome a 'Julia0' fork :-)
  - ViralBShah 11y ago
    I feel that 1-based indexing has been the right choice, and I am aware that many do not like it. I really haven't run into off-by-one issues much, but this is probably more about how one is used to writing code. I like that you can iterate over a vector of size n, by just going from 1 to n. Coming from a regular programming background, as opposed to a technical computing background, can certainly make this look alien. However, I think it starts feeling natural quickly. There are many other things in Julia that can take a lot longer to get used to.
    - gct 11y ago
      It sucks for slicing though. In python (zero based), you can use 0:n to slice the entire array with a half-open interval. With one based, you have to either have a closed interval, so 1:n is the whole array (which sucks because what does an empty interval starting at 1 look like? 1:0?) Or you can have a half-open interval and index 1:n+1. I don't see what you saved yourself other than making things uglier.
  - chrispeel 11y ago
    For those who don't like one-based indexing, your problem is solved: there is now a package which provides two-based indexing https://github.com/simonster/TwoBasedIndexing.jl https://github.com/simonster/TwoBasedIndexing.jl ;-) Edit: For a history of 0-based indexing see http://exple.tive.org/blarg/2013/10/22/citation-needed/ http://exple.tive.org/blarg/2013/10/22/citation-needed/ My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer. 0-base is applicable to people creating chips or writing hardware drivers. Most scientific programmers are not so closely tied to hardware; 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic.
    - masklinn 11y ago
      > My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer. It's never been needed, Pascal had arbitrary array bounds (including negative) 50 years ago. > 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic. What does binary arithmetic have to do with anything? And have there been any study pertaining to the accessibility of 0-indexed versus 1-indexed arrays to high-scool programmers which would warrant such an assertion?
    - JulianMorrison 11y ago
      Zero based indexing counts offsets. Literally in C, you are saying "add n times sizeof(T) to the base pointer and dereference it". It rather implies arrays are contiguous in storage and lack any "holes". It's a crude but close-to-the-metal approach. C arrays are not general, they are one specific structure. An array could be implemented as a tree, but C doesn't have built-in support for that. One based indexing counts, or labels boxes. Counting implies no "holes", but abandons any implication about how they are laid out in storage. Labelling allows treating numerical arrays as sparse hash tables, with as many "holes" as needed. It's a high level view, suited to mathy ways of thinking about data.
      - 11y ago
- tmalsburg2 11y ago
  My impression is that you aim to produce a perfectly clean and elegant – perhaps in order to avoid a Python-3-type scenario. However, if that is what you are doing, I am afraid that Julia could become the GNU Hurd or programming languages. Most languages, at least those that have practical relevance, have design flaws. Heck, R is one giant mess of design flaws, and yet it is actually amazingly useful. Another example would be Perl. I think design flaws can be a sign that the creators of these languages are reasonable people who are capable of making pragmatic decisions (instead of just academic decisions). I’m clearly not an expert and I’m not up-to-date with what is going on in Julia but as a potential user I have the impression that we’re not moving closer to a point where Julia is ready for production use, and I’m worried that Julia will lose momentum while R is growing stronger and deeper roots. Personally, I really like Julia and would prefer if it would replace R but over time this is becoming less and less likely. The world is full with examples showing that inferior technologies can become dominant just because they had a head start. I think this article is relevant here: http://paulgraham.com/startuplessons.html http://paulgraham.com/startuplessons.html
  - twic 11y ago
    On the other hand, Rust spent a long time undergoing tumultuous upheaval, considerably more radical than this, i think, and yet it did eventually stabilise and produce a 1.0.
    - steveklabnik 11y ago
      Oh yeah. Rust was really four different programming languages over a roughly eight year period. Always the same focus, but radically different ways of accomplishing it.
  - Xcelerate 11y ago
    Personally, I would much rather see Julia undergo serious upheavals in an attempt to make an elegant language that works really well for scientific/HPC computing. Too many languages get trapped trying to provide backwards compatibility for everything and you end up with a lot of unintuitive cruft and quirks. If you want a language for "getting stuff done", use Python. But I find that I get a lot done with Julia in spite of constant change (in fact, I don't use Python at all).

by 3JPLW 11y ago

ViralBShah 11y ago

A little bit of history here. Back when we started Julia 6 years ago, I personally was in a much more of a Matlab/Octave frame of mind from my own experience. That is why the early array implementation followed that style. We had no idea where Julia would go. In some sense, it has been a personal journey of maturity in my own thinking, and if I can take the liberty to say so, the same is true for the community as well.

exDM69 11y ago
Has your stance on one-based array indexing changed in those 6 years? How many off-by-one issues have you encountered? I'm not from a Matlab/Octave background, so one-based indexing feels really foreign to me.
- jules 11y ago
  For what it's worth, I've ran into this a couple of times. Maybe it's just because I'm more familiar with 0-based indexing, but I think it's not just that. In every case the 0-based indexing had fewer (+1)'s and (-1)'s than the 1-based code. For instance k%n returns a number in the range [0,n), or in Julia notation [0:n-1]. When you want to do array indexing with that, you need (k%n)+1 in Julia. Btw, Matlab also uses 1-based indexing, no? I thought that Julia chose 1-based indexing to get brain backwards compatibility with Matlab.
  - ViralBShah 11y ago
    Not just Matlab, but many technical computing languages use 1-based indexing. Julia was never intended to have any compatibility with Matlab.
    - jjaredsimpson 11y ago
      It's just bikeshedding IMO. When I write C# code, array indices are like memory operation in my mind. When I go to Octave indices are like subscripts on sequences to me. Anyone who think it really makes a difference is being ridiculous. I think zero based is "prettier" because of half open intervals, but it really doesn't matter.
      - eggy 11y ago
        I don't think it is bikeshedding IMO from a pure mathematics point of view. In set theory you begin with the empty set as zero, and the next integer is N+1. I find it interesting that the Julia devs have gone with APLs style of slicing, and did not move to 0-based indexing. Ken Iverson, who created APL and later the J programming language, chose 0-based indexing for J. APL has a system variable QUAD-IO that allows you to choose your indexing value of 0 or 1, which leads to many issues. Iverson's books on math (Arithmetic, Calculus, and the Concrete Math Companion (Knuth's book done in J)) all to demonstrate mathematics in J, all begin at Chapter 0. I think the choice of index in APL is a bad idea. I have run other people's code, and realized it doesn't work because they set their indexing to 1 vs. 0 for me. People and programmers can work around most things, but I think if you are a functional programmer, and mathematics is your thing, you are more comfortable with 0-based indexing. This of course does not address scientists, for whom pure mathematics is not at the forefront of whatever they are working on. I do think it leads to problems down the road for 1-based indexing.
  - onnoonno 11y ago
    Julia looks very interesting but honestly, exactly the 1-based indexing makes it feel really awkward and turns me off every time I think about using it somewhere. Wouldn't it be possible to create array classes in Julia that all indexed zero-based? I would welcome a 'Julia0' fork :-)
- ViralBShah 11y ago
  I feel that 1-based indexing has been the right choice, and I am aware that many do not like it. I really haven't run into off-by-one issues much, but this is probably more about how one is used to writing code. I like that you can iterate over a vector of size n, by just going from 1 to n. Coming from a regular programming background, as opposed to a technical computing background, can certainly make this look alien. However, I think it starts feeling natural quickly. There are many other things in Julia that can take a lot longer to get used to.
  - gct 11y ago
    It sucks for slicing though. In python (zero based), you can use 0:n to slice the entire array with a half-open interval. With one based, you have to either have a closed interval, so 1:n is the whole array (which sucks because what does an empty interval starting at 1 look like? 1:0?) Or you can have a half-open interval and index 1:n+1. I don't see what you saved yourself other than making things uglier.
- chrispeel 11y ago
  For those who don't like one-based indexing, your problem is solved: there is now a package which provides two-based indexing https://github.com/simonster/TwoBasedIndexing.jl https://github.com/simonster/TwoBasedIndexing.jl ;-) Edit: For a history of 0-based indexing see http://exple.tive.org/blarg/2013/10/22/citation-needed/ http://exple.tive.org/blarg/2013/10/22/citation-needed/ My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer. 0-base is applicable to people creating chips or writing hardware drivers. Most scientific programmers are not so closely tied to hardware; 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic.
  - masklinn 11y ago
    > My understanding is that with modern compilers, there's no need for 0-based indexing; you can use 0-base, 1-base, or 69-base indexing as you prefer. It's never been needed, Pascal had arbitrary array bounds (including negative) 50 years ago. > 1-based indexing is one of the things that helps make Julia more accessible to high-school programmers or PhD students who don't want to learn binary arithmetic. What does binary arithmetic have to do with anything? And have there been any study pertaining to the accessibility of 0-indexed versus 1-indexed arrays to high-scool programmers which would warrant such an assertion?
  - JulianMorrison 11y ago
    Zero based indexing counts offsets. Literally in C, you are saying "add n times sizeof(T) to the base pointer and dereference it". It rather implies arrays are contiguous in storage and lack any "holes". It's a crude but close-to-the-metal approach. C arrays are not general, they are one specific structure. An array could be implemented as a tree, but C doesn't have built-in support for that. One based indexing counts, or labels boxes. Counting implies no "holes", but abandons any implication about how they are laid out in storage. Labelling allows treating numerical arrays as sparse hash tables, with as many "holes" as needed. It's a high level view, suited to mathy ways of thinking about data.
    - 11y ago
tmalsburg2 11y ago
My impression is that you aim to produce a perfectly clean and elegant – perhaps in order to avoid a Python-3-type scenario. However, if that is what you are doing, I am afraid that Julia could become the GNU Hurd or programming languages. Most languages, at least those that have practical relevance, have design flaws. Heck, R is one giant mess of design flaws, and yet it is actually amazingly useful. Another example would be Perl. I think design flaws can be a sign that the creators of these languages are reasonable people who are capable of making pragmatic decisions (instead of just academic decisions). I’m clearly not an expert and I’m not up-to-date with what is going on in Julia but as a potential user I have the impression that we’re not moving closer to a point where Julia is ready for production use, and I’m worried that Julia will lose momentum while R is growing stronger and deeper roots. Personally, I really like Julia and would prefer if it would replace R but over time this is becoming less and less likely. The world is full with examples showing that inferior technologies can become dominant just because they had a head start. I think this article is relevant here: http://paulgraham.com/startuplessons.html http://paulgraham.com/startuplessons.html
- twic 11y ago
  On the other hand, Rust spent a long time undergoing tumultuous upheaval, considerably more radical than this, i think, and yet it did eventually stabilise and produce a 1.0.
  - steveklabnik 11y ago
    Oh yeah. Rust was really four different programming languages over a roughly eight year period. Always the same focus, but radically different ways of accomplishing it.
- Xcelerate 11y ago
  Personally, I would much rather see Julia undergo serious upheavals in an attempt to make an elegant language that works really well for scientific/HPC computing. Too many languages get trapped trying to provide backwards compatibility for everything and you end up with a lot of unintuitive cruft and quirks. If you want a language for "getting stuff done", use Python. But I find that I get a lot done with Julia in spite of constant change (in fact, I don't use Python at all).