5 ms·

Rank Is actually implicitly done by numpy using a mechanism called broadcasting. For example: >>> np.array([10, 20, 30]) + np.array([[1,2,3], [4,5,6], [7,8,9

• ipsum2 6y ago
  To do sieves in J like the blog post mentioned, the equivalent numpy would be: >>> arr = np.array([10, 20, 30]) >>> arr[arr > 15] array([20, 30])
• moonchild 6y ago
  > Rank Is actually implicitly done by numpy using a mechanism called broadcasting. Numpy's broadcasting is scalar conformability, to which the rank operator (and general conformability) provides a general case. Example in j: ] x =. i. 2 3 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ] y =. i. 3 4 0 1 2 3 4 5 6 7 8 9 10 11 x + y NB. this will error because + expects that, if its arguments' shapes are not the same, one will be a prefix of the other |length error | x +y NB. this is easy enough to fix, however x +"2 y NB. +"2 is shorthand for +"2 2; meaning, choose rank-2 arrays from both the left and right arguments 0 2 4 6 8 10 12 14 16 18 20 22 12 14 16 18 20 22 24 26 28 30 32 34 Numpy will actually do this without the rank operator, because it uses suffix agreement rather than prefix agreement (which is absolutely bonkers—j used suffix agreement for about 5 minutes in 1990, before realising it was an awful idea). For for numpy, see if you can add: np.array([[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]], [[12, 13, 14, 15], [16, 17, 18, 19], [20, 21, 22, 23]]]) + np.array([[0, 1, 2], [3, 4, 5]]) Intelligently. (I'm sure it's not overly difficult to come up with a solution, but can you do it with a single higher-order function call which generalises to other argument shapes?) (The j equivalent, (i. 2 3 4) + (i. 2 3) also works without trouble.) ------------------------------------------------------------------------ Another example, which may be more illustrative, is the ability to perform reductions along arbitrary axes. For example: ] x =. i. 4 3 0 1 2 3 4 5 6 7 8 9 10 11 +/ x NB. sum reduced along leading axis, the default, producing an array of shape 3 18 22 26 +/"1 x NB. sum each rank-1 array (vector); or, reduce last axis, producing an array of shape 4 3 12 21 30 ------------------------------------------------------------------------ Another curiosity, which I have thus far neglected, is the extent to which numpy's being 'a little more verbose' is actually incredibly important in shaping the way you approach and think about problems. The great-uncle comment also addresses this, but Iverson probably says it better than either of us can: read https://www.jsoftware.com/papers/tot.htm https://www.jsoftware.com/papers/tot.htm
  • yuppiemephisto 6y ago
    What is prefix/suffix agreement? Could you give a simple example of why prefix beats suffix? I'm curious and have never heard these terms before, and googling didn't help.
    • 00ajcr 6y ago
      To broadcast operations (such as addition) between arrays in NumPy, trailing dimensions have to be equal (or be of length 1). In the example given above the 3D array and 2D array have shape (lengths of dimensions): (2, 3, 4) (2, 3) That is - the suffixes do not agree (4 != 3 and 3 != 2) and NumPy raises an error. However, for the same operation in J the prefixes agree: (2, 3, 4) (2, 3) and the addition gives the expected result. To add the arrays with these shapes in NumPy, one method is transpose each array (reverse order of the dimensions), add these arrays, and then transpose back: (a.T + b.T).T
      • repsilat 6y ago
        Thinking about these as a "verbose imperative language" programmer, I'd say suffix agreement seems to make more sense to me at first glance, and I'd like to hear more about why it might be worse. For why I think it makes sense: I think of multidimensional arrays as being arrays of arrays, and "normal" index lookup operating on the first dimension. If I have a float[100][3] in some context I might think of it as 100 vec3s, and I might want to do some vec3 operation on each of them. I might want to dot them all with my some other vector, or add them all to some other vector. I almost never have 100 scalars and want to apply one scalar to all elements of the corresponding vec3. But I guess maybe this is all widely agreed on, and maybe the contentious part is just index order? Like, maybe you'd say "100 vec3s" is actually float[3][100] in which case prefix agreement would make more sense.
    • moonchild 6y ago
      Sibling explains the difference. To explain why prefix is better—first let's look at the base case where argument shapes match. Let's assume that addition is defined on scalars; we don't have to explain that 11=5+6. Then let's consider vectors: there are two forms of scalar broadcasting for vectors. The first is addition of two equal-length vectors, for which each item of the left argument will be broadcast to the corresponding item of the right argument: 10 14=3 5+7 9. The second is the addition of a scalar to a vector (or vice versa), where the scalar is matched up with each item of the vector: 10 14=6 10+4. For the first case, we can say that: R[i] = x[i] + y[i] (when x and y are vectors) (Where i is any valid array subscript; and R, x, and y are the names conventionally given to the result, left argument, and right argument of some function, respectively.) Assume that we extend our scalar rule to arbitrary dimensions (that is, scalar+n-dimensional array will do what we expect)—there is actually a good reason for this, but we'll just assume it for now; it's a pretty intuitive rule. Then the simple recursive rule I gave above gives you prefix agreement for any two argument shapes; just replace 'vector' with 'nonscalar'. Here are the rules for suffix agreement: R[i] = x[i] + y[i] (when x and y have the same rank) R[i] = x + y[i] (when y has bigger rank) R[i] = x[i] + y (when x has bigger rank) The prefix agreement rule is simple: add each element of x to its corresponding element in y. The suffix agreement rule is much more complex (3 rules, as opposed to just 1), and with higher-ranked arrays it gets harder to reason about which elements go together. There's an even deeper synergy, though, which goes between prefix agreement and forks. Forks are a generalisation of a convention from calculus. There's a convention that, given functions f and g, (f+g) is a legal function such that (f+g)(x) <=> f(x)+g(x). (And ditto for other arithmetic operators.) Apl does not distinguish between user-defined functions (like f) and infix builtins (like +); x f y denotes the calling of function f with arguments x and y, same as x + y denotes the calling of function + with arguments x and y. So the f+g rule is generalised: we say that, given any functions f, g, and h, (f g h) x is equivalent to (f x) g (h x). What does this have to do with prefix agreement? Well, all we have to do is remember that an array, as a relation of indices to elements, is very close to a function. In fact, I think that it's appropriate to say that an array is semantically a function, though its syntactic role is different. So saying x[i] is like saying 'apply function x to argument i'. Which leads very nicely—right into prefix agreement: (f g h) x <=> (f x) g (h x) . . . (x + y)[i] <=> (x[i]) + (y[i])
• jonahx 6y ago
  > are a little more verbose That's putting it mildly.
  • formerly_proven 6y ago
    NOTABUG CLOSED WONTFIX