3 ms·

What do you mean by closed ranges composition ?

Iwan-Zotow 8y ago
Suppose I have to compute sum for array of length N. In python it is something like s = 0 for k in range(0, N): s += a[k] Ok, I have more cores/CPUs, could I do it in parallel? Sure v1 = Sum(0, N/2) v2 = Sum(N/2, N) s = v1 + v2 I even could do it on asymmetric cores (like most phone CPUs today) v1 = Sum(0, K) v2 = Sum(K, N) s = v1 + v2 I could do a bit more complicated things, like calculating means. For that, I have to make a monoid def mean(from, to, a): m = 0 for k in range(from, to): m += a[k] N = to - from return (m/N, N) Here I'm returning tuple, and for that to be a monoid I have to state composition law: def compose(M1, M2): A1, N1 = M1 A2, N2 = M2 N = N1+N2 return ((A1N1 + A2N2)/N, N) If my identity is (0,0) tuple, it is quite easy to verify that indeed I have a monoid. Nice, simple, composable ranges. For closed [1...N] ranges this is NOT nice, NOT simple, just plain fugly exercise. Sorry, looks like code samples are screwed up a bit, don't know how to fix it
- mcabbott 8y ago
  But why should you have to care, why not just sum(range)? Or @threads for i in eachindex(object) and let the details about how many cores be written once, correctly, elsewhere?
  - Iwan-Zotow 8y ago
    > But why should you have to care, why not just sum(range)? Sure, if you have single infinitely fast CPU. In real life, we have to decompose problem, run it in parallel and compose it back. > Or @threads for i in eachindex(object) & let the details about how many cores be written once, correctly, elsewhere? Here I'm explicitly talking about details, how it should be done, importance of composition, monoids etc. I understand, that if someone did it for you, you might not care - well, more power to you then. But some basics rules about Pyton ranges are pretty good: 1. [0...K) + [K...N) = [0...N) 2. Given range [K...N), number of elements in the range is N-K, in case of [0...N) number of elements is N-0=N. Simple, elegant, composable. I can't say the same about Julia
    - hhmc 8y ago
      1-indexed ranges can still be composable if we define them as open on the right at N+1 (which is how I imagine one would want to define things in their implementation). [1...N] = [1...N+1) = [1...K+1) + [K+1..N+1) So the real loss is your point (2) - which, I agree, makes the implementation much less elegant and simple.
      - Iwan-Zotow 8y ago
        > 1-indexed ranges can still be composable you could make them work, but improper abstraction leaks out in so many fugly ways. F.e. in Python you could compose functions, not only ranges. What do you pass in? Simple, [0, len(a)). What you get out? Simple, len(a). So you could operate on ranges composing function calls like in FP. Even works for unknown beforehand sequences/streams, just count along the sequence how many events you processed, and return it as length, and it could go into composition function. With Julia you have to think what is passed and what is returned. Shall I always return len(a)? Or maybe len(a)+1? What to return in case of streaming events? Sometimes len(a) and sometimes len(a)+1 with tons of comments and warnings? It is not a good way to deal with all that and not a good way to build API. Could be done and probably was already done, sure. BUt simplicity, elegance and composability is missing in the base design.
        newen 8y ago
        I don't buy it. Inclusive ranges are much more intuitive than left side inclusive and right side exclusive ranges. And they compose just fine; you just have to add a plus one at the right places. E.g. function mean(a, l, r) s = 0 N = r-l+1 for k in l:r s += a[k] end s/N, N end function mean2(a) N = length(a) lr1 = (1,div(N,2)) lr2 = (div(N,2)+1,N) v1, N1 = mean(a, lr1...) v2, N2 = mean(a, lr2...) N = N1+N2 (v1 * N1 + v2 * N2) / N, N end function compose(M1, M2) v1, N1 = M1 v2, N2 = M2 N = N1+N2 (v1 * N1 + v2 * N2) / N end function mean3(a) N = length(a) lr1 = (1,div(N,2)) lr2 = (div(N,2)+1,N) compose(mean(a, lr1...), mean(a, lr2...)) end You have to think about what is passed in and out anyway. Arguments of elegance etc. are most of the time personal preferences and very biased.

Suppose I have to compute sum for array of length N. In python it is something like s = 0 for k in range(0, N): s += a[k] Ok, I have more cores/CPUs, could I do it in parallel? Sure v1 = Sum(0, N/2) v2 = Sum(N/2, N) s = v1 + v2 I even could do it on asymmetric cores (like most phone CPUs today) v1 = Sum(0, K) v2 = Sum(K, N) s = v1 + v2 I could do a bit more complicated things, like calculating means. For that, I have to make a monoid def mean(from, to, a): m = 0 for k in range(from, to): m += a[k] N = to - from return (m/N, N) Here I'm returning tuple, and for that to be a monoid I have to state composition law: def compose(M1, M2): A1, N1 = M1 A2, N2 = M2 N = N1+N2 return ((A1N1 + A2N2)/N, N) If my identity is (0,0) tuple, it is quite easy to verify that indeed I have a monoid. Nice, simple, composable ranges. For closed [1...N] ranges this is NOT nice, NOT simple, just plain fugly exercise. Sorry, looks like code samples are screwed up a bit, don't know how to fix it

mcabbott 8y ago
But why should you have to care, why not just sum(range)? Or @threads for i in eachindex(object) and let the details about how many cores be written once, correctly, elsewhere?
- Iwan-Zotow 8y ago
  > But why should you have to care, why not just sum(range)? Sure, if you have single infinitely fast CPU. In real life, we have to decompose problem, run it in parallel and compose it back. > Or @threads for i in eachindex(object) & let the details about how many cores be written once, correctly, elsewhere? Here I'm explicitly talking about details, how it should be done, importance of composition, monoids etc. I understand, that if someone did it for you, you might not care - well, more power to you then. But some basics rules about Pyton ranges are pretty good: 1. [0...K) + [K...N) = [0...N) 2. Given range [K...N), number of elements in the range is N-K, in case of [0...N) number of elements is N-0=N. Simple, elegant, composable. I can't say the same about Julia
  - hhmc 8y ago
    1-indexed ranges can still be composable if we define them as open on the right at N+1 (which is how I imagine one would want to define things in their implementation). [1...N] = [1...N+1) = [1...K+1) + [K+1..N+1) So the real loss is your point (2) - which, I agree, makes the implementation much less elegant and simple.
    - Iwan-Zotow 8y ago
      > 1-indexed ranges can still be composable you could make them work, but improper abstraction leaks out in so many fugly ways. F.e. in Python you could compose functions, not only ranges. What do you pass in? Simple, [0, len(a)). What you get out? Simple, len(a). So you could operate on ranges composing function calls like in FP. Even works for unknown beforehand sequences/streams, just count along the sequence how many events you processed, and return it as length, and it could go into composition function. With Julia you have to think what is passed and what is returned. Shall I always return len(a)? Or maybe len(a)+1? What to return in case of streaming events? Sometimes len(a) and sometimes len(a)+1 with tons of comments and warnings? It is not a good way to deal with all that and not a good way to build API. Could be done and probably was already done, sure. BUt simplicity, elegance and composability is missing in the base design.
      - newen 8y ago
        I don't buy it. Inclusive ranges are much more intuitive than left side inclusive and right side exclusive ranges. And they compose just fine; you just have to add a plus one at the right places. E.g. function mean(a, l, r) s = 0 N = r-l+1 for k in l:r s += a[k] end s/N, N end function mean2(a) N = length(a) lr1 = (1,div(N,2)) lr2 = (div(N,2)+1,N) v1, N1 = mean(a, lr1...) v2, N2 = mean(a, lr2...) N = N1+N2 (v1 * N1 + v2 * N2) / N, N end function compose(M1, M2) v1, N1 = M1 v2, N2 = M2 N = N1+N2 (v1 * N1 + v2 * N2) / N end function mean3(a) N = length(a) lr1 = (1,div(N,2)) lr2 = (div(N,2)+1,N) compose(mean(a, lr1...), mean(a, lr2...)) end You have to think about what is passed in and out anyway. Arguments of elegance etc. are most of the time personal preferences and very biased.