3 ms·

> There is also a performance overhead with iterables Though, for the sake of pedantry, it's not something that can't be handled by an optimizing compiler that

> There is also a performance overhead with iterables Though, for the sake of pedantry, it's not something that can't be handled by an optimizing compiler that understands them. Haskell, for example, is good at this.

celrod 8y ago
For what it's worth, just sticking to the Fib example, since that's a fairly standard benchmark: struct FibonacciIterable{I} s0::I s1::I end import Base: iterate iterate(iter::FibonacciIterable) = iter.s0, (iter.s0, iter.s1) iterate(iter::FibonacciIterable, state) = state[2], (state[2], sum(state)) function iteratorfib(N) i = 0 x = 0 for F in FibonacciIterable(0,1) i += 1 x = F i > N && break end x end vs function loopfib(N) x0 = 0; x1 = 1 N == 0 && return 0 for i in 2:N x0, x1 = x1, x0+x1 end x1 end Then I get: julia> N = 30; julia> using BenchmarkTools julia> @benchmark iteratorfib($N) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 31.351 ns (0.00% GC) median time: 32.723 ns (0.00% GC) mean time: 32.891 ns (0.00% GC) maximum time: 54.525 ns (0.00% GC) -------------- samples: 10000 evals/sample: 994 julia> @benchmark loopfib($N) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 37.831 ns (0.00% GC) median time: 37.871 ns (0.00% GC) mean time: 38.616 ns (0.00% GC) maximum time: 105.626 ns (0.00% GC) -------------- samples: 10000 evals/sample: 992 If instead of writing "N=30" and interpolating it ($N) I just used N, I'd be benchmarking the dynamic dispatch. If I declared N constant, or just wrote 30 directly, the compiler turns into a cheater: julia> @benchmark iteratorfib(30) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 2.990 ns (0.00% GC) median time: 2.997 ns (0.00% GC) mean time: 3.030 ns (0.00% GC) maximum time: 56.348 ns (0.00% GC) -------------- samples: 10000 evals/sample: 1000 julia> @benchmark loopfib(30) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 2.990 ns (0.00% GC) median time: 2.997 ns (0.00% GC) mean time: 3.006 ns (0.00% GC) maximum time: 8.908 ns (0.00% GC) -------------- samples: 10000 evals/sample: 1000 Anyway, my points here are: 1) In this benchmark that should be focused on iteration, the custom iteration protocol actually seems faster than the for loop for some reason. 2) While for iterative algorithms like conjugate gradient, it does seem like a nice fit, it's probably not the clearest approach everywhere.
- bunderbunder 8y ago
  And Julia, too, apparently. I've really got to take a closer look at it. How does it compare to Python, ecosystem-wise? Pandas might be the thing I'd be most likely to miss in the stuff I'm doing.
  - ChrisRackauckas 8y ago
    There's a lot of generic numerical analysis libraries which is where it shines over Python. It matches pretty well in areas like data science and machine learning. For example, for tables there JuliaDB which is more akin to dask since it works out-of-core (but it's also able to use built-in online algorithms), DataFrames.jl which matches R, and Pandas.jl which is a straight wrapper for Python pandas.

celrod 8y ago

For what it's worth, just sticking to the Fib example, since that's a fairly standard benchmark: struct FibonacciIterable{I} s0::I s1::I end import Base: iterate iterate(iter::FibonacciIterable) = iter.s0, (iter.s0, iter.s1) iterate(iter::FibonacciIterable, state) = state[2], (state[2], sum(state)) function iteratorfib(N) i = 0 x = 0 for F in FibonacciIterable(0,1) i += 1 x = F i > N && break end x end vs function loopfib(N) x0 = 0; x1 = 1 N == 0 && return 0 for i in 2:N x0, x1 = x1, x0+x1 end x1 end Then I get: julia> N = 30; julia> using BenchmarkTools julia> @benchmark iteratorfib($N) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 31.351 ns (0.00% GC) median time: 32.723 ns (0.00% GC) mean time: 32.891 ns (0.00% GC) maximum time: 54.525 ns (0.00% GC) -------------- samples: 10000 evals/sample: 994 julia> @benchmark loopfib($N) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 37.831 ns (0.00% GC) median time: 37.871 ns (0.00% GC) mean time: 38.616 ns (0.00% GC) maximum time: 105.626 ns (0.00% GC) -------------- samples: 10000 evals/sample: 992 If instead of writing "N=30" and interpolating it ($N) I just used N, I'd be benchmarking the dynamic dispatch. If I declared N constant, or just wrote 30 directly, the compiler turns into a cheater: julia> @benchmark iteratorfib(30) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 2.990 ns (0.00% GC) median time: 2.997 ns (0.00% GC) mean time: 3.030 ns (0.00% GC) maximum time: 56.348 ns (0.00% GC) -------------- samples: 10000 evals/sample: 1000 julia> @benchmark loopfib(30) BenchmarkTools.Trial: memory estimate: 0 bytes allocs estimate: 0 -------------- minimum time: 2.990 ns (0.00% GC) median time: 2.997 ns (0.00% GC) mean time: 3.006 ns (0.00% GC) maximum time: 8.908 ns (0.00% GC) -------------- samples: 10000 evals/sample: 1000 Anyway, my points here are: 1) In this benchmark that should be focused on iteration, the custom iteration protocol actually seems faster than the for loop for some reason. 2) While for iterative algorithms like conjugate gradient, it does seem like a nice fit, it's probably not the clearest approach everywhere.

bunderbunder 8y ago
And Julia, too, apparently. I've really got to take a closer look at it. How does it compare to Python, ecosystem-wise? Pandas might be the thing I'd be most likely to miss in the stuff I'm doing.
- ChrisRackauckas 8y ago
  There's a lot of generic numerical analysis libraries which is where it shines over Python. It matches pretty well in areas like data science and machine learning. For example, for tables there JuliaDB which is more akin to dask since it works out-of-core (but it's also able to use built-in online algorithms), DataFrames.jl which matches R, and Pandas.jl which is a straight wrapper for Python pandas.