12 ms·

nickel's performance definitely will need some work. It's several orders of magnitude slower than nix for even simple tasks: $ time nix eval --expr "builti

nickel's performance definitely will need some work. It's several orders of magnitude slower than nix for even simple tasks: $ time nix eval --expr "builtins.foldl' (l: r: if l > r then l else r) 0 (builtins.genList (x: x) 5000000)" 0.839s memory: 627 MB $ time nickel eval <<<"std.array.fold_left std.number.max 0 (std.array.generate (fun x => x) 5000000)" 1:20.06s memory: 10540 MB I know we don't actually need to deal with 5 million element lists in practical code, but this is also not nickel's most pathological case, and it's easy to write fairly reasonable and normal code which is prohibitively slow. I rewrote some code I had laying around from nix, where it evaluated in a mere 3s or so (using perhaps a GiB of memory), into nickel. At first the nickel version crashed after using all my memory, but with hours of optimization, I managed to get it to run in just under 2 hours with only 60GiB of memory usage.

yagrouze 2y ago
That looks bad indeed. I must say that contracts, while very nice for error reporting and all, also brings additional challenge related to performance (cf https://dl.acm.org/doi/10.1145/2914770.2837630 https://dl.acm.org/doi/10.1145/2914770.2837630). Our stance is that it's a reasonable trade off for configuration, but the cost would be unacceptable in a general purpose language. Also, it's true that we've focused on language design and tooling before performance, and then working our way when users report unacceptable perfs on their concrete use-case. There's been some improvement recently-ish. For example, with the latest Nickel, I get (by inlining `max` to mimic the Nix version): $ time nix eval --expr "builtins.foldl' (l: r: if l > r then l else r) 0 (builtins.genList (x: x) 5000000)" 0,74s user 0,11s $ time nickel eval <<<"std.array.fold_left (fun x y => if x > y then x else y) 0 (std.array.generate (fun x => x) 5000000)" 36,63s user 3,40s Nothing to be too excited about, though. Although it doesn't make it unusable in small and medium-sized projects (we have users with 120kLoc of Nickel - with they'd definitively like to make it faster, it's still usable), I agree that performance is a big area of improvement, to say the least.
yagrouze 2y ago
(also, it seems `foldl'` is an ad-hoc builtin operation in Nix, probably for perf reasons, so the comparison isn't entirely fair either)

yagrouze 2y ago

That looks bad indeed. I must say that contracts, while very nice for error reporting and all, also brings additional challenge related to performance (cf https://dl.acm.org/doi/10.1145/2914770.2837630 https://dl.acm.org/doi/10.1145/2914770.2837630). Our stance is that it's a reasonable trade off for configuration, but the cost would be unacceptable in a general purpose language. Also, it's true that we've focused on language design and tooling before performance, and then working our way when users report unacceptable perfs on their concrete use-case. There's been some improvement recently-ish. For example, with the latest Nickel, I get (by inlining `max` to mimic the Nix version): $ time nix eval --expr "builtins.foldl' (l: r: if l > r then l else r) 0 (builtins.genList (x: x) 5000000)" 0,74s user 0,11s $ time nickel eval <<<"std.array.fold_left (fun x y => if x > y then x else y) 0 (std.array.generate (fun x => x) 5000000)" 36,63s user 3,40s Nothing to be too excited about, though. Although it doesn't make it unusable in small and medium-sized projects (we have users with 120kLoc of Nickel - with they'd definitively like to make it faster, it's still usable), I agree that performance is a big area of improvement, to say the least.

yagrouze 2y ago

(also, it seems `foldl'` is an ad-hoc builtin operation in Nix, probably for perf reasons, so the comparison isn't entirely fair either)