4 ms·
Thanks. This is really helpful. I tried to think of some specific example. I think most of what I'd generally come up with is tied to the available datastructur
by latch 8y ago
Thanks. This is really helpful. I tried to think of some specific example. I think most of what I'd generally come up with is tied to the available datastructures. We recently tested some code to essentially do a group-by in code, with various aggregation functions, and it was considerably slower in Elixir vs D and C. Very simplified, something like:
Enum.reduce(rows, %{}, fn [user_id, hits], agg ->
Map.update(agg, user_id, hits, &(&1 + hits))
end)
I didn't apply any of your suggestions.
If I had another performance-like related pain point, it's throwing multiple cores at some data. I think it's somewhat common to have a large piece of data that you want to concurrently access. For example, keeping a large suffix tree in memory for an autocomplete endpoint, and letting each request concurrently access it. This isn't really a specific performance issue, but it's a performance sensitive pattern that is very antithetical to everything Erlang. ETS has limited structures, requires copying of values, and has no atomic replace (so you can't build a new version in a temp table and just rename it).
- zinclozenge 8y agoSpecifically for the suffix tree in memory, you would store the tree as a constant in a module and dynamically recompile at runtime. By doing so it puts the tree in a special heap that can be read by all processes and doesn't get copied around. There's a library called 'mochiglobal' that does that for you. Discord ported it to Elixir and called it 'fastglobal' (although their implementation can be sped up by about ~25%).
- elcritch 8y agoTryout using Rustler and passing Refs. It’s a handy combination and Rust avoids most memory management issues one would face with NIF’s in C. I’m working on some data calculation code and think I’ll go that approach.