3 ms·

To play Devil’s Advocate, this is a good test of “how many concurrent tasks will an uncritical developer relying on ChatGPT end up achieving with low effort”.

• latency-guy2 3y ago
  Time to fire a bunch of people then
• tialaramex 3y ago
  Right, realistically your product will not primarily be used by the most talented experts, who are optimising as best as possible given intimate knowledge of internals. People will do what basically works, and it's arguably doing a disservice when "benchmarks" avoid showing us also naive performance without the tuning. The tuned performance is worth knowing, but realistically most customers will get the naive performance, ain't nobody got time to learn to tune everything. I think this applies all the way down to hardware features and programming languages. For example the naive use of sort() in C++ will go faster than in Rust (in C++ that's an unstable sort in Rust it's stable) but may astonish naive programmers (if you don't know what an unstable sort is, or didn't realise that's what the C++ function does). Or opposite example, Rust's Vec::reserve is actively good to call in your code that adds a bunch of things to a Vec, it never destroys amortized growth, but C++ std::vector reserve does destroy amortized growth so you should avoid calling it unless you know the final size of the std::vector.
  • fwlr 3y ago
    That is a fair point. I don’t think it saves this particular analysis, of course. Probably what’s important to know is something we might call the “Pareto naive performance”; with just a little bit of effort (the first 20% or so), you can get some significant performance improvement (the first 80% or so). Even the naive programmer just banging together something that works in Elixir is going to quickly find out how to raise Erlang’s default 260k process limit when they hit that, after all.
    • pkolaczk 3y ago
      > Even the naive programmer just banging together something that works in Elixir is going to quickly find out how to raise Erlang’s default 260k process limit when they hit that, after all. Sure. Will redo this part. I doubt it would allow Elixir to get a better result in the benchmark, though, as it was already losing significantly at 100k tasks. Any hints on how to decrease Elixir memory usage? Many people criticize using defaults in the comments, but don't suggest any certain settings that would improve the results. And a part of blogging experience is too learn things - also from the author perspective ;)
      • deleted 3y ago
        [deleted]
      • oshanz 3y ago
        1..num_tasks |> Task.async_stream(fn _ -> :timer.sleep(10000) end, max_concurrency: num_tasks) |> Stream.run() https://elixirforum.com/t/how-much-memory-do-you-need-to-run-1-million-concurrent-tasks/56009/9 https://elixirforum.com/t/how-much-memory-do-you-need-to-run...
    • pkolaczk 3y ago
      Trying to raise the limit, but it didn't work: $ /usr/bin/time -v elixir --erl "-P 10000000" main.exs 1000000 08:42:56.594 [error] Too many processes ** (SystemLimitError) a system limit has been reached Any other ideas how to raise the limit?
      • fwlr 3y ago
        I asked ChatGPT and it said “This limit can be adjusted with the `+P` flag when starting the BEAM… To adjust the limit, you could start the Elixir application using a command like the following: elixir --erl "+P 1000000" “If you are getting this error not because of the BEAM limit, but rather because of your operating system limit (like the limit on the number of open files or the number of child processes), you will need to look into how to increase these limits on your specific operating system.”