pkolaczk

22 ms·

271.
by pkolaczk 3y ago
Trying to raise the limit, but it didn't work: $ /usr/bin/time -v elixir --erl "-P 10000000" main.exs 1000000 08:42:56.594 [error] Too many processes ** (SystemLimitError) a system limit has been
272.
by pkolaczk 3y ago
Thanks for the suggestion. I fixed it and indeed it improved memory consumption of NodeJS.
273.
by pkolaczk 3y ago
Thank you! I fixed the code and updated the benchmarks. However, your 166 MB doesn't match my testing (I'm getting about 240M at 100k tasks) and 10x more at 1M. Are you sure you haven't misread your RSS?
274.
by pkolaczk 3y ago
Then this should be applied to all benchmarks not just Java.
275.
by pkolaczk 3y ago
In real code I saw Rust beating Go and Java based code by 10x-50x in memory use. But too complex and too many factors to analyze, so not a material for a blog post. Actually this tiny benchmark favors GC based runtimes because it just sleep
276.
by pkolaczk 3y ago
Stack memory is allocated lazily on Linux. I thought this benchmarks shows it pretty clearly. Otherwise 10k threads would be already in GBs territory, when in fact it is below 100 MB.
277.
by pkolaczk 3y ago
GIL is not a problem for concurrency. It is a problem for parallelism when doing CPU bound work.
278.
by pkolaczk 3y ago
> Even the naive programmer just banging together something that works in Elixir is going to quickly find out how to raise Erlang’s default 260k process limit when they hit that, after all. Sure. Will redo this part. I doubt it would all
279.
by pkolaczk 3y ago
You can always pass the async executor to the sync code and spawn async coroutines into it. And you can keep it in a global context as well to avoid parameters. E.g. there is `Handle::current()` for exactly this purpose in Tokio. Function c
280.
by pkolaczk 3y ago
Just read the text. It lists versions and states it was using defaults in the release mode. Most of those compilers don't have any fancy options actually. E.g go is just `go build`, Python is just `python3 ...`. Rust options are in the
281.
by pkolaczk 3y ago
Because goroutine stacks start at minimum 2 kB. But it is not obvious if this is the only factor that contributes to memory usage so I believe experimenting to verify theory is still quite valuable.
282.
by pkolaczk 3y ago
Ok, this one is funny. Actually ChatGPT got it right and there was a create_task before, but right before committing I "inlined it" and introduced a bug. Thanks for letting me know. Will double check if it didn't messed up th
283.
by pkolaczk 3y ago
I don't know how C#, but Rust async/await doesn't allocate anything on the heap by itself. So it is not a universal property of all async/await implementations contrary to what you imply in your blog post.
284.
by pkolaczk 3y ago
If it was true, the results would be much worse. The average from my measurements is a bit above 2 kB. This website says it is 2 kB: https://tpaschalis.me/goroutines-size/
285.
How much memory do you need to run 1M concurrent tasks?(pkolaczk.github.io)
253 points by pkolaczk 3y ago|214 comments
286.
by pkolaczk 3y ago
It's not any worse than introducing a new abstraction when there previously wasn't one. At least in a statically typed language one will quickly identify those touch points. When introducing a new abstraction there is no such help
287.
by pkolaczk 3y ago
Hard disagree here. Encapsulation is just one aspect of implementing the abstraction. And I'm not even convinced it is the best way here. Leaving the ranges as simple structures with two public fields, but introducing an "infinity
288.
by pkolaczk 3y ago
But that's still insufficient abstraction and my general point holds.
289.
by pkolaczk 3y ago
Sure you can probably also do it, but this is not the way how the code was originally written. The original ranges present the bounds in their public API, and most code just operated on them.
290.
by pkolaczk 3y ago
Scala is great, but the tooling is what ruined it for me. SBT is far from simple, Scala 3 basically reset the IDE suport, the compiler is abysmally slow (if you think Rust compiler is not fast, don't look at Scala which is like order o
291.
by pkolaczk 3y ago
Rust can have some bad effects on your brains. I noticed I can't read Java code as easily as before. E.g. yesterday I had to read it multiple times before I understood what was going on and why they are calling setters on an item that
292.
by pkolaczk 3y ago
I wasnt talking specifically about real (float) numbers, but yes - this is that abstraction. And it generalizes to any type with ordering (can work with integral types as well).
293.
by pkolaczk 3y ago
I don't remember who said that, but mathematics is all about building abstractions, not about computation. So many times mathematics helped me make code simpler.
294.
by pkolaczk 3y ago
The three cases force every code using ranges to deal with them. Apply this way if thinking many times for multiple concepts and you end up with a spaghetti of multiply nested if statements that's near impossible to analyze for corre
295.
by pkolaczk 3y ago
While I agree with the statements made in the original post, I'm afraid this thinking can be used as an excuse for avoiding any attempts at finding proper abstractions. Similarly to how the term "premature optimisation" is so
296.
by pkolaczk 3y ago
If so few people say yes/no then this means to me that either the difficulty is not really as bad as some people try to paint it here (under assumption people don't want to continue using a language that is frustrating), or that i
297.
by pkolaczk 3y ago
> Safe Rust cannot represent circular data structures which makes entire classes of algorithms and architectures unimplementable. That's not accurate. Safe Rust can represent circular data structures. It cannot represent circular
298.
by pkolaczk 3y ago
You can always tell it to shut up by using pointers and unsafe.
299.
by pkolaczk 3y ago
How come Rust appears at the top of most loved languages in the SO survey so many years in a row? And doesn't even make into the "most dreaded" list at all (when many simpler languages do make into that list)? Also, there are
300.
by pkolaczk 3y ago
Prior experience and preferred style of programming can affect severely the difficulty of learning Rust. For me it was mostly 1 week of struggling a bit with the borrow-checker and even that only in places where I really wanted to avoid cop

More ›