10 ms·

Erlang Garbage Collection Details and Why It Matters (2015)

114 points by jlturner 6y ago

dang 6y ago
If curious, past threads: Erlang Garbage Collection Details and Why It Matters - https://news.ycombinator.com/item?id=15095868 https://news.ycombinator.com/item?id=15095868 - Aug 2017 (24 comments) Erlang Garbage Collection Details and Why They Matter - https://news.ycombinator.com/item?id=10218343 https://news.ycombinator.com/item?id=10218343 - Sept 2015 (23 comments)
- arthurcolle 6y ago
  your browser extension sounds so sick (heard about it 20 minutes ago on the Crystal post) - any chance you'd make it available, or are you concerned some villain would use it to perform title based post manipulation?
  - stanislavb 6y ago
    Out of curiosity, what “title based” post manipulation?
  - calpaterson 6y ago
    If you want to see previous discussions for a link, you can try out my own tool if you like. eg for this post: https://quarchive.com/shares/bJ3iG3rg3UbvT5fxgjtCSUgr https://quarchive.com/shares/bJ3iG3rg3UbvT5fxgjtCSUgr EDIT: how to do it for yourself: register, bookmark the page in your browser and then open quarchive.com to see the comments on hn/reddit etc. If you have ideas send me an email
    - macintux 6y ago
      Or you can click on the "past" link under the title at the top of the page. (Note: not the "past" link at the very top of the page.)
  - brianush1 6y ago
    I use Refined Hacker News[0] to get a list of past threads at the bottom of the page. dang himself has said[1] it overlaps surprisingly with their internal extension, so it might prove useful [0]: https://github.com/plibither8/refined-hacker-news https://github.com/plibither8/refined-hacker-news [1]: https://news.ycombinator.com/item?id=20176460 https://news.ycombinator.com/item?id=20176460
  - xojoc 6y ago
    You may be interested in this form I'm building: https://discussions.xojoc.pw/?url=https%3A%2F%2Fhamidreza-s.github.io%2Ferlang%2520garbage%2520collection%2520memory%2520layout%2520soft%2520realtime%2F2015%2F08%2F24%2Ferlang-garbage-collection-details-and-why-it-matters.html https://discussions.xojoc.pw/?url=https%3A%2F%2Fhamidreza-s.... which searches for discussions on HN, Reddit, Lobsters, etc for a given url. It also has a bookmarklet so it can be used on any page very easly.
  - dang 6y ago
    The bottleneck is that it's a single codebase in which moderation functionality and ordinary functionality (browsing, voting, commenting, editing) are blended together. I'd need to tease those apart in order to release a public version, and it's not a high-enough priority compared to the other things that need doing. One of these days! The upside will be that when it does finally get released, it will be the fruit of many years of development, so there will be a lot of goodies in there.
bluedays 6y ago
> *Even if we are using a language that manages memory itself like Erlang, nothing prevents us from understanding how memory is allocated and deallocated. Unlike Go Language Memory Model Documentation Page that advices “If you must read the rest of this document to understand the behavior of your program, you are being too clever. Don’t be clever.”, I believe that we must be clever enough to make our system faster and safer, and sometimes it doesn’t happen unless we dig deeper into what is going on.* I dunno, article seems right about this. Golang docs sound condescending.
- blacktriangle 6y ago
  Golang itself is condescending. It was basically invented as a way to get fresh CS grads productive ASAP with as little understanding as possible.
  - brudgers 6y ago
    Google is an enterprise. Go is an enterprise language. For the usual enterprise reasons: bean counting, internal politics, and cubicles. It is a great language because it solves Google's problems. That's true of all Google tools and services. To the degree a person has the same problems as Google, their tools are great. Most people and organizations don't have Google's problems to a significant degree.
    - bsaul 6y ago
      "Most people and organizations don't have Google's problems to a significant degree" Except in this particular case, i believe the average junior Googler is far more capable of handling advanced language as the average junior employee of any regular corp. Which means go's tradeoff toward "simplicity" is probably the wisest for them as well.
      - blacktriangle 6y ago
        Honestly I'd argue that that is more of a Google/large corporate problem. The thing about junior devs is that in order to hire a bunch of them and get anything productive out of them, you need a ton of people above them. You need to spend more time on design to give them more clear requirements, you need more senior devs to provide mentorship, you need more management. Smaller dev shops don't have that, they can only bring on 1-2 juniors at a time and hope for the best.
dnautics 6y ago
I think also super interesting is how erlang gives you hooks in it's FFI to create and destroy C ABI entities that are refcounted and thus tied to the GC.
crazydoggers 6y ago
A useful trick for long lived processes where you also care about GC pauses is to run erlang:garbage_collect(pid) If you’ve got real time requirements you can often minimize GC impact by choosing when in the process lifecycle the GC occurs. https://erlang.org/doc/man/erlang.html#garbage_collect-1 https://erlang.org/doc/man/erlang.html#garbage_collect-1
- hinkley 6y ago
  I had a waveform animation that would get jerky periodically due to the GC sawtooth. Given a relatively fixed cost per frame, the GC would tend to come every N frames, which is the sort of pattern the human brain is good at picking up on. This meant the behavior was perceptually 'worst' when you just sat and stared at the screen, doing nothing else. So I just started forcing a GC any time the buffers were full at the end of the previous frame. The next frame could be drawn entirely from buffer, which only took a couple of milliseconds, reducing the potential jitter from a few frames to maybe half a frame. You'd still get random stutters but there was no pattern to it anymore, unless the network was being very badly behaved (which in this case was useful information). Not only did that smooth out the display but it let us increase the max number of things you could display by a little over half.

114 points by jlturner 6y ago

dang 6y ago

If curious, past threads: Erlang Garbage Collection Details and Why It Matters - https://news.ycombinator.com/item?id=15095868 https://news.ycombinator.com/item?id=15095868 - Aug 2017 (24 comments) Erlang Garbage Collection Details and Why They Matter - https://news.ycombinator.com/item?id=10218343 https://news.ycombinator.com/item?id=10218343 - Sept 2015 (23 comments)

arthurcolle 6y ago
your browser extension sounds so sick (heard about it 20 minutes ago on the Crystal post) - any chance you'd make it available, or are you concerned some villain would use it to perform title based post manipulation?
- stanislavb 6y ago
  Out of curiosity, what “title based” post manipulation?
- calpaterson 6y ago
  If you want to see previous discussions for a link, you can try out my own tool if you like. eg for this post: https://quarchive.com/shares/bJ3iG3rg3UbvT5fxgjtCSUgr https://quarchive.com/shares/bJ3iG3rg3UbvT5fxgjtCSUgr EDIT: how to do it for yourself: register, bookmark the page in your browser and then open quarchive.com to see the comments on hn/reddit etc. If you have ideas send me an email
  - macintux 6y ago
    Or you can click on the "past" link under the title at the top of the page. (Note: not the "past" link at the very top of the page.)
- brianush1 6y ago
  I use Refined Hacker News[0] to get a list of past threads at the bottom of the page. dang himself has said[1] it overlaps surprisingly with their internal extension, so it might prove useful [0]: https://github.com/plibither8/refined-hacker-news https://github.com/plibither8/refined-hacker-news [1]: https://news.ycombinator.com/item?id=20176460 https://news.ycombinator.com/item?id=20176460
- xojoc 6y ago
  You may be interested in this form I'm building: https://discussions.xojoc.pw/?url=https%3A%2F%2Fhamidreza-s.github.io%2Ferlang%2520garbage%2520collection%2520memory%2520layout%2520soft%2520realtime%2F2015%2F08%2F24%2Ferlang-garbage-collection-details-and-why-it-matters.html https://discussions.xojoc.pw/?url=https%3A%2F%2Fhamidreza-s.... which searches for discussions on HN, Reddit, Lobsters, etc for a given url. It also has a bookmarklet so it can be used on any page very easly.
- dang 6y ago
  The bottleneck is that it's a single codebase in which moderation functionality and ordinary functionality (browsing, voting, commenting, editing) are blended together. I'd need to tease those apart in order to release a public version, and it's not a high-enough priority compared to the other things that need doing. One of these days! The upside will be that when it does finally get released, it will be the fruit of many years of development, so there will be a lot of goodies in there.

bluedays 6y ago

> *Even if we are using a language that manages memory itself like Erlang, nothing prevents us from understanding how memory is allocated and deallocated. Unlike Go Language Memory Model Documentation Page that advices “If you must read the rest of this document to understand the behavior of your program, you are being too clever. Don’t be clever.”, I believe that we must be clever enough to make our system faster and safer, and sometimes it doesn’t happen unless we dig deeper into what is going on.* I dunno, article seems right about this. Golang docs sound condescending.

blacktriangle 6y ago
Golang itself is condescending. It was basically invented as a way to get fresh CS grads productive ASAP with as little understanding as possible.
- brudgers 6y ago
  Google is an enterprise. Go is an enterprise language. For the usual enterprise reasons: bean counting, internal politics, and cubicles. It is a great language because it solves Google's problems. That's true of all Google tools and services. To the degree a person has the same problems as Google, their tools are great. Most people and organizations don't have Google's problems to a significant degree.
  - bsaul 6y ago
    "Most people and organizations don't have Google's problems to a significant degree" Except in this particular case, i believe the average junior Googler is far more capable of handling advanced language as the average junior employee of any regular corp. Which means go's tradeoff toward "simplicity" is probably the wisest for them as well.
    - blacktriangle 6y ago
      Honestly I'd argue that that is more of a Google/large corporate problem. The thing about junior devs is that in order to hire a bunch of them and get anything productive out of them, you need a ton of people above them. You need to spend more time on design to give them more clear requirements, you need more senior devs to provide mentorship, you need more management. Smaller dev shops don't have that, they can only bring on 1-2 juniors at a time and hope for the best.

dnautics 6y ago

I think also super interesting is how erlang gives you hooks in it's FFI to create and destroy C ABI entities that are refcounted and thus tied to the GC.

crazydoggers 6y ago

A useful trick for long lived processes where you also care about GC pauses is to run erlang:garbage_collect(pid) If you’ve got real time requirements you can often minimize GC impact by choosing when in the process lifecycle the GC occurs. https://erlang.org/doc/man/erlang.html#garbage_collect-1 https://erlang.org/doc/man/erlang.html#garbage_collect-1

hinkley 6y ago
I had a waveform animation that would get jerky periodically due to the GC sawtooth. Given a relatively fixed cost per frame, the GC would tend to come every N frames, which is the sort of pattern the human brain is good at picking up on. This meant the behavior was perceptually 'worst' when you just sat and stared at the screen, doing nothing else. So I just started forcing a GC any time the buffers were full at the end of the previous frame. The next frame could be drawn entirely from buffer, which only took a couple of milliseconds, reducing the potential jitter from a few frames to maybe half a frame. You'd still get random stutters but there was no pattern to it anymore, unless the network was being very badly behaved (which in this case was useful information). Not only did that smooth out the display but it let us increase the max number of things you could display by a little over half.