6 ms·
They debugged the problem to be excessive allocations in the tight loop of the TSV parser. If you allocate a separate object per each field, I can imagine how i
by pkolaczk 3y ago
They debugged the problem to be excessive allocations in the tight loop of the TSV parser. If you allocate a separate object per each field, I can imagine how it can result in throughput counted in kB/s instead of GB/s.
We have had very similar issues with Java - excessive heap allocations can bring the system to a crawl and there is often no easy way out.
- DanielHB 3y agoA common technique (at least in Java, never seen it in nodejs) is to have a pool of pre-allocated objects and just reuse them as they are discarded. Essentially manual GC
- DrDroop 3y agoHappens JS world aswel SimonDev did a video about it, this one I think: https://www.youtube.com/watch?v=easvMCCBFkQ https://www.youtube.com/watch?v=easvMCCBFkQ This channel is a real gem, show what talent really is: https://www.youtube.com/watch?v=a1L7k35EHIc https://www.youtube.com/watch?v=a1L7k35EHIc
- KeyXiote 3y agoThanks for the vid links, always looking for new channels with informative content.
- pkolaczk 3y agoIt is easier said than done, when the whole ecosystem is based on libraries and frameworks that allocate like crazy. Preallocating and reusing introduces complexity and increases likelihood of bugs. And at least in Java you can run into surprising GC troubles with pooling as well. Object pools tend to be promoted to a tenured generation and if you do it too much, you essentially break the generational hypothesis many GCs are optimized for. Generational GCs don't like dealing with objects with long lifetimes.
- didntcheck 3y agoI recall hearing there's something of a parallel ecosystem of near-zero-allocation Java libraries, used in HFT among other things. I'm unsure why people use a GC'd language in the first place if they're going to go to those lengths to avoid it, but presumably there are good reasons (perhaps memory safety, before Rust came along)
- DanielHB 3y agoYou use it only for the one class you are allocating a million times per second and let the other ones be GC, often this technique is used within a single heavy function with a tight loop Most of your application probably doesn't need it and the tight loop might not be big enough to be worth using JNI to write in a non-GC language (as well as maintaining another compiler stack for) Ideally the runtime should allow you to manually manage memory when you need to, like some keyword that lets you manually keep track of an object lifetime. But as far as I know most GC languages don't have that
- JackFr 3y agoObject pooling is typically not a common pattern in Java these days outside of HFT stuff, and even in those cases it’s often not performance, but determinism that’s the issue. (Connection pooling is still common as with other expensive objects, just not plain object pooling.) Java’s garbage collectors are excellent if you cut with the grain.
- DanielHB 3y agoYeah, you would only use it in very tight loops in special cases, ideally only within a single heavy function. It would be madness to do object pooling across a large scale codebase
- Cthulhu_ 3y agoThat reminds me of how iOS could achieve fast scrolling of lists, which Facebook and Twitter struggled with for years because they tried to stick with HTML. Basically if you scroll down, a list element from above is moved to the bottom and has its contents replaced. Anyway, I'm not sure if this is in Java already or is upcoming, but Java will finally add value objects, that is, stack variables instead of just making everything a heap object. For tight loops that will be much more efficient, since it can just reuse the same memory, stay within the cpu registers, etc. (I'm talking out of my ass btw from a superficial knowledge of Go and Java and half-remembered theory from college).
- MrJohz 3y agoVirtual scrolling is a very common technique for rendering large tables or lists in HTML (and pretty much any other environment), and is by no means unique to iOS...
- lawgimenez 3y agoAnd also common in Android from the very beginning of their ListView now called RecyclerView.
- masfuerte 3y agoThe Windows 3.1 list box supported it.
- orf 3y agoThey didn't debug anything. They just stuck a job queue on it and then rewrote part of it in Rust. That's the point of my comment. Using their code in the post (gist here[1]): $ pv lol.tsv | node lol.js 1.71GiB 0:00:07 [ 225MiB/s] [================================>] 100% Finished reading the file. Do we live in a world where engineers using a mature language like NodeJS with a pretty simple line-by-line file reading process and a string split think they hit a hard wall at 700kb/s, and the problem is not their bad code? 1. https://gist.github.com/orf/92bc26381e3dc00c8e96af262768591b https://gist.github.com/orf/92bc26381e3dc00c8e96af262768591b
- nerdponx 3y agoBut is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. The point of the article is that you can write really boring normal code in NodeJS and it's really slow, or you can write really boring normal code in Rust and it's a lot faster, and it's not that hard to use the latter to replace hot paths in the former.
- disgruntledphd2 3y agoIt's probably all the memory allocations. I know that when I do work like this, creating a huge array up front and assigning to that typically results in massive speedups. I know basically nothing about Node though, so I could be completely wrong here.
- Karrot_Kream 3y agoYeah that's one way to do it. Another is to reuse structures where possible to not force reallocation. There's a bunch of ways.
- xmcqdpt2 3y agoNode.js has a decent JIT, or so I've heard. I have experience with the HotSpot JVM. For the JVM this code would absolutely stop allocating on the heap past 10k iterations of the loop. "Split a string then grab the parts in a hot loop" is ground stakes for JIT compilers. If they are seeing any GC activity once the loop gets hot, something else is going very wrong.
- xmcqdpt2 3y agoBy the way the very same code in Java would absolutely rip through the files with no extra allocations of arrays once the JIT gets seriously involved. A big loop with temporary arrays in it that are used to populate the fields of a record every iteration is HotSpot C2's favourite thing. Does node not similarly allocate on the stack in hot loops?
- pkolaczk 3y agoDid you actually test it? Some time ago someone at another forum made a similar claim that returning an Optional from a method in Java will be surely optimized as we well, but it turned out that it wasn't (https://pkolaczk.github.io/overhead-of-optional/ https://pkolaczk.github.io/overhead-of-optional/).
- xmcqdpt2 3y agoGood point, I should probably check.