9 ms·

> having 25 Node.js instances running for three hours to parse 200GB of data seems like an indication that something isn't quite right. That’s about 700kb/s.

• pkolaczk 3y ago
  They debugged the problem to be excessive allocations in the tight loop of the TSV parser. If you allocate a separate object per each field, I can imagine how it can result in throughput counted in kB/s instead of GB/s. We have had very similar issues with Java - excessive heap allocations can bring the system to a crawl and there is often no easy way out.
  • DanielHB 3y ago
    A common technique (at least in Java, never seen it in nodejs) is to have a pool of pre-allocated objects and just reuse them as they are discarded. Essentially manual GC
    • DrDroop 3y ago
      Happens JS world aswel SimonDev did a video about it, this one I think: https://www.youtube.com/watch?v=easvMCCBFkQ https://www.youtube.com/watch?v=easvMCCBFkQ This channel is a real gem, show what talent really is: https://www.youtube.com/watch?v=a1L7k35EHIc https://www.youtube.com/watch?v=a1L7k35EHIc
      • KeyXiote 3y ago
        Thanks for the vid links, always looking for new channels with informative content.
    • pkolaczk 3y ago
      It is easier said than done, when the whole ecosystem is based on libraries and frameworks that allocate like crazy. Preallocating and reusing introduces complexity and increases likelihood of bugs. And at least in Java you can run into surprising GC troubles with pooling as well. Object pools tend to be promoted to a tenured generation and if you do it too much, you essentially break the generational hypothesis many GCs are optimized for. Generational GCs don't like dealing with objects with long lifetimes.
      • didntcheck 3y ago
        I recall hearing there's something of a parallel ecosystem of near-zero-allocation Java libraries, used in HFT among other things. I'm unsure why people use a GC'd language in the first place if they're going to go to those lengths to avoid it, but presumably there are good reasons (perhaps memory safety, before Rust came along)
        • DanielHB 3y ago
          You use it only for the one class you are allocating a million times per second and let the other ones be GC, often this technique is used within a single heavy function with a tight loop Most of your application probably doesn't need it and the tight loop might not be big enough to be worth using JNI to write in a non-GC language (as well as maintaining another compiler stack for) Ideally the runtime should allow you to manually manage memory when you need to, like some keyword that lets you manually keep track of an object lifetime. But as far as I know most GC languages don't have that
      • JackFr 3y ago
        Object pooling is typically not a common pattern in Java these days outside of HFT stuff, and even in those cases it’s often not performance, but determinism that’s the issue. (Connection pooling is still common as with other expensive objects, just not plain object pooling.) Java’s garbage collectors are excellent if you cut with the grain.
      • DanielHB 3y ago
        Yeah, you would only use it in very tight loops in special cases, ideally only within a single heavy function. It would be madness to do object pooling across a large scale codebase
    • Cthulhu_ 3y ago
      That reminds me of how iOS could achieve fast scrolling of lists, which Facebook and Twitter struggled with for years because they tried to stick with HTML. Basically if you scroll down, a list element from above is moved to the bottom and has its contents replaced. Anyway, I'm not sure if this is in Java already or is upcoming, but Java will finally add value objects, that is, stack variables instead of just making everything a heap object. For tight loops that will be much more efficient, since it can just reuse the same memory, stay within the cpu registers, etc. (I'm talking out of my ass btw from a superficial knowledge of Go and Java and half-remembered theory from college).
      • MrJohz 3y ago
        Virtual scrolling is a very common technique for rendering large tables or lists in HTML (and pretty much any other environment), and is by no means unique to iOS...
        • lawgimenez 3y ago
          And also common in Android from the very beginning of their ListView now called RecyclerView.
          • masfuerte 3y ago
            The Windows 3.1 list box supported it.
  • orf 3y ago
    They didn't debug anything. They just stuck a job queue on it and then rewrote part of it in Rust. That's the point of my comment. Using their code in the post (gist here[1]): $ pv lol.tsv | node lol.js 1.71GiB 0:00:07 [ 225MiB/s] [================================>] 100% Finished reading the file. Do we live in a world where engineers using a mature language like NodeJS with a pretty simple line-by-line file reading process and a string split think they hit a hard wall at 700kb/s, and the problem is not their bad code? 1. https://gist.github.com/orf/92bc26381e3dc00c8e96af262768591b https://gist.github.com/orf/92bc26381e3dc00c8e96af262768591b
    • nerdponx 3y ago
      But is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. The point of the article is that you can write really boring normal code in NodeJS and it's really slow, or you can write really boring normal code in Rust and it's a lot faster, and it's not that hard to use the latter to replace hot paths in the former.
      • disgruntledphd2 3y ago
        It's probably all the memory allocations. I know that when I do work like this, creating a huge array up front and assigning to that typically results in massive speedups. I know basically nothing about Node though, so I could be completely wrong here.
        • Karrot_Kream 3y ago
          Yeah that's one way to do it. Another is to reuse structures where possible to not force reallocation. There's a bunch of ways.
        • xmcqdpt2 3y ago
          Node.js has a decent JIT, or so I've heard. I have experience with the HotSpot JVM. For the JVM this code would absolutely stop allocating on the heap past 10k iterations of the loop. "Split a string then grab the parts in a hot loop" is ground stakes for JIT compilers. If they are seeing any GC activity once the loop gets hot, something else is going very wrong.
  • xmcqdpt2 3y ago
    By the way the very same code in Java would absolutely rip through the files with no extra allocations of arrays once the JIT gets seriously involved. A big loop with temporary arrays in it that are used to populate the fields of a record every iteration is HotSpot C2's favourite thing. Does node not similarly allocate on the stack in hot loops?
    • pkolaczk 3y ago
      Did you actually test it? Some time ago someone at another forum made a similar claim that returning an Optional from a method in Java will be surely optimized as we well, but it turned out that it wasn't (https://pkolaczk.github.io/overhead-of-optional/ https://pkolaczk.github.io/overhead-of-optional/).
      • xmcqdpt2 3y ago
        Good point, I should probably check.
• mhh__ 3y ago
  Realizing that Memory & Memory allocation is almost always where the performance goes wrong in practice takes some wisdom.
  • discreteevent 3y ago
    "almost always [...] wisdom" - Is this sarcasm? Maybe you work in a domain where that is the case but in my experience it's almost always not memory allocation or GC. "Wisdom" if there is any comes from measuring things properly and finding out the hard way that it's not what you think it is.
    • mhh__ 3y ago
      I think it's true more often than not. Also note that I said memory and memory allocation e.g. allocating too much hurts your page tables and your L1 cache. What field do you have in mind? If it's typically well optimized then fair enough but my bread and butter is compilers which are a pretty typical approximately zero compute, branchy pointer chasing type of workload.
      • insanitybit 3y ago
        I think many people on HN are used to throughput oriented horizontally scalable services, where GC is not particularly relevant because: a) You don't care about pause times since latency isn't your key metric b) You can almost always just throw more money at compute power to increase throughput It's almost always the case in such a system that your bottleneck is on the databases that you're pushing data through, as your main approach of "scale horizontally" typically leads to those sorts of bottlenecks.
        • mhh__ 3y ago
          Forgive me for talking about single programs then. Databases do typically have exactly the same memory problems though, again this time I mean memory locality rather than allocating it, obviously with the transactional elements on top.
• alex_lav 3y ago
  > I’m all for playing with rust, but this post strikes me as engineers being distracted by shiny new things because they are only able to parse a TSV at 700kb/s? They definitely just wanted to A.) Play with Rust, B.) Write a post about it. This is the key line: > As an intermediate Rustacean (I do have a fairly successful open-source project written in Rust, called fnm), It's fine to say "I rewrote this code in Rust because I like Rust". We don't have to do a whole song and dance about why X needed a rewrite in Rust because X wasn't doing a good enough job. People do exactly this work in Node every day successfully. They just wanted to write Rust instead of optimizing some Javascript. _Which is fine_, just totally not newsworthy.
  • _nhh 3y ago
    Couldnt agree more.
• anonzzzies 3y ago
  I agree, 700kb/s is just impossibly slow; I mean personally I wouldn't use node to do it, but with node you still should be orders faster. Something is not right.
• z3t4 3y ago
  Async programming is not easy. For example the following line for await ... will will process the whole file, then give you all lines at once. So the problem is probably that all lines in all files are parsed at the time creating a bottleneck somewhere and ballooning memory usage. The solution is to use streams, pause the stream in order to parse the line, then continue the stream. And pipe the parsed result to stdout or a tcp stream. That will use minimal memory, and if the GC doesn't do it's job Node.js now supports manual GC. So you could probably parse the whole thing using only a few kilobytes of memory.
  • gchq-7703 3y ago
    I don't believe that's true. Consider this infinite generator: async function* infinite() { while (1) { yield 'example' } } (async function () { for await (const value of infinite()) { console.log(value) } })() If the generator was consumed all at once, this would never print (because the generator is infinite). `for await... of` should only consume a single step of the stream / generator at a time. It's just syntactic sugar for the usual calls to .next() and etc. See the docs here[0] [0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
  • matharmin 3y ago
    "for await" uses async iterators, which is pretty much the same thing as a stream - just way simpler to use.
  • nightlyherb 3y ago
    N.B. Assuming `readLineStream` in that for await loop is node's readline object, the code is iterating over an asyncIterator[1] which works exactly as what you describe in your solution. (AsyncIterators have a next method that returns a Promise of the next item) [1] https://nodejs.org/api/readline.html#rlsymbolasynciterator https://nodejs.org/api/readline.html#rlsymbolasynciterator
• BiteCode_dev 3y ago
  Agree. Espacially, a TSV file is very easy to process in chunks, you don't need a specialized parser like with JSON where most libs just loads the whole thing. 700kb/s smell super fishy. I can parse that with one Python VM and a few MB of RAM way faster. And node has a JIT. Besides, let's imagine it would be effectively the case that Python and Node are too slow for this, that would be the perfect case to delegate the task to infra instead of introducing a whole new language and ecosystem plus a rewrite. E.G: a perfect use case for DuckDB.
  • didntcheck 3y ago
    Agreed that there's something else going on here, but it's worth noting that (C)Python uses reference counting (with periodic cycle breaking), rather than tracing GC. Not sure if that actually affects throughput in this scenario though
• whostolemyhat 3y ago
  Amazing, you read a fairly brief article with a terse description of the problem, and are now convinced you know better than the author.
  • orf 3y ago
    If you read an article with a terse description about the author struggling to ride a bike with 1 wheel missing, would you not assume a second wheel might help?
• pornel 3y ago
  700KB/s is embarrassingly slow! Modern computers are really fast. That's a couple of orders of magnitude slower than a WiFi transfer. 1000x slower than SSD read speeds. TSV is possible to parse at RAM speeds — 5000x faster. I suspect that widespread use of scripting languages, and hosting on micro instances containerized like a turducken, with disks over network, set people's expectations pretty low. You don't need a cluster of 25 machines for simple tabulation of 200GB of data.
• hestefisk 3y ago
  Exactly my thoughts. I wonder how fast that could be processed by a Perl script or even awk/sed.
  • allarm 3y ago
    AWK/Sed are actually very efficient. See this post for example, it describes parsing 25Tb of data with awk: https://livefreeordichotomize.com/posts/2019-06-04-using-awk-and-r-to-parse-25tb/index.html https://livefreeordichotomize.com/posts/2019-06-04-using-awk...
• JackFr 3y ago
  > Hand-waving at the garbage collector whilst seemingly doing very little to understand what’s happening doesn’t convince me that there isn’t another, possibly pretty obvious issue going on. This is pure voodoo profiling. “The job is slow. I guess it’s. . . . here!” Although the author has my sympathy - I spent a month rewriting a module, optimizing every line of code, squeezing every last bit of performance out to discover that the slowness was entirely due to a downstream SQL statement to which I had made a minor “innocuous” change.
  • insanitybit 3y ago
    You call it voodoo profiling, but isn't it kind of just being experienced? Like, if you show me code I can probably tell you where the expensive bits are just by looking. I could be wrong, but it's usually not very hard. Complex runtimes make it a lot harder because then I have to reason through things like JIT and any implicit yielding, but even still. So I'm pretty sympathetic to someone looking at a fairly small section of code, going "oh yeah, that's gonna create a lot of allocations", and then doing work to avoid those allocations. The alternative is that they write benchmarks and start profiling, which is actually quite hard to do well, and takes time. I'm not convinced that the author would have been better served by doing so here.
    • smaudet 3y ago
      I'm inclined to agree - when there's a simple solution (easier more reliable language), use it, the right answer is not "oh you are doing it wrong just follow these contortions and you'll be fine". Maybe your language that requires contortions is the problem.
    • _gabe_ 3y ago
      I’m curious, have you done much profiling? And what tools have you used if you have? This isn’t a judgment against you, it’s a genuine question because I’m wondering if perhaps I really am just this inexperienced. But the one thing I’ve learned is I’m always very surprised by where the hotspots in my code come from. I always assume big allocations, or bad cache coherency, or O(1) loops when I could use a hash map will be the killers, but then it turns out it’s something else entirely. Because of this, I’ve just learned to never trust my intuition when it comes to this. Compilers are crazy these days, and it’s very difficult to tell if something will be slow or if the compiler is going to clean it up behind the scenes and make it a non issue. Add to that JIT for certain languages and I don’t even know how you would begin to intuit where a hotspot would be. Edit: And just for additional context, I’ve used Intel’s VTune profiler, Tracy, Optick, and written a few manual benchmarks. Using these tools isn’t always easy, but they give a lot of insights that I wouldn’t pick up on my own.
• insanitybit 3y ago
  > You should be able to understand why your current system performs so absolutely terribly before you decide to rewrite it in any language. This is the case if the rewrite would take more time than the time it takes to debug and fix the issue. If you already have evidence that this is just an issue with the runtime (ie: "I have a very hard time controlling allocations in this language and allocations are a significant part of the compute time") I don't think it's the wrong call to just pick a language where you have very high confidence that it'll be "just fast". I've run into this with Python. I just didn't care to figure out what was going on that was making things slow. Rewriting the logic in Rust was not hard, so I did it, and things were fast. I look back now, in this moment, and absolutely don't give a shit about why the Python code was slow or if I could have made it fast.
  • satvikpendem 3y ago
    I'm all for rewriting things in Rust too but if it's a fundamental architectural problem that you just port over to the new language, you're essentially brute forcing it with an interpreted to compiled language change rather than solving the underlying issue. For example, running a O(n^2) operation rather than an O(n) one.
  • martinald 3y ago
    I'm not a nodejs expert at all, but couldn't they just call the garbage collection manually? Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'.
    • insanitybit 3y ago
      Calling it manually wouldn't stop it from being called on its own. Also there are two types of GC calls in node, one for the short lived memory and one for long lived memory (this is a generational gc) so you're going to have to understand where your memory allocations live if you want to do that. That's not too hard, thankfully, thanks to profilers. But anyway, you'd have to first stop it from being called at all, then call it manually after. This seems very hard, I don't know how much Node would like this, especially as you would have concurrent calls to the GC potentially. There's a very scary difference between "turn off, turn back on" and "turn back on, turn off", which can easily happen if you have multiple concurrent calls to the GC. Imagine if you threw an exception after turning the GC off and never turned it back on? Oof. I'd frankly prefer someone just uses Rust than starts to fiddle with GC at runtime in such a way. > Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? Maybe. It's possible that V8 will do that. But also you're adding extra lines to manage your buffer if you want to take control over that allocation. It's probably what I would do tbh, although the post makes it clear that just using Rust, where such management is trivial, is lower cost than expected thanks to the FFI library used. > The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'. I think the main issue I have with the post is this line: "Perhaps it's time to consider that JavaScript may be the problem." Javascript isn't so much "the problem" as it is simply a tool that makes optimizing around allocations difficult. It's very hard to know when you're properly managing your memory or not, and you're subject to the whims of V8 to optimize things for you. If the author had phrased this instead as "I'm sure there's some way that I could get V8 to optimize the allocations, or avoid them altogether, I'd have faster code - but the cost of doing that is far higher than simply using a language where these problems don't exist" I would have no real objections.
• afavour 3y ago
  Strongly agree. It’s entirely possible that the problems are attributable to GC but there are many alternatives within Node-land that could help: streams? A library that stores data in a typed array rather than raw objects? Adding an entirely new language and toolchain to your development process reeks of over engineering. Now your project needs a Rust dev on hand, what if you leave? Will there definitely be someone to pick up the slack?
• marcosdumay 3y ago
  No, I do claim that attitude is completely counterproductive. This is not some case of bad pick of algorithms. The entire optimization work will happen through microoptimizations spread through the code. If you insist on doing the microoptimizations on a language that is actively antagonistic to them, you will only create a lot of useless work, and have only mildly useful results to show for it, that will go away as soon as anything on your ecosystem changes (in JS that is measured in what? minutes?). And also, you will end up with an horrible codebase that's probably much less legible than what a lower level language will give you. And that besides not having any good tooling or community support, because all the tools and the community are focused on developer productivity, not software performance, as that's the entire focus of the language.
  • orf 3y ago
    I can parse a 200gb TSV with NodeJS in under a minute. Could it be done in 1 millisecond with assembly? Probably. But we are talking about over a days runtime here. That is not a micro optimisation, regardless of the language you’re using. You’re just using the language wrong, the same as if it took a day to parse with C. Your attitude, that it’s pointless to even try and investigate performance issues because it’s written in JS rather than whatever language you see as optimal, is totally counterproductive.
    • hnfong 3y ago
      > Could it be done in 1 millisecond with assembly? Probably. I know you're probably just knowingly spewing hyperbole to make your point, but unless your L1 cache is 200GB memory bandwidth alone makes this impossible.