4 ms·

But is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. The point of the article is that you can write real

But is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. The point of the article is that you can write really boring normal code in NodeJS and it's really slow, or you can write really boring normal code in Rust and it's a lot faster, and it's not that hard to use the latter to replace hot paths in the former.

disgruntledphd2 3y ago
It's probably all the memory allocations. I know that when I do work like this, creating a huge array up front and assigning to that typically results in massive speedups. I know basically nothing about Node though, so I could be completely wrong here.
- Karrot_Kream 3y ago
  Yeah that's one way to do it. Another is to reuse structures where possible to not force reallocation. There's a bunch of ways.
- xmcqdpt2 3y ago
  Node.js has a decent JIT, or so I've heard. I have experience with the HotSpot JVM. For the JVM this code would absolutely stop allocating on the heap past 10k iterations of the loop. "Split a string then grab the parts in a hot loop" is ground stakes for JIT compilers. If they are seeing any GC activity once the loop gets hot, something else is going very wrong.
  - disgruntledphd2 3y ago
    Yeah but if you're appending in a loop you'll need to allocate memory every time, right? Can a JIT avoid or rewrite this?
    - xmcqdpt2 3y ago
      Yep the Hotspot JIT will absolutely allocate on the stack and reuse memory allocations inside loops. It also has escape analysis and it will detect you don't need the array of strings after the iteration. In that case it usually just allocates the strings directly at their final locations on the record object.
      - disgruntledphd2 3y ago
        Wow, that's super cool. Almost makes me wish i did stuff on the JVM (but until someone rewrites LAPACK for it, that seems unlikely).
orf 3y ago
> But is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. 1. It should never be that stupidly slow 2. That should be obvious to anyone with a napkin and some math 3. Excessive allocations are highly unlikely to be the root cause of the slowness 4. Instead of debugging, they seemingly spent a lot of time putting it on Kubernetes, scaling it out to 25 instances and then finally rewriting a portion of it in Rust. The whole thing smells of confusion and multiple issues being conflated together, with the rewrite in rust fixing these by virtue of deletion. And if the root is was the GC, then some `--trace-gc` output in the post would be nice at the absolute minimum. Showing a for loop with a string split doesn't cut it.
- maurice2k 3y ago
  Exactly. I'm not a super-expert in Node.js but worked a lot with it and also know a lot of internals. And I can definitely say: IO speed is pretty good. To be honest, I've never come across a language with bad IO speed and I've seen quite a lot of languages. I also did some benchmarks (like orf did) and speed is pretty good. Maybe Rust is a little bit better because it's nearer to syscalls than Node.js is but we're not talking about 760kb/s vs. 250mb/s. 200GB of logs and 25 instances means 8GB per instance. Not sure what they're doing, but processing of 8GB taking 3h is like doing it with my 80286 processor ;-) Even with inserting stuff in MongoDB this seems way to slow. And they only replaced the file IO part with Rust... I guess the records array is getting pretty huge which might result in swapping to disk. But then: don't let it happen, just flush it to the database or whatever. I believe the problem could have been solved with Node.js without issues.
mrighele 3y ago
The point the parent is making is that the boring normal code in NodeJS is not slow, at least not as presented in the article. The bottleneck is probably somewhere else. Allocation is usually not slow in a garbage collected language. Deallocation can be, but that is usually easy to check (for example with the --trace-gc flag, or running with profiling enabled and checking the results with chrome devtools). It would have been nice if they shared such analysis. With the limited information provided in the article, there are other points that I would check before going as far as writing a module in another language. For example > records.push({ They are collecting all the parsed rows in a single array. This is going to consume memory proportional to the size of the file, potentially in the same order of magnitude (depends on how much information is being collected and how much discarded).
ryanjshaw 3y ago
The low throughput is definitely a code smell. In my experience, a common mistake in handling files too large for memory is to read the data faster than you process it, consequently running out of memory. For example, maybe they have a loop reading lines and doing a bunch of async DB writes: your process will read at disk speed and write at database speed (relatively slower), and you'll run out of memory due to async tasks/context queuing up in memory. Perhaps this is why it worked to split the file into 8GB units X 25 servers: it didn't blow their memory budget per server.

disgruntledphd2 3y ago

It's probably all the memory allocations. I know that when I do work like this, creating a huge array up front and assigning to that typically results in massive speedups. I know basically nothing about Node though, so I could be completely wrong here.

Karrot_Kream 3y ago
Yeah that's one way to do it. Another is to reuse structures where possible to not force reallocation. There's a bunch of ways.
xmcqdpt2 3y ago
Node.js has a decent JIT, or so I've heard. I have experience with the HotSpot JVM. For the JVM this code would absolutely stop allocating on the heap past 10k iterations of the loop. "Split a string then grab the parts in a hot loop" is ground stakes for JIT compilers. If they are seeing any GC activity once the loop gets hot, something else is going very wrong.
- disgruntledphd2 3y ago
  Yeah but if you're appending in a loop you'll need to allocate memory every time, right? Can a JIT avoid or rewrite this?
  - xmcqdpt2 3y ago
    Yep the Hotspot JIT will absolutely allocate on the stack and reuse memory allocations inside loops. It also has escape analysis and it will detect you don't need the array of strings after the iteration. In that case it usually just allocates the strings directly at their final locations on the record object.
    - disgruntledphd2 3y ago
      Wow, that's super cool. Almost makes me wish i did stuff on the JVM (but until someone rewrites LAPACK for it, that seems unlikely).

orf 3y ago

> But is the code shitty? It looks like boring normal code to me. I'm not sure what point you're making here. 1. It should never be that stupidly slow 2. That should be obvious to anyone with a napkin and some math 3. Excessive allocations are highly unlikely to be the root cause of the slowness 4. Instead of debugging, they seemingly spent a lot of time putting it on Kubernetes, scaling it out to 25 instances and then finally rewriting a portion of it in Rust. The whole thing smells of confusion and multiple issues being conflated together, with the rewrite in rust fixing these by virtue of deletion. And if the root is was the GC, then some `--trace-gc` output in the post would be nice at the absolute minimum. Showing a for loop with a string split doesn't cut it.

maurice2k 3y ago
Exactly. I'm not a super-expert in Node.js but worked a lot with it and also know a lot of internals. And I can definitely say: IO speed is pretty good. To be honest, I've never come across a language with bad IO speed and I've seen quite a lot of languages. I also did some benchmarks (like orf did) and speed is pretty good. Maybe Rust is a little bit better because it's nearer to syscalls than Node.js is but we're not talking about 760kb/s vs. 250mb/s. 200GB of logs and 25 instances means 8GB per instance. Not sure what they're doing, but processing of 8GB taking 3h is like doing it with my 80286 processor ;-) Even with inserting stuff in MongoDB this seems way to slow. And they only replaced the file IO part with Rust... I guess the records array is getting pretty huge which might result in swapping to disk. But then: don't let it happen, just flush it to the database or whatever. I believe the problem could have been solved with Node.js without issues.

mrighele 3y ago

The point the parent is making is that the boring normal code in NodeJS is not slow, at least not as presented in the article. The bottleneck is probably somewhere else. Allocation is usually not slow in a garbage collected language. Deallocation can be, but that is usually easy to check (for example with the --trace-gc flag, or running with profiling enabled and checking the results with chrome devtools). It would have been nice if they shared such analysis. With the limited information provided in the article, there are other points that I would check before going as far as writing a module in another language. For example > records.push({ They are collecting all the parsed rows in a single array. This is going to consume memory proportional to the size of the file, potentially in the same order of magnitude (depends on how much information is being collected and how much discarded).

ryanjshaw 3y ago

The low throughput is definitely a code smell. In my experience, a common mistake in handling files too large for memory is to read the data faster than you process it, consequently running out of memory. For example, maybe they have a loop reading lines and doing a bunch of async DB writes: your process will read at disk speed and write at database speed (relatively slower), and you'll run out of memory due to async tasks/context queuing up in memory. Perhaps this is why it worked to split the file into 8GB units X 25 servers: it didn't blow their memory budget per server.