4 ms·

> You should be able to understand why your current system performs so absolutely terribly before you decide to rewrite it in any language. This is the case if

> You should be able to understand why your current system performs so absolutely terribly before you decide to rewrite it in any language. This is the case if the rewrite would take more time than the time it takes to debug and fix the issue. If you already have evidence that this is just an issue with the runtime (ie: "I have a very hard time controlling allocations in this language and allocations are a significant part of the compute time") I don't think it's the wrong call to just pick a language where you have very high confidence that it'll be "just fast". I've run into this with Python. I just didn't care to figure out what was going on that was making things slow. Rewriting the logic in Rust was not hard, so I did it, and things were fast. I look back now, in this moment, and absolutely don't give a shit about why the Python code was slow or if I could have made it fast.

satvikpendem 3y ago
I'm all for rewriting things in Rust too but if it's a fundamental architectural problem that you just port over to the new language, you're essentially brute forcing it with an interpreted to compiled language change rather than solving the underlying issue. For example, running a O(n^2) operation rather than an O(n) one.
martinald 3y ago
I'm not a nodejs expert at all, but couldn't they just call the garbage collection manually? Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'.
- insanitybit 3y ago
  Calling it manually wouldn't stop it from being called on its own. Also there are two types of GC calls in node, one for the short lived memory and one for long lived memory (this is a generational gc) so you're going to have to understand where your memory allocations live if you want to do that. That's not too hard, thankfully, thanks to profilers. But anyway, you'd have to first stop it from being called at all, then call it manually after. This seems very hard, I don't know how much Node would like this, especially as you would have concurrent calls to the GC potentially. There's a very scary difference between "turn off, turn back on" and "turn back on, turn off", which can easily happen if you have multiple concurrent calls to the GC. Imagine if you threw an exception after turning the GC off and never turned it back on? Oof. I'd frankly prefer someone just uses Rust than starts to fiddle with GC at runtime in such a way. > Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? Maybe. It's possible that V8 will do that. But also you're adding extra lines to manage your buffer if you want to take control over that allocation. It's probably what I would do tbh, although the post makes it clear that just using Rust, where such management is trivial, is lower cost than expected thanks to the FFI library used. > The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'. I think the main issue I have with the post is this line: "Perhaps it's time to consider that JavaScript may be the problem." Javascript isn't so much "the problem" as it is simply a tool that makes optimizing around allocations difficult. It's very hard to know when you're properly managing your memory or not, and you're subject to the whims of V8 to optimize things for you. If the author had phrased this instead as "I'm sure there's some way that I could get V8 to optimize the allocations, or avoid them altogether, I'd have faster code - but the cost of doing that is far higher than simply using a language where these problems don't exist" I would have no real objections.
- giancarlostoro 3y ago
  I'm convinced that most of the time when someone blames Garbage Collection, they're just not digging deeper into their code for a different strategy. It's like when you finally realize why you need database indexes, or the benefits of memoization. You will find it and it will all feel so obvious, and you will likely refactor everywhere else you made the same mistake, and be amazed how much better everything is when you take the time to learn why your performance has downgraded. Chances are insanely low that the GC is actually your problem.
orf 3y ago
I agree in principal, but they first put it in k8s and then split it into a job queue. They even state that it took over a day to run at one point, implying a lengthy and prolonged attempt at fixing the issue. If you’re working on something and it’s slow, and during some iteration cycle you find that changing some lines makes it fast, cool. No need to debug it or spend time digging into it. And it’s good that it’s faster, the post screams “we don’t know how to debug these issues” rather than “we don’t care why it’s slow”.
- insanitybit 3y ago
  I think we basically agree, some debugging and profiling was probably warranted here, since they were going to go through the effort of horizontal scaling.
afavour 3y ago
> Rewriting the logic in Rust was not hard For you perhaps but it’s a pretty massive change in the longer term, though. You now have Rust in your codebase. How many engineers on the team will be able to work on it? If you leave tomorrow will the code become a liability? With the numbers the author is providing it sounds very much like the problem could be solved in Node, even if it means using an unconventional programming style and there would be a smaller maintenance overhead than reaching for Rust.
- insanitybit 3y ago
  > How many engineers on the team will be able to work on it? In my experience, basically all of them. Rust isn't that hard to learn on the job, it's maybe hard for some people to learn in isolation. Obviously adding a new language requires understanding that there's going to need to be build system support, dev support, etc. I guess I assumed that was there? Unless Wiz is still in the 'wild west' phase where you can just do whatever you want.
  - bheadmaster 3y ago
    Personal anecdote: I've learned Go in a day. Did the Go tour, and immediately felt comfortable writing it. I've tried learning Rust multiple times, did tours, tutorials, even started reading the book - still don't feel comfortable writing it. Rust is not a simple language. I say that, and my first language was C++.
    - earthling8118 3y ago
      Another anecdote: I learn new languages all the time. I make it a personal goal to never let not knowing a language get in the way of what I do. Rust was one of the easiest I've ever picked up. There is a ton of good information, examples, books, and tooling out there that helps me. On the other hand I've had to pick up Go several times in the past. It's never stuck. Basically every project I have seen written in it is convoluted and just messed up to read and interact with. People ALWAYS overstate the effort that Rust takes. There are a lot of spots that you can get messed up on. But not all of us are out here writing code that hits these things. People act like the existence of lifetimes means you have to write them everywhere and think about it. That's not the case for almost any of us.

satvikpendem 3y ago

I'm all for rewriting things in Rust too but if it's a fundamental architectural problem that you just port over to the new language, you're essentially brute forcing it with an interpreted to compiled language change rather than solving the underlying issue. For example, running a O(n^2) operation rather than an O(n) one.

martinald 3y ago

I'm not a nodejs expert at all, but couldn't they just call the garbage collection manually? Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'.

insanitybit 3y ago
Calling it manually wouldn't stop it from being called on its own. Also there are two types of GC calls in node, one for the short lived memory and one for long lived memory (this is a generational gc) so you're going to have to understand where your memory allocations live if you want to do that. That's not too hard, thankfully, thanks to profilers. But anyway, you'd have to first stop it from being called at all, then call it manually after. This seems very hard, I don't know how much Node would like this, especially as you would have concurrent calls to the GC potentially. There's a very scary difference between "turn off, turn back on" and "turn back on, turn off", which can easily happen if you have multiple concurrent calls to the GC. Imagine if you threw an exception after turning the GC off and never turned it back on? Oof. I'd frankly prefer someone just uses Rust than starts to fiddle with GC at runtime in such a way. > Also, wouldn't it be better to have the fields (and httpstatus) variable be declared above the for loop and just replace the contents of the fields variable instead of allocating a new one each time? Maybe. It's possible that V8 will do that. But also you're adding extra lines to manage your buffer if you want to take control over that allocation. It's probably what I would do tbh, although the post makes it clear that just using Rust, where such management is trivial, is lower cost than expected thanks to the FFI library used. > The thought process this blog seems to use is 'we have a lot of allocations and nodejs is garbage collected, therefore there is no alternative to using a different language'. I think the main issue I have with the post is this line: "Perhaps it's time to consider that JavaScript may be the problem." Javascript isn't so much "the problem" as it is simply a tool that makes optimizing around allocations difficult. It's very hard to know when you're properly managing your memory or not, and you're subject to the whims of V8 to optimize things for you. If the author had phrased this instead as "I'm sure there's some way that I could get V8 to optimize the allocations, or avoid them altogether, I'd have faster code - but the cost of doing that is far higher than simply using a language where these problems don't exist" I would have no real objections.
giancarlostoro 3y ago
I'm convinced that most of the time when someone blames Garbage Collection, they're just not digging deeper into their code for a different strategy. It's like when you finally realize why you need database indexes, or the benefits of memoization. You will find it and it will all feel so obvious, and you will likely refactor everywhere else you made the same mistake, and be amazed how much better everything is when you take the time to learn why your performance has downgraded. Chances are insanely low that the GC is actually your problem.

orf 3y ago

I agree in principal, but they first put it in k8s and then split it into a job queue. They even state that it took over a day to run at one point, implying a lengthy and prolonged attempt at fixing the issue. If you’re working on something and it’s slow, and during some iteration cycle you find that changing some lines makes it fast, cool. No need to debug it or spend time digging into it. And it’s good that it’s faster, the post screams “we don’t know how to debug these issues” rather than “we don’t care why it’s slow”.

insanitybit 3y ago
I think we basically agree, some debugging and profiling was probably warranted here, since they were going to go through the effort of horizontal scaling.

afavour 3y ago

> Rewriting the logic in Rust was not hard For you perhaps but it’s a pretty massive change in the longer term, though. You now have Rust in your codebase. How many engineers on the team will be able to work on it? If you leave tomorrow will the code become a liability? With the numbers the author is providing it sounds very much like the problem could be solved in Node, even if it means using an unconventional programming style and there would be a smaller maintenance overhead than reaching for Rust.

insanitybit 3y ago
> How many engineers on the team will be able to work on it? In my experience, basically all of them. Rust isn't that hard to learn on the job, it's maybe hard for some people to learn in isolation. Obviously adding a new language requires understanding that there's going to need to be build system support, dev support, etc. I guess I assumed that was there? Unless Wiz is still in the 'wild west' phase where you can just do whatever you want.
- bheadmaster 3y ago
  Personal anecdote: I've learned Go in a day. Did the Go tour, and immediately felt comfortable writing it. I've tried learning Rust multiple times, did tours, tutorials, even started reading the book - still don't feel comfortable writing it. Rust is not a simple language. I say that, and my first language was C++.
  - earthling8118 3y ago
    Another anecdote: I learn new languages all the time. I make it a personal goal to never let not knowing a language get in the way of what I do. Rust was one of the easiest I've ever picked up. There is a ton of good information, examples, books, and tooling out there that helps me. On the other hand I've had to pick up Go several times in the past. It's never stuck. Basically every project I have seen written in it is convoluted and just messed up to read and interact with. People ALWAYS overstate the effort that Rust takes. There are a lot of spots that you can get messed up on. But not all of us are out here writing code that hits these things. People act like the existence of lifetimes means you have to write them everywhere and think about it. That's not the case for almost any of us.