6 ms·

> This entire article is a pretty damning report on JavaScript in general How so?

tyingq 8y ago
A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.
- austincheney 8y ago
  I guess I just don't see it. NPM is a massive infrastructure where one little piece was rewritten in Rust and then the Rust team wrote a promotional paper about it. The paper isn't bad, but it also isn't a technical white paper. I don't see how any of this is critical of JavaScript, which isn't even really discussed in the paper and still runs the rest of the infrastructure. If anything the paper is more damning of C, C++, and Java but I still think damning is far too extreme to describe what the paper said.
  - johannes1234321 8y ago
    > If anything the paper is more damning of C, C++ I don't see it as damning on either of these. On C++ it says "we didn't want to learn it." Which is fine. Maybe after learning they would have decided different, or not. On Java they said "we didn't want to learn how to operate it" as they feared the complexity of an Java application server for a single small service, which they can create in a way which hooks into their monitoring infrastructure. No damning their either. However their company's purpose is to push Javascript and they are saying "operating JavaScript is hard, doing this in Rust is easy" which directly goes against their business.
    - austincheney 8y ago
      I think you and I read completely different papers. I didn’t see anything about not wanting to learn or motivations of easy.
- derefr 8y ago
  Does every language have to be suitable for every task? “Operating a business-logic-embedding CDN at scale” isn’t ever something Node claimed to be capable of. I wouldn’t expect any other dynamic runtime-garbage-collected language, e.g. Ruby or Python or PHP, to be suited to that use-case either. Use the right tool for the job. The PHP website is running PHP, but it isn’t running a web server written in PHP. Web servers are system software; PHP isn’t for writing system software. Same thing applies here. Node does fine with business logic, but isn’t really architecturally suited to serving the “data layer” when you have control/data separation.
  - tyingq 8y ago
    From the paper it appears to be an authorization service that decides what rights a particular user has. Not a webserver or CDN. It mentions it being CPU bound, though it isn't clear to me why it would be, or why JS wouldn't work well enough for that.
    - Outpox 8y ago
      Well I'd say that if it's an authorization service there will be cryptographic calculation which is "heavy" on the CPU.
      - tyingq 8y ago
        I don't think that's what it does. "the authorization service that determines whether a user is allowed to, say, publish a particular package" Also, I would assume any crypto in v8 is already written in C, with JS calling into it.
      - throwaway66666 8y ago
        Users who suggest cryptography are getting downvoted (myself included). I am curious as to why. Is it perceived as spam or an attack on nodejs? Encrypted cookies and JWT (json web tokens) rely on a similar strategy so this is pretty standard. It would be pretty secure as long as the encryption key is not unique, so definitely not criticism on the NPM team but mere theories as to why a (what one would assume is a) database or memory bottle-neck is being presented as a CPU bottleneck in this scenario.
- CamouflagedKiwi 8y ago
  > Imagine the K8S team porting from Go to some other language for similar reasons. k8s was originally written in Java, so they have already done this once.
  - tyingq 8y ago
    I didn't know that, but it makes it a more interesting comparison. I imagine at least part of rewriting it in Go was to "eat your own dogfood".
    - pjmlp 8y ago
      Actually it was more like "new team got in charge" actually. There is a FOSDEM 2019 talk about it.
      - tyingq 8y ago
        Thanks! The talk: "The clusterfuck hidden in the Kubernetes code base" https://fosdem.org/2019/schedule/event/kubernetesclusterfuck/ https://fosdem.org/2019/schedule/event/kubernetesclusterfuck...
        tannhaeuser 8y ago
        Wow. Didn't realize it was that bad: > We look at what it would take to begin undoing the spaghetti code that is the various Kubernetes binaries Well at least the developers are being frank about it I guess.
        patrec 8y ago
        One wonders if there'll be a similar talk in a year... > The audience walks away feeling empathetic that they aren’t alone in their journey to writing idiomatic Go and is now equipped with strong refactoring techniques developed by some of the world’s top engineers for the Kuberentes project. As an occasional user of kubernetes, minikube etc. it's not something I would have guessed to have been developed by the word's top engineers. I mean kubernetes tries to, and probably manages to provide an useful abstraction, but at a few million LOC and a few man-month of full time senior engineering effort to run anything in production it's not exactly a epitome of elegant and efficient engineering.
- zinclozenge 8y ago
  The k8s team probably should, judging from the state of their codebase.
  - steveklabnik 8y ago
    I’m interested to see what becomes of https://www.cloudatomiclab.com/rustyk8s/ https://www.cloudatomiclab.com/rustyk8s/
    - zinclozenge 8y ago
      I'd kill for a rust client with feature parity. Writing k8s operators in rust would be so much better.
  - tyingq 8y ago
    If you feel that way, this is pretty funny then: https://news.ycombinator.com/item?id=19295841 https://news.ycombinator.com/item?id=19295841
  - Thaxll 8y ago
    Easy to criticize a projects that has x millions line of codes. Every projects of that scale are going to have issues.
    - zinclozenge 8y ago
      There's a lot of circumstantial evidence suggesting that efforts to refactor the codebase were torpedoed because of developer elitism.
- IsaacSchlueter 8y ago
  I mean, we still write the overwhelming majority of our code in JavaScript. We port to Rust when CPU-heavy task becomes a bottleneck to the rest of the system. It's not as if this paper is saying (nor is it the case) that we've ported the whole registry to Rust. JS has lots of advantages we appreciate.
  - tyingq 8y ago
    I'm curious why this authorization service is CPU intensive. The article says it's basically deciding if you're authorized to publish a package. It sounds like the sort of thing that would talk to a database, or a cache like redis, and therefore mostly be IO bound itself. Is this maybe parsing data structures itself?
    - IsaacSchlueter 8y ago
      It's been a few years since I was directly involved in engineering, but my fairly educated understanding is that it's more around reading of possibly-private packages than publishing. Publishing is a relatively rare event compared with reading, but in a world of private packages, orgs, and teams, the "can {user} read {object}" gets more complicated. It probably wouldn't be CPU bound if not for the sheer scale we're dealing with, but once all the IO bottlenecks are resolved, you still have to check to make sure that a login token is valid, then get the user associated with it, then check the teams/orgs/users with access to a thing (which might be public, in which case, the problem is a lot simpler, but you still have to get that info and check it), and then whether the user is in any of those groups. So there's a straightforward but relevant bit of CPU work to be done, and that's where Rust shines.

A team that likely has lots of JavaScript expertise basically stated that JavaScript is unsuitable for their task. And that the operational improvement once written in Rust was notable enough to write a paper. Imagine the K8S team porting from Go to some other language for similar reasons.

austincheney 8y ago
I guess I just don't see it. NPM is a massive infrastructure where one little piece was rewritten in Rust and then the Rust team wrote a promotional paper about it. The paper isn't bad, but it also isn't a technical white paper. I don't see how any of this is critical of JavaScript, which isn't even really discussed in the paper and still runs the rest of the infrastructure. If anything the paper is more damning of C, C++, and Java but I still think damning is far too extreme to describe what the paper said.
- johannes1234321 8y ago
  > If anything the paper is more damning of C, C++ I don't see it as damning on either of these. On C++ it says "we didn't want to learn it." Which is fine. Maybe after learning they would have decided different, or not. On Java they said "we didn't want to learn how to operate it" as they feared the complexity of an Java application server for a single small service, which they can create in a way which hooks into their monitoring infrastructure. No damning their either. However their company's purpose is to push Javascript and they are saying "operating JavaScript is hard, doing this in Rust is easy" which directly goes against their business.
  - austincheney 8y ago
    I think you and I read completely different papers. I didn’t see anything about not wanting to learn or motivations of easy.
derefr 8y ago
Does every language have to be suitable for every task? “Operating a business-logic-embedding CDN at scale” isn’t ever something Node claimed to be capable of. I wouldn’t expect any other dynamic runtime-garbage-collected language, e.g. Ruby or Python or PHP, to be suited to that use-case either. Use the right tool for the job. The PHP website is running PHP, but it isn’t running a web server written in PHP. Web servers are system software; PHP isn’t for writing system software. Same thing applies here. Node does fine with business logic, but isn’t really architecturally suited to serving the “data layer” when you have control/data separation.
- tyingq 8y ago
  From the paper it appears to be an authorization service that decides what rights a particular user has. Not a webserver or CDN. It mentions it being CPU bound, though it isn't clear to me why it would be, or why JS wouldn't work well enough for that.
  - Outpox 8y ago
    Well I'd say that if it's an authorization service there will be cryptographic calculation which is "heavy" on the CPU.
    - tyingq 8y ago
      I don't think that's what it does. "the authorization service that determines whether a user is allowed to, say, publish a particular package" Also, I would assume any crypto in v8 is already written in C, with JS calling into it.
    - throwaway66666 8y ago
      Users who suggest cryptography are getting downvoted (myself included). I am curious as to why. Is it perceived as spam or an attack on nodejs? Encrypted cookies and JWT (json web tokens) rely on a similar strategy so this is pretty standard. It would be pretty secure as long as the encryption key is not unique, so definitely not criticism on the NPM team but mere theories as to why a (what one would assume is a) database or memory bottle-neck is being presented as a CPU bottleneck in this scenario.
CamouflagedKiwi 8y ago
> Imagine the K8S team porting from Go to some other language for similar reasons. k8s was originally written in Java, so they have already done this once.
- tyingq 8y ago
  I didn't know that, but it makes it a more interesting comparison. I imagine at least part of rewriting it in Go was to "eat your own dogfood".
  - pjmlp 8y ago
    Actually it was more like "new team got in charge" actually. There is a FOSDEM 2019 talk about it.
    - tyingq 8y ago
      Thanks! The talk: "The clusterfuck hidden in the Kubernetes code base" https://fosdem.org/2019/schedule/event/kubernetesclusterfuck/ https://fosdem.org/2019/schedule/event/kubernetesclusterfuck...
      - tannhaeuser 8y ago
        Wow. Didn't realize it was that bad: > We look at what it would take to begin undoing the spaghetti code that is the various Kubernetes binaries Well at least the developers are being frank about it I guess.
      - patrec 8y ago
        One wonders if there'll be a similar talk in a year... > The audience walks away feeling empathetic that they aren’t alone in their journey to writing idiomatic Go and is now equipped with strong refactoring techniques developed by some of the world’s top engineers for the Kuberentes project. As an occasional user of kubernetes, minikube etc. it's not something I would have guessed to have been developed by the word's top engineers. I mean kubernetes tries to, and probably manages to provide an useful abstraction, but at a few million LOC and a few man-month of full time senior engineering effort to run anything in production it's not exactly a epitome of elegant and efficient engineering.
zinclozenge 8y ago
The k8s team probably should, judging from the state of their codebase.
- steveklabnik 8y ago
  I’m interested to see what becomes of https://www.cloudatomiclab.com/rustyk8s/ https://www.cloudatomiclab.com/rustyk8s/
  - zinclozenge 8y ago
    I'd kill for a rust client with feature parity. Writing k8s operators in rust would be so much better.
- tyingq 8y ago
  If you feel that way, this is pretty funny then: https://news.ycombinator.com/item?id=19295841 https://news.ycombinator.com/item?id=19295841
- Thaxll 8y ago
  Easy to criticize a projects that has x millions line of codes. Every projects of that scale are going to have issues.
  - zinclozenge 8y ago
    There's a lot of circumstantial evidence suggesting that efforts to refactor the codebase were torpedoed because of developer elitism.
IsaacSchlueter 8y ago
I mean, we still write the overwhelming majority of our code in JavaScript. We port to Rust when CPU-heavy task becomes a bottleneck to the rest of the system. It's not as if this paper is saying (nor is it the case) that we've ported the whole registry to Rust. JS has lots of advantages we appreciate.
- tyingq 8y ago
  I'm curious why this authorization service is CPU intensive. The article says it's basically deciding if you're authorized to publish a package. It sounds like the sort of thing that would talk to a database, or a cache like redis, and therefore mostly be IO bound itself. Is this maybe parsing data structures itself?
  - IsaacSchlueter 8y ago
    It's been a few years since I was directly involved in engineering, but my fairly educated understanding is that it's more around reading of possibly-private packages than publishing. Publishing is a relatively rare event compared with reading, but in a world of private packages, orgs, and teams, the "can {user} read {object}" gets more complicated. It probably wouldn't be CPU bound if not for the sheer scale we're dealing with, but once all the IO bottlenecks are resolved, you still have to check to make sure that a login token is valid, then get the user associated with it, then check the teams/orgs/users with access to a thing (which might be public, in which case, the problem is a lot simpler, but you still have to get that info and check it), and then whether the user is in any of those groups. So there's a straightforward but relevant bit of CPU work to be done, and that's where Rust shines.