7 ms·

> “Nginx is an incredible software and platform, but I was wondering if it wouldn't be time to just bite the bullet and create a more modern solution based on R

> “Nginx is an incredible software and platform, but I was wondering if it wouldn't be time to just bite the bullet and create a more modern solution based on Rust.” I actually don't understand why I am seeing arguments like this all the time. Rewrite takes time and effort. It may be OK for new projects. But for a mature project that has a quarter million lines of code and load-tested for cross-platform compatibility and security, it is way more than "just biting the bullet". (Not to mention that memory safety in Rust is ... still improvable. [1]) Rust is not a panacea. [2] When a corporate tries to seize control of a open source project, some one (or some entity?) has to do something, and rewriting in Rust by no means reclaims the freedom and openness of the project. [1] https://news.ycombinator.com/item?id=39440808 https://news.ycombinator.com/item?id=39440808 [2] https://ariadne.space/2023/12/07/most-breaches-actually-begin-in-corp/ https://ariadne.space/2023/12/07/most-breaches-actually-begi...

jhgb 3y ago
Is it a rewrite...or a completely new piece of software that simply takes the NginX experience into consideration when designing its features?
- wccrawford 3y ago
  The blogger keeps saying "Fork", but I can't find anything from the actual developer that says that. Instead, they keep saying it's a new project and that it "understands the nginx configuration language". So I think it's pretty clear that it's not a fork. To add to the confusion, it seems like they copied the site with all the bug reports as-is, which obviously would only apply to a fork, not to a project that hasn't even got any code yet.
  - rascul 3y ago
    A quick glance at the commit log (or whatever it's called for mercurial) makes it look like a fork to me, but so far not much more beside name changing. http://freenginx.org/hg/nginx/ http://freenginx.org/hg/nginx/ https://hg.nginx.org/nginx https://hg.nginx.org/nginx
GrayShade 3y ago
> When a corporate tries to seize control of a open source project Yeah, they wanted to have some CVEs for a bunch of bugs in experimental code, the author of the fork didn't.
- dns_snek 3y ago
  What a weird hill to die on. Perhaps someone can illuminate the situation better, but I'm not keen to trust a project that seems to take CVE assignment against their software personally.
  - codetrotter 3y ago
    Was the code in question included in stable releases? If not, I might agree that assigning CVEs are not helpful. But if the code was included in one or more stable release then I think it should probably have CVEs assigned. Depending on how serious or real the problems were.
    - ktosobcy 3y ago
      AFAIR it was im stable release BUTin a test code thats's behind a flag (disabled by default) so... it's a bit "shrodinger"
    - px43 3y ago
      I have personally reported bugs in the past that got closed as WONTFIX because they required some obscure configuration that the developer didn't believe would exist in the real world, but the way I found them was by breaking into actual real world servers owned by a client that I couldn't talk about due to an NDA. This happens a lot, and getting blown off constantly is a big reason why a lot of researchers just don't report bugs they find. I don't know much about these nginx bugs, but it seems like they were found in real live production systems, not by researchers trying to bump their CVE credits. Also it seems like quite a few people in the world are using nginx for HTTP/3 and quic, so trying to say "it doesn't count because it's not on by default" seems like a bit of a stretch. https://trac.nginx.org/nginx/ticket/2585 https://trac.nginx.org/nginx/ticket/2585 https://trac.nginx.org/nginx/ticket/2586 https://trac.nginx.org/nginx/ticket/2586
kristopolous 3y ago
The large "rewrite in language X" projects, I can't think of any that succeeded without the significant cost of time, effort and users. Maybe there's a survivorship bias in that only the ones that go poorly people talk about?
- MrJohz 3y ago
  Fish shell would be the big counterexample right now. C(++?) to Rust in about a year, IIRC. I believe they were still pushing out updates to the original codebase during that time, although I assume they could have done more if they weren't also rewriting.
  - secondcoming 3y ago
    I'm not sure a shell comes anywhere close to the complexity of a modern fully fledged web server.
  - callalex 3y ago
    With all due respect, is there anything that relies on production grade, highly scalable fish shell scripts, and in an adversarial security context?
    - MrJohz 3y ago
      With all due respect, I don't think that was the question! ;)
- darylteo 3y ago
  Lots of JS ecosystem is going to "rebuild in Rust" right now, ironically. JS has a different lifecycle though, I suppose. Everything gets rebuilt like 5-10 times more often than other on platforms.
  - DrewRWx 3y ago
    Everything in JS gets rebuilt until they are no longer in JS.
masfoobar 3y ago
Rust has been getting A LOT of traction in the last few years. It seems many programs are getting Rust replacements in one shape or another. If not being replaced it is, at the very least, being included in existing projects. The linux kernel now supports Rust which I believe, at this time, for Device Drivers. I also heared Rust compiler is included in the Windows Kernel. Whether you/we like it or not, Rust is a serious contender! As a developer myself, would prefer to focus my energy on Odin or Zig but the reality is Rust is where job security is likely to be.. alongside languages like Java, Python, C#, etc. Over the years, Job adverts have evolved from "must know OOP" to "must know design patterns" to "SOLID principles" and... now is likely to be "must focus on memory safety!" For many, which Paul Graham referred as "the pointy haired boss" in his Revenge of the Nerd peice, is just cool buzz words of the time. It is just the current thing we need to do/use. Rust is going to gain the edge with these buzzwords. This brings me back to projects like freenginx. The reality is new developers coming in are not focusing on C or C++. Academia is likely moving away from them in the coming years, in favour of Rust. The reality is new projects in C or C++ might not last when alternatives are being written in Rust. As it has "memory safety" it will likely be the default choice many look for, especially in a business environment. As I have mentioned, I have been focusing my energy on Odin and (a bit of) Zig. However, the reality is there are not going to be jobs using them.. compared to Rust, C#, Python, etc. This means I will have to bite the bullet... and soon... learning Rust.
viraptor 3y ago
> I actually don't understand why I am seeing arguments like this all the time. Have a look at: https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_charset_filter_module.c https://github.com/nginx/nginx/blob/master/src/http/modules/... It's got the whole checklist: nginx idiosyncratic module system, inline parsing, custom utf conversion, buffer preallocation and adjustments, linked lists, comments about side effects of custom allocator, and probably other things. It's not easy to deal with source like that and any serious improvement to that area would effectively be a rewrite anyway. Since anything doing work in nginx is a module anyway, it wouldn't even have to be a full rewrite in one go.
- anonymoushn 3y ago
  Anecdotally in the cache module, it's unclear what invariants are supposed to be maintained for a big ol struct full of information about this cache lookup, which functions expect to only be called while holding a lock, etc. The module design and the issues you mentioned also causes suboptimal performance :(
ergonaught 3y ago
The person who actually wrote your quote explicitly referred to doing this as a “new project” within the quote.
wvh 3y ago
A lot of the needed functionality might already be available in the Rust ecosystem, while a large amount of the code in Nginx predates general HTTP utility or even Unicode libraries. I assume that software that has grown organically with the web has gathered a lot of cruft and hacks that might not be needed anymore and make further development a lot harder. It might be a worthwhile experiment to take a common set of HTTP libraries in Rust and add Nginx-specific configuration parsing and some other bits together and see how far that kind of hackery would get. With a good BDFL and some community involvement, it would be a great way to get the ball rolling.
traceroute66 3y ago
> Rust is not a panacea. Amen to that. I don't understand why more people don't highlight Rust's main weakness. The lack of strong stdlib. No stdlib means you have two choices: (a) Re-invent the wheel; or (b) Use a crate. Most people choose (b) because they don't have the time, inclination or expertise to do (a). Which means you open yourself up to supply chain attacks, unless you constantly audit the upstream crates, which, let's face it, most people won't, they probably won't even audit the original version they pulled in, let alone the updates. Which means with something like an nginx re-write in Rust, which is a complex piece of software with many moving parts, you end up importing a whole ton of random crates. Half of which will probably become abandonware within a few years, because that's the way of the world with open-source, maintainers move on to bigger and better things. There is a difference between rewrite in Rust and rewrite properly in Rust. No doubt there are people out there doing solid open source work with Rust (e.g. the guys at tweede golf) but that takes time and effort, and in the case of tweede golf, the extra effort is financially sponsored, something that is not the case with most open source projects, especially the single-maintainer ones.
- bborud 3y ago
  Yeah, the poor stdlib is the main reason Go is my go-to (no pun intended) language for writing servers rather than Rust. That and the slow compiler, which a lot of people think shouldn't matter, but does matter.
  - traceroute66 3y ago
    > language for writing servers Or indeed clients. In 2024 you shouldn't need to import third-party code to make a call to a REST endpoint and parse the JSON. Or the same with crypto. All the stuff that's bread-and-butter in today's cloud-centric world, you shouldn't need to farm out to third-party code, it should be stdlib.
    - callalex 3y ago
      That’s a bit of a slippery slope though and causes maintinence nightmares for mature projects. Let’s say I want to upgrade to a new more secure TLS protocol. If it’s bundled to the language version that may mean changing the entire concurrency model, or something equally burdensome on a million line code base.
bborud 3y ago
While I agree that a rewrite of Nginx in Rust probably isn't the best use of resources, I wouldn't be so quick to dismiss the idea of writing an Nginx equivalent. And you might as well do that in Rust. There are things I'd love to see in this space. For instance I'd love a proxy that explores the idea of being fully configurable via a gRPC API and which can delegate decisions to some other service (over gRPC). (Doesn't the commercial Nginx have some of these features?)
sophacles 3y ago
I think you're confusing two things. "create a more modern solution" is not the same statement as "rewrite it in rust". The later implies "make the same software in a new language", the former suggests "create new software because the use cases and problems the software solves have changed". The software was initially released 20 years ago. It was designed in an era where most http requests served static assets and html w/o js, and https was for special situations like banks and order forms. At that time multicore CPUs were just emerging, and servers still shipped with 1Gb ethernet. JSON was a quirky idea, php was king, and Gmail was shaking up web-dev with this funky new Ajax thing[1]. Point being, things have changed - these days there are a lot of situations where you wouldn't even consider nginx since it's not the right tool, there are situations where nginx is no longer the easiest component to use and/or deploy, there are situations where it's a giant hassle to maintain an nginx deployment compared to tools written for a "servers are cattle" world. Put another way, the web was 13 years old and the internet was a little over 30, and both had really only gotten big over the previous decade. If we were talking aviation, it would be arguing to retrofit the best fighters from WW1 on the dawn of WW2 or asking why we can't just strap jet engines on a DC3. nginx has been an amazing bit of software and a solid workhorse getting us here. But more and more people are looking at other solutions, because the problem to solve isn't the problem nginx was designed for. That's ok - the world changes all the time. Software isn't done, we haven't even had computers for a hundred years. Heck we haven't even finished figuring out all the ways to use computers yet. I'm leary of any claim that we fully solved some aspect of it, and more so when the claim is applied to tech from the early days of that field. [1] it wasn't new new, but this was the first time most people really started to grasp what you could do on the web with javascript.

jhgb 3y ago

Is it a rewrite...or a completely new piece of software that simply takes the NginX experience into consideration when designing its features?

wccrawford 3y ago
The blogger keeps saying "Fork", but I can't find anything from the actual developer that says that. Instead, they keep saying it's a new project and that it "understands the nginx configuration language". So I think it's pretty clear that it's not a fork. To add to the confusion, it seems like they copied the site with all the bug reports as-is, which obviously would only apply to a fork, not to a project that hasn't even got any code yet.
- rascul 3y ago
  A quick glance at the commit log (or whatever it's called for mercurial) makes it look like a fork to me, but so far not much more beside name changing. http://freenginx.org/hg/nginx/ http://freenginx.org/hg/nginx/ https://hg.nginx.org/nginx https://hg.nginx.org/nginx

GrayShade 3y ago

> When a corporate tries to seize control of a open source project Yeah, they wanted to have some CVEs for a bunch of bugs in experimental code, the author of the fork didn't.

dns_snek 3y ago
What a weird hill to die on. Perhaps someone can illuminate the situation better, but I'm not keen to trust a project that seems to take CVE assignment against their software personally.
- codetrotter 3y ago
  Was the code in question included in stable releases? If not, I might agree that assigning CVEs are not helpful. But if the code was included in one or more stable release then I think it should probably have CVEs assigned. Depending on how serious or real the problems were.
  - ktosobcy 3y ago
    AFAIR it was im stable release BUTin a test code thats's behind a flag (disabled by default) so... it's a bit "shrodinger"
  - px43 3y ago
    I have personally reported bugs in the past that got closed as WONTFIX because they required some obscure configuration that the developer didn't believe would exist in the real world, but the way I found them was by breaking into actual real world servers owned by a client that I couldn't talk about due to an NDA. This happens a lot, and getting blown off constantly is a big reason why a lot of researchers just don't report bugs they find. I don't know much about these nginx bugs, but it seems like they were found in real live production systems, not by researchers trying to bump their CVE credits. Also it seems like quite a few people in the world are using nginx for HTTP/3 and quic, so trying to say "it doesn't count because it's not on by default" seems like a bit of a stretch. https://trac.nginx.org/nginx/ticket/2585 https://trac.nginx.org/nginx/ticket/2585 https://trac.nginx.org/nginx/ticket/2586 https://trac.nginx.org/nginx/ticket/2586

kristopolous 3y ago

The large "rewrite in language X" projects, I can't think of any that succeeded without the significant cost of time, effort and users. Maybe there's a survivorship bias in that only the ones that go poorly people talk about?

MrJohz 3y ago
Fish shell would be the big counterexample right now. C(++?) to Rust in about a year, IIRC. I believe they were still pushing out updates to the original codebase during that time, although I assume they could have done more if they weren't also rewriting.
- secondcoming 3y ago
  I'm not sure a shell comes anywhere close to the complexity of a modern fully fledged web server.
- callalex 3y ago
  With all due respect, is there anything that relies on production grade, highly scalable fish shell scripts, and in an adversarial security context?
  - MrJohz 3y ago
    With all due respect, I don't think that was the question! ;)
darylteo 3y ago
Lots of JS ecosystem is going to "rebuild in Rust" right now, ironically. JS has a different lifecycle though, I suppose. Everything gets rebuilt like 5-10 times more often than other on platforms.
- DrewRWx 3y ago
  Everything in JS gets rebuilt until they are no longer in JS.

masfoobar 3y ago

Rust has been getting A LOT of traction in the last few years. It seems many programs are getting Rust replacements in one shape or another. If not being replaced it is, at the very least, being included in existing projects. The linux kernel now supports Rust which I believe, at this time, for Device Drivers. I also heared Rust compiler is included in the Windows Kernel. Whether you/we like it or not, Rust is a serious contender! As a developer myself, would prefer to focus my energy on Odin or Zig but the reality is Rust is where job security is likely to be.. alongside languages like Java, Python, C#, etc. Over the years, Job adverts have evolved from "must know OOP" to "must know design patterns" to "SOLID principles" and... now is likely to be "must focus on memory safety!" For many, which Paul Graham referred as "the pointy haired boss" in his Revenge of the Nerd peice, is just cool buzz words of the time. It is just the current thing we need to do/use. Rust is going to gain the edge with these buzzwords. This brings me back to projects like freenginx. The reality is new developers coming in are not focusing on C or C++. Academia is likely moving away from them in the coming years, in favour of Rust. The reality is new projects in C or C++ might not last when alternatives are being written in Rust. As it has "memory safety" it will likely be the default choice many look for, especially in a business environment. As I have mentioned, I have been focusing my energy on Odin and (a bit of) Zig. However, the reality is there are not going to be jobs using them.. compared to Rust, C#, Python, etc. This means I will have to bite the bullet... and soon... learning Rust.

viraptor 3y ago

> I actually don't understand why I am seeing arguments like this all the time. Have a look at: https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_charset_filter_module.c https://github.com/nginx/nginx/blob/master/src/http/modules/... It's got the whole checklist: nginx idiosyncratic module system, inline parsing, custom utf conversion, buffer preallocation and adjustments, linked lists, comments about side effects of custom allocator, and probably other things. It's not easy to deal with source like that and any serious improvement to that area would effectively be a rewrite anyway. Since anything doing work in nginx is a module anyway, it wouldn't even have to be a full rewrite in one go.

anonymoushn 3y ago
Anecdotally in the cache module, it's unclear what invariants are supposed to be maintained for a big ol struct full of information about this cache lookup, which functions expect to only be called while holding a lock, etc. The module design and the issues you mentioned also causes suboptimal performance :(

ergonaught 3y ago

The person who actually wrote your quote explicitly referred to doing this as a “new project” within the quote.

wvh 3y ago

A lot of the needed functionality might already be available in the Rust ecosystem, while a large amount of the code in Nginx predates general HTTP utility or even Unicode libraries. I assume that software that has grown organically with the web has gathered a lot of cruft and hacks that might not be needed anymore and make further development a lot harder. It might be a worthwhile experiment to take a common set of HTTP libraries in Rust and add Nginx-specific configuration parsing and some other bits together and see how far that kind of hackery would get. With a good BDFL and some community involvement, it would be a great way to get the ball rolling.

traceroute66 3y ago

> Rust is not a panacea. Amen to that. I don't understand why more people don't highlight Rust's main weakness. The lack of strong stdlib. No stdlib means you have two choices: (a) Re-invent the wheel; or (b) Use a crate. Most people choose (b) because they don't have the time, inclination or expertise to do (a). Which means you open yourself up to supply chain attacks, unless you constantly audit the upstream crates, which, let's face it, most people won't, they probably won't even audit the original version they pulled in, let alone the updates. Which means with something like an nginx re-write in Rust, which is a complex piece of software with many moving parts, you end up importing a whole ton of random crates. Half of which will probably become abandonware within a few years, because that's the way of the world with open-source, maintainers move on to bigger and better things. There is a difference between rewrite in Rust and rewrite properly in Rust. No doubt there are people out there doing solid open source work with Rust (e.g. the guys at tweede golf) but that takes time and effort, and in the case of tweede golf, the extra effort is financially sponsored, something that is not the case with most open source projects, especially the single-maintainer ones.

bborud 3y ago
Yeah, the poor stdlib is the main reason Go is my go-to (no pun intended) language for writing servers rather than Rust. That and the slow compiler, which a lot of people think shouldn't matter, but does matter.
- traceroute66 3y ago
  > language for writing servers Or indeed clients. In 2024 you shouldn't need to import third-party code to make a call to a REST endpoint and parse the JSON. Or the same with crypto. All the stuff that's bread-and-butter in today's cloud-centric world, you shouldn't need to farm out to third-party code, it should be stdlib.
  - callalex 3y ago
    That’s a bit of a slippery slope though and causes maintinence nightmares for mature projects. Let’s say I want to upgrade to a new more secure TLS protocol. If it’s bundled to the language version that may mean changing the entire concurrency model, or something equally burdensome on a million line code base.

bborud 3y ago

While I agree that a rewrite of Nginx in Rust probably isn't the best use of resources, I wouldn't be so quick to dismiss the idea of writing an Nginx equivalent. And you might as well do that in Rust. There are things I'd love to see in this space. For instance I'd love a proxy that explores the idea of being fully configurable via a gRPC API and which can delegate decisions to some other service (over gRPC). (Doesn't the commercial Nginx have some of these features?)

sophacles 3y ago

I think you're confusing two things. "create a more modern solution" is not the same statement as "rewrite it in rust". The later implies "make the same software in a new language", the former suggests "create new software because the use cases and problems the software solves have changed". The software was initially released 20 years ago. It was designed in an era where most http requests served static assets and html w/o js, and https was for special situations like banks and order forms. At that time multicore CPUs were just emerging, and servers still shipped with 1Gb ethernet. JSON was a quirky idea, php was king, and Gmail was shaking up web-dev with this funky new Ajax thing[1]. Point being, things have changed - these days there are a lot of situations where you wouldn't even consider nginx since it's not the right tool, there are situations where nginx is no longer the easiest component to use and/or deploy, there are situations where it's a giant hassle to maintain an nginx deployment compared to tools written for a "servers are cattle" world. Put another way, the web was 13 years old and the internet was a little over 30, and both had really only gotten big over the previous decade. If we were talking aviation, it would be arguing to retrofit the best fighters from WW1 on the dawn of WW2 or asking why we can't just strap jet engines on a DC3. nginx has been an amazing bit of software and a solid workhorse getting us here. But more and more people are looking at other solutions, because the problem to solve isn't the problem nginx was designed for. That's ok - the world changes all the time. Software isn't done, we haven't even had computers for a hundred years. Heck we haven't even finished figuring out all the ways to use computers yet. I'm leary of any claim that we fully solved some aspect of it, and more so when the claim is applied to tech from the early days of that field. [1] it wasn't new new, but this was the first time most people really started to grasp what you could do on the web with javascript.