8 ms·

Show HN: Make Ubuntu/Debian packages for Nginx with your patches in one command

53 points by inlineint 11y ago

dozzie 11y ago
dpkg-buildbackage in use. Good. Much better than I expected.
feld 11y ago
What patches do people want applied that are missing? I know there are several patches as port options on FreeBSD.
- inlineint 11y ago
  Some of third party modules (from https://www.nginx.com/resources/wiki/modules/ https://www.nginx.com/resources/wiki/modules/) require patching, moreover if you want to just add a third party module to nginx you should recomplile nginx with configure flag --add-module=<directory> pointing to the module directory because nginx doesn't support dynamic loading of modules.
  - fidget 11y ago
    Soon it shall. Tengine is the solution to that in the meantime though
    - deleted 11y ago
      [deleted]
    - Sephr 11y ago
      It's already out in nginx 1.9.11.
      - helper 11y ago
        Unfortunately loadable modules in nginx still suck. Instead of having an ABI like apache does, nginx relies on signature checking for each module. This means that your modules still need to be built at the same time you build your nginx binary. If you decide later on you want to add a new module, you still need to rebuild nginx. Its so frustrating because it really introduces a lot of operational costs to an otherwise excellent piece of software. It was the most surprising thing about nginx since Apache has had loadable modules for a long time. There was some discussion about this when 1.9.11 was released that suggested that loadable modules wouldn't even be all that beneficial to package mantainers[1]. [1]: https://news.ycombinator.com/item?id=11067664 https://news.ycombinator.com/item?id=11067664
- deleted 11y ago
  [deleted]
- enzanki_ars 11y ago
  Personally, I use the nginx-rtmp module [0] a lot, but building nginx for that isn't too much of a hassle. [0]: https://github.com/arut/nginx-rtmp-module https://github.com/arut/nginx-rtmp-module
- nikolay 11y ago
  Many. My pretty standard setup has about 10 custom modules cache tools, LuaJIT, etc.
deleted 11y ago
[deleted]
cm3 11y ago
I'm curious how people do this at scale. If many administrators/devops build custom nginx for production servers, how isn't that already a security risk for most of them? In such a case, using Gentoo, where everything is fully automated and built from source, ready for deployment on servers, seems like an easier time to have. Or a similar solution for that matter. I've done this in the past with Debian stable plus custom build of a critical network service package, and you're basically playing mini distro builder every month or so. pkgsrc, nix, bsd ports, gentoo all seem like the more natural solution rather than everybody doing the same work of upgrading a build recipe for a newer upstream release and hopefully getting a working package out of it. There's also the problem that you have to tell the package database to treat your custom package differently or name it differently so that you won't get official distro packages installed accidentally.
- btown 11y ago
  Existing or everyone-collaborates-on-a-single-upstream nginx packages are just as much of a risk, though. Things can fail subtly based on the order of the compilation flags used in, for instance, mainline https://packages.debian.org/sid/nginx-extras https://packages.debian.org/sid/nginx-extras ... because the order of compilation flags, not the order of directives in the nginx config files, determines (for instance) whether gunzipping is done before or after string substitution. I'd much rather know and document exactly what flags are going in, in what order, and when, rather than accept defaults that might not even work for my given case. Make this into an Ansible role, make sure all source code is coming from a trusted location (maybe a fork you own), and you've got a reproducible way to generate a reliable nginx deployment.
  - cm3 11y ago
    Fair enough but a ports or portage recipe with use flags that are known to work are more likely to produce the expected results and do so for everyone rather one for the few custom builders who do it right. My concern is based on the fact that I see rebuilding nginx as part of getting started guides for many things, including rtmp streaming and since these guides are usually followed by less knowledgeable nginx users who say just want to stream, the numbers of broken or vulnerable nginx setups increases substantially. How many of those setups would be up to date if they were installed via a ports tree compared to the current manual configure and make steps and need to follow nginx upstream for new releases. In pkgsrc, gentoo, nix, etc. others already took care of that and you just have to run e.g. emerge.
tavert 11y ago
Or you could use a distro that makes this easy. Sign in at https://build.opensuse.org/package/show/server:http/nginx https://build.opensuse.org/package/show/server:http/nginx, click "Branch package," add your new patch and 2 lines to the spec file to apply it, turn on publishing under Repositories. Then `zypper addrepo obs://home:$USER:branches:server:http/openSUSE_$VER $USER` and you're done.
yourabi 11y ago
FPM pretty much solves this https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm
- nikolay 11y ago
  Not really. There's much more than just to compile it. There's an init script, a logrotate configuration, etc. The script by the OP is great, I just wish it had the ability to overwrite the bundled version of Nginx as well.
  - yourabi 11y ago
    Sorry, but are you familiar with fpm? All of what you mention is supported with fpm (you can have multiple post install files added...etc).
    - vacri 11y ago
      In which case you end up with an FPM command line with so many flags, there's not a lot of difference to other methods.
    - nikolay 11y ago
      Yes, believe it or not, I am. And did you look at all at the script above? It uses the standard package and just custom compiles with other modules and everything works just perfectly. To replicate this with FPM, I need to do an extra effort and possibly keep making changes with new versions of Ubuntu, test, etc. Why do that when I just want custom modules baked into the package?!

53 points by inlineint 11y ago

dozzie 11y ago

dpkg-buildbackage in use. Good. Much better than I expected.

feld 11y ago

What patches do people want applied that are missing? I know there are several patches as port options on FreeBSD.

inlineint 11y ago
Some of third party modules (from https://www.nginx.com/resources/wiki/modules/ https://www.nginx.com/resources/wiki/modules/) require patching, moreover if you want to just add a third party module to nginx you should recomplile nginx with configure flag --add-module=<directory> pointing to the module directory because nginx doesn't support dynamic loading of modules.
- fidget 11y ago
  Soon it shall. Tengine is the solution to that in the meantime though
  - deleted 11y ago
    [deleted]
  - Sephr 11y ago
    It's already out in nginx 1.9.11.
    - helper 11y ago
      Unfortunately loadable modules in nginx still suck. Instead of having an ABI like apache does, nginx relies on signature checking for each module. This means that your modules still need to be built at the same time you build your nginx binary. If you decide later on you want to add a new module, you still need to rebuild nginx. Its so frustrating because it really introduces a lot of operational costs to an otherwise excellent piece of software. It was the most surprising thing about nginx since Apache has had loadable modules for a long time. There was some discussion about this when 1.9.11 was released that suggested that loadable modules wouldn't even be all that beneficial to package mantainers[1]. [1]: https://news.ycombinator.com/item?id=11067664 https://news.ycombinator.com/item?id=11067664
deleted 11y ago
[deleted]
enzanki_ars 11y ago
Personally, I use the nginx-rtmp module [0] a lot, but building nginx for that isn't too much of a hassle. [0]: https://github.com/arut/nginx-rtmp-module https://github.com/arut/nginx-rtmp-module
nikolay 11y ago
Many. My pretty standard setup has about 10 custom modules cache tools, LuaJIT, etc.

deleted 11y ago

[deleted]

cm3 11y ago

I'm curious how people do this at scale. If many administrators/devops build custom nginx for production servers, how isn't that already a security risk for most of them? In such a case, using Gentoo, where everything is fully automated and built from source, ready for deployment on servers, seems like an easier time to have. Or a similar solution for that matter. I've done this in the past with Debian stable plus custom build of a critical network service package, and you're basically playing mini distro builder every month or so. pkgsrc, nix, bsd ports, gentoo all seem like the more natural solution rather than everybody doing the same work of upgrading a build recipe for a newer upstream release and hopefully getting a working package out of it. There's also the problem that you have to tell the package database to treat your custom package differently or name it differently so that you won't get official distro packages installed accidentally.

btown 11y ago
Existing or everyone-collaborates-on-a-single-upstream nginx packages are just as much of a risk, though. Things can fail subtly based on the order of the compilation flags used in, for instance, mainline https://packages.debian.org/sid/nginx-extras https://packages.debian.org/sid/nginx-extras ... because the order of compilation flags, not the order of directives in the nginx config files, determines (for instance) whether gunzipping is done before or after string substitution. I'd much rather know and document exactly what flags are going in, in what order, and when, rather than accept defaults that might not even work for my given case. Make this into an Ansible role, make sure all source code is coming from a trusted location (maybe a fork you own), and you've got a reproducible way to generate a reliable nginx deployment.
- cm3 11y ago
  Fair enough but a ports or portage recipe with use flags that are known to work are more likely to produce the expected results and do so for everyone rather one for the few custom builders who do it right. My concern is based on the fact that I see rebuilding nginx as part of getting started guides for many things, including rtmp streaming and since these guides are usually followed by less knowledgeable nginx users who say just want to stream, the numbers of broken or vulnerable nginx setups increases substantially. How many of those setups would be up to date if they were installed via a ports tree compared to the current manual configure and make steps and need to follow nginx upstream for new releases. In pkgsrc, gentoo, nix, etc. others already took care of that and you just have to run e.g. emerge.

tavert 11y ago

Or you could use a distro that makes this easy. Sign in at https://build.opensuse.org/package/show/server:http/nginx https://build.opensuse.org/package/show/server:http/nginx, click "Branch package," add your new patch and 2 lines to the spec file to apply it, turn on publishing under Repositories. Then `zypper addrepo obs://home:$USER:branches:server:http/openSUSE_$VER $USER` and you're done.

yourabi 11y ago

FPM pretty much solves this https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm

nikolay 11y ago
Not really. There's much more than just to compile it. There's an init script, a logrotate configuration, etc. The script by the OP is great, I just wish it had the ability to overwrite the bundled version of Nginx as well.
- yourabi 11y ago
  Sorry, but are you familiar with fpm? All of what you mention is supported with fpm (you can have multiple post install files added...etc).
  - vacri 11y ago
    In which case you end up with an FPM command line with so many flags, there's not a lot of difference to other methods.
  - nikolay 11y ago
    Yes, believe it or not, I am. And did you look at all at the script above? It uses the standard package and just custom compiles with other modules and everything works just perfectly. To replicate this with FPM, I need to do an extra effort and possibly keep making changes with new versions of Ubuntu, test, etc. Why do that when I just want custom modules baked into the package?!