5 ms·

There's a difference between (re-)building stuff, for fun, learning purpose or even to use in production in your own projects, and dunning-kruger demanding (exa

There's a difference between (re-)building stuff, for fun, learning purpose or even to use in production in your own projects, and dunning-kruger demanding (exaggerated) that existing solutions that provide stable APIs for multiple OS are thrown out, causing a massive amount of forced work (i.e., not fun) to others while getting a worse end result. Also, it can be fun to explore existing ideas and build upon those too - nobody forces you to maintain anything existing "for fun".

josephg 4y ago
My problem with this is that I feel like we (people coming along later) have three options when it comes to designing systems given the existing state of posix / linux / the web / etc: 1. Build inside the ecosystem as it was designed. Eg, make a static webpage using HTML. Make a linux binary using the standard linux tooling + apt (or whatever). This is almost always a good idea when you can, but sometimes the ecosystem is a bit shonky, or misaligned with what you're building. Eg, I want my program to have a fixed execution environment and instead I have debian and macos. I want a react-like API but the browser doesn't provide that. Etc. 2. Build a new thing on top of the ecosystem as it exists. Eg, build docker on top of linux. Build web frameworks. Invent the web browser, or couchdb views, or electron and build your software on top of that. The problem with this approach is that if each generation layers their own rubbish on top of the underlying layers then software will keep getting slower, more complex and more buggy over time. 3. Change the underlying system to work how you want it to work. Instead of using docker on top of ubuntu, replace ubuntu with nixos. Instead of using dpdk for higher storage performance, add io-uring to the linux kernel. I think I would prefer it if more people did option 3 rather than option 2. I think there's lots of ways modern computing systems could be better. I'd rather if linux gained the ability to run binaries from the web (for example) than we invent a new kind of binary in userland (docker containers) and run those. I don't want people in 100 years to still be using linux (with all the bad decisions it contains), but with 10 more layers of different generations' ideas stacked on top. (Oh, its linux - but with docker, and then most people run this specific container to run languageX, and that lets us run electo2050 apps, which in turn you can use to run IRC. It only uses 50 gigs of ram. So small and light!) Option 3 may require us to throw out or deprecate parts of the POSIX API. That may be overdue.
- benj111 4y ago
  You present 3 independent Choices, but surely these are stages. 1. Can you do X with existing tooling? If not 2. Can you add a tool to your set to do X? If not 3. Get a new tool set. 2 should also serve as testing to show the utility and iron out the links of things that make it to 3. Jumping straight to 3 likely means you end up with an ill thought out solution that soon needs another 3.
  - josephg 4y ago
    Sorry; that’s not quite the point I’m making. The difference I want to emphasise between 2 and 3 is whether the new tool is layered on top of the existing tool, or made by modifying the existing tool. If you want a new approach to storing data, do you make it by writing a new kernel module, or by doing some new thing in userland on top of the filesystem? All of docker’s features could be built into the Linux kernel. And it would probably be better as a result, and have ancillary benefits to existing Linux programs. But docker was built as a separate userland tool instead. Building a new thing is usually easier, since you don’t need to dive into the politics of an existing community. But if each new generation of software engineers builds new software by layering on top of the old stuff, then we end up with layers on layers of cruft and our computers run slower and are more buggy as a result. The POSIX apis aren’t very good in lots of ways. With a lot of complex effort in userland I can make a database with atomic write semantics. But maybe we should just add atomic file operations to Linux instead. In the long run I think it would be safer and result in faster programs in most cases.
    - benj111 4y ago
      Yes. Docker has proved there's utility in what it does. Assuming kinks in the implementation have been ironed out, now is the time to start thinking about making it a kernel module. If you just put every half thought out, poorly implemented idea straight in the kernel, you'd have a 5 petabyte kernel.

josephg 4y ago

My problem with this is that I feel like we (people coming along later) have three options when it comes to designing systems given the existing state of posix / linux / the web / etc: 1. Build inside the ecosystem as it was designed. Eg, make a static webpage using HTML. Make a linux binary using the standard linux tooling + apt (or whatever). This is almost always a good idea when you can, but sometimes the ecosystem is a bit shonky, or misaligned with what you're building. Eg, I want my program to have a fixed execution environment and instead I have debian and macos. I want a react-like API but the browser doesn't provide that. Etc. 2. Build a new thing on top of the ecosystem as it exists. Eg, build docker on top of linux. Build web frameworks. Invent the web browser, or couchdb views, or electron and build your software on top of that. The problem with this approach is that if each generation layers their own rubbish on top of the underlying layers then software will keep getting slower, more complex and more buggy over time. 3. Change the underlying system to work how you want it to work. Instead of using docker on top of ubuntu, replace ubuntu with nixos. Instead of using dpdk for higher storage performance, add io-uring to the linux kernel. I think I would prefer it if more people did option 3 rather than option 2. I think there's lots of ways modern computing systems could be better. I'd rather if linux gained the ability to run binaries from the web (for example) than we invent a new kind of binary in userland (docker containers) and run those. I don't want people in 100 years to still be using linux (with all the bad decisions it contains), but with 10 more layers of different generations' ideas stacked on top. (Oh, its linux - but with docker, and then most people run this specific container to run languageX, and that lets us run electo2050 apps, which in turn you can use to run IRC. It only uses 50 gigs of ram. So small and light!) Option 3 may require us to throw out or deprecate parts of the POSIX API. That may be overdue.

benj111 4y ago
You present 3 independent Choices, but surely these are stages. 1. Can you do X with existing tooling? If not 2. Can you add a tool to your set to do X? If not 3. Get a new tool set. 2 should also serve as testing to show the utility and iron out the links of things that make it to 3. Jumping straight to 3 likely means you end up with an ill thought out solution that soon needs another 3.
- josephg 4y ago
  Sorry; that’s not quite the point I’m making. The difference I want to emphasise between 2 and 3 is whether the new tool is layered on top of the existing tool, or made by modifying the existing tool. If you want a new approach to storing data, do you make it by writing a new kernel module, or by doing some new thing in userland on top of the filesystem? All of docker’s features could be built into the Linux kernel. And it would probably be better as a result, and have ancillary benefits to existing Linux programs. But docker was built as a separate userland tool instead. Building a new thing is usually easier, since you don’t need to dive into the politics of an existing community. But if each new generation of software engineers builds new software by layering on top of the old stuff, then we end up with layers on layers of cruft and our computers run slower and are more buggy as a result. The POSIX apis aren’t very good in lots of ways. With a lot of complex effort in userland I can make a database with atomic write semantics. But maybe we should just add atomic file operations to Linux instead. In the long run I think it would be safer and result in faster programs in most cases.
  - benj111 4y ago
    Yes. Docker has proved there's utility in what it does. Assuming kinks in the implementation have been ironed out, now is the time to start thinking about making it a kernel module. If you just put every half thought out, poorly implemented idea straight in the kernel, you'd have a 5 petabyte kernel.