13 ms·

Right, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted program

Right, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted programs (some might call them microservices) it lets you interact with to solve real world problems. If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainability wise? Sounds like a nightmare to me.

lmm 8y ago
> If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainability wise? You mean like BSD?
- boramalper 8y ago
  Same with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. Busybox is a slightly better example (even though it’s also a userspace program).
  - jcelerier 8y ago
    > Same with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. they're part of the same repo and built at the same time than the kernel. Run a big "make universe" here : https://github.com/freebsd/freebsd https://github.com/freebsd/freebsd and see for yourself. That they are different binaries does not matter a lot, it's just a question of user interface. See for instance busybox where all the userspace is in a single binary.
    - zerd 8y ago
      I would argue that the fact that they're in the same repo does not make it a monolith at all. All of google and facebook is in a single repo and built at the same time, but it doesn't make them monoliths.
thermodynthrway 8y ago
With web apps the main concern is data consistency between relations. On the OS level you have these same concerns with memory and disk, and there's database-like systems in the kernel and drivers to handle it. Essentially all these utilities are running within the same "database" which is disk and memory management handled by the kernel. Usually microservices have their own databases, which is where consistency hell begins
- kolpa 8y ago
  > Usually microservices have their own databases That's news to me, and seems insane. Unless you mean "their own database tables", not "database servers". But that's just the same as having multiple directories and files in a Unix filesystem.
  - VBprogrammer 8y ago
    That architecture is commonly known as a distributed monolith. If you put two services on the same DB you can guarantee that someone will be joining onto a table they shouldn't have before the week is out.
    - ljw1001 8y ago
      This is another way of saying that micro-services are for easy problems, where you can avoid having many processes share the same complex database.
      - wsy 8y ago
        It's exactly the other way around: If you can afford all your components sharing the same database without creating a big dependency hell, then your problem is _too small_ for microservices. If your problem is so large that you have to split it up to manage its complexity, start considering microservices (it might still not be the right option for you).
        8y ago
blablabla123 8y ago
But there is a big difference. These small targeted programs are invoked in user land, usually by the user. Microservices get invoked directly by the user when debugging is going on. Otherwise they are expected to automagically talk to each other and depending on the abstraction even discovery each other automatically. Also I can pipe these tools together from the same terminal session, like tail -f foo | grep something | awk ... You don't have that in general with Microservices. Unix tools are Lego, Microservices aren't. They are Domino at best. Probably one could come up with an abstraction to do Lego with Microservices but we're not there yet.
- deleted 8y ago
  [deleted]
- paxys 8y ago
  Look at the package dependency tree of an average linux program. They are absolutely examples of "microservices" talking to each other.
  - digi_owl 8y ago
    And they can be a right mess to wrangle because upstream gets into some artistic frenzy, and shouts down anyone that worries about the ensuing breakages as luddites and haters.
    - wsy 8y ago
      And still nobody would argue to pack everything together into one executable because of these issues. As everything in software engineering, it is about finding the right trade-off.
      - digi_owl 8y ago
        Oh, but why do that when we have containers...
  - lmm 8y ago
apeace 8y ago
My understanding is that OpenBSD is constructed this way, and I have also heard that their code is well-organized and easy to follow. There is the "base" system which is the OS itself and common packages (all the ones you mentioned), then there is the "ports" repo which contains many open-source applications with patches to make them work with OpenBSD. Here is a Github mirror of their repos: https://github.com/openbsd https://github.com/openbsd I think OpenBSD has reaped many of the same advantages described by Segment with their monorepo approach, such as easily being able to add pledge[1] to many ports relatively quickly. https://www.slideshare.net/GiovanniBechis/pledge-in-openbsd https://www.slideshare.net/GiovanniBechis/pledge-in-openbsd
- fastball 8y ago
  "packages" and "ports" are both collections of "microservices" in your analogy, so you can't really assert the whole thing is a monolith.
  - lmm 8y ago
    The whole thing is released and deployed together, which is the usual distinction between microservices and not. I don't think anyone's advocating not using modules and folders to structure your code in your repository (certainly I hope not), but a single deployment artifact makes life a lot easier.
joshvm 8y ago
One example I'm familiar with that sounds like microservices is the Robot Operating System (ROS). At its heart it's just a framework for pub/sub over IP, it just happens to be targeted towards robotics. A ROS system comprises of 'nodes' for each logical operation e.g. image acquisition -> camera calibration -> analysis -> output. The system is defined by a graph of these nodes, since you can pipe messages wherever they're needed; all nodes can be many-many. Each node is a self-contained application which communicates to a master node via tcp/ip (like most pub/sub systems a master node is requried to tell nodes where to send messages). So you can do cool stuff like have lots of seprate networked computers all talking to each other (fairly) easily. It works pretty well and once you've got a particular node stable - e.g. the node that acquires images - you don't need to touch it. If you need to refactor or bugfix, you only edit that code. If you need to test new things, you can just drop them into an existing system because there's separation between the code (e.g. you just tell the system what your new node will publish/subscribe and it'll do the rest). There is definitely a feeling of duct tape and glue, since you're often using nodes made by lots of different people, some of which are maintainted, others aren't, different naming conventions, etc. However, I think that's just because ROS is designed to be as generic as possible, rather than a side effect of it running like a microservice.
- chrisco255 8y ago
  There's actually no reason you can't architect microservices like this. You can put RabbitMQ or some AMQP service as a comm layer between services. But then you have to architect your system to be event-driven. It's not a bad approach.
  - thermodynthrway 8y ago
    I've worked on a system that used Kafka in-between. It makes consistency issues a ton worse because everything is async. At least with HTTP you can do synchronous calls
    - bsder 8y ago
      Repeat after me: there is no such thing as a synchronous network. I get where you are coming from. But you will go async sooner or later if you need any reasonable error recovery or reliability. The only question is how much pain you will suffer before you do so.
mirceal 8y ago
Linux is the kernel.
inetknght 8y ago
> I would say what makes it useful is all the various small, targeted programs You mean GNU coreutils?

lmm 8y ago

> If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainability wise? You mean like BSD?

boramalper 8y ago
Same with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. Busybox is a slightly better example (even though it’s also a userspace program).
- jcelerier 8y ago
  > Same with BSD; the userspace programs are not part of the kernel, and developed separately by the same entity. they're part of the same repo and built at the same time than the kernel. Run a big "make universe" here : https://github.com/freebsd/freebsd https://github.com/freebsd/freebsd and see for yourself. That they are different binaries does not matter a lot, it's just a question of user interface. See for instance busybox where all the userspace is in a single binary.
  - zerd 8y ago
    I would argue that the fact that they're in the same repo does not make it a monolith at all. All of google and facebook is in a single repo and built at the same time, but it doesn't make them monoliths.

thermodynthrway 8y ago

With web apps the main concern is data consistency between relations. On the OS level you have these same concerns with memory and disk, and there's database-like systems in the kernel and drivers to handle it. Essentially all these utilities are running within the same "database" which is disk and memory management handled by the kernel. Usually microservices have their own databases, which is where consistency hell begins

kolpa 8y ago
> Usually microservices have their own databases That's news to me, and seems insane. Unless you mean "their own database tables", not "database servers". But that's just the same as having multiple directories and files in a Unix filesystem.
- VBprogrammer 8y ago
  That architecture is commonly known as a distributed monolith. If you put two services on the same DB you can guarantee that someone will be joining onto a table they shouldn't have before the week is out.
  - ljw1001 8y ago
    This is another way of saying that micro-services are for easy problems, where you can avoid having many processes share the same complex database.
    - wsy 8y ago
      It's exactly the other way around: If you can afford all your components sharing the same database without creating a big dependency hell, then your problem is _too small_ for microservices. If your problem is so large that you have to split it up to manage its complexity, start considering microservices (it might still not be the right option for you).
      - 8y ago

blablabla123 8y ago

But there is a big difference. These small targeted programs are invoked in user land, usually by the user. Microservices get invoked directly by the user when debugging is going on. Otherwise they are expected to automagically talk to each other and depending on the abstraction even discovery each other automatically. Also I can pipe these tools together from the same terminal session, like tail -f foo | grep something | awk ... You don't have that in general with Microservices. Unix tools are Lego, Microservices aren't. They are Domino at best. Probably one could come up with an abstraction to do Lego with Microservices but we're not there yet.

deleted 8y ago
[deleted]
paxys 8y ago
Look at the package dependency tree of an average linux program. They are absolutely examples of "microservices" talking to each other.
- digi_owl 8y ago
  And they can be a right mess to wrangle because upstream gets into some artistic frenzy, and shouts down anyone that worries about the ensuing breakages as luddites and haters.
  - wsy 8y ago
    And still nobody would argue to pack everything together into one executable because of these issues. As everything in software engineering, it is about finding the right trade-off.
    - digi_owl 8y ago
      Oh, but why do that when we have containers...
- lmm 8y ago

apeace 8y ago

My understanding is that OpenBSD is constructed this way, and I have also heard that their code is well-organized and easy to follow. There is the "base" system which is the OS itself and common packages (all the ones you mentioned), then there is the "ports" repo which contains many open-source applications with patches to make them work with OpenBSD. Here is a Github mirror of their repos: https://github.com/openbsd https://github.com/openbsd I think OpenBSD has reaped many of the same advantages described by Segment with their monorepo approach, such as easily being able to add pledge[1] to many ports relatively quickly. https://www.slideshare.net/GiovanniBechis/pledge-in-openbsd https://www.slideshare.net/GiovanniBechis/pledge-in-openbsd

fastball 8y ago
"packages" and "ports" are both collections of "microservices" in your analogy, so you can't really assert the whole thing is a monolith.
- lmm 8y ago
  The whole thing is released and deployed together, which is the usual distinction between microservices and not. I don't think anyone's advocating not using modules and folders to structure your code in your repository (certainly I hope not), but a single deployment artifact makes life a lot easier.

joshvm 8y ago

One example I'm familiar with that sounds like microservices is the Robot Operating System (ROS). At its heart it's just a framework for pub/sub over IP, it just happens to be targeted towards robotics. A ROS system comprises of 'nodes' for each logical operation e.g. image acquisition -> camera calibration -> analysis -> output. The system is defined by a graph of these nodes, since you can pipe messages wherever they're needed; all nodes can be many-many. Each node is a self-contained application which communicates to a master node via tcp/ip (like most pub/sub systems a master node is requried to tell nodes where to send messages). So you can do cool stuff like have lots of seprate networked computers all talking to each other (fairly) easily. It works pretty well and once you've got a particular node stable - e.g. the node that acquires images - you don't need to touch it. If you need to refactor or bugfix, you only edit that code. If you need to test new things, you can just drop them into an existing system because there's separation between the code (e.g. you just tell the system what your new node will publish/subscribe and it'll do the rest). There is definitely a feeling of duct tape and glue, since you're often using nodes made by lots of different people, some of which are maintainted, others aren't, different naming conventions, etc. However, I think that's just because ROS is designed to be as generic as possible, rather than a side effect of it running like a microservice.

chrisco255 8y ago
There's actually no reason you can't architect microservices like this. You can put RabbitMQ or some AMQP service as a comm layer between services. But then you have to architect your system to be event-driven. It's not a bad approach.
- thermodynthrway 8y ago
  I've worked on a system that used Kafka in-between. It makes consistency issues a ton worse because everything is async. At least with HTTP you can do synchronous calls
  - bsder 8y ago
    Repeat after me: there is no such thing as a synchronous network. I get where you are coming from. But you will go async sooner or later if you need any reasonable error recovery or reliability. The only question is how much pain you will suffer before you do so.

mirceal 8y ago

Linux is the kernel.

inetknght 8y ago

> I would say what makes it useful is all the various small, targeted programs You mean GNU coreutils?