3 ms·

Yeah... It took 4 years to properly integrate Kafka into our pipelines. Everything, like everything is complicated with it: cluster management, numerous semi-t

Yeah... It took 4 years to properly integrate Kafka into our pipelines. Everything, like everything is complicated with it: cluster management, numerous semi-tested configurations, etc. My final conclusion with it is that the project just doesn't really know what it wants to be. Instead it tries to provide everything for everybody, and ends up being an unbelievably complicated mess. You know, there are systems that know what they want to be (Amazon S3, Postres, etc), and then there are systems that try to eat the world (Kafka, k8s, systemd).

1oooqooq 1y ago
systemd knows very well what it wants to be, they just don't tell anyone. it's real goal is to make Linux administration as useless as windows so RH can sell certifications. tell me the output of systemctl is not as awful as opening the windows service panel.
- hbogert 1y ago
  Tell me systemctl output isn't more beneficial than per distro bash-mess
  - vkazanov 1y ago
    Well, systemd IS useful, the same way Kafka is. I don't want to back to crappy bash for service management, and Kafka is a de fact standard event streaming solution. But both are kind of hard to understand end-to-end, especially for an occasional user.
  - 1oooqooq 1y ago
    not really. both requires that you know obscure and badly documented stuff. systemd whole premise is "people will not read the distro or bash scripting manual"... then nobody read systemd's (you have even less reason, since it's badly written, ever changing in conflicting ways, and a single use tool) so you went from complaining your coworkers can't write bash to complaining they don't know they have to use EXEC= EXEC=/bin/x because random values, without any hint, are lists of commands instead of string values.
    - tarasglek 1y ago
      Bash scripts are write-only software. one read systemd c
      - 1oooqooq 1y ago
        that's, like, your opinion.
  - zaphirplane 1y ago
    I have no dog in the systemd wars. The bash output is what? Distro use other open source software.
- chupasaurus 1y ago
  There are 2 service panels in Windows since 8 and they are quite different...
  - 1oooqooq 1y ago
    both (and systemctl) will show two to three times the same useless info on the screen. e.g. service x - start the service x.
EdwardDiego 1y ago
> that the project just doesn't really know what it wants to be It's a distributed log? What else is it trying to do?
- zaphirplane 1y ago
  Calling it a distributed log may just be a Reductio ad absurdum
  - EdwardDiego 1y ago
    How?
pphysch 1y ago
> You know, there are systems that know what they want to be (Amazon S3, Postres, etc), and then there are systems that try to eat the world (Kafka, k8s, systemd). I am not sure about this taxonomy. K8s, systemd, and (I would add) the Linux kernel are all taking on the ambitious task of central, automatic orchestration of general purpose computing systems. It's an extremely complex problem and I think all those technologies have done a reasonably good job of choosing the right abstractions to break down that (ever-changing) mess. People tend to criticize projects with huge scope because they are obviously complex, and complexity is the enemy, but most of the complexity is necessary in these cases. If Kafka's goal is to be a general purpose "operating system" for generic data systems, then that explains its complexity. But it's less obvious to me that this premise is a good one.

1oooqooq 1y ago

systemd knows very well what it wants to be, they just don't tell anyone. it's real goal is to make Linux administration as useless as windows so RH can sell certifications. tell me the output of systemctl is not as awful as opening the windows service panel.

hbogert 1y ago
Tell me systemctl output isn't more beneficial than per distro bash-mess
- vkazanov 1y ago
  Well, systemd IS useful, the same way Kafka is. I don't want to back to crappy bash for service management, and Kafka is a de fact standard event streaming solution. But both are kind of hard to understand end-to-end, especially for an occasional user.
- 1oooqooq 1y ago
  not really. both requires that you know obscure and badly documented stuff. systemd whole premise is "people will not read the distro or bash scripting manual"... then nobody read systemd's (you have even less reason, since it's badly written, ever changing in conflicting ways, and a single use tool) so you went from complaining your coworkers can't write bash to complaining they don't know they have to use EXEC= EXEC=/bin/x because random values, without any hint, are lists of commands instead of string values.
  - tarasglek 1y ago
    Bash scripts are write-only software. one read systemd c
    - 1oooqooq 1y ago
      that's, like, your opinion.
- zaphirplane 1y ago
  I have no dog in the systemd wars. The bash output is what? Distro use other open source software.
chupasaurus 1y ago
There are 2 service panels in Windows since 8 and they are quite different...
- 1oooqooq 1y ago
  both (and systemctl) will show two to three times the same useless info on the screen. e.g. service x - start the service x.

EdwardDiego 1y ago

> that the project just doesn't really know what it wants to be It's a distributed log? What else is it trying to do?

zaphirplane 1y ago
Calling it a distributed log may just be a Reductio ad absurdum
- EdwardDiego 1y ago
  How?

pphysch 1y ago

> You know, there are systems that know what they want to be (Amazon S3, Postres, etc), and then there are systems that try to eat the world (Kafka, k8s, systemd). I am not sure about this taxonomy. K8s, systemd, and (I would add) the Linux kernel are all taking on the ambitious task of central, automatic orchestration of general purpose computing systems. It's an extremely complex problem and I think all those technologies have done a reasonably good job of choosing the right abstractions to break down that (ever-changing) mess. People tend to criticize projects with huge scope because they are obviously complex, and complexity is the enemy, but most of the complexity is necessary in these cases. If Kafka's goal is to be a general purpose "operating system" for generic data systems, then that explains its complexity. But it's less obvious to me that this premise is a good one.