4 ms·

All is fine and dandy and BSDs can solve many use cases. Unfortunately for the solution we are working on, which imply many microservices we need Kubernetes and

All is fine and dandy and BSDs can solve many use cases. Unfortunately for the solution we are working on, which imply many microservices we need Kubernetes and no BSD equivalent to Kubernetes exists.

roydivision 2y ago
Same here, otherwise I'd be considering the BSDs.
- draga79 2y ago
  Things are improving...stay tuned :-)
INTPenis 2y ago
Can't you just run k8s on bsd? You might have to build and maintain your own release of it, but I'm sure someone has done it already.
- blueflow 2y ago
  The BSD's don't have the namespaces/unshare syscall functionality like Linux does.
  - arianvanp 2y ago
    Use microvms instead of containers perhaps?
  - yjftsjthsd-h 2y ago
    FreeBSD has jails, which AFAIK are fully a match in terms of functionality, but you would need to finish implementing containerd support (maybe via runj?) and then get k8s to run on top of that. I seriously doubt there's any actual technical blockers, it's just that nobody's finished implementing all the pieces and putting them together.
- dbolgheroni 2y ago
  No, FreeBSD has jails but k8s uses different types of runtimes for OCI containers (containerd, CRI-O, Docker Engine, etc).
kev009 2y ago
I've helped build two top 10 service provider networks (10s of Tbps). One on FreeBSD, and one on Linux with Kubernetes. I don't really see Kubernetes as being a game changer. The biggest pro, it makes it easier to onboard both development and operations personnel having a quasi-standard for how a lot of things like scheduling and application networking work. But it also seems to come with a magnitude of accidental and ornamental complexity. I would imply the same about microservices versus, say, figuring out your repository, language, and deployment pipelines to provide a smooth developer and operator experience. Too much of this industry is fashion and navel gazing instead of thinking about the core problems and standing behind a methodology that works for the business. Unless google moves its own infrastructure to Kubernetes, then maybe there's something to be had that couldn't reasonably be done otherwise :)
- hggigg 2y ago
  Agree with this on all points. We went from a virtualized server model to managed Kubernetes and costs have escalated considerably. The additional complexity and maintenance overheads of Kubernetes are not trivial and required additional staff hires just to keep things ticking. I think the cost so far from moving from two cages in separate datacentres running blades to AWS is approximately a 6x multiplier including staff. This was all driven on the back of "we must have microservices to scale", something we have failed entirely to do. It's a complete own goal.
  - threeseed 2y ago
    That's because you've moved to AWS which is an expensive cloud. You could have easily run Kubernetes on your virtualised servers.
    - hggigg 2y ago
      Define easily?
      - stackskipton 2y ago

roydivision 2y ago

Same here, otherwise I'd be considering the BSDs.

draga79 2y ago
Things are improving...stay tuned :-)

INTPenis 2y ago

Can't you just run k8s on bsd? You might have to build and maintain your own release of it, but I'm sure someone has done it already.

blueflow 2y ago
The BSD's don't have the namespaces/unshare syscall functionality like Linux does.
- arianvanp 2y ago
  Use microvms instead of containers perhaps?
- yjftsjthsd-h 2y ago
  FreeBSD has jails, which AFAIK are fully a match in terms of functionality, but you would need to finish implementing containerd support (maybe via runj?) and then get k8s to run on top of that. I seriously doubt there's any actual technical blockers, it's just that nobody's finished implementing all the pieces and putting them together.
dbolgheroni 2y ago
No, FreeBSD has jails but k8s uses different types of runtimes for OCI containers (containerd, CRI-O, Docker Engine, etc).

kev009 2y ago

I've helped build two top 10 service provider networks (10s of Tbps). One on FreeBSD, and one on Linux with Kubernetes. I don't really see Kubernetes as being a game changer. The biggest pro, it makes it easier to onboard both development and operations personnel having a quasi-standard for how a lot of things like scheduling and application networking work. But it also seems to come with a magnitude of accidental and ornamental complexity. I would imply the same about microservices versus, say, figuring out your repository, language, and deployment pipelines to provide a smooth developer and operator experience. Too much of this industry is fashion and navel gazing instead of thinking about the core problems and standing behind a methodology that works for the business. Unless google moves its own infrastructure to Kubernetes, then maybe there's something to be had that couldn't reasonably be done otherwise :)

hggigg 2y ago
Agree with this on all points. We went from a virtualized server model to managed Kubernetes and costs have escalated considerably. The additional complexity and maintenance overheads of Kubernetes are not trivial and required additional staff hires just to keep things ticking. I think the cost so far from moving from two cages in separate datacentres running blades to AWS is approximately a 6x multiplier including staff. This was all driven on the back of "we must have microservices to scale", something we have failed entirely to do. It's a complete own goal.
- threeseed 2y ago
  That's because you've moved to AWS which is an expensive cloud. You could have easily run Kubernetes on your virtualised servers.
  - hggigg 2y ago
    Define easily?
    - stackskipton 2y ago