6 ms·

I'm excited to see how this compares to SmartOS. I'm pretty heavily invested in SmartOS in my personal infrastructure but its future, post-Joyent acquisition, h

I'm excited to see how this compares to SmartOS. I'm pretty heavily invested in SmartOS in my personal infrastructure but its future, post-Joyent acquisition, has been worrying me. I really wish I did work for an org big enough to use Oxide's gear. Not having to futz around with bogus IBM PC AT-type compatibility edifice, janky BMCs and iDRACs, hardware RAID controllers, etc, would be so unbelievably nice.

_rs 3y ago
I had been using SmartOS for a long time but finally had to bite the bullet and give up. I ended up deciding on Proxmox on a ZFS root and am quite happy with it.
- geek_at 3y ago
  the nice thing about the Proxmox + ZFS setup is that it works and is even recommended without using hardware raid controllers. Less headaches either way. I recently wrote a guide [1] how to use proxmox with ZFS over iSCSI so you can use the snapshot features from a SAN [1] https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html
- rjzzleep 3y ago
  I feel the same. I used a SmartOS distro called Danube Cloud for a long time and am looking to move and looked at Harvester[1] and OpenNebula, but with everything I know about Kubernetes(and LongHorn) I'm reluctant to use something so heavily based on Kubernetes. At its peak I reached out multiple times to Joyent to fix their EFI support for virtualization. The Danube team had similar experiences with them, working on live migrations for VMs, and a few months back I did a rebase of the platform image to a more recent illumos stack. Two of the fundamental issues with Illumos is that they don't seem to understand that they need to fix the horrendous platform build to get community support to keep up with the pace of development of other OS's. The platform build is a huge nasty mess of custom shell scripts, file based status snapshots, which includes the entire userspace in the kernel build. Basically if your openssl version is out of wack the entire thing will fail. Not because it has to, but because it was never adapted to modern needs of someone just wanting to hack on a kernel. It's fixable, but I don't see any desire to fix it, and even if that desire eventually shows up it might just be too little, too late. [1] https://harvesterhci.io/ https://harvesterhci.io/
- icybox 3y ago
  I've been running smartos at least since 2015 where I co-located my server. There have been times where I felt like giving up, but people like danmcd, jperkin and others always stepped in and fixed what needed to be fixed for LX to be usable and working. (Keeping java updated and running is hard, uphill battle. Thanks!) I always ran a mixture of OS and LX zones and bcantrill's t-shirt with "Save the whales, kill your VM" made sense. I've used zones in Solaris 10 even before and they just click with me. FreeBSD's jails are nice, but far from it. And linux's cgroups are a joke. And using KVM/VMs for security containerization is just insane. At dayjob, I've implemented multiple proxmox clusters, because we're linux shop and there's no way to "sell" smartos or tritonDC to die-hard debian colleagues, but I've managed to sell them ZFS. With personal stuff, I like my systems to take care of themselves without constant babysitting and SmartOS or OpenBSD provide just that. I don't dislike windows, I love UNIX. You could really feel those extra 20y UNIX had compared to linux. I migrated all my stuff to proxmox for like 2 months. And then went back to SmartOS, because there was something missing ... probably elegance, sanity, simplicity or even something you'd call "hack value".
  - deleted 3y ago
    [deleted]
  - unethical_ban 3y ago
    And here I am, having compared the SmartOS documentation and ease of installation to Proxmox... and with very few complaints am using Proxmox to host a file server on bare metal/Samba container and a OPNSense on VM. I remember buying the OpenSolaris Bible in 2008, getting really excited to dig into my second Unix (after FreeBSD). And then, the Sun went down on me... and I stuck with Ubuntu 10 years.
    - rbanffy 3y ago
      > and I stuck with Ubuntu 10 years. For a while Nexenta had an Ubuntu running on the OpenSolaris kernel.
  - sswezey 3y ago
    Do you have a write-up of your SmartOS and/or OpenBSD set up(s)?
nwilkens 3y ago
SmartOS is being actively developed since the aquisition from Joyent[1] in April 2022. We've released a new version every two weeks post acquisition, and are continuing to develop and invest. We also hold office hours events roughly every two weeks on Discord[2], and would love for you to stop by and ask any questions, or just listen along! [1]: https://www.tritondatacenter.com/blog/a-new-chapter-begins-for-triton-and-smartos https://www.tritondatacenter.com/blog/a-new-chapter-begins-f... [2]: https://discord.gg/v4NwA3Hqay https://discord.gg/v4NwA3Hqay
- rjzzleep 3y ago
  IllumOS needs to attract new developers. To do that, the platform build needs to become a lot more straightforward. It's a pretty huge endeavour in my opinion. I'd be happy to help out on that regard, but in the past Joyent has not been very open to outside support.
  - jclulow 3y ago
    I've noticed you make this comment repeatedly when illumos is mentioned on HN. I think you're underestimating the irreducible complexity of the build process for what is essentially a whole UNIX operating system, save for a few external dependencies. It's not just a kernel, but an extensive set of user mode libraries and executables. The build is complex in part because it's a complex body of software. I also think you're overestimating the extent to which make(1S) is the reason we're not more popular than Linux. There are any number of more relevant factors that make someone choose one operating system or another. Also, certainly for me personally my goal is not world domination, merely the sustainable maintenance of a body of software that helps me solve the problems that I work on, and which I enjoy using and developing as a result. I agree we need (as do all projects!) new developers, both now, and over the long term. We work as we can to make improvements to the build process, and the documentation. We are a relatively niche project, but we do attract new developers from time to time, and we're making changes at least as rapidly as we ever have in the past. There are a number of actively maintained illumos distributions (OmniOS, SmartOS, Tribblix, OpenIndiana, and now Helios) and there are a variety of commercial interests that ship more proprietary appliances on top of an illumos base. For our part at Oxide we continue to encourage our staff to get involved with illumos development as it makes sense for them, and we try to offer resources and assistance to the broader community as well. If you would like to contribute, we have a guide to getting started: https://illumos.org/docs/contributing/ https://illumos.org/docs/contributing/ Please, though, it's "illumos", not "IllumOS"!

_rs 3y ago

I had been using SmartOS for a long time but finally had to bite the bullet and give up. I ended up deciding on Proxmox on a ZFS root and am quite happy with it.

geek_at 3y ago
the nice thing about the Proxmox + ZFS setup is that it works and is even recommended without using hardware raid controllers. Less headaches either way. I recently wrote a guide [1] how to use proxmox with ZFS over iSCSI so you can use the snapshot features from a SAN [1] https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html
rjzzleep 3y ago
I feel the same. I used a SmartOS distro called Danube Cloud for a long time and am looking to move and looked at Harvester[1] and OpenNebula, but with everything I know about Kubernetes(and LongHorn) I'm reluctant to use something so heavily based on Kubernetes. At its peak I reached out multiple times to Joyent to fix their EFI support for virtualization. The Danube team had similar experiences with them, working on live migrations for VMs, and a few months back I did a rebase of the platform image to a more recent illumos stack. Two of the fundamental issues with Illumos is that they don't seem to understand that they need to fix the horrendous platform build to get community support to keep up with the pace of development of other OS's. The platform build is a huge nasty mess of custom shell scripts, file based status snapshots, which includes the entire userspace in the kernel build. Basically if your openssl version is out of wack the entire thing will fail. Not because it has to, but because it was never adapted to modern needs of someone just wanting to hack on a kernel. It's fixable, but I don't see any desire to fix it, and even if that desire eventually shows up it might just be too little, too late. [1] https://harvesterhci.io/ https://harvesterhci.io/
icybox 3y ago
I've been running smartos at least since 2015 where I co-located my server. There have been times where I felt like giving up, but people like danmcd, jperkin and others always stepped in and fixed what needed to be fixed for LX to be usable and working. (Keeping java updated and running is hard, uphill battle. Thanks!) I always ran a mixture of OS and LX zones and bcantrill's t-shirt with "Save the whales, kill your VM" made sense. I've used zones in Solaris 10 even before and they just click with me. FreeBSD's jails are nice, but far from it. And linux's cgroups are a joke. And using KVM/VMs for security containerization is just insane. At dayjob, I've implemented multiple proxmox clusters, because we're linux shop and there's no way to "sell" smartos or tritonDC to die-hard debian colleagues, but I've managed to sell them ZFS. With personal stuff, I like my systems to take care of themselves without constant babysitting and SmartOS or OpenBSD provide just that. I don't dislike windows, I love UNIX. You could really feel those extra 20y UNIX had compared to linux. I migrated all my stuff to proxmox for like 2 months. And then went back to SmartOS, because there was something missing ... probably elegance, sanity, simplicity or even something you'd call "hack value".
- deleted 3y ago
  [deleted]
- unethical_ban 3y ago
  And here I am, having compared the SmartOS documentation and ease of installation to Proxmox... and with very few complaints am using Proxmox to host a file server on bare metal/Samba container and a OPNSense on VM. I remember buying the OpenSolaris Bible in 2008, getting really excited to dig into my second Unix (after FreeBSD). And then, the Sun went down on me... and I stuck with Ubuntu 10 years.
  - rbanffy 3y ago
    > and I stuck with Ubuntu 10 years. For a while Nexenta had an Ubuntu running on the OpenSolaris kernel.
- sswezey 3y ago
  Do you have a write-up of your SmartOS and/or OpenBSD set up(s)?

nwilkens 3y ago

SmartOS is being actively developed since the aquisition from Joyent[1] in April 2022. We've released a new version every two weeks post acquisition, and are continuing to develop and invest. We also hold office hours events roughly every two weeks on Discord[2], and would love for you to stop by and ask any questions, or just listen along! [1]: https://www.tritondatacenter.com/blog/a-new-chapter-begins-for-triton-and-smartos https://www.tritondatacenter.com/blog/a-new-chapter-begins-f... [2]: https://discord.gg/v4NwA3Hqay https://discord.gg/v4NwA3Hqay

rjzzleep 3y ago
IllumOS needs to attract new developers. To do that, the platform build needs to become a lot more straightforward. It's a pretty huge endeavour in my opinion. I'd be happy to help out on that regard, but in the past Joyent has not been very open to outside support.
- jclulow 3y ago
  I've noticed you make this comment repeatedly when illumos is mentioned on HN. I think you're underestimating the irreducible complexity of the build process for what is essentially a whole UNIX operating system, save for a few external dependencies. It's not just a kernel, but an extensive set of user mode libraries and executables. The build is complex in part because it's a complex body of software. I also think you're overestimating the extent to which make(1S) is the reason we're not more popular than Linux. There are any number of more relevant factors that make someone choose one operating system or another. Also, certainly for me personally my goal is not world domination, merely the sustainable maintenance of a body of software that helps me solve the problems that I work on, and which I enjoy using and developing as a result. I agree we need (as do all projects!) new developers, both now, and over the long term. We work as we can to make improvements to the build process, and the documentation. We are a relatively niche project, but we do attract new developers from time to time, and we're making changes at least as rapidly as we ever have in the past. There are a number of actively maintained illumos distributions (OmniOS, SmartOS, Tribblix, OpenIndiana, and now Helios) and there are a variety of commercial interests that ship more proprietary appliances on top of an illumos base. For our part at Oxide we continue to encourage our staff to get involved with illumos development as it makes sense for them, and we try to offer resources and assistance to the broader community as well. If you would like to contribute, we have a guide to getting started: https://illumos.org/docs/contributing/ https://illumos.org/docs/contributing/ Please, though, it's "illumos", not "IllumOS"!