4 ms·
I used freebsd for my last project, I can’t deny that support for BSD is spotty on cloud providers; but as far as performance and so forth it’s not far behind i
by dijit 6y ago
I used freebsd for my last project, I can’t deny that support for BSD is spotty on cloud providers; but as far as performance and so forth it’s not far behind if at all.
The compelling argument for FreeBSD has always been: zfs, pf and dtrace.
There’s also kqueues which are a much better interface to high throughput I/O than epoll.
The problem is that every time this kind of question is asked the ecosystem in Linux is different. Right now Linux has bpftrace, ZFS (kinda) and IO_Uring; so the gap has closed considerably.
- lllr_finger 6y agoIt's historically been a very similar proposition to Solaris: zfs, dtrace, containers, and higher performance. For as much as I like both FreeBSD and Solaris, Brendan Gregg (mentioned elsewhere in the comments) has a very convincing argument as to why the scales have tipped in favor of Linux in the last few years: http://www.brendangregg.com/blog/2017-09-05/solaris-to-linux-2017.html http://www.brendangregg.com/blog/2017-09-05/solaris-to-linux...
- jaekash 6y agoZFS integration was worse in Solaris 10 than in FreeBSD. In Solaris the page cache and ARC was not properly integrated and when using mmap data would be cached twice in both page cache and arc cache and there was weird issues with these contending with each other. I think this was still the case in Solaris 11 also but hard to say since it is closed source and their marketing material was fraudulent. As for the higher performance, I would like to see evidence that Solaris has better performance than Linux. The one saving grace of Solaris was that it gave you amazing debugging capabilities but the practical outcome of that was just giving you the ability to gain deep insight to see see exactly how bad Solaris was as an operating system. In all my years of using it (10+) I never experienced it as being more performant than Linux and in most cases it was way slower if you did not tune it significantly. There were some very good ideas that went into Solaris but they were almost invariably poorly implemented.
- Annatar 6y agoSolaris regularly beats Linux in performance on the same hardware, and the subsystems which are "slower" aren't really: they just don't lie and trade safety for performance, like Linux does. Linux is fake performance at the cost of correctness of operation.
- ThA0x2 6y agoSolaris hasn't beaten Linux in performance in the past 5 years in anything. As a matter of fact, you can't point to a single source that proves your point. "That which is purported without evidence can be disregarded without evidence."
- jaekash 6y agoI would like to see some evidence to back this claim.
- Annatar 6y agoThe burden of proof is on you, since you're the one clearly advocating for Linux here.
- jaekash 6y ago> The burden of proof is on you, since you're the one clearly advocating for Linux here. Ehh... if you want proof for a specific claim then ask for it. I am asking you for proof of the specific positive claim you made, which is: > Solaris regularly beats Linux in performance on the same hardware, and the subsystems which are "slower" aren't really: they just don't lie and trade safety for performance, like Linux does. Linux is fake performance at the cost of correctness of operation. The claims I did make is that in my experience Solaris had to be tuned to be reasonably performant. One example where tuning was needed by default was malloc. The default malloc from Solaris had horrible multi threaded performance . IIRC ZFS also had quite horrible performance by default if not tuned and I recall there were multiple tuning we had to do on new installations specifically relating to mmap to somewhat mitigate the poor integration of ZFS ARC cache and page cache which are documented here: https://github.com/openzfs/zfs/issues/225 https://github.com/openzfs/zfs/issues/225 and https://lists.freebsd.org/pipermail/freebsd-hackers/2016-July/049665.html https://lists.freebsd.org/pipermail/freebsd-hackers/2016-Jul... there was also a whole wiki on ZFS tuning for various applications including MySQL.
- Annatar 6y agoBrendan's "convincing" argument can be summed up as "Linux is better in all areas" and listing some relatively new technologies like BPF but which are horrid to program much less understand, and he lists no technical details. Whatever short-circuit suddenly happened in his head, he can be treated as a hostile witness for all intents and purposes. Him and I had a long one on one discussion and I'm not in the least convinced by his "arguments".
- 8fingerlouie 6y ago> The compelling argument for FreeBSD has always been: zfs, pf and dtrace. You forgot to mention the "completeness" of the base system. Where Linux distributions is a "mix and match" of various tools/versions, that may or may not be the same across each of them. FreeBSD provides a solid base that "just works". There's just one version (per release). Jails is also another great feature. Linux has cgroups and namespaces, with various tools built on top of it like Docker, LXC and LXD, but (IMO) Jails still beat them in a heartbeat by security and ease of use. Docker has had more severe vulnerabilities in a single year (2018 IIRC) than Jails has had in it's entire lifetime.
- dijit 6y agoI never saw docker as being built for security, I think it's a common concept that's attributed to them because "contain(ers)" sounds like it contains things, the reality is that they're talking about shipping containers pretty exclusively in the context of docker. Shipping containers are not secure.
- marios 6y agoBut the content of shipping container #1 does not spill into shipping container #2. If it does, there's a problem. Similarly with software containers, you don't want data flowing into nearby containers (unless you do want that, in which case you have setup the necessary networking and that should be the only way for the containers in question to share data). I'm mostly interested in docker et al for the isolation aspect. The "ships the app with all its dependencies" is an interesting byproduct, although all too often it translates it means there's a mess in the amount of dependencies required. And that it's going to be an equal mess come update time -- unless of course the image is never updated in which case there's a chance the image becomes vulnerable (for example, outdated libssl).
- 8fingerlouie 6y ago> I'm mostly interested in docker et al for the isolation aspect. The "ships the app with all its dependencies" is an interesting byproduct And that's why i use Jails instead (or LXC if you're on Linux). None of the layers. Any vulnerability is in the "base" image, and easily spotted, and can be hotfixed using normal package tools. Docker is a mess. Most containers consist of minimum 4 "base images", each of which can contain vulnerabilities. All done to save the "overhead" of installing Python on top of a base Debian image, or whatever your flavor of poison is. Instead i use Ansible for reproducibility, and scripts for updating/upgrading. Another advantage to jails is that they're just plain directories in a hosts filesystem, allowing existing vulnerability scanners to scan the entire stack from a single host, instead of using containers that needs to be scanned individually.
- jaekash 6y ago> There’s also kqueues which are a much better interface to high throughput I/O than epoll. Can you clarify why you say this? One area where kqueue will be more performance is in that it supports multiple updates on interest set in one operation, while epoll does not, which means that doing multiple updates to an interest set will likely be less performant with epoll, however high throughput I/O does not translate to high frequency of updates to an interest set.
- dijit 6y agoI never mentioned performance, I'm suggesting that kqueue is a better 'interface'. epoll is an interface designed by someone who didn't know what they were doing at the time, that's the only way it can be described.. if you look at alternative implementations (such as IO-Completion ports on Windows) then you can see there's a lot to be desired, one of the things epoll doesn't do, for instance is tell you how many bytes are waiting on a buffer; so you have to read the whole buffer and determine if you're beginning a new unrelated message or not. There's also the fact that epoll will evict things from the buffer on read; so you can't do a partial read and come back. It's just.. bad. io_uring solves a lot of these issues.
- patrec 6y agoFor someone who has good grounding in FreeBSD and the above technologies already, I can see how that decision would make sense (if you know dtrace well, you probably invested several man weeks into it -- why throw that away?). Typically the situation will be the reverse however (almost anyone you can hire at any level will know more Linux than FreeBSD) and the technological benefits you list do not seem like they would make up for that. ZFS: these days it's developed on and for linux first and foremost, as far as I can tell (FreeBSD moved to ZFS on linux, correct?). Dtrace: linux finally caught up and basically went one better with perf/ebpf. kqueues: as you said, io_uring looks like it will take care of that (and probably put linux ahead). pf: I can see how pf is compelling if what you want is first and foremost some firewall box, and you take whatever OS comes with the firewall (I haven't used pf, but most people seem to agree its significantly nicer than iptables et al).
- dijit 6y agoiptables has been supplanted by nftables; though it's not as highly scalable as pf and uses a lot of iptables terminology. I begrudge the notion though, that desiring something like pf is because you primarily want a firewall box; the syntax is much cleaner and ultimately the performance is better (without bolt-ons like conntrack). For example these are basic firewalls with rate limiting pf: https://git.drk.sc/darkscience/ds-salt/-/blob/master/states/base/security/freebsd/firewall https://git.drk.sc/darkscience/ds-salt/-/blob/master/states/... iptables: https://git.drk.sc/darkscience/ds-salt/-/blob/master/states/base/security/debian/ipv4-iptables https://git.drk.sc/darkscience/ds-salt/-/blob/master/states/... here's what we have for nft: https://git.drk.sc/darkscience/playbooks/-/blob/a09507e2a9b6e44dcba47b5d4c36a1901a2beac1/roles/firewall/templates/nftables.conf https://git.drk.sc/darkscience/playbooks/-/blob/a09507e2a9b6...
- patrec 6y agoI'm not saying you wouldn't desire something like pf when using linux. Of course you would. But in the grand scheme of things, I don't see many people choosing FreeBSD over Linux for pf unless they want a firewall box.
- haecceity 6y agoio_uring is a lot more complex than kqueue though.