4 ms·

FreeBSD ate my RAM

jmclnx 4mo ago
Interesting post, it made me wonder. At one time FreeBSD swap usage/logic was far better than what Linux did. Is that still the case ?
- shevy-java 4mo ago
  I remember how NetBSD promoted itself as running on many more toasters than Linux once. Then some NetBSD dev wrote on their mailing list that this is no longer true. Linux runs on more toasters now. (And also top 500 supercomputers, but toasters are the real metal to the petal test.) These fights always remind me of: https://www.jwz.org/doc/worse-is-better.html https://www.jwz.org/doc/worse-is-better.html It's an interesting piece of history too. I kind of evaluate it a bit differently, e. g. my summary is "momentum beats academic perfection". Which is not completely what it is about, but it is my own imperfect TL;DR summary.
  - bee_rider 4mo ago
    This basically fits my stereotype of BSD being a little bit more hardcore while Linux is a little more accessible… when the question was “can you install an OS on a toaster,” BSD had an advantage. Now that normal engineers have to make IOT toasters (for some reason) Linux should have the advantage, right?
    - sublinear 4mo ago
      Normal engineers don't do that either.
  - Levitating 4mo ago
    why did that url point me to a scrotum in an egg cup
    - UnlockedSecrets 4mo ago
      Seems like at a glance that website if it sees a referral from ycombinator, it redirects to that image.... In a private window it loads the 'intended' page.....
    - m0llusk 4mo ago
      HN has disappointed JWZ
      - saghm 4mo ago
        Just blackholing the request would be kinder than taking it out on the person following the link, given that their gripe seems to be more with the one who posted it (and that as a deterrent, it doesn't seem to be particularly effective given that it's been years since the first time I've seen this conversation play out in comments)
  - Someone 4mo ago
    > but toasters are the real metal to the petal test. Or is that a new tongue in cheek idiom? AFAIK, it’s peDal, not peTal, and the other way around: pedal to the metal. The literal meaning is to push your accelerator pedal down to the floor of your car, which is made of metal.
    - goodmythical 4mo ago
      "metal to the petal" gives me more of a "gilding the lily" flavor, especially in cnotrast to the original construction. Still not sure if it was on purpose, but I read it as a reference to a "but can it run DOOM?" benchmark.
- 0x457 4mo ago
  Yes, It's just not every tool is aware of ZFS ARC. Which is what this post is about. Author just describes in an odd way.
  - mike_hock 4mo ago
    It's a bit odd that you have to query a filesystem-specific value to get simple memory usage stats.
    - uniqueuid 4mo ago
      Which is exactly (part of) why people call ZFS a "rampant layering violation"!
    - 0x457 4mo ago
      Despite having File System, ZFS is a lot more than just file system. Sooner you understand and make peace with it, easier it is to use it.
- man8alexd 4mo ago
  FreeBSD didn’t have memory overcommit and instead used strict swap reservation - each allocated anonymous memory page was supposed to have a corresponding swap page. This required 2x RAM swap space, otherwise you would get “out of swap” when forking a large process. FreeBSD implemented memory overcommit around 2000.
  - jandrese 4mo ago
    Oh, so that's where that old nugget of wisdom came from! I've heard the rule about making your swap at least 2x your RAM for ages and thought it was just some old rule of thumb from the 80s. I didn't know there used to be a legitimate reason for it.
    - man8alexd 4mo ago
      See this retrocomputing.SE question https://retrocomputing.stackexchange.com/questions/32492/origin-of-the-rule-that-swap-size-should-be-2x-of-the-physical-memory https://retrocomputing.stackexchange.com/questions/32492/ori...
naturalmovement 4mo ago
[flagged]
- toast0 4mo ago
  Sure, but also some tools needed fixing.
- deleted 4mo ago
  [deleted]
- craftkiller 4mo ago
  I've noticed that the various tools report different amounts of free memory. I appreciate finally having an explanation for why.
m463 4mo ago
the end struck me - a picture of an os book. I wonder if students these days retain their books after college, or do they get returned as a rental?
- linguae 4mo ago
  I'm a professor at a community college in Silicon Valley, and my students use online textbooks. I try to use Creative Commons or other libre textbooks, but sometimes I use paid textbooks when they are heads-and-shoulders better than their libre alternatives. Some e-textbooks can be accessed on a subscription basis. I admit I prefer non-subscription materials, but a colleague advised me that often the book that students learn from is different from a good reference book that students can use once they've already learned the material. For example, my colleagues and I have had great success with an online, interactive textbook for discrete math. While the subscription is unfortunately only valid for the duration of the course, once students have learned discrete math, they could buy a used copy of Rosen's discrete math textbook as a reference. The nice thing about e-textbooks is not needing to carry around a bunch of heavy books. I remember the tomes I had in my college days, such as Stewart's Calculus.
  - NooneAtAll3 4mo ago
    just hint students towards anna's archive and then sky's the limit
  - saghm 4mo ago
    > The nice thing about e-textbooks is not needing to carry around a bunch of heavy books. I remember the tomes I had in my college days, such as Stewart's Calculus. I'm not sure how relatable this is for people outside of my age group (late millennial), but growing up we used to get chided about stuffing too much in our backpacks and shown presentations about how we were going to give ourselves scoliosis, but then had like two minutes between classes that might be on the other side of the building (meaning no time to stop at our lockers other than at lunch) and we'd get chided even worse if we didn't have our textbooks available for the days we happened to need them in class (which of course we were never told in advance, and some classes had multiple textbooks that wouldn't all get used every time). I wish that we had iPads or Chromebooks or whatever back when I was in school not even because I would have wanted to have been able to surf the web or play games or whatever, but just to have a solution to having to pick between having a sore back or getting worse scolding (with a side course of hypocrisy either way).
duendefm 4mo ago
Thank you for such a quality post.
hnloser 4mo ago
[flagged]
tomeow 4mo ago
[flagged]
shevy-java 4mo ago
[flagged]
- edoceo 4mo ago
  The OS Crusades are over man
  - irishcoffee 4mo ago
    For real though, what is the point of running a bsd on a personal computer? Seems like intentionally handicapping yourself.
    - soupbowl 4mo ago
      Consistent ZFS tooling around the OS and ZFS support out of the box, which always works.
      - irishcoffee 4mo ago
        So more so because of the storage drivers? I don’t have the specific concern, maybe that’s why I don’t understand the draw.
    - BSDobelix 4mo ago
      I have FreeBSD on all my personal machines and i feel the opposite of handicapped, however i use Linux (CachyOS) for gaming (as a VM with pass-thru).
- vermaden 4mo ago
  No ZFS no problem :)
- acheron 4mo ago
  Netcraft confirms it.
  - BSDobelix 4mo ago
tom2ow 4mo ago
[flagged]
tom2ow 4mo ago
[flagged]
tiffanyh 4mo ago
If you like this kind of post, you might like this “htop explained” post. https://peteris.rocks/blog/htop/ https://peteris.rocks/blog/htop/
- NooneAtAll3 4mo ago
  htop explained* I was looking forward for web protocol, but alas...
  - tiffanyh 4mo ago
    It was a typo. Thanks, fixed. But there is an http headers article you might like. https://peteris.rocks/blog/exotic-http-headers/ https://peteris.rocks/blog/exotic-http-headers/
    - teddyh 4mo ago
      I”d instead suggest RFC 9110, HTTP Semantics: <https://www.rfc-editor.org/info/rfc9110/ https://www.rfc-editor.org/info/rfc9110/>
efxhoy 4mo ago
Great job on getting the fixes merged!
drdexebtjl 4mo ago
I don’t understand the part about using heuristics and deciding what counts as used memory… Used memory for the system is always total minus available. Heuristics? I would hope that the system knows precisely what is using every single byte of physical and virtual memory. Is this a reporting problem? Why do we have to settle for heuristics and not the exact number?
- CrociDB 4mo ago
  The thing is it's easy to define free, unused memory. But a lot of the used memory is your system caching stuff that would be free if you needed more than what's actually free. So you can see you have 1g of free memory out of your 4g, but then you allocate 3g and it will do without a sweat and you'd be confused. So you have to go and dig for what those caches are and report that they're effectively free too.
  - drdexebtjl 4mo ago
    Instantly reclaimable disk caches should count as available, and they do. This isn’t hard. The OS should just expose a counter for available memory instead of having applications understand every type of memory reservation. edit: Linux does this, but it has its own share of issues with memory counters. The “cached” memory includes tmpfs and ramfs for seemingly no reason.
    - jkrejcha 4mo ago
      > The “cached” memory includes tmpfs and ramfs for seemingly no reason. If you're curious why that is by the way, it's because that's actually how these are implemented (tmpfs/ramfs is just a mount to a filesystem where the files never get marked clean[1]) [1]: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt https://www.kernel.org/doc/Documentation/filesystems/ramfs-r...
      - drdexebtjl 4mo ago
        That’s clever. Makes for terrible UX, though. AFAIK the only way for you to figure out how much of your disks is actually cached involves enumerating all tmpfs and ramfs mounts, summing their sizes, and subtracting the sum from the cache size reported by the kernel.
wolvoleo 4mo ago
I wonder if these btop fixes got into the standard ports collection? Or even upstream? I like the command for viewing the ARC cache size, never knew that. It's only 2GB on my system (of 64GB RAM).
- BSDobelix 4mo ago
  >I like the command for viewing the ARC cache size zfs-stats is also nice for more zfs internals like hit-rate L2 etc.. https://www.freshports.org/sysutils/zfs-stats https://www.freshports.org/sysutils/zfs-stats
  - wolvoleo 4mo ago
    Thanks! I didn't know that one.
- JdeBP 4mo ago
  Ports is on 1.4.7. * https://freshports.org/sysutils/btop/ https://freshports.org/sysutils/btop/ The fix is still pending being even made to the origin version. * https://github.com/aristocratos/btop/pull/1728 https://github.com/aristocratos/btop/pull/1728
  - wolvoleo 4mo ago
    Thanks! I will keep an eye on that.
haunter 4mo ago
As for the “unauthorized” sign: Book publishers used to print India/SEA-only editions of books only sold in those countries, significantly cheaper than in the US or Europe. Then a Thai guy realized that this would be a good business opportunity: buy cheaper books and then import them to the US. Wiley brought him to court, went to the SCOTUS, Wiley lost the case. So they ended up printing cheaper edition books. https://en.wikipedia.org/wiki/Kirtsaeng_v._John_Wiley_%26_Sons,_Inc https://en.wikipedia.org/wiki/Kirtsaeng_v._John_Wiley_%26_So.... Mind you "price discrimination" like this still exist in the digital world where locality is easier to be enforced. For example Steam has extensive regional pricing across countries so the same game can be significantly cheaper in Russia, India, Brazil etc. compared to US or EU An example: https://steamdb.info/app/413150/ https://steamdb.info/app/413150/ https://partner.steamgames.com/pricing/explorer https://partner.steamgames.com/pricing/explorer https://partner.steamgames.com/doc/store/pricing/currencies https://partner.steamgames.com/doc/store/pricing/currencies
- Joker_vD 4mo ago
  When Kirtsaeng moved to the US in 1997 to pursue an undergraduate degree in mathematics at Cornell University,[4] he discovered that textbooks (not just those published by Wiley, but of other publishers too) were considerably more expensive to buy in the United States than in his home country. Kirtsaeng asked his relatives from Thailand to buy such books at home and ship them to him to sell at a profit. He sold the imported books on eBay, making $1.2 million in revenue, although the parties disputed the net profit amount. A text book case (ha!) about one of the mechanisms that enable the free markets and trade to bring the prices of goods down.
- stavros 4mo ago
  Wow, how did we go from "it's impossible to enforce region locking online because you can transmit information instantly across the world" to "locality more easily enforced in the digital world"?
  - exe34 4mo ago
    Most people haven't started using vpns yet.

Interesting post, it made me wonder. At one time FreeBSD swap usage/logic was far better than what Linux did. Is that still the case ?

shevy-java 4mo ago
I remember how NetBSD promoted itself as running on many more toasters than Linux once. Then some NetBSD dev wrote on their mailing list that this is no longer true. Linux runs on more toasters now. (And also top 500 supercomputers, but toasters are the real metal to the petal test.) These fights always remind me of: https://www.jwz.org/doc/worse-is-better.html https://www.jwz.org/doc/worse-is-better.html It's an interesting piece of history too. I kind of evaluate it a bit differently, e. g. my summary is "momentum beats academic perfection". Which is not completely what it is about, but it is my own imperfect TL;DR summary.
- bee_rider 4mo ago
  This basically fits my stereotype of BSD being a little bit more hardcore while Linux is a little more accessible… when the question was “can you install an OS on a toaster,” BSD had an advantage. Now that normal engineers have to make IOT toasters (for some reason) Linux should have the advantage, right?
  - sublinear 4mo ago
    Normal engineers don't do that either.
- Levitating 4mo ago
  why did that url point me to a scrotum in an egg cup
  - UnlockedSecrets 4mo ago
    Seems like at a glance that website if it sees a referral from ycombinator, it redirects to that image.... In a private window it loads the 'intended' page.....
  - m0llusk 4mo ago
    HN has disappointed JWZ
    - saghm 4mo ago
      Just blackholing the request would be kinder than taking it out on the person following the link, given that their gripe seems to be more with the one who posted it (and that as a deterrent, it doesn't seem to be particularly effective given that it's been years since the first time I've seen this conversation play out in comments)
- Someone 4mo ago
  > but toasters are the real metal to the petal test. Or is that a new tongue in cheek idiom? AFAIK, it’s peDal, not peTal, and the other way around: pedal to the metal. The literal meaning is to push your accelerator pedal down to the floor of your car, which is made of metal.
  - goodmythical 4mo ago
    "metal to the petal" gives me more of a "gilding the lily" flavor, especially in cnotrast to the original construction. Still not sure if it was on purpose, but I read it as a reference to a "but can it run DOOM?" benchmark.
0x457 4mo ago
Yes, It's just not every tool is aware of ZFS ARC. Which is what this post is about. Author just describes in an odd way.
- mike_hock 4mo ago
  It's a bit odd that you have to query a filesystem-specific value to get simple memory usage stats.
  - uniqueuid 4mo ago
    Which is exactly (part of) why people call ZFS a "rampant layering violation"!
  - 0x457 4mo ago
    Despite having File System, ZFS is a lot more than just file system. Sooner you understand and make peace with it, easier it is to use it.
man8alexd 4mo ago
FreeBSD didn’t have memory overcommit and instead used strict swap reservation - each allocated anonymous memory page was supposed to have a corresponding swap page. This required 2x RAM swap space, otherwise you would get “out of swap” when forking a large process. FreeBSD implemented memory overcommit around 2000.
- jandrese 4mo ago
  Oh, so that's where that old nugget of wisdom came from! I've heard the rule about making your swap at least 2x your RAM for ages and thought it was just some old rule of thumb from the 80s. I didn't know there used to be a legitimate reason for it.
  - man8alexd 4mo ago
    See this retrocomputing.SE question https://retrocomputing.stackexchange.com/questions/32492/origin-of-the-rule-that-swap-size-should-be-2x-of-the-physical-memory https://retrocomputing.stackexchange.com/questions/32492/ori...

naturalmovement 4mo ago

[flagged]

toast0 4mo ago
Sure, but also some tools needed fixing.
deleted 4mo ago
[deleted]
craftkiller 4mo ago
I've noticed that the various tools report different amounts of free memory. I appreciate finally having an explanation for why.

m463 4mo ago

the end struck me - a picture of an os book. I wonder if students these days retain their books after college, or do they get returned as a rental?

linguae 4mo ago
I'm a professor at a community college in Silicon Valley, and my students use online textbooks. I try to use Creative Commons or other libre textbooks, but sometimes I use paid textbooks when they are heads-and-shoulders better than their libre alternatives. Some e-textbooks can be accessed on a subscription basis. I admit I prefer non-subscription materials, but a colleague advised me that often the book that students learn from is different from a good reference book that students can use once they've already learned the material. For example, my colleagues and I have had great success with an online, interactive textbook for discrete math. While the subscription is unfortunately only valid for the duration of the course, once students have learned discrete math, they could buy a used copy of Rosen's discrete math textbook as a reference. The nice thing about e-textbooks is not needing to carry around a bunch of heavy books. I remember the tomes I had in my college days, such as Stewart's Calculus.
- NooneAtAll3 4mo ago
  just hint students towards anna's archive and then sky's the limit
- saghm 4mo ago
  > The nice thing about e-textbooks is not needing to carry around a bunch of heavy books. I remember the tomes I had in my college days, such as Stewart's Calculus. I'm not sure how relatable this is for people outside of my age group (late millennial), but growing up we used to get chided about stuffing too much in our backpacks and shown presentations about how we were going to give ourselves scoliosis, but then had like two minutes between classes that might be on the other side of the building (meaning no time to stop at our lockers other than at lunch) and we'd get chided even worse if we didn't have our textbooks available for the days we happened to need them in class (which of course we were never told in advance, and some classes had multiple textbooks that wouldn't all get used every time). I wish that we had iPads or Chromebooks or whatever back when I was in school not even because I would have wanted to have been able to surf the web or play games or whatever, but just to have a solution to having to pick between having a sore back or getting worse scolding (with a side course of hypocrisy either way).

duendefm 4mo ago

Thank you for such a quality post.

hnloser 4mo ago

[flagged]

tomeow 4mo ago

[flagged]

shevy-java 4mo ago

[flagged]

edoceo 4mo ago
The OS Crusades are over man
- irishcoffee 4mo ago
  For real though, what is the point of running a bsd on a personal computer? Seems like intentionally handicapping yourself.
  - soupbowl 4mo ago
    Consistent ZFS tooling around the OS and ZFS support out of the box, which always works.
    - irishcoffee 4mo ago
      So more so because of the storage drivers? I don’t have the specific concern, maybe that’s why I don’t understand the draw.
  - BSDobelix 4mo ago
    I have FreeBSD on all my personal machines and i feel the opposite of handicapped, however i use Linux (CachyOS) for gaming (as a VM with pass-thru).
vermaden 4mo ago
No ZFS no problem :)
acheron 4mo ago
Netcraft confirms it.
- BSDobelix 4mo ago

tom2ow 4mo ago

[flagged]

tom2ow 4mo ago

[flagged]

tiffanyh 4mo ago

If you like this kind of post, you might like this “htop explained” post. https://peteris.rocks/blog/htop/ https://peteris.rocks/blog/htop/

NooneAtAll3 4mo ago
htop explained* I was looking forward for web protocol, but alas...
- tiffanyh 4mo ago
  It was a typo. Thanks, fixed. But there is an http headers article you might like. https://peteris.rocks/blog/exotic-http-headers/ https://peteris.rocks/blog/exotic-http-headers/
  - teddyh 4mo ago
    I”d instead suggest RFC 9110, HTTP Semantics: <https://www.rfc-editor.org/info/rfc9110/ https://www.rfc-editor.org/info/rfc9110/>

efxhoy 4mo ago

Great job on getting the fixes merged!

drdexebtjl 4mo ago

I don’t understand the part about using heuristics and deciding what counts as used memory… Used memory for the system is always total minus available. Heuristics? I would hope that the system knows precisely what is using every single byte of physical and virtual memory. Is this a reporting problem? Why do we have to settle for heuristics and not the exact number?

CrociDB 4mo ago
The thing is it's easy to define free, unused memory. But a lot of the used memory is your system caching stuff that would be free if you needed more than what's actually free. So you can see you have 1g of free memory out of your 4g, but then you allocate 3g and it will do without a sweat and you'd be confused. So you have to go and dig for what those caches are and report that they're effectively free too.
- drdexebtjl 4mo ago
  Instantly reclaimable disk caches should count as available, and they do. This isn’t hard. The OS should just expose a counter for available memory instead of having applications understand every type of memory reservation. edit: Linux does this, but it has its own share of issues with memory counters. The “cached” memory includes tmpfs and ramfs for seemingly no reason.
  - jkrejcha 4mo ago
    > The “cached” memory includes tmpfs and ramfs for seemingly no reason. If you're curious why that is by the way, it's because that's actually how these are implemented (tmpfs/ramfs is just a mount to a filesystem where the files never get marked clean[1]) [1]: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt https://www.kernel.org/doc/Documentation/filesystems/ramfs-r...
    - drdexebtjl 4mo ago
      That’s clever. Makes for terrible UX, though. AFAIK the only way for you to figure out how much of your disks is actually cached involves enumerating all tmpfs and ramfs mounts, summing their sizes, and subtracting the sum from the cache size reported by the kernel.

wolvoleo 4mo ago

I wonder if these btop fixes got into the standard ports collection? Or even upstream? I like the command for viewing the ARC cache size, never knew that. It's only 2GB on my system (of 64GB RAM).

BSDobelix 4mo ago
>I like the command for viewing the ARC cache size zfs-stats is also nice for more zfs internals like hit-rate L2 etc.. https://www.freshports.org/sysutils/zfs-stats https://www.freshports.org/sysutils/zfs-stats
- wolvoleo 4mo ago
  Thanks! I didn't know that one.
JdeBP 4mo ago
Ports is on 1.4.7. * https://freshports.org/sysutils/btop/ https://freshports.org/sysutils/btop/ The fix is still pending being even made to the origin version. * https://github.com/aristocratos/btop/pull/1728 https://github.com/aristocratos/btop/pull/1728
- wolvoleo 4mo ago
  Thanks! I will keep an eye on that.

haunter 4mo ago

As for the “unauthorized” sign: Book publishers used to print India/SEA-only editions of books only sold in those countries, significantly cheaper than in the US or Europe. Then a Thai guy realized that this would be a good business opportunity: buy cheaper books and then import them to the US. Wiley brought him to court, went to the SCOTUS, Wiley lost the case. So they ended up printing cheaper edition books. https://en.wikipedia.org/wiki/Kirtsaeng_v._John_Wiley_%26_Sons,_Inc https://en.wikipedia.org/wiki/Kirtsaeng_v._John_Wiley_%26_So.... Mind you "price discrimination" like this still exist in the digital world where locality is easier to be enforced. For example Steam has extensive regional pricing across countries so the same game can be significantly cheaper in Russia, India, Brazil etc. compared to US or EU An example: https://steamdb.info/app/413150/ https://steamdb.info/app/413150/ https://partner.steamgames.com/pricing/explorer https://partner.steamgames.com/pricing/explorer https://partner.steamgames.com/doc/store/pricing/currencies https://partner.steamgames.com/doc/store/pricing/currencies

Joker_vD 4mo ago
When Kirtsaeng moved to the US in 1997 to pursue an undergraduate degree in mathematics at Cornell University,[4] he discovered that textbooks (not just those published by Wiley, but of other publishers too) were considerably more expensive to buy in the United States than in his home country. Kirtsaeng asked his relatives from Thailand to buy such books at home and ship them to him to sell at a profit. He sold the imported books on eBay, making $1.2 million in revenue, although the parties disputed the net profit amount. A text book case (ha!) about one of the mechanisms that enable the free markets and trade to bring the prices of goods down.
stavros 4mo ago
Wow, how did we go from "it's impossible to enforce region locking online because you can transmit information instantly across the world" to "locality more easily enforced in the digital world"?
- exe34 4mo ago
  Most people haven't started using vpns yet.