8 ms·

"70% of all Microsoft patches are for memory-related bugs". This says a lot. I'm sure Microsoft has very smart and competent engineers, yet they still get all

"70% of all Microsoft patches are for memory-related bugs". This says a lot. I'm sure Microsoft has very smart and competent engineers, yet they still get all these bugs. It's because manual memory management is really, really hard. I'm baffled to keep hearing some people saying that manual memory management is okay when the evidence is this huge. Perhaps for very simple programs is okay to do manual memory management, but for complex ones like most of them nowadays, it seems close impossible for humans to write safe programs.

gwbas1c 7y ago
Keep in mind that, in kernel mode, things like garbage collection just won't work!
- merb 7y ago
  maybe it would? maybe somebody could create a co-processor just for that. sadly cpu's won't actually have specific extensions to have a good way of doing garbadge collection. but I guess it would be pretty helpful.
  - 0815test 7y ago
    So, LISP machines?
    - merb 7y ago
      maybe, without the brackets, of course.
- sametmax 7y ago
  Good thing rust doesn't have one.
- __s 7y ago
  Feel like that's a common misconception: https://en.wikipedia.org/wiki/Midori_(operating_system) https://en.wikipedia.org/wiki/Midori_(operating_system) http://programatica.cs.pdx.edu/House http://programatica.cs.pdx.edu/House
- hi41 7y ago
  Could you please expand on this. When garbage collection works in applications and webservers that handle large number of requests, why wouldn't it work in OS kernel?
  - arcticbull 7y ago
    It introduces indeterminism into a system that strives to have as little as possible. You could of course, but you probably shouldn’t.
  - inlined 7y ago
    GC can create noticeable pauses in the whole windows UX because windows scales vertically in a machine not horizontally in a DC. And per another comment I made, if your OOM handler needs to JIT it can be catastrophic.
- pjmlp 7y ago
  Xerox PARC, ETHZ, DEC and MSR had other opinion regarding that.
- Matthias247 7y ago
  I would argue that it actually might not matter too much: 1. In a constraint context (embedded/kernel) you want to preallocate most things anyway, since even normal malloc/free isn't deterministic. Once you preallocate objects, it doesn't matter if those are coming from a long-lived GCed heap or from somewhere else. 2. Kernels like Linux rely on the fact that some small allocations never fail. Plugging in a GC and asserting that it won't fail would have the same effect (with some impact on latencies). What might be most important is that the language supports explicit stack allocations, which avoids the need for any kind of heap when necessary. That is given for languages like C# and Go.
avinium 7y ago
> I'm baffled to keep hearing some people saying that manual memory management is okay when the evidence is this huge. It's a lot easier for someone to scoff at memory bugs on an internet forum than it is to write memory-safe code.
- raverbashing 7y ago
  Correct answer People need to "humble down" and see that if people keep falling into the same traps even with training, it's not as simple as "people sucks"
  - johncolanduoni 7y ago
    What makes you think they don’t include themselves in the term “humans”?
  - ziftface 7y ago
    This seems self contradictory. If people keep falling into the same traps even with training, how is "this is a human limitation" not a reasonable conclusion?
    - raverbashing 7y ago
      I don't see what are you reading there, but "this is a human limitation" is the correct conclusion Hence why we need languages that are better at memory management than C. It's ergonomics. In the same way you can't put a racing gearbox/clutch in a street car and expect everybody to deal with it. They won't.
  - CountSessine 7y ago
    People need to "humble down" and see that if people keep falling into the same traps even with training, it's not as simple as "people sucks" The parent isn't saying "people suck" or even that "people who write memory bugs suck". They're saying that 'programmer machismo' sucks. Saying, "we don't need better tools, we just need better people, and I'm one of those better people," sucks. Closely related is saying, "manual memory management is hard, so I guess most programmers should use better tools. But I'm a really advanced 10x 'leet programmer, (in spite of how much of my code has wound up in CVE's), and I'm clearly capable of writing code using manual memory management because of my 'leet skills." It's one of the peculiarities of the field that we're completely ok with practitioners using dangerous tools and building techniques when better tools and techniques are available.
_bxg1 7y ago
"Good programmers don't need help managing memory" is the "global warming isn't real" of the programming world
- bdamm 7y ago
  This is also a “10x and I’m sorry” issue. The unicorns write a prototype which is then handed to lesser mortals for maintenance. Will the mortals do as well as the initial author that had complete control? No. I was just thinking it’s basically denialism at this point yet I still run into engineers that think memory management is no biggie. Everyone has a blind spot, and this is one often found in the most talented.
Scuds 7y ago
You don't want to have to rely on people being competent all the time. You want people to 'fall into the pit of success' instead of opting into several safety mechanisms that'll get you something that's 'pretty good' Besides, if a language guarantees that all pointers are not null, initialized, not dangling, etc you have a solid foundation in which to build large systems. Also - why are people even mentioning C#? Rust is a language for writing SQL servers, drivers, browsers, and other foundational components where a garbage collector wouldn't make sense. Just about all of my stuff deals with HTTP, which is farther up the stack from Rust's sweet spot. Yes, you can ducttape rust style memory safety features into C# or C++, but I'd imagine using it is kind of like introducing asynchrony into a C# codebase - it wants to affect everything and if you don't you'll always be contending with the sync/async border. Better to make it a requirement and reap the rewards from knock-on effects.
- arcticbull 7y ago
  There’s nothing intrinsic to rust that makes it better suited for “foundational components” over anywhere else in the stack. Just because it’s capable of being used lower down doesn’t mean it’s a better or worse language higher up the stack. It’s probably got all the APIs or great crates you need for your work. It’s not a language “for” anything per se, it’s ergonomic anywhere up the stack.
  - mitchty 7y ago
    > There’s nothing intrinsic to rust that makes it better suited for “foundational components” over anywhere else in the stack. Actually, there is, not having a GC means you can better reason about stack/heap memory usage. That matters for performance critical and things like embedded systems where you might not even use malloc() in c.
    - arcticbull 7y ago
      I think I may not have phrased it well, I agree with you, it’s better than many other languages for low level software development. What I’m saying is there’s nothing that makes it worse for high-level software development.
inlined 7y ago
I don’t understand how SAL and RAII couldn’t help with this. In three years I never had memory issues reported against my C++ code. The closest thing that happened to me was corruption while prototyping a Zip utility that had new features I was inventing. There I had to build a variable sized input bitstream writer and had corruption on some ISAs due to differences in the interpretation of <<.
- nemaar 7y ago
  if you don't have memory issues in your code then either it's so small that everyone participating in its development can actually keep it in a perfect state or no one searched hard enough yet. After a certain complexity bugs just appear, you don't have to do anything, they are just there.
  - inlined 7y ago
    I’m not saying I never wrote bugs, but tools like SAL or RAII types that manage sharing help. By the time I checked in, they were either squashed or not found until after I left Microsoft. I built a domain agnostic virtualization and streaming platform that generalized the “Click2Run” features of Office and could apply retroactively to any traditional app that didn’t include kernel code (e.g. worked on Adobe Creative Suite) I also worked on the early phases of the sandbox that runs WinRT for JS, which also required paying my dues and fixing bugs or implementing new features (e.g. the fetch API) for the Trident engine in IE. I wouldn’t consider these trivial projects.
- yawaramin 7y ago
  Because modern C++ won't save us: https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/ https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/
Vinnl 7y ago
> This says a lot. I'm sure Microsoft has very smart and competent engineers, yet they still get all these bugs. But let's say they were very smart and did not get all those bugs. Even then, why would someone claim the compiler double-checking is not useful? I have a pretty good grasp of Javascript and can pretty much always predict what type something is going to be. Of course I make mistakes, but even if I didn't: using TypeScript is still great, because I don't have to manually keep track of all those types. Sure, I could do it, but it's more useful to spend my cognitive effort and energy on other things.
la_barba 7y ago
Its hard, but you're trading memory safety for performance. Rust cheerleaders will tell you that its possible to avoid that trade-off, but the proof is in the pudding. Its become almost impossible to see a new OS project succeed in the current landscape, so maybe an entire AAA game written in Rust will be an interesting challenge. Also, being memory safe doesn't get you much (or rather very very little IMHO), because complex code still contains logic bugs, UI bugs, performance bugs, etc, etc. Personally, I want the program to crash rather than keep running only to find out it was silently messing up data or otherwise leaving things in an inconsistent state.
- cwyers 7y ago
  The showcase project for Rust is a web browser (after all, Rust started out at Mozilla), which is certainly one of the larger undertakings one can start nowadays.
  - la_barba 7y ago
    Well, I personally wouldn't put any web-browser in the performance applications category. They are important applications, no doubt, and increasing their memory safety is always a plus. But there is something "soft realtime"/definitive about fitting the rendering of millions of textured-polygons, and running AI, Physics, data streaming, audio, gameplay code in a total budget of 16ms per-frame for hours and hours.
  - hu3 7y ago
    How is Firefox a showcase of Rusts performance when Chrome beats it in benchmarks? https://www.phoronix.com/scan.php?page=news_item&px=Firefox-66-Chrome-73-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=Firefox-... Not to mention Firefox's poor state in macOS.
    - wtetzner 7y ago
      What do you mean by it’s poor state in macOs? I use it as my main browser on macOs both at work and at home, and much prefer it over Chrome.

gwbas1c 7y ago

Keep in mind that, in kernel mode, things like garbage collection just won't work!

merb 7y ago
maybe it would? maybe somebody could create a co-processor just for that. sadly cpu's won't actually have specific extensions to have a good way of doing garbadge collection. but I guess it would be pretty helpful.
- 0815test 7y ago
  So, LISP machines?
  - merb 7y ago
    maybe, without the brackets, of course.
sametmax 7y ago
Good thing rust doesn't have one.
__s 7y ago
Feel like that's a common misconception: https://en.wikipedia.org/wiki/Midori_(operating_system) https://en.wikipedia.org/wiki/Midori_(operating_system) http://programatica.cs.pdx.edu/House http://programatica.cs.pdx.edu/House
hi41 7y ago
Could you please expand on this. When garbage collection works in applications and webservers that handle large number of requests, why wouldn't it work in OS kernel?
- arcticbull 7y ago
  It introduces indeterminism into a system that strives to have as little as possible. You could of course, but you probably shouldn’t.
- inlined 7y ago
  GC can create noticeable pauses in the whole windows UX because windows scales vertically in a machine not horizontally in a DC. And per another comment I made, if your OOM handler needs to JIT it can be catastrophic.
pjmlp 7y ago
Xerox PARC, ETHZ, DEC and MSR had other opinion regarding that.
Matthias247 7y ago
I would argue that it actually might not matter too much: 1. In a constraint context (embedded/kernel) you want to preallocate most things anyway, since even normal malloc/free isn't deterministic. Once you preallocate objects, it doesn't matter if those are coming from a long-lived GCed heap or from somewhere else. 2. Kernels like Linux rely on the fact that some small allocations never fail. Plugging in a GC and asserting that it won't fail would have the same effect (with some impact on latencies). What might be most important is that the language supports explicit stack allocations, which avoids the need for any kind of heap when necessary. That is given for languages like C# and Go.

avinium 7y ago

> I'm baffled to keep hearing some people saying that manual memory management is okay when the evidence is this huge. It's a lot easier for someone to scoff at memory bugs on an internet forum than it is to write memory-safe code.

raverbashing 7y ago
Correct answer People need to "humble down" and see that if people keep falling into the same traps even with training, it's not as simple as "people sucks"
- johncolanduoni 7y ago
  What makes you think they don’t include themselves in the term “humans”?
- ziftface 7y ago
  This seems self contradictory. If people keep falling into the same traps even with training, how is "this is a human limitation" not a reasonable conclusion?
  - raverbashing 7y ago
    I don't see what are you reading there, but "this is a human limitation" is the correct conclusion Hence why we need languages that are better at memory management than C. It's ergonomics. In the same way you can't put a racing gearbox/clutch in a street car and expect everybody to deal with it. They won't.
- CountSessine 7y ago
  People need to "humble down" and see that if people keep falling into the same traps even with training, it's not as simple as "people sucks" The parent isn't saying "people suck" or even that "people who write memory bugs suck". They're saying that 'programmer machismo' sucks. Saying, "we don't need better tools, we just need better people, and I'm one of those better people," sucks. Closely related is saying, "manual memory management is hard, so I guess most programmers should use better tools. But I'm a really advanced 10x 'leet programmer, (in spite of how much of my code has wound up in CVE's), and I'm clearly capable of writing code using manual memory management because of my 'leet skills." It's one of the peculiarities of the field that we're completely ok with practitioners using dangerous tools and building techniques when better tools and techniques are available.

_bxg1 7y ago

"Good programmers don't need help managing memory" is the "global warming isn't real" of the programming world

bdamm 7y ago
This is also a “10x and I’m sorry” issue. The unicorns write a prototype which is then handed to lesser mortals for maintenance. Will the mortals do as well as the initial author that had complete control? No. I was just thinking it’s basically denialism at this point yet I still run into engineers that think memory management is no biggie. Everyone has a blind spot, and this is one often found in the most talented.

Scuds 7y ago

You don't want to have to rely on people being competent all the time. You want people to 'fall into the pit of success' instead of opting into several safety mechanisms that'll get you something that's 'pretty good' Besides, if a language guarantees that all pointers are not null, initialized, not dangling, etc you have a solid foundation in which to build large systems. Also - why are people even mentioning C#? Rust is a language for writing SQL servers, drivers, browsers, and other foundational components where a garbage collector wouldn't make sense. Just about all of my stuff deals with HTTP, which is farther up the stack from Rust's sweet spot. Yes, you can ducttape rust style memory safety features into C# or C++, but I'd imagine using it is kind of like introducing asynchrony into a C# codebase - it wants to affect everything and if you don't you'll always be contending with the sync/async border. Better to make it a requirement and reap the rewards from knock-on effects.

arcticbull 7y ago
There’s nothing intrinsic to rust that makes it better suited for “foundational components” over anywhere else in the stack. Just because it’s capable of being used lower down doesn’t mean it’s a better or worse language higher up the stack. It’s probably got all the APIs or great crates you need for your work. It’s not a language “for” anything per se, it’s ergonomic anywhere up the stack.
- mitchty 7y ago
  > There’s nothing intrinsic to rust that makes it better suited for “foundational components” over anywhere else in the stack. Actually, there is, not having a GC means you can better reason about stack/heap memory usage. That matters for performance critical and things like embedded systems where you might not even use malloc() in c.
  - arcticbull 7y ago
    I think I may not have phrased it well, I agree with you, it’s better than many other languages for low level software development. What I’m saying is there’s nothing that makes it worse for high-level software development.

inlined 7y ago

I don’t understand how SAL and RAII couldn’t help with this. In three years I never had memory issues reported against my C++ code. The closest thing that happened to me was corruption while prototyping a Zip utility that had new features I was inventing. There I had to build a variable sized input bitstream writer and had corruption on some ISAs due to differences in the interpretation of <<.

nemaar 7y ago
if you don't have memory issues in your code then either it's so small that everyone participating in its development can actually keep it in a perfect state or no one searched hard enough yet. After a certain complexity bugs just appear, you don't have to do anything, they are just there.
- inlined 7y ago
  I’m not saying I never wrote bugs, but tools like SAL or RAII types that manage sharing help. By the time I checked in, they were either squashed or not found until after I left Microsoft. I built a domain agnostic virtualization and streaming platform that generalized the “Click2Run” features of Office and could apply retroactively to any traditional app that didn’t include kernel code (e.g. worked on Adobe Creative Suite) I also worked on the early phases of the sandbox that runs WinRT for JS, which also required paying my dues and fixing bugs or implementing new features (e.g. the fetch API) for the Trident engine in IE. I wouldn’t consider these trivial projects.
yawaramin 7y ago
Because modern C++ won't save us: https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/ https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/

Vinnl 7y ago

> This says a lot. I'm sure Microsoft has very smart and competent engineers, yet they still get all these bugs. But let's say they were very smart and did not get all those bugs. Even then, why would someone claim the compiler double-checking is not useful? I have a pretty good grasp of Javascript and can pretty much always predict what type something is going to be. Of course I make mistakes, but even if I didn't: using TypeScript is still great, because I don't have to manually keep track of all those types. Sure, I could do it, but it's more useful to spend my cognitive effort and energy on other things.

la_barba 7y ago

Its hard, but you're trading memory safety for performance. Rust cheerleaders will tell you that its possible to avoid that trade-off, but the proof is in the pudding. Its become almost impossible to see a new OS project succeed in the current landscape, so maybe an entire AAA game written in Rust will be an interesting challenge. Also, being memory safe doesn't get you much (or rather very very little IMHO), because complex code still contains logic bugs, UI bugs, performance bugs, etc, etc. Personally, I want the program to crash rather than keep running only to find out it was silently messing up data or otherwise leaving things in an inconsistent state.

cwyers 7y ago
The showcase project for Rust is a web browser (after all, Rust started out at Mozilla), which is certainly one of the larger undertakings one can start nowadays.
- la_barba 7y ago
  Well, I personally wouldn't put any web-browser in the performance applications category. They are important applications, no doubt, and increasing their memory safety is always a plus. But there is something "soft realtime"/definitive about fitting the rendering of millions of textured-polygons, and running AI, Physics, data streaming, audio, gameplay code in a total budget of 16ms per-frame for hours and hours.
- hu3 7y ago
  How is Firefox a showcase of Rusts performance when Chrome beats it in benchmarks? https://www.phoronix.com/scan.php?page=news_item&px=Firefox-66-Chrome-73-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=Firefox-... Not to mention Firefox's poor state in macOS.
  - wtetzner 7y ago
    What do you mean by it’s poor state in macOs? I use it as my main browser on macOs both at work and at home, and much prefer it over Chrome.