11 ms·

When did we stop caring about memory management?

91 points by matthewwarren 11y ago

eva1984 11y ago
Since it is a web server we are talking about here, let's assume it is the backend space :) I guess the answer is obvious: it is nice, but might not worth the effort. 1.2 million requests/s is pretty good, but in reality, you probably would still need 2-3 other hosts as a failsafe backups, let alone there are other factors, like memory constraint, in which case, less optimized solution might be just acceptable because it is sufficient. So under common usage cases, the need for scalability outweighs the urge to squash the maximum performance from a single machine.
- kasey_junk 11y ago
  Just to be clear kestrel benchmarks at 1.2 million requests per second. You'd need 10 of your servers to match 1 of theirs. That kind of scalability difference does still matter.
  - eva1984 11y ago
    Thanks for pointing that out. It did seem like that could make a difference by this metric alone in some scenario.
jasonjei 11y ago
I think this quote summarizes hackers today: ``[T]ime spent sweating memory is time you're not writing your app. The hard part is developing the experience is that you need to know when you need to care." Not having to worry about memory is now a luxury we can afford in the age of plenty. While writing an efficient spellchecker would have been a great feat in the 80s, there's no point of that when you can load the whole dictionary file in memory. In other words, memory management is a #highclassproblem for many developers--you worry about efficiency once you have a lot of adoption. (Within reason; if your app is unbearably slow that isn't good.)
- PavlovsCat 11y ago
  We also live in the age of plenty in regards to how many things we have running at the same time, or how many tabs there are open in a browser. Things can add up pretty quickly. And why settle for "without optimizing it's slightly faster than things used to be when people had to optimize", when you can have "optimized, it's orders of magnitude faster than things used to be"? > you worry about efficiency once you have a lot of adoption. Sometimes things can be tricky to refactor if you didn't move carefully at every step. Initially not worrying about efficiency at all might mean having to rewrite from scratch once you do. Of course, this can't be generalized. Sometimes it really, really doesn't matter, unless you care about craftsmanship as a value in and of itself -- taking 10-20% more time to write code that makes you happy and proud might very well make you more effective in the long run.
  - fixermark 11y ago
    If you can sell it as faster than the alternatives, it makes sense to optimize. But in the web-focused marketplace, adding 6 months to your launch calendar to get something fast to market may mean someone else's good-enough solution becomes the entrenched, de-facto standard that you are now fighting to prove you're superior enough to that it justifies people's time cost to move their data to your solution. That's the risk tradeoff.
    - PavlovsCat 11y ago
      It really depends, avoiding some pitfalls isn't always that much more work, but merely requires knowing (and caring) about them in the first place.
      - kylequest 11y ago
        It's easy to point out the places where you could've done a bit more optimization after the fact. And, yes, sometimes those optimizations don't cost an arm and a leg, but it's very easy to fall into the premature optimization mode that kills the product and the company. Not everybody has tons of funding in the bank. For some startups shipping a day later can put you in the death spiral trajectory. Timing is a huge factor in a startup success. It's easy to think why not just do it right in the first place. Aside from the timing you, as a startup, don't really know what "it" is or "it" is constantly moving. Committing to perfection at that stage can be a sure way to kill the company. Even if "it" is known and it's not changing you still need to keep in mind the Gall's Law.
PavlovsCat 11y ago
> How useful is it to know about C-style memory management when you're a front-end JavaScript Developer? Try making your own particle engine and a simple game running at 60fps. I guess "avoiding GC" is only a very small subset of memory management, but if you're a noob like me, it surely will open your eyes about some things you would never have guessed could cause problems.
- fixermark 11y ago
  JavaScript is a brutally poor target for that application space though. It'd be better to target Unity and their web plugin; that's designed to be a game engine.
  - PavlovsCat 11y ago
    That's why you learn so much about more efficient JavaScript when you do it regardless.
  - Tiktaalik 11y ago
    You have to concern yourself with memory management in Unity.
zoul 11y ago
I just don’t understand why there aren’t more languages using automatic reference counting as featured in modern Objective-C and Swift. ARC has very low overhead and predictable performance profile (no GC waking up to choke your app for hundreds of ms). I think it makes almost no sense to consider going back to manual memory management because of performance, not even in games. Unless you are writing for some quite limited embedded hardware.
- tamana 11y ago
  Is ARC actually safe from memory leaks? CMS GC doesn't choke your all for 100ms
  - whyever 11y ago
    I think you can leak memory by having cycles.
  - setpatchaddress 11y ago
    No, you can still have memory leaks with ARC. In practice, the GC Apple was using prior to ARC could collect cycles, but it consumed a nontrivial number of CPU cycles compared to ARC. Apple's tools people seem to be unconvinced that GC is efficient enough for production use on slow CPUs.
- pcwalton 11y ago
  Because atomic reference counting has much worse throughput than tracing GC. This has been known for a long time. It makes sense for Swift because of Swift's unique constraints—interoperability with an atomic-reference-counted world (Objective-C) and, less importantly, mobile UIs putting demands on latency. But most applications would be better off with a tracing GC.
  - AlisdairO 11y ago
    > Because atomic reference counting has much worse throughput than tracing GC. This has been known for a long time. I know this to be true for ubiquitously heap allocated systems, but I'm curious if this has been studied extensively for stack+heap reference counted systems, where only a relatively small number of items are heap allocated? Seems like reference counting would gain in this kind of environment, as the stack effectively gives you many of the advantages generational gcs enjoy from their nursery, and you don't have to worry about write boundaries, tracing and the like.
deleted 11y ago
[deleted]
Tiquor 11y ago
When we got the wonderful blend of plentiful ram, inexpensive CPU and storage along with enough variety in languages and separation of architecture concerns that not everything you built had to care about memory.
frik 11y ago
So many were wrong that by Moore's law we will have plentiful ram and inexpensive CPU to run bloated Java or C# programs. In 2016, desktop applications are still 99% C/C++ and fast mobile apps in C/C++/Object-C. Languages like Rust and leaner new languages with JIT that care more about resources like Go, Julia, Lua, Swift, PHP7, Hack or even newer iterations like C++14 are certainly good way to go.
- pcwalton 11y ago
  1. I don't see any reason why Lua (presumably you mean LuaJIT?) and PHP 7 would be faster than Java or C#. The compilation techniques are the same, except that Java and C# have more mature JITs. 2. Java and C# JITs perform many more optimizations than Go 6g/8g do. 3. Objective-C message passing is a lot slower than method calls in Java and C#.
  - chc4 11y ago
    Not defending the grandparent's post, but LuaJIT is actually faster than Java or C#. Hell, it's faster than raw C in certain cases because it can optimize better due to being a simpler language. Mike Pall is a god amongst man. I think it's due to LuaJIT being one of the very few tracing JITs that are still in development, since they are notoriously difficult to write and maintain.
    - xxs 11y ago
      Tracing compiler might add a bit of performance in the naive cases where the developers don't know how the JIT optimizes. If I care about performance I'd check the generated assembly and modify accordingly till I get decent result. Baring that: do you have any reputable source claiming LuaJIT outperforms C#/java/C?
      - chc4 11y ago
        Unfortunately the Benchmarks Game seems to have removed LuaJIT in favor of Lua 5.3, but https://stackoverflow.com/questions/2588071/is-luajit-really-faster-than-every-other-jit-ed-dynamic-languages https://stackoverflow.com/questions/2588071/is-luajit-really... mentions it as being an order of magnitude faster. I've seen it said multiple times that LuaJIT is the gold standard of JITing, but can't find very many concrete benchmarks anymore.
pcwalton 11y ago
This article has a hint of "C-style memory management is good for you"—implying that you might want to write in something like JS and then go back and write in C++ in order to "tighten the screws". I think that's wrong—by and large, it's been a very good thing that people have stopped caring about memory management in production. That's because correct C-style memory management never† scales up to large software projects in a way that doesn't result in crashes and, worse, vulnerabilities. So, yes, we've traded performance for automatic memory management. But we've gained reliability, safety, and security. For the vast majority of server-side apps, that's a worthwhile trade. † Unless you use the processes like NASA and the like use, which for most apps are infeasible from a cost point of view.
- ArkyBeagle 11y ago
  It scales, and you don't have to use NASA-like process. The article fairly clearly says that making this work is a learned skill. And it is. If course it's possible to write stuff using manual memory management in a manner such that it's nearly impossible to do well. Doctor, doctor, it hurts when I do that... I wrote a 'C' based thing around 2012 that: - is about 50,000 lines; 50ish by 1000ish lines. - uses no dynamic memory. - does very "dynamic memory" things. Each type/shape/class has its own statically allocated 'C' array of structs that is declared to worst-case, and has a "new". Behind the "new" is a lot of behavior, including pointing to other such things/objects. A set of "delete" operations would be also relatively easy. But because it's an array and it is in 'C', this is relatively little code. Yes, there is duplication involved but the object creation code is probably 20% of the total thing and each object creation is darn near provably correct. It's built on RAII principles ( perhaps foolishly, since there's no deletion ). Dependencies between objects are completely explicit in the code. Object creation must be in the correct order to manage dependencies The magic happens because scripts* are used to create all things, once at startup. it also helps because by its very nature, the system is fully serialized - after a creation script, the state of the tables of objects can be written to a text file. *scripts are, in this case nothing but sequences of verbs which send text files full of XML to a service at a TCP port. It could have been JSON or attribute-value pairs, but instead it's XML. I do take extra care to make a very comprehensive test suite or it. but since the allocation scheme is 1) cheap and 2) fully exposed as state in the system, there's nothing actually opaque about it. I don't have to do "trust falls" with a garbage collector or even the heap. You can't have a malloc() or new() failure - it'll violate a constraint and quit/exit(). You then make the table larger. From the frame of reference of the other objects, it's abstracted and opaque - after the initial setup. From the "scripting" interface, it's fully transparent.
rdtsc 11y ago
Needing to think memory management turned out be a bit un-expected you might say. There were a few things that went well and few that were a bit unexpected. Things that went well: 1) RAM got ridiculously cheap: 2x 8GB is now what $60-70? So it would seem, in that regard, yeah you shouldn't care about memory. 2) GC languages and runtimes. Believe or not there was heavy skepticism that GC runtime could be ever be a serious strategy in practice. It was an exotic technology that many thought would left for academia and fringe research area. But most popular frameworks and languages today are GC based. Some things were a bit un-expected: 1) Memory access speed is fast but it didn't keep up with CPU speed. A cache miss will costs hundreds of nanoseconds. CPU speeds may have been doubling every 2 years or so, but memory access speeds have only been increasing by 10-20% or so. In the 80s and 90s you just didn't care as much about cache hits and misses. But those are the forefront of performance optimization strategies. "Align your struct members","use structs of arrays instead of arrays of structs","think about cache lines and strides when you loop through your data arrays..." etc. And because CPU speeds go so much faster, memory access also stands out on performance charts. So GC languages and runtimes can still seems slow and sluggish for applications that don't pay attention to memory. 2) Applications got more concurrent and servers are expected to handle a larger number of clients / connections / requests. That means each one needs a context, so it needs memory. So memory is again at the forefront. Accessing it and its latency, GC pause times are important. 3) Mobile happened. Mobile devices have proliferated so the needs for fast, small footprint applications never went away. Even as there are serves witih terabytes of RAM around, there are even more little arm cores and then everyone talks about IoT. That means langauges/runtimes which compile ahead of time will never be 100% displaced. Heck even on Android, ART (AoT copiling) is replacing JIT.
- asuffield 11y ago
  There's a secondary dimension to this: You can still micro-optimise for memory and cpu performance. That's certainly a thing that is possible to do. Alternatively, you can buy more cpus and memory. What has changed in the past few years is that for many workloads, the cost of buying more cpus and memory is significantly lower than the cost of all the development work to optimise away the need. Hardware is cheap compared to developers, for many applications. There are still some applications where this is not the case, and in those cases people still care about memory management.
georgefrick 11y ago
As I sit here on Sunday afternoon, trying to get a Phaser game to run on an iPad 2; I read this and think "We haven't stopped caring, we care when it's a specified business requirement". This bites us in the ass, because I can't make Wistia (for example) stop eating all the memory on this thing with my app running next to it. So, personally, I stopped caring when it was not a priority and didn't, on average, affect user experience. But; it was a mistake and I am watching memory/etc more and more these days. This is for various reasons (old hardware, more than one app on a page, growing amount of bundled media content, etc).
- forrestthewoods 11y ago
  We stopped caring because it didn't matter. Except when it began to matter it was too late. Somethings you have to plan for from the start. You can't go back later and fix it after the fact.
  - kuschku 11y ago
    Even switching from "copy and allocate objects everywhere" to non-copying native IO can improve your memory efficiency and usability. I am working on an Android app, and rewrote the part that handles the connection via a binary protocol over a socket with a remote server. Rewriting that to take advantage of NIO already improved connection times (build up a connection, initialize, etc) from almost 2 minutes to only 8 or 9 seconds. RAM usage of the backend was reduced by 50% – and it is easier readable. More often than not it’s not that hard to add it, if one carefully thinks about the requirements and future extensions at every step.
    - forrestthewoods 11y ago
      > More often than not it’s not that hard to add it, if one carefully thinks about the requirements and future extensions at every step. Is that argument to think about memory from the start or to not think about it until absolutely necessary? :)
      - 11y ago
wonkaWonka 11y ago
First with Smalltalk, then with Java, and eventually Objective-C as well.
nroose 11y ago
It seems to me that we have been doing this more and more as memory has gotten bigger and bigger, we care more and more about performance, and more and more people are writing more and more code that will be used for shorter and shorter times under tighter and tighter deadlines.
redwood 11y ago
Premature optimization is. ..
- maximilianburke 11y ago
  a) The whole quote is "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%" b) The quote isn't talking about data structuring/memory layout, but rather micro-optimizations of instruction sequences (removing goto statements, in the case of the original article)
- ansgri 11y ago
  .. is 50% chance to die by thousand paper cuts, 25% simply wasted time, AND 25% chance to avoid death by thousand paper cuts. At least in image processing and robotics, from my experience. It's very annoying when you have optimized all obvious hotspots 10x only to get 3x speedup because the remaining 30% code consistently abuses « simple ways », e.g. return big objects by value, use std::vector instead of proper Eigen arrays, constantly serialize objects everywhere, etc.
allard 11y ago
If your closet or garage increased in volume at the rate of Moore's law, you'd pay less attention to how you put stuff away.
- colordrops 11y ago
  Especially if you had a robot that would organize and cleanup your garage for you.
  - zanny 11y ago
    yc2017 anyone?
    - bobbles 11y ago
      It'd be a cool idea to sell warehouse storage with drone delivery of individual items if needed, that way you dont need to care as much about how much you can fit into your apartment at any one time
- firethief 11y ago
  What if you could only carry things to/from that closet in a fixed-size box? Typical L1 cache size for the x86 family is only a few times larger than on the 486. Cache is getting faster, but transistor shrinkage should affect cache speed and instruction execution time in roughly equal proportion. Of course, the marginal value of spending development time improving performance can vary a great deal; but when performance matters, the key considerations are going to remain location, location, and location for the foreseeable future.
shmerl 11y ago
Some didn't stop caring about it. It's all about how much you care about performance, and whether you think no garbage collection is preferable (or not). > How low should we go? How useful is it to know about C-style memory management when you're a front-end JavaScript Developer? There are things in between as well (such as RAII). That's about using it. Usefulness of knowing it is another thing. Someone will have to implement things on the lower level either way. I.e. garbage collector itself is going to manage memory after all.
chewyfruitloop 11y ago
Enterprise applications moved to Java on purpose to lose the issues that memory management causes. In my last job I first came in contact with people who had never had to do any type of memory management, that was only 8 years ago (that guy probably has his PhD by now and will be Dr Coole...seriously). If you can stand the speed of GC languages then, there's a whole bunch of savings you get in development time and especially in support.
- xxs 11y ago
  As a matter of fact GC is godsend when writing concurrent code - there is no ABA issue.
72deluxe 11y ago
C-style memory management is important to know about but in C++ land, RAII and C++11 features are here to help. Move semantics, use of containers, references etc. all mean many of these bugs are a thing of the past. In addition, extra tools and profiling / static code analysis has helped me to write better code.
- 0xcde4c3db 11y ago
  The language has gotten to a pretty nice place, but the ecosystem is nowhere near caught up. A lot of APIs still haven't even incorporated the idiom of passing const references, almost exclusively taking pointers to non-const and defining their own idiosyncratic ownership schemes. Qt4 is a good example: RAII of widgets is hazardous because windows/containers will delete their "children" (i.e. things that are under them in the widget hierarchy, not members), even if they're on the stack. This means that simply declaring local variables in the wrong order can cause a double delete. I encountered this in tutorial code on the Qt site, which kind of put me off choosing Qt for new projects.
  - 72deluxe 11y ago
    This is very true. What did you use instead? wxWidgets is the same - when a wxWindow gets deleted, it will remove its children in the view hierarchy, which are likely not children of it as data members. I think a lot of this comes from the fact that underneath native pointers are being used (eg handles on Windows) and widgets/controls are not necessarily going to be data members of a parent. It also means you can reparent widgets/controls to other GUI items at runtime. I do this (for example) when switching notebook pages to rehouse a GUI element in order to save RAM; I just assign it new data to display and it means less RAM usage and creation time for each notebook page. For your own data controls, we can still ensure that they clean themselves up alright and safely, just use pointers for the GUI items. In some sense, the GUI toolkit takes care of deleting its own members, eg. wxWidgets will delete all of a parent wxWindow children at teardown, so it isn't a leak situation.
    - 0xcde4c3db 11y ago
lordnacho 11y ago
To answer the headline, we stopped caring when a bunch of off-the-shelf solutions became available. For a lot of problems, once you have this off-the-shelf solution, you're just not going to have a memory issue. It's like cooking without having to manage where the pots, pans, and plates are. Say you hire some guy to clean up for you, and the guy just implements some simple strategy. It's a lot more pleasant, and if you have a large enough kitchen, it won't make a difference to the quality (and timeliness) of the food. In some cases you may have to consider the guy running around, but most of the time you net win anyway. For things that are truly performance sensitive, you still need to think about cache issues and such, and you definitely still care about memory management.
- mmsmatt 11y ago
  Complete agreement with the last sentence. I can think of two projects in my past five years that really showed performance sensitivity. I knew that at design time so I chose to care about where data sat in memory and how it would behave in multilevel caches. All other projects, couldn't care less where the kitchen dude stored my pots, and he never lost one. None of those projects jumped from performance agnostic to performance sensitive overnight.
jordwalke 11y ago
This article conflates two separate issues: Automatic memory management, and Virtual Machine runtimes. People often make the mistake of thinking that some sort of garbage collection strategy (such as reference counting[1] collection (ARC) or tracing collectors) implies that you must be running in a VM (which typically employ some kind of JIT to achieve reasonable performance). This is not true - see Haskell, OCaml, SML, and Swift (for reference counting collection). In my experience, many of the issues with debugging performance issues are not related to memory management, but understanding JIT heuristics which are moving targets (especially in the dynamically typed languages). I think we need to collectively separate the idea of some kind of automated (or semi-automated) collection strategy, from a language level JITed Virtual Machine. You may have one without the other. [1]https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf
- aidenn0 11y ago
  I reread the article after seeing your post, and could not find where they made this conflation. I've not worked on performance critical code running under a JIT, but have worked on performance critical code running under a GC, and having an intimate knowledge of memory can be vital to understanding the performance of your application.
  - jordwalke 11y ago
    The article divides programming languages into two groups: Those that require you to manage memory manually and those that rely on language level managed memory. However, the examples of the managed memory languages also rely on language level Virtual Machines and/or JITs (which are critical to achieving excellent performance in those respective languages). Then, the article points out that the people using these "garbage collected languages" (which are also JITed (in a VM likely)) tend to have problems debugging performance issues. If he's right, then the examples aren't making the case very convincingly because the realm of languages with garbage collection isn't divided granularly enough in his analysis. The result is attributing the problem to a common language feature that they all share, without acknowledging that they also all share other very important features. I'd never claim that understanding memory usage/allocation/freeing isn't important. It's important to understand how memory is used whether that be understanding how your collector handles generations of memory in a GC'd language, or whether it be understanding how your programming patterns cause a waterfall of destructor chains in a manually managed language. Often, understanding these things is difficult. My claim was that, in my experience, the biggest difficulties in debugging performance were in understanding JIT heuristics in language level Virtual Machines. This is because (especially in dynamic languages) the heuristics aren't documented anywhere, they're subject to change, and are likely not consistent between particular runtimes you're hosted within. (Those optimizations you made just for V8, could even hurt perf if you run your library in JSC). There's also the issue with language level virtual machines in that they usually impose some system wide tax on many operations (a layer of indirection), so I could see the people mentioned in the article having a hard time squeezing out that last portion of performance due to that indirection - again, that might be the fault of there being a language level Virtual Machine - and not necessarily as much related to memory (it depends on your application, and amount you allocate, of course). In my experience, optimizing collection behavior has been relatively straightforward in comparison and the techniques you learn for one collector seem to translate to others. Perhaps my experience is biased towards JIT/VM/GC in dynamic languages where the JITs are so much more complex in comparison. I have seen core hot paths in critical client side JS libraries be slowed down 9x because of obscure JIT deopts caused by properties names beginning with a dollar sign. I'm not just talking about a 9x slowdown of field lookups - our entire end to end benchmarks tanked (since fixed). Perhaps the VM architects decided that something with a leading dollar sign would not likely benefit from being made into a hidden JIT class? I've never had anything so obscure happen with any kind of garbage collection, whether that be reference counted ARC, or a tracing collector. Edit: spelling
andmarios 11y ago
Memory management still matters but by how much differs based on your specifications. Are you programming for a microcontroller? For a low end VPS? For a low-volume high-turnover web app? Do you have a deadline? How many people should be able to come up to speed and contribute to your codebase? The author's example is interesting. He talks about a webserver that after optimizations can reach 1.2 million requests per second. In reality though, your gigabit ethernet interface can't handle that much anyway.
pslam 11y ago
I never stopped caring. It just turns out it never really mattered for a large fraction of cases. There's still plenty of cases where you absolutely do care: low level firmware, kernels, absolute performance, and high reliability. It's more a case of not worrying about it when there's much bigger risks in a project than having loose guarantees about allocation performance or fragmentation.
bsaul 11y ago
Developping on smartphone and tablet makes you still caring a lot about memory management if you don't want your app to be killed by the os... I guess the question was asked to server side web developpers...
deleted 11y ago
[deleted]
mgrennan 11y ago
Software growth is not keeping up with hardware growth. My parents told me to "Clear your plate." That was because they lived through the depression. I remember writing code in assembler and 64k and feel the say way now. Programs I wrote in that space now take 64m and I think "What a waist".
forbin_meet_hal 11y ago
Having worked in the memory chip business for 10 years: It's cheap!! Hell, for a while, companies were dumping memories below cost. I remember a Fry's newspaper ad that said something like: "$50 1GB DIMM, $0 after in-store discount. (Limit 2 per customer.)" Kind of similar to the old (and possibly apocryphal) Sun Microsystems legend: Frustrated that too many domestic programmers were spoiled with ample CPU speeds, they hired a bunch of Russian programmers who dealt with the export restrictions on computing gear and thus had learned to write tight, elegant code.

91 points by matthewwarren 11y ago

eva1984 11y ago

Since it is a web server we are talking about here, let's assume it is the backend space :) I guess the answer is obvious: it is nice, but might not worth the effort. 1.2 million requests/s is pretty good, but in reality, you probably would still need 2-3 other hosts as a failsafe backups, let alone there are other factors, like memory constraint, in which case, less optimized solution might be just acceptable because it is sufficient. So under common usage cases, the need for scalability outweighs the urge to squash the maximum performance from a single machine.

kasey_junk 11y ago
Just to be clear kestrel benchmarks at 1.2 million requests per second. You'd need 10 of your servers to match 1 of theirs. That kind of scalability difference does still matter.
- eva1984 11y ago
  Thanks for pointing that out. It did seem like that could make a difference by this metric alone in some scenario.

jasonjei 11y ago

I think this quote summarizes hackers today: ``[T]ime spent sweating memory is time you're not writing your app. The hard part is developing the experience is that you need to know when you need to care." Not having to worry about memory is now a luxury we can afford in the age of plenty. While writing an efficient spellchecker would have been a great feat in the 80s, there's no point of that when you can load the whole dictionary file in memory. In other words, memory management is a #highclassproblem for many developers--you worry about efficiency once you have a lot of adoption. (Within reason; if your app is unbearably slow that isn't good.)

PavlovsCat 11y ago
We also live in the age of plenty in regards to how many things we have running at the same time, or how many tabs there are open in a browser. Things can add up pretty quickly. And why settle for "without optimizing it's slightly faster than things used to be when people had to optimize", when you can have "optimized, it's orders of magnitude faster than things used to be"? > you worry about efficiency once you have a lot of adoption. Sometimes things can be tricky to refactor if you didn't move carefully at every step. Initially not worrying about efficiency at all might mean having to rewrite from scratch once you do. Of course, this can't be generalized. Sometimes it really, really doesn't matter, unless you care about craftsmanship as a value in and of itself -- taking 10-20% more time to write code that makes you happy and proud might very well make you more effective in the long run.
- fixermark 11y ago
  If you can sell it as faster than the alternatives, it makes sense to optimize. But in the web-focused marketplace, adding 6 months to your launch calendar to get something fast to market may mean someone else's good-enough solution becomes the entrenched, de-facto standard that you are now fighting to prove you're superior enough to that it justifies people's time cost to move their data to your solution. That's the risk tradeoff.
  - PavlovsCat 11y ago
    It really depends, avoiding some pitfalls isn't always that much more work, but merely requires knowing (and caring) about them in the first place.
    - kylequest 11y ago
      It's easy to point out the places where you could've done a bit more optimization after the fact. And, yes, sometimes those optimizations don't cost an arm and a leg, but it's very easy to fall into the premature optimization mode that kills the product and the company. Not everybody has tons of funding in the bank. For some startups shipping a day later can put you in the death spiral trajectory. Timing is a huge factor in a startup success. It's easy to think why not just do it right in the first place. Aside from the timing you, as a startup, don't really know what "it" is or "it" is constantly moving. Committing to perfection at that stage can be a sure way to kill the company. Even if "it" is known and it's not changing you still need to keep in mind the Gall's Law.

PavlovsCat 11y ago

> How useful is it to know about C-style memory management when you're a front-end JavaScript Developer? Try making your own particle engine and a simple game running at 60fps. I guess "avoiding GC" is only a very small subset of memory management, but if you're a noob like me, it surely will open your eyes about some things you would never have guessed could cause problems.

fixermark 11y ago
JavaScript is a brutally poor target for that application space though. It'd be better to target Unity and their web plugin; that's designed to be a game engine.
- PavlovsCat 11y ago
  That's why you learn so much about more efficient JavaScript when you do it regardless.
- Tiktaalik 11y ago
  You have to concern yourself with memory management in Unity.

zoul 11y ago

I just don’t understand why there aren’t more languages using automatic reference counting as featured in modern Objective-C and Swift. ARC has very low overhead and predictable performance profile (no GC waking up to choke your app for hundreds of ms). I think it makes almost no sense to consider going back to manual memory management because of performance, not even in games. Unless you are writing for some quite limited embedded hardware.

tamana 11y ago
Is ARC actually safe from memory leaks? CMS GC doesn't choke your all for 100ms
- whyever 11y ago
  I think you can leak memory by having cycles.
- setpatchaddress 11y ago
  No, you can still have memory leaks with ARC. In practice, the GC Apple was using prior to ARC could collect cycles, but it consumed a nontrivial number of CPU cycles compared to ARC. Apple's tools people seem to be unconvinced that GC is efficient enough for production use on slow CPUs.
pcwalton 11y ago
Because atomic reference counting has much worse throughput than tracing GC. This has been known for a long time. It makes sense for Swift because of Swift's unique constraints—interoperability with an atomic-reference-counted world (Objective-C) and, less importantly, mobile UIs putting demands on latency. But most applications would be better off with a tracing GC.
- AlisdairO 11y ago
  > Because atomic reference counting has much worse throughput than tracing GC. This has been known for a long time. I know this to be true for ubiquitously heap allocated systems, but I'm curious if this has been studied extensively for stack+heap reference counted systems, where only a relatively small number of items are heap allocated? Seems like reference counting would gain in this kind of environment, as the stack effectively gives you many of the advantages generational gcs enjoy from their nursery, and you don't have to worry about write boundaries, tracing and the like.

deleted 11y ago

[deleted]

Tiquor 11y ago

When we got the wonderful blend of plentiful ram, inexpensive CPU and storage along with enough variety in languages and separation of architecture concerns that not everything you built had to care about memory.

frik 11y ago

So many were wrong that by Moore's law we will have plentiful ram and inexpensive CPU to run bloated Java or C# programs. In 2016, desktop applications are still 99% C/C++ and fast mobile apps in C/C++/Object-C. Languages like Rust and leaner new languages with JIT that care more about resources like Go, Julia, Lua, Swift, PHP7, Hack or even newer iterations like C++14 are certainly good way to go.

pcwalton 11y ago
1. I don't see any reason why Lua (presumably you mean LuaJIT?) and PHP 7 would be faster than Java or C#. The compilation techniques are the same, except that Java and C# have more mature JITs. 2. Java and C# JITs perform many more optimizations than Go 6g/8g do. 3. Objective-C message passing is a lot slower than method calls in Java and C#.
- chc4 11y ago
  Not defending the grandparent's post, but LuaJIT is actually faster than Java or C#. Hell, it's faster than raw C in certain cases because it can optimize better due to being a simpler language. Mike Pall is a god amongst man. I think it's due to LuaJIT being one of the very few tracing JITs that are still in development, since they are notoriously difficult to write and maintain.
  - xxs 11y ago
    Tracing compiler might add a bit of performance in the naive cases where the developers don't know how the JIT optimizes. If I care about performance I'd check the generated assembly and modify accordingly till I get decent result. Baring that: do you have any reputable source claiming LuaJIT outperforms C#/java/C?
    - chc4 11y ago
      Unfortunately the Benchmarks Game seems to have removed LuaJIT in favor of Lua 5.3, but https://stackoverflow.com/questions/2588071/is-luajit-really-faster-than-every-other-jit-ed-dynamic-languages https://stackoverflow.com/questions/2588071/is-luajit-really... mentions it as being an order of magnitude faster. I've seen it said multiple times that LuaJIT is the gold standard of JITing, but can't find very many concrete benchmarks anymore.

pcwalton 11y ago

This article has a hint of "C-style memory management is good for you"—implying that you might want to write in something like JS and then go back and write in C++ in order to "tighten the screws". I think that's wrong—by and large, it's been a very good thing that people have stopped caring about memory management in production. That's because correct C-style memory management never† scales up to large software projects in a way that doesn't result in crashes and, worse, vulnerabilities. So, yes, we've traded performance for automatic memory management. But we've gained reliability, safety, and security. For the vast majority of server-side apps, that's a worthwhile trade. † Unless you use the processes like NASA and the like use, which for most apps are infeasible from a cost point of view.

ArkyBeagle 11y ago
It scales, and you don't have to use NASA-like process. The article fairly clearly says that making this work is a learned skill. And it is. If course it's possible to write stuff using manual memory management in a manner such that it's nearly impossible to do well. Doctor, doctor, it hurts when I do that... I wrote a 'C' based thing around 2012 that: - is about 50,000 lines; 50ish by 1000ish lines. - uses no dynamic memory. - does very "dynamic memory" things. Each type/shape/class has its own statically allocated 'C' array of structs that is declared to worst-case, and has a "new". Behind the "new" is a lot of behavior, including pointing to other such things/objects. A set of "delete" operations would be also relatively easy. But because it's an array and it is in 'C', this is relatively little code. Yes, there is duplication involved but the object creation code is probably 20% of the total thing and each object creation is darn near provably correct. It's built on RAII principles ( perhaps foolishly, since there's no deletion ). Dependencies between objects are completely explicit in the code. Object creation must be in the correct order to manage dependencies The magic happens because scripts* are used to create all things, once at startup. it also helps because by its very nature, the system is fully serialized - after a creation script, the state of the tables of objects can be written to a text file. *scripts are, in this case nothing but sequences of verbs which send text files full of XML to a service at a TCP port. It could have been JSON or attribute-value pairs, but instead it's XML. I do take extra care to make a very comprehensive test suite or it. but since the allocation scheme is 1) cheap and 2) fully exposed as state in the system, there's nothing actually opaque about it. I don't have to do "trust falls" with a garbage collector or even the heap. You can't have a malloc() or new() failure - it'll violate a constraint and quit/exit(). You then make the table larger. From the frame of reference of the other objects, it's abstracted and opaque - after the initial setup. From the "scripting" interface, it's fully transparent.

rdtsc 11y ago

Needing to think memory management turned out be a bit un-expected you might say. There were a few things that went well and few that were a bit unexpected. Things that went well: 1) RAM got ridiculously cheap: 2x 8GB is now what $60-70? So it would seem, in that regard, yeah you shouldn't care about memory. 2) GC languages and runtimes. Believe or not there was heavy skepticism that GC runtime could be ever be a serious strategy in practice. It was an exotic technology that many thought would left for academia and fringe research area. But most popular frameworks and languages today are GC based. Some things were a bit un-expected: 1) Memory access speed is fast but it didn't keep up with CPU speed. A cache miss will costs hundreds of nanoseconds. CPU speeds may have been doubling every 2 years or so, but memory access speeds have only been increasing by 10-20% or so. In the 80s and 90s you just didn't care as much about cache hits and misses. But those are the forefront of performance optimization strategies. "Align your struct members","use structs of arrays instead of arrays of structs","think about cache lines and strides when you loop through your data arrays..." etc. And because CPU speeds go so much faster, memory access also stands out on performance charts. So GC languages and runtimes can still seems slow and sluggish for applications that don't pay attention to memory. 2) Applications got more concurrent and servers are expected to handle a larger number of clients / connections / requests. That means each one needs a context, so it needs memory. So memory is again at the forefront. Accessing it and its latency, GC pause times are important. 3) Mobile happened. Mobile devices have proliferated so the needs for fast, small footprint applications never went away. Even as there are serves witih terabytes of RAM around, there are even more little arm cores and then everyone talks about IoT. That means langauges/runtimes which compile ahead of time will never be 100% displaced. Heck even on Android, ART (AoT copiling) is replacing JIT.

asuffield 11y ago
There's a secondary dimension to this: You can still micro-optimise for memory and cpu performance. That's certainly a thing that is possible to do. Alternatively, you can buy more cpus and memory. What has changed in the past few years is that for many workloads, the cost of buying more cpus and memory is significantly lower than the cost of all the development work to optimise away the need. Hardware is cheap compared to developers, for many applications. There are still some applications where this is not the case, and in those cases people still care about memory management.

georgefrick 11y ago

As I sit here on Sunday afternoon, trying to get a Phaser game to run on an iPad 2; I read this and think "We haven't stopped caring, we care when it's a specified business requirement". This bites us in the ass, because I can't make Wistia (for example) stop eating all the memory on this thing with my app running next to it. So, personally, I stopped caring when it was not a priority and didn't, on average, affect user experience. But; it was a mistake and I am watching memory/etc more and more these days. This is for various reasons (old hardware, more than one app on a page, growing amount of bundled media content, etc).

forrestthewoods 11y ago
We stopped caring because it didn't matter. Except when it began to matter it was too late. Somethings you have to plan for from the start. You can't go back later and fix it after the fact.
- kuschku 11y ago
  Even switching from "copy and allocate objects everywhere" to non-copying native IO can improve your memory efficiency and usability. I am working on an Android app, and rewrote the part that handles the connection via a binary protocol over a socket with a remote server. Rewriting that to take advantage of NIO already improved connection times (build up a connection, initialize, etc) from almost 2 minutes to only 8 or 9 seconds. RAM usage of the backend was reduced by 50% – and it is easier readable. More often than not it’s not that hard to add it, if one carefully thinks about the requirements and future extensions at every step.
  - forrestthewoods 11y ago
    > More often than not it’s not that hard to add it, if one carefully thinks about the requirements and future extensions at every step. Is that argument to think about memory from the start or to not think about it until absolutely necessary? :)
    - 11y ago

wonkaWonka 11y ago

First with Smalltalk, then with Java, and eventually Objective-C as well.

nroose 11y ago

It seems to me that we have been doing this more and more as memory has gotten bigger and bigger, we care more and more about performance, and more and more people are writing more and more code that will be used for shorter and shorter times under tighter and tighter deadlines.

redwood 11y ago

Premature optimization is. ..

maximilianburke 11y ago
a) The whole quote is "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%" b) The quote isn't talking about data structuring/memory layout, but rather micro-optimizations of instruction sequences (removing goto statements, in the case of the original article)
ansgri 11y ago
.. is 50% chance to die by thousand paper cuts, 25% simply wasted time, AND 25% chance to avoid death by thousand paper cuts. At least in image processing and robotics, from my experience. It's very annoying when you have optimized all obvious hotspots 10x only to get 3x speedup because the remaining 30% code consistently abuses « simple ways », e.g. return big objects by value, use std::vector instead of proper Eigen arrays, constantly serialize objects everywhere, etc.

allard 11y ago

If your closet or garage increased in volume at the rate of Moore's law, you'd pay less attention to how you put stuff away.

colordrops 11y ago
Especially if you had a robot that would organize and cleanup your garage for you.
- zanny 11y ago
  yc2017 anyone?
  - bobbles 11y ago
    It'd be a cool idea to sell warehouse storage with drone delivery of individual items if needed, that way you dont need to care as much about how much you can fit into your apartment at any one time
firethief 11y ago
What if you could only carry things to/from that closet in a fixed-size box? Typical L1 cache size for the x86 family is only a few times larger than on the 486. Cache is getting faster, but transistor shrinkage should affect cache speed and instruction execution time in roughly equal proportion. Of course, the marginal value of spending development time improving performance can vary a great deal; but when performance matters, the key considerations are going to remain location, location, and location for the foreseeable future.

shmerl 11y ago

Some didn't stop caring about it. It's all about how much you care about performance, and whether you think no garbage collection is preferable (or not). > How low should we go? How useful is it to know about C-style memory management when you're a front-end JavaScript Developer? There are things in between as well (such as RAII). That's about using it. Usefulness of knowing it is another thing. Someone will have to implement things on the lower level either way. I.e. garbage collector itself is going to manage memory after all.

chewyfruitloop 11y ago

Enterprise applications moved to Java on purpose to lose the issues that memory management causes. In my last job I first came in contact with people who had never had to do any type of memory management, that was only 8 years ago (that guy probably has his PhD by now and will be Dr Coole...seriously). If you can stand the speed of GC languages then, there's a whole bunch of savings you get in development time and especially in support.

xxs 11y ago
As a matter of fact GC is godsend when writing concurrent code - there is no ABA issue.

72deluxe 11y ago

C-style memory management is important to know about but in C++ land, RAII and C++11 features are here to help. Move semantics, use of containers, references etc. all mean many of these bugs are a thing of the past. In addition, extra tools and profiling / static code analysis has helped me to write better code.

0xcde4c3db 11y ago
The language has gotten to a pretty nice place, but the ecosystem is nowhere near caught up. A lot of APIs still haven't even incorporated the idiom of passing const references, almost exclusively taking pointers to non-const and defining their own idiosyncratic ownership schemes. Qt4 is a good example: RAII of widgets is hazardous because windows/containers will delete their "children" (i.e. things that are under them in the widget hierarchy, not members), even if they're on the stack. This means that simply declaring local variables in the wrong order can cause a double delete. I encountered this in tutorial code on the Qt site, which kind of put me off choosing Qt for new projects.
- 72deluxe 11y ago
  This is very true. What did you use instead? wxWidgets is the same - when a wxWindow gets deleted, it will remove its children in the view hierarchy, which are likely not children of it as data members. I think a lot of this comes from the fact that underneath native pointers are being used (eg handles on Windows) and widgets/controls are not necessarily going to be data members of a parent. It also means you can reparent widgets/controls to other GUI items at runtime. I do this (for example) when switching notebook pages to rehouse a GUI element in order to save RAM; I just assign it new data to display and it means less RAM usage and creation time for each notebook page. For your own data controls, we can still ensure that they clean themselves up alright and safely, just use pointers for the GUI items. In some sense, the GUI toolkit takes care of deleting its own members, eg. wxWidgets will delete all of a parent wxWindow children at teardown, so it isn't a leak situation.
  - 0xcde4c3db 11y ago

lordnacho 11y ago

To answer the headline, we stopped caring when a bunch of off-the-shelf solutions became available. For a lot of problems, once you have this off-the-shelf solution, you're just not going to have a memory issue. It's like cooking without having to manage where the pots, pans, and plates are. Say you hire some guy to clean up for you, and the guy just implements some simple strategy. It's a lot more pleasant, and if you have a large enough kitchen, it won't make a difference to the quality (and timeliness) of the food. In some cases you may have to consider the guy running around, but most of the time you net win anyway. For things that are truly performance sensitive, you still need to think about cache issues and such, and you definitely still care about memory management.

mmsmatt 11y ago
Complete agreement with the last sentence. I can think of two projects in my past five years that really showed performance sensitivity. I knew that at design time so I chose to care about where data sat in memory and how it would behave in multilevel caches. All other projects, couldn't care less where the kitchen dude stored my pots, and he never lost one. None of those projects jumped from performance agnostic to performance sensitive overnight.

jordwalke 11y ago

This article conflates two separate issues: Automatic memory management, and Virtual Machine runtimes. People often make the mistake of thinking that some sort of garbage collection strategy (such as reference counting[1] collection (ARC) or tracing collectors) implies that you must be running in a VM (which typically employ some kind of JIT to achieve reasonable performance). This is not true - see Haskell, OCaml, SML, and Swift (for reference counting collection). In my experience, many of the issues with debugging performance issues are not related to memory management, but understanding JIT heuristics which are moving targets (especially in the dynamically typed languages). I think we need to collectively separate the idea of some kind of automated (or semi-automated) collection strategy, from a language level JITed Virtual Machine. You may have one without the other. [1]https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf

aidenn0 11y ago
I reread the article after seeing your post, and could not find where they made this conflation. I've not worked on performance critical code running under a JIT, but have worked on performance critical code running under a GC, and having an intimate knowledge of memory can be vital to understanding the performance of your application.
- jordwalke 11y ago
  The article divides programming languages into two groups: Those that require you to manage memory manually and those that rely on language level managed memory. However, the examples of the managed memory languages also rely on language level Virtual Machines and/or JITs (which are critical to achieving excellent performance in those respective languages). Then, the article points out that the people using these "garbage collected languages" (which are also JITed (in a VM likely)) tend to have problems debugging performance issues. If he's right, then the examples aren't making the case very convincingly because the realm of languages with garbage collection isn't divided granularly enough in his analysis. The result is attributing the problem to a common language feature that they all share, without acknowledging that they also all share other very important features. I'd never claim that understanding memory usage/allocation/freeing isn't important. It's important to understand how memory is used whether that be understanding how your collector handles generations of memory in a GC'd language, or whether it be understanding how your programming patterns cause a waterfall of destructor chains in a manually managed language. Often, understanding these things is difficult. My claim was that, in my experience, the biggest difficulties in debugging performance were in understanding JIT heuristics in language level Virtual Machines. This is because (especially in dynamic languages) the heuristics aren't documented anywhere, they're subject to change, and are likely not consistent between particular runtimes you're hosted within. (Those optimizations you made just for V8, could even hurt perf if you run your library in JSC). There's also the issue with language level virtual machines in that they usually impose some system wide tax on many operations (a layer of indirection), so I could see the people mentioned in the article having a hard time squeezing out that last portion of performance due to that indirection - again, that might be the fault of there being a language level Virtual Machine - and not necessarily as much related to memory (it depends on your application, and amount you allocate, of course). In my experience, optimizing collection behavior has been relatively straightforward in comparison and the techniques you learn for one collector seem to translate to others. Perhaps my experience is biased towards JIT/VM/GC in dynamic languages where the JITs are so much more complex in comparison. I have seen core hot paths in critical client side JS libraries be slowed down 9x because of obscure JIT deopts caused by properties names beginning with a dollar sign. I'm not just talking about a 9x slowdown of field lookups - our entire end to end benchmarks tanked (since fixed). Perhaps the VM architects decided that something with a leading dollar sign would not likely benefit from being made into a hidden JIT class? I've never had anything so obscure happen with any kind of garbage collection, whether that be reference counted ARC, or a tracing collector. Edit: spelling

andmarios 11y ago

Memory management still matters but by how much differs based on your specifications. Are you programming for a microcontroller? For a low end VPS? For a low-volume high-turnover web app? Do you have a deadline? How many people should be able to come up to speed and contribute to your codebase? The author's example is interesting. He talks about a webserver that after optimizations can reach 1.2 million requests per second. In reality though, your gigabit ethernet interface can't handle that much anyway.

pslam 11y ago

I never stopped caring. It just turns out it never really mattered for a large fraction of cases. There's still plenty of cases where you absolutely do care: low level firmware, kernels, absolute performance, and high reliability. It's more a case of not worrying about it when there's much bigger risks in a project than having loose guarantees about allocation performance or fragmentation.

bsaul 11y ago

Developping on smartphone and tablet makes you still caring a lot about memory management if you don't want your app to be killed by the os... I guess the question was asked to server side web developpers...

deleted 11y ago

[deleted]

mgrennan 11y ago

Software growth is not keeping up with hardware growth. My parents told me to "Clear your plate." That was because they lived through the depression. I remember writing code in assembler and 64k and feel the say way now. Programs I wrote in that space now take 64m and I think "What a waist".

forbin_meet_hal 11y ago

Having worked in the memory chip business for 10 years: It's cheap!! Hell, for a while, companies were dumping memories below cost. I remember a Fry's newspaper ad that said something like: "$50 1GB DIMM, $0 after in-store discount. (Limit 2 per customer.)" Kind of similar to the old (and possibly apocryphal) Sun Microsystems legend: Frustrated that too many domestic programmers were spoiled with ample CPU speeds, they hired a bunch of Russian programmers who dealt with the export restrictions on computing gear and thus had learned to write tight, elegant code.