11 ms·
So We've Got a Memory Leak
- DonHopkins 3y ago"I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say 'Yeah it works but you're leaking memory everywhere. Perhaps we should fix that.' I’ll just restart Apache every 10 requests." -Rasmus Lerdorf, PHP Non-Designer https://en.wikiquote.org/wiki/Rasmus_Lerdorf https://en.wikiquote.org/wiki/Rasmus_Lerdorf
- fragmede 3y agoThat explains PHP
- DonHopkins 3y agohttps://news.ycombinator.com/item?id=40256878 https://news.ycombinator.com/item?id=40256878 >It's not "supposed" to be that way. >It just happened to end up that way because Rasmus Lerdorf just doesn't give a shit. ¯\_(ツ)_/¯ [...]
- im3w1l 3y agoThere is a scene in the pirates of the carribean I think of a lot. "You are without a doubt the worst pirate I have ever heard of" "Ah, but you have heard of me." He kept the scope down. He shipped. It was hugely successful. In the end it was overtaken and rightly so, but that doesn't invalidate the success it had.
- DonHopkins 3y agoBut it doesn't justify the arrogance. "For all the folks getting excited about my quotes. Here is another - Yes, I am a terrible coder, but I am probably still better than you :)" -Rasmus Lerdorf OR the continued negligence. https://news.ycombinator.com/item?id=40256878 https://news.ycombinator.com/item?id=40256878 And who remembers how careless, reckless, and blithe he was with the PHP 5.3.7 release he didn't bother to test because running tests was too much of a hassle because there were already so many test failures that wading through them all to see if there were any new ones was just too much to ask of him, the leader of the widely used project, in charge of cutting releases? >5.3.7 upgrade warning: [22-Aug-2011] Due to unfortunate issues with 5.3.7 (see bug#55439) users should postpone upgrading until 5.3.8 is released (expected in a few days). No seriously, he's literally as careless as he claims to be (when he says that repeatedly, you should believe him!), and his lack of giving a shit about things like tests and encryption and security that are extremely important has caused actual serious security problems, like breaking crypt() by checking in sloppy buggy code that would have caused a unit test to fail, but without bothering to run the unit tests (because so many of them failed anyway, so who cares??), and then MAKING A RELEASE of PHP 5.3.7 with, OF ALL THINGS, a broken untested crypt()! http://i.imgur.com/cAvSr.jpg http://i.imgur.com/cAvSr.jpg Do you think that's just his sense of humor, a self deprecating joke, breaking then releasing crypt() without testing, that's funny in some context? What context would that be? Do you just laugh and shrug it off with "Let Rasmus be Rasmus!" https://www.reddit.com/r/programming/comments/jsudd/you_see_rasmus_lerdorf_creator_of_php_wrecking/ https://www.reddit.com/r/programming/comments/jsudd/you_see_... >r314434 (rasmus): Make static analyzers happy >r315218 (stas): Unbreak crypt() (fix bug #55439) # If you want to remove static analyser messages, be my guest, but please run unit tests after http://svn.php.net/viewvc/php/php-src/trunk/ext/standard/php_crypt_r.c?r1=314438&r2=314437&pathrev=314438 http://svn.php.net/viewvc/php/php-src/trunk/ext/standard/php... https://plus.google.com/113641248237520845183/posts/g68d9RvRA1i https://plus.google.com/113641248237520845183/posts/g68d9RvR... [broken link] >Rasmus Lerdorf >+Lorenz H.-S. We do. See http://gcov.php.net http://gcov.php.net >You can see the code coverage, test case failures, Valgrind reports and more for each branch. >The crypt change did trigger a test to fail, we just went a bit too fast with the release and didn't notice the failure. This is mostly because we have too many test failures which is primarily caused by us adding tests for bug reports before actually fixing the bug. I still like the practice of adding test cases for bugs and then working towards making the tests pass, however for some of these non-critical bugs that are taking a while to change we should probably switch them to XFAIL (expected fail) so they don't clutter up the test failure output and thus making it harder to spot new failures like this crypt one. And don't even get me started about mysql_real_escape_string! It has the word "real" in it. I mean, come on, who would ever name a function "real", and why? That implies the existence of a not-so-real mysql escape string function. Why didn't they simply FIX the gaping security hole in the not-so-real mysql escape string function, instead of maintaining one that was real that you should use, and one that was not so real that you should definitely not use, in the name of backwards compatibility? Or were there actually people out there using the non-real mysql escape string function, and they didn't want to ruffle their feathers by forcing those people with code that had a security hole so big you could fly a space shuttle through to fix their gaping security holes? The name of the function "mysql_real_escape_string" says all you need to know about the culture and carelessness and lack of security consciousness of the PHP community. Melania Trump's "I REALLY DON'T CARE DO U?" nihilistic fashion statement sums up Rasmus Lerdorf's and the PHP community's attitude towards security, software quality, programming, standards, computer science, and unit tests. ¯\_(ツ)_/¯ https://www.youtube.com/watch?v=l5imY2oQauE https://www.youtube.com/watch?v=l5imY2oQauE
- lelanthran 3y agoYour examples of arrogance has nuance: > "For all the folks getting excited about my quotes. Here is another - Yes, I am a terrible coder, but I am probably still better than you :)" -Rasmus Lerdorf Well, he shipped, met his user's needs, met the markets needs and generally hit all the necessary bullet points to make a successful and lasting impact on the world. If he didn't meet some requirement that you have, like memory safety, its because it wasn't necessary. Checking off the required stuff and leaving the optional stuff for later is the sign of a good coder. > his lack of giving a shit about things like tests and encryption and security that are extremely important Woah there, cowboy! It turned out his take was correct, because it continued dominating over all those other technologies which cared about the thing you cared about. Being correct is better than being elegant, clean, or bug-free. > This is mostly because we have too many test failures which is primarily caused by us adding tests for bug reports before actually fixing the bug. I still like the practice of adding test cases for bugs and then working towards making the tests pass, however for some of these non-critical bugs that are taking a while to change That's just a different way of saying "We didn't have enough resources to make the fixes go quicker". What's the alternative here? Don't log the bug? Don't make a test to repro the bug? > Why didn't they simply FIX the gaping security hole in the not-so-real mysql escape string function, You sound like you've never been in a professional development shop at all. The reason that things hang around seemingly forever is because someone is using it!. It's the amateur mickey-mouse outfits that remove stuff which users are still using. It really is the equivalent of "Don't break userland". No professional worth their salt breaks their existing users without a very good reason. This is why Microsoft is still shipping broken win32 functions that were written in 1998. It's why Linus insists "Don't break userland". If you want to level up to professional level when shipping software, you're going to be shipping a lot of mistakes that you already know about. On the whole, your comment makes Rasmus look like more of a professional than you.
- DonHopkins 3y agoYour argument that it's correct to not give a shit about tests and security and encryption and memory leaks simply because lots of people are using the project is upside-down, extremely unprofessional, and downright dangerous. He shipped, but he didn't meet his users or the Internet community's needs, because his users and the Internet at large need safe reliable systems that somebody's actually bothered running the existing unit tests on, instead of security theater, Dunning-Kruger evangelism, and knee-jerk apologetics like your defeatist and fatalistic acceptance and justification of the status quo. Trotting out Win32 to justify PHP's flaws is pretty unhinged. I'm unsure you're not just a parody account. A serious person would realize they've run out of valid arguments and re-examine their priors before making such an embarrassingly bottom-of-the-barrel justification. Thomas Midgley Jr. also shipped and made a lasting impact on the world. Changing the world is not the only measure of success, nor justification of harmful impact. https://en.wikipedia.org/wiki/Thomas_Midgley_Jr https://en.wikipedia.org/wiki/Thomas_Midgley_Jr. >His legacy is one of inventing the two chemicals that did the greatest environmental damage. Environmental historian J. R. McNeill stated that he "had more adverse impact on the atmosphere than any other single organism in Earth's history." Author Bill Bryson remarked that he possessed "an instinct for the regrettable that was almost uncanny." Science writer Fred Pearce described him as a "one-man environmental disaster". PHP is the CFC of programming languages.
- pavlov 3y agoNever calling free() is a valid memory management strategy if you know your process lifetime exactly.
- helpfulContrib 3y ago[dead]
- Hamuko 3y agoOldie but a goodie: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98125 https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...
- alternatex 3y agoI think the complaint is that there was no strategy. The dude just wanted to build some websites.
- rwmj 3y agoI tried to do this in a real program once. The program only ever runs for a fraction of a second and then exits, so it seemed like a good candidate. Unfortunately what happened after is that our company started running tools like Coverity which complained about leaked memory. The path of least resistance was to fix that by adding free()'s everywhere.
- matja 3y agoAnd then you accidently add a use-after-free bug by trying to satisfy a "correctness" tool...
- SubjectToChange 3y agoIt’s hard to fault the static analysis tooling here. Not freeing memory is, almost always, unintended behavior. Besides, if a business is going to blindly apply static analysis to their projects and then demand that every warning be “fixed”, then the tool really isn’t the problem. Still, any serious developer should, at the very least, have the patience to interrogate their code.
- chubs 3y agoWhen I was a rails developer, the ‘done thing’ was to simply throw hardware at issues like these as an acceptable tradeoff for productivity. If you cared about this sort of thing, you’d use something more formal. I find it personally difficult to calm my pearl-clutching perfectionist tendencies to embrace that approach but I can’t deny it does work :)
- jsheard 3y agoLifehack: instead of admitting you are rebooting the server every 10 minutes to clear the memory leaks, call it a "phased arena allocation strategy" and then it's fine.
- projektfu 3y agoOddly, Apache uses a pool allocator[1], so it is using essentially the same strategy as Rasmus already. 1. or it did 20 years ago, i might be out of date :)
- gembeMx 3y ago[dead]
- invisitor 3y agoI didn't read it all but I noticed I enjoyed the way you write. I don't know if it's the emojis or the overall formatting you use.
- stevenharman 3y agoThank you! It is a long read, to be sure.
- smallstepforman 3y agoI just dont understand the fear with manual memory management. With RAII and and simple diligance (clear ownership rules), managing memory is an easy engineering task. I actually find it *more* challenging to deal with frameworks that insist or reference counting and shared pointers since ownership is now obscure. I create it, I free it. I transfer, I no longer care. Its part of engineering discipline. Memory bugs are no worse than logic bugs, we fix the logic bugs, makes sense to fix the memory bugs. Disclaimer: I do embedded complex systems that run 24/7. We do the same for OS resources (handles, sockets, etc) and dont use automatic resource managers, we do it manually. So why complicate the design with automatic memory management?
- yosefk 3y ago35% of vulnerabilities in the biggest tech companies being due to use after free bugs are a part of the answer. (More than 90% of severe vulnerabilities are due to memory bugs impossible in memory-safe languages.)
- samatman 3y agoEvery memory-safe language has a runtime written in a memory-managed language. Yes, even Rust: Rust is implemented using many (well-vetted) unsafe blocks. So projects to improve the quality, and lower the defect rate, in memory-managed programming, are far from wasted. Even if they only get used to write fast garbage collectors so that line coders can get on with the work of delivering value to customers.
- loeg 3y agoManual memory management vs GC is orthogonal to memory safe vs unsafe.
- AgentME 3y agoIn practice the lines of each issue are placed very close together. Other than Rust, there are no popular memory safe languages that use manual memory management.
- SomeoneFromCA 3y ago"Much has been written about various tools for profiling a leak, understanding heap dumps, common causes of leaks". Eww.. leaks and heap dumps. Someone needs a healthier diet.
- gavinhoward 3y agoValgrind makes finding leaks so easy in C. Fixing them is harder, but it's usually easy if your design is right. I usually allocate and free in the same function unless the function is meant to allocate for the caller. (And then that call is considered allocation in the caller.)
- canucker2016 3y agoIt's the "repro the bug" that's hard... When I worked on static analysis of codebases, the error handling codepath was the most likely source of problems.
- gavinhoward 3y agoYes. To do that, I fuzz like crazy, and every path becomes a test, even if it means a lot of manual work checking each one. That alone exercises tons of error paths. To cover 99% of the rest, I use SQLite's error injection [1] on all tests. Just doing error injection on allocation gets you 90% of the way there. [1]: https://sqlite.org/testing.html#anomaly_testing https://sqlite.org/testing.html#anomaly_testing
- lelanthran 3y ago> I usually allocate and free in the same function unless the function is meant to allocate for the caller. (And then that call is considered allocation in the caller.) Yeah. I do a similar thing in C, but think of it as 'differing level of scopes in the abstraction'[1]. It's pretty easy to visually spot when a scope acquires a resource in $SCOPE::foo() and never releases that resource in the $SCOPE::cleanup(). [1] Like the way code has block scope, function scope, file scope and global scope, a problem domain or an abstraction over the solution (the model) also has varying levels of scope. I've never found this to be a taught thing, however. Modeling the problem domain and any proposed solution is a useful skill before jumping into coding a solution.
- 1024core 3y agoReminds me of this story I heard about Yahoo. Their ads server had a memory leak and it would OOM after something like 10000 requests. Their solution: restart the server after 8000 requests. This worked for a year or two. And then it started OOM-ing after 8000 requests. Next solution: restart the server after 6000 requests.
- xandrius 3y agoIf that gives you extra time to move the problem to the future, I'd say that's a win :D
- otabdeveloper4 3y ago> Their solution: restart the server after 8000 requests. 8000 requests is something like 500 milliseconds for the average ad server. You need exceptionally fast restarts for that to work.
- wging 3y agoPerhaps now, but the story is about Yahoo, which means it could be from the early 2000s or late 90s. Traffic volumes were probably lower, computers were definitely slower, internet advertising was not as big as it is now, etc.
- toast0 3y agoThis is in the context of (y)Apache. You set MaxRequestsPerChild, so when the request limit is hit, the child is killed, a new one started, and requests can be served by other children until the replacement is ready. In a pure config, idle children block on accept, so the kernel does all the coordination work required to get requests to children, the child exits after hitting the cap, and the parent catches the exit and spawns another. As long as all the children don't hit their cap at once, there's no gap in service. If they do all hit the cap at once, it's still not so bad. I don't know about ads, but on Yahoo Travel, I had no qualms about solving memory leaks that took months to show up with MaxRequestsPerChild 100000. I gave valgrind or something a whirl, but didn't find it right away and was tired of getting paged once a quarter for it, so... I did do some scaleout work for Shopping once, and found theirs set at 3. I don't remember where I left it, but much higher. Nobody knew why it was 3, so I took the chance that it would become aparrent if I set it to 100, and nothing obvious was wrong and the servers were much happier. fork() is fast, but it's not fast enough to do it every 3 requests.
- seanhunter 3y agoSo one place I worked might win some sort of prize for the dumbest way to burn $5m with a memory leak. So back in the 90s the printer driver in Solaris had a memory leak[1]. At the time I was a contractor for a big bank that is sadly/not sadly no longer with us any more. This was when the status of faxes in confirming contracts hadn't been sufficiently tested in court so banks used to book trades by fax and the system that sent the fax would also send a document to a particular printer which would print the trade confirm out and there was some poor sap whos job consisted of picking up the confirms from this printer, calling the counterparty and reading them out so that they were all on tape[2] and legally confirmed. Anyhow one day the memory leak caused the printer driver to fall over and fail to print out one of the confirms so the person didn't read it out on the telephone. The market moved substantially and the counterparty DKd the trade[3]. A lot of huffing and puffing by senior executives at the bank had no effect and they booked a $5m loss and made a policy to never trade with that particular bank again[4]. The fax printer job was moved to windows NT. [1] According to the excellent "Expert C programming" this was eventually fixed because Scott McNealy, then CEO of Sun Microsystems had been given a very underpowered workstation (as CEO) so was affected a lot by this problem and eventually made enough of a fuss the devs got around to fixing it https://progforperf.github.io/Expert_C_Programming.pdf https://progforperf.github.io/Expert_C_Programming.pdf [2] Calls in the securities division of banks are pretty much always recorded for legal and compliance reasons [3] DK stands for "Don't know". If the other side says they "don't know" a trade they are disputing the fact that a contract was made. [4] Which I'm sure hurt us more than it hurt them as they could just trade somewhere else and pay some other bank their commission
- fbdab103 3y ago>...they booked a $5m loss and made a policy to never trade with that particular bank again Maybe I am too cynical, but would many businesses retroactively agree to a deal which would cost them a ton of money? If the Process requires Is dotted, Ts crossed, and a phone call confirmation which was never placed -why eat the loss when the other party should own the error? Citi just had a lawsuit because of paying back a loan too quickly. I expect everyone in finance to play hard ball on written agreements when it works in their favor.
- narag 3y agoI've used languages with and without garbage collection. Usually, manual is more difficult to write, automatic is more difficult to troubleshoot. I would love to use a language that allows me to do both. Writing exploratory code is more convenient using automatic memory management. There are certain kinds of code that benefit from manual memory management. I find appalling that people can't find a middle ground between forbidden and mandatory.
- amedvednikov 3y agoV uses a GC by default, but it's easily disabled per function/module via the @[manualfree] attribute or for the entire project via `v -gc none` https://vlang.io https://vlang.io
- Negitivefrags 3y agoThat langauge is C++. I almost never do manual memory management in that language, but you can do it if you want.
- deleted 3y ago[deleted]