4 ms·
Nope, just stories from my day job. One time, to solve the problem of "we want floating point numbers that are close to be considered equal" on an object, the
by cogman10 4y ago
Nope, just stories from my day job.
One time, to solve the problem of "we want floating point numbers that are close to be considered equal" on an object, the programmer decided to take every field of that object (reflectively of course) turn that into a string, and then compare the two strings together.
Another time, in order to convert a LocalDate to our own internal date representation, a dev decided to toString the local date, split on the `-`s to extract the day month year. (Yes, localdate has day, month, year methods).
One time, to convert an int to a double a dev decided to turn that int into a string and then used "Double.parseDouble" to turn that int into a double (I wish I were joking).
All these and more were found through the joys of profiling :D
- Waterluvian 4y agoI’m in pain reading this. But I feel the need to confess my own sin: I once didn’t want to write or add a dependency for a Python deep equality checker so I just sorted, JSONified, and string compared the two. It worked fine for years. It wasn’t performant important code. But it still keeps me up at night.
- cogman10 4y agoIt's stuff like this that makes me mad when I see "Optimization is the root of all evil" quotes thrown at me. The code I'm most typically fixing for performance isn't some complex algorithm that needs inline assembly and AVX register weaving... No, the kind of code I most often fix for performance is code where someone does `Double.parseDouble(foo.toString())` and turn it into `double bar = (double)foo;`. Or code where someone does n^2 when they could have had n by using a dictionary/hashmap. Or code where someone uses a dictionary/hashmap instead of using a simple POJO. It drives me bonkers when someone apparently goes out of their way to write code in the slowest form possible.
- Waterluvian 4y agoThere’s no black and white. I’ve known really good optimizers who do harm to the company and their peers by obsessing over gains that simply don’t matter. And I’ve known people who make programs unnecessarily slow and painful because of sloppiness. That’s not to say you can’t endeavour to “do it right the first time” but very regularly that is not a realistic goal. Good engineering requires understanding the big picture, particularly what matters and doesn’t matter to a business (and how those priorities change over time).
- acdha 4y agoYeah, I generally favor the approach of first making the code easy to understand/validate, and easy to replace later if you find a performance problem. I’ve found it’s usually only a small percentage of the time that your first guess for what’s going to be a bottleneck is accurate.
- girvo 4y agoEngineering has pretty much always been the art and science of "good enough". Anyone can make a bridge with infinite resources, but an engineer is who can make a bridge just good enough to not fall down within budget and spec. The challenge with software is the underlying "physics" of our world continue to get more powerful year over year (unlike real physics), so "good enough" ends up being sloppier and sloppier as time goes on, and for a lot of applications it doesn't matter. Of course, until that day where it turns out it does matter. Good developers I think can anticipate some of those cases, and great developers can pick the few that they can sink the time into getting Right (tm) (c) 2022, to get the most bang for their (time) bucks.
- horsawlarway 4y agoEh - It's good to know when you're taking the slow route, but most times the slow route is fine. It's easy to point to the title piece and assume the right advice to is try to optimize early - but the reality is that the gains mentioned here basically only matter because these libraries are used by millions of developers, and run often. essentially - don't try to optimize until you have a reason to make that effort worth it. For these libraries - it's worth it. For the times you're getting asked to fix performance, it's because performance has become a feature that matters enough to have someone ask you to fix it. It's fine to wait until that time to swing back around and pay attention to it. There are thousands of times where that never happens, because the performance was fine taking the scenic route through a string. Plus - that leaves you plenty of low hanging fruit to pick up some easy wins. --- I think the real story here is not that folks are taking the slow route, I think the story is that the low hanging fruit hasn't been knocked out of these libraries. And really, that goes to show that a lot of software really is supported by a very small number of library authors, who aren't doing it as their day job, for very little in the way of real compensation. Ex: SVGO is used by 9.1 million people (on github alone) but only has 163 contributors, and only 4 people have more than 10 commits. That means only one in 55,000 users has bothered to add anything. And there's only one real dev for every 2.25 million users. And "users" here also includes entire companies.
- TheCoelacanth 4y agoIt's fine to not optimize from the start, but the examples they gave are of cases where the code is doing something more complex that's also slower. Don't make your solution more complex than it needs to be. Start with the simplest solution and you will often get a faster solution as a side effect.
- horsawlarway 4y agoI disagree with this take. In both examples (the original article, as well as the one posted right above) the code under review was slow but simple. Taking just the two from the comment right above this: Double.parseDouble(foo.toString()) This will work for basically any possible value/type of foo, assuming foo supports .toString(). It will work for NaN, it will work for infinity, it will work if the value is already a string. It's explicitly avoiding the complexity of having to consider those cases at programming time, in exchange for taking the slow route (code someone else has written to already do that for you). Simply casting to double will probably work, but not always - worse, the warnings you get, and the edge cases you might need to deal with will vary significantly across languages. Ex: Java will do different things than C++, which will do different things than C#, Python will vary by flavor (CPython and Jython use different underlying types). Basically - it's saving the programmer a boat-load of complexity (reduces the edge cases down to just the exceptions parseDouble() might throw), in exchange for doing something slow. --- Take the hashmap example right after the double example. Will a hashmap let you speed up some algorithms by avoiding n2 in favor of n? Sure - at the expense of memory size and longer initialization. In most places that's probably a sane call. Is it always the right call? Nope. Would not be ideal for my arduino - the memory is so limited n2 is unlikely to matter most times, and again - the memory is so limited I'd rather go slow and finish than try to go fast and segfault. It's also not the right call when the expected length of the data is always going to be small values of n. If n is always going to be less than 5, I'd bet you good money the hash implementation will be slower in most cases. ---- Basically - optimize when optimizing becomes important, because something is slow and it sucks. Until then... slow is smooth and smooth is fast.
- egberts1 4y ago> It drives me bonkers when someone apparently goes out of their way to write code in the slowest form possible. We used to call that BASIC.
- tevon 4y agowait wait wait, the actual quote is "PREMATURE optimization is the root of all evil". Completely different meaning
- youngtaff 4y agoWe also shouldn't forget the full quote “There is no doubt that the holy grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.” — From “Structured Programming with go to Statements” (Knuth, 1974),
- wiredfool 4y agoI just yesterday found a case in my codebase of linear searching dict.keys() for the dictionary key.
- akdor1154 4y agoI had a candidate try that in an interview with me yesterday.
- deleted 4y ago[deleted]
- lucumo 4y ago> I once didn’t want to write or add a dependency for a Python deep equality checker so I just sorted, JSONified, and string compared the two. It worked fine for years. It wasn’t performant important code. I think this one is actually fine. You thought about the solution, considered your options and made a design decision based on the balance of your priorities. That's what engineering is: a bunch of trade-offs solidified into a product. The situation where someone uses Double.parseDouble(number.toString()) to translate an int into a double is different. There's really no reason to do it like that: it's harder to understand, uses more memory and cpu and is slower. There's no trade-off that makes that line come out as the best decision.
- specialist 4y agoYou did the right thing. Deep clone and equals on deeply nested, probably heterogenous, object graphs can be challenging. And certainly error prone.
- memco 4y agoThat’s a tough one because Python lacks a standard convention for deep comparisons so your only options are some bespoke function you (probably copy from some random website) stick in every library you use or pull in some third party dependency or use a standard module with a fairly straightforward serialization method that does what you need which I would guess is either pickle or json. Is there a better option?
- more-coffee 4y agoDepending on how much that equality check is deciding the application's performance, jsonifying to compare stuff is fairly ok. Just beware of any lists which you'll need to sort manually (on both sides) to prevent false negatives :)
- acdha 4y agoI’m having flashbacks to some contract Java developers who were calling toString() on deserialized JSON objects and then parsing the output (old school index/substring, not even regex) because they insisted it was too hard to use the built-in accessors. Bonus points for then trying more kludges when they got a reminder that JSON does not guarantee object key order so they couldn’t write code assuming values would be adjacent. They also did that exact same thing parsing dates, except that they flubbed the logic and IIRC had an off-by-one on the month for the constructed Date object.
- Waterluvian 4y agoLots of JSON libraries can be told to enforce key order. And you’ve gotta remember that if you’re going to do something so ghoulish!
- acdha 4y agoThese were definitely not the kind of developer who read the documentation looking for options like that. I've mostly avoided Java for years and still found myself going “this is like 20 lines of code [badly] re-implementing a built-in library function” on almost every code review.
- v-erne 4y agoCould this just be incompenence? I have seen this kind of code many years ago when external company was brought to sell us their implementation of CMS on some (even then) outdated Java/js-on-server-before-node-era platform. I was asked to do code review to chcek if their code can be integrated into our codebase. I was quite unexperienced at that point in time but enough so to notice that there was something off with the code. There was a lot js that looked almost like exercise in obfuscation. Things like doing join, append and split on array to do push - and it was not abstracted into function - this three operations were repeated all over the code. And then I met the programmer - it was ovious after few questions that he did not know what he was doing and his direct manager, who was also present, know nothing about programming (he was more od a business person as he himself proclaimed). It was as if this company hired the cheapes guy they could find, that on paper looked competent, give him task of implementing CMS, and only looked at results (the CMS actually worked), but no one ever checked the code that was hiding under (and the guy never worked with anyone else and never read any other code besides his!)