5 ms·
Heh, so many of these fixes boil down to "I'm converting this thing to a string and then searching the string". And, funnily, I've often stumbled on this probl
by cogman10 4y ago
Heh, so many of these fixes boil down to "I'm converting this thing to a string and then searching the string". And, funnily, I've often stumbled on this problem with Java programmers. They'll take a nice rich type, turn it into a string and then spend a bunch of code/time turning that string back into something else.
Super wasteful when you start thinking about what all that means. Rather then having a representation that neatly fits into registers, now you have this blob of memory that has to be linearly searched to pull information from it. Just. yuck.
And why do devs do this? TBH, I really don't know. Because regexes are easier to understand? IDK.
- Waterluvian 4y agoI’ve never seen this approach. I imagine I’m just not working in a domain that touches on problems “solved” this way. Any popular examples to point to?
- cogman10 4y agoNope, just stories from my day job. One time, to solve the problem of "we want floating point numbers that are close to be considered equal" on an object, the programmer decided to take every field of that object (reflectively of course) turn that into a string, and then compare the two strings together. Another time, in order to convert a LocalDate to our own internal date representation, a dev decided to toString the local date, split on the `-`s to extract the day month year. (Yes, localdate has day, month, year methods). One time, to convert an int to a double a dev decided to turn that int into a string and then used "Double.parseDouble" to turn that int into a double (I wish I were joking). All these and more were found through the joys of profiling :D
- Waterluvian 4y agoI’m in pain reading this. But I feel the need to confess my own sin: I once didn’t want to write or add a dependency for a Python deep equality checker so I just sorted, JSONified, and string compared the two. It worked fine for years. It wasn’t performant important code. But it still keeps me up at night.
- cogman10 4y agoIt's stuff like this that makes me mad when I see "Optimization is the root of all evil" quotes thrown at me. The code I'm most typically fixing for performance isn't some complex algorithm that needs inline assembly and AVX register weaving... No, the kind of code I most often fix for performance is code where someone does `Double.parseDouble(foo.toString())` and turn it into `double bar = (double)foo;`. Or code where someone does n^2 when they could have had n by using a dictionary/hashmap. Or code where someone uses a dictionary/hashmap instead of using a simple POJO. It drives me bonkers when someone apparently goes out of their way to write code in the slowest form possible.
- Waterluvian 4y agoThere’s no black and white. I’ve known really good optimizers who do harm to the company and their peers by obsessing over gains that simply don’t matter. And I’ve known people who make programs unnecessarily slow and painful because of sloppiness. That’s not to say you can’t endeavour to “do it right the first time” but very regularly that is not a realistic goal. Good engineering requires understanding the big picture, particularly what matters and doesn’t matter to a business (and how those priorities change over time).
- acdha 4y agoYeah, I generally favor the approach of first making the code easy to understand/validate, and easy to replace later if you find a performance problem. I’ve found it’s usually only a small percentage of the time that your first guess for what’s going to be a bottleneck is accurate.
- girvo 4y agoEngineering has pretty much always been the art and science of "good enough". Anyone can make a bridge with infinite resources, but an engineer is who can make a bridge just good enough to not fall down within budget and spec. The challenge with software is the underlying "physics" of our world continue to get more powerful year over year (unlike real physics), so "good enough" ends up being sloppier and sloppier as time goes on, and for a lot of applications it doesn't matter. Of course, until that day where it turns out it does matter. Good developers I think can anticipate some of those cases, and great developers can pick the few that they can sink the time into getting Right (tm) (c) 2022, to get the most bang for their (time) bucks.
- acdha 4y agoI’m having flashbacks to some contract Java developers who were calling toString() on deserialized JSON objects and then parsing the output (old school index/substring, not even regex) because they insisted it was too hard to use the built-in accessors. Bonus points for then trying more kludges when they got a reminder that JSON does not guarantee object key order so they couldn’t write code assuming values would be adjacent. They also did that exact same thing parsing dates, except that they flubbed the logic and IIRC had an off-by-one on the month for the constructed Date object.
- Waterluvian 4y agoLots of JSON libraries can be told to enforce key order. And you’ve gotta remember that if you’re going to do something so ghoulish!
- acdha 4y agoThese were definitely not the kind of developer who read the documentation looking for options like that. I've mostly avoided Java for years and still found myself going “this is like 20 lines of code [badly] re-implementing a built-in library function” on almost every code review.
- v-erne 4y agoCould this just be incompenence? I have seen this kind of code many years ago when external company was brought to sell us their implementation of CMS on some (even then) outdated Java/js-on-server-before-node-era platform. I was asked to do code review to chcek if their code can be integrated into our codebase. I was quite unexperienced at that point in time but enough so to notice that there was something off with the code. There was a lot js that looked almost like exercise in obfuscation. Things like doing join, append and split on array to do push - and it was not abstracted into function - this three operations were repeated all over the code. And then I met the programmer - it was ovious after few questions that he did not know what he was doing and his direct manager, who was also present, know nothing about programming (he was more od a business person as he himself proclaimed). It was as if this company hired the cheapes guy they could find, that on paper looked competent, give him task of implementing CMS, and only looked at results (the CMS actually worked), but no one ever checked the code that was hiding under (and the guy never worked with anyone else and never read any other code besides his!)
- _aavaa_ 4y agoI have, and it boggles the mind. You'd think Java was a string-oriented language.
- Dachande663 4y agoCommon example I’ve seen is checking if an IP is in a range. Most (JS) libs I’ve seen do something like regex or, more commonly, split on periods and then have a set of nested ifs. Versus if you have a proper IP type, you check if its long value is between ranges. (Not saying you can’t do this, but because so many times an IP is stored as its string representation, that seems to be the default.
- usrusr 4y agoAlso haven't seen it, but I could imagine it happening if someone desperately misses duck-typing due to too many similar classes not under their control (can't make them share interfaces) and they have a lot of routine with e.g. json from/to but none with reflection. (I do enjoy a hearty dose of tojson in my println debugging)
- rr888 4y agoAnother common one is converting booleans to "Y" and "N".
- Zardoz84 4y agoCan be worse. On the old codebase where I work, I found booleans as : S, s, si, Yes, yes, Si, 1 ... I managed to unified it to be 1 and 0, but sometimes I keep seeing and old "Si" on some place
- karmelapple 4y agoWe saw it with contractor-developed code. They took JSON from an API, then converted it to a big blob of text, and did some gnarly searching. It made no sense. But we were also in a rush to get this app built, so we didn’t push back on this tortured logic. But we were saddled with this technical debt for an embarrassingly long time.
- WorldMaker 4y agoThat sounds almost more excusable than the rush code I found the other day. They were storing something (to local storage and then to a DB) via JSON.stringify() but for whatever reason they hadn't thought after retrieving it back from the database to just JSON.parse() it and instead used a weird mash of string replacements, splits, and regexes to parse an ad hoc brittle subset of JSON.
- ErikCorry 4y agoPeople will convert a float to an integer by using parseInt. That function automatically converts its argument to a string, then parses it as an int, ignoring anything after the decimal point. Completely fails for a number big enough to be stringified to 1.34e20. Plot twist, if your number is in a reasonable range this is very fast because there was one a popularJS benchmark that did this.
- mcculley 4y agoI have had to do it where all of the member methods are private (or otherwise inaccessible to me) and toString() is the only useful way to get information out of the object.
- jiggywiggy 4y agoMost of these actually come down to supporting logic in comments to turn thing on / off, etc. Or enhancing parsing. Not as much what you describe.
- notatoad 4y ago>And why do devs do this? TBH, I really don't know. it's one of the pillars of the unix philosophy at work - text is the universal interface. your nice rich type requires figuring out how that type works, maybe even reading documentation. if you make it a string, you can just treat it like a string. and we all know how to handle strings, or at least how to copy code from somewhere else that handles strings.
- Someone1234 4y agoC#'s System.Object base type contains a ToString, allowing any richer object to fallback into such inefficient systems if required. I'd also caution that "everything is a string so no need to read the documentation" is unrealistic and even outright dangerous. *NIX's string philosophy has directly resulted in multiple security vulnerabilities.
- Semaphor 4y agoJust FYI, I’m reasonably certain that parent is being satirical.
- KronisLV 4y agoI'm not so sure about the documentation part, but this seems valid at face value: > it's one of the pillars of the unix philosophy at work - text is the universal interface. In general, text is indeed the lowest common denominator that's available to you and fits with the majority of GNU tools nicely (and how most *nix distros are structured). Of course, some structure helps, since dealing with something like Nginx or Apache2 text logs might not be as nice as having your logs in JSON, though in regards to configuration files or program output there's lots of nuance - structured text (like JSON) will be easier to work with programmatically but might not be as nice to look at. Working with binary data in comparison can be asking for problems, to the point where using SQLite for your application data might make more sense: https://sqlite.org/appfileformat.html https://sqlite.org/appfileformat.html In this particular case, however, the conversions were made within the confines of a single program, so the interop is indeed not a valid concern here. A properly written type might indeed allow you to avoid various inefficiencies. If you're not serializing your objects and whatnot to make the life of the consumer of your data easier (be it another tool, or a REST/SOAP API client that needs JSON/XML, or a configuration file on the file system), then there's not that much point in doing those conversions.
- rk06 4y agoOne thing is that there is no deep comparison for objects. Say if you want an object as a key in Map, how would you go? In c#, I would use a Tuple. Or override equals to do a deep check.
- avmich 4y agoCan you write deep comparison? For a particular object? Or for many objects representable as a JSON, use a generic JSON comparison?
- rk06 4y agoThe problem is how to make Map use the deep comparison? I can do it manually yes. But if standard libraries are going to do referential check, then string is the only option left
- shakow 4y agoMy Java is a bit rusty, but can't you define the hashing function for this class?
- WorldMaker 4y agoIn JS there are a pair of Stage 1 proposals to increase support for richer/more complex keys in collections like Map: https://github.com/tc39/proposal-richer-keys https://github.com/tc39/proposal-richer-keys Strings seem really easy for composite keys and "deep comparison" for objects, but you could do more like C# actually does under the hood which is compute simple hash numbers (getting back to the article's point that computers are often faster with number calculations than string work). C# does most of its equality "magic" with GetHashCode() (surprisingly not Equals() and obviously not ToString()). (Relatedly this is why C# linters will yell at you if you override Equals but not also GetHashCode. Equals seems the more "useful" override but GetHashCode is the actual work horse.) Most implementations of GetHashCode (even for deep comparison situations) are generally starting with some random prime constant XOR bits of all the data and return the integer you have left after all the XORs. You can do the same thing in JS today if you wish. Modern .NET now has a nice System.HashCode helper class for doing it quickly with a lot of good best practices baked in (a strong starting constant, for instance), so it's easy to forget the bad old days of writing all those XORs by hand, and .NET has always had GetHashCode a required method of System.Object so "recursion" of hashcodes is guaranteed to be simple. But those things aside, it's still really easy to compute hash codes by hand in JS to use for comparisons than to use string manipulation, if you put your mind to it.
- esperent 4y ago> regexes are easier to understand Not a statement I ever expected to see.
- rising-sky 4y agoSame, I mean they are tricky to get a handle of but seemingly logical once you gain familiarity, until you have a use case that does not fit into a previously encountered pattern, and then it's back to the drawing board. But I guess, it's easier then reading the manual but what do I know
- jbverschoor 4y agoWhy? Because it’s all they know.
- EMM_386 4y agoJava has (had?) such beauties as "InternalFrameInternalFrameTitlePaneInternalFrameTitlePaneMaximizeButtonWindowNotFocusedStates". Now imagine a bit of reflection on that to do some string manipulation. Good times!
- pjmlp 4y agoSmalltalk, Objective-C and Swift top that, because the parameter names are part of the method name as well.
- pjc50 4y ago> They'll take a nice rich type, turn it into a string and then spend a bunch of code/time turning that string back into something else. They find the strongly typed system "inflexible", so they fall back to a "stringly typed" one.
- hombre_fatal 4y agoI think it’s just because everyone knows how to work with strings, it’s so tangible, and you can see what data a string contains right there in the print(thing). You don’t even have to know what the type is of the thing to work on it’s string representation.
- maccard 4y agoI'm guilty of having written code like this on occasion, as I'm sure we all are. It's easy to look at these changes and say "of course it's slow", but for every instance of this that OP fixed there's probably 99 other occurances of the same thing that _don't_ matter for performance. In a perfect world, sure, everyone would know whether they're on the hot path or the slow path and prioritise accordingly - it doesn't matter if something that is called once a day takes 15ms instead of 0.15ms, until someone adds it to the hot path and it gets called 290 times.
- josephg 4y agoI stubbornly hate code like this - but largely because of the security and correctness nightmare this is when combined with user generated data. For example, in a library one of my coworkers wrote, object paths were flattened into strings (eg “user.address.city) and then regular expressions and things worked on that. Sometimes user data ended up in the path (eg “cookies.<uuid>”). But then - what happens if the user puts a dot in their cookie? Or a newline? Does the program crash? Are there security implications we aren’t seeing? What do we do if dots are meaningful? Do we filter them out? Error? Escape them (and add a flurry of unit tests?) Are there bugs in the regex if a path segment is an empty string? It’s a nightmare. Much better to have paths be a list of keys (eg [“cookies”, uuid]). That’s what they represent anyway, and the code will be faster, and this whole class of string parsing bugs disappears.
- maccard 4y agoOh I totally agree. It's also my gripe with using Unix tools in general and chaining together tools in bash with pipes. Perfect is the enemy of good though, and probably 90% of my code is done properly, and I cut corners where I think it's appropriate - sometimes thats ok, and sometimes in hindsight it's not.