5 ms·

One of my pathologies as a developer is that I want to do things "properly" at all times. When I see things like fn tokenize(expr: String) -> Vec<String> {

• mumblemumble 5y ago
  Spending a month or two doing really scrupulous test-driven development, perhaps? I don't personally like TDD as an end in and of itself, but I love it as a sort of kata-like practice to help ingrain good habits. In this case, where I see it potentially helping is that the tests give you a very clear, objective way of deciding when the code is done, and it's time to stop futzing with it. Though you would have to be careful about remembering that the "refactor" phase is about refactoring for maintainability, not performance. I don't know what the tooling situation is like in Rust, but another one is to try and be really scrupulous about never making a change whose purpose is performance without first profiling the code, so that you can objectively measure the impact of the change. You don't need to do this forever, either. But, if your experience is anything like my own, I'm guessing that a lot of what motivates you is the feeling that you've done something of tangible value when you optimize the code. Actually seeing some hard numbers might change your feelings a bit. It's a lot easier to feel good about, "I spent an afternoon making the code more efficient," than it is to feel good about, "I spent an afternoon reducing the time it takes this program to execute from 50ms to 49ms."
  • 6keZbCECT2uB 5y ago
    "But, if your experience is anything like my own, I'm guessing that a lot of what motivates you is the feeling that you've done something of tangible value when you optimize the code." This is not at all how it is for me. The execution model of the code is core to how I read programs in languages where I am familiar with their execution model. So reading code that has dirty data or control flow, such as copying an entire array twice to change one character in two passes is as repulsive to me as maintaining and synchronizing two separate functions, one for each character swap.
    • mumblemumble 5y ago
      In that case, perhaps the best option is to lean into it and get onto a more Chuck Moore career trajectory? (By which I mean, there are some corners of the profession where this sort of mindset is very highly valued. Aerospace, some high frequency trading shops, embedded development. Nothing wrong with playing to your strengths.)
• rpsw 5y ago
  I think in a real parser your instincts would likely serve you well. For example, if a string data type was added, our tokenize method could not be this simple. To the actual question, I think one good approach when cutting corners or doing a quicker sub-optimal approach, is to document the limitations and assumptions in some way. You could then evaluate and prioritise you efforts and time from this, rather than getting stuck in the weeds early in the process.
  • nerpderp82 5y ago
    Barring the use of unsafe, Rust gives you more guarantees about containing low quality code behind good interfaces, so in Rust, one should do the thing in the most expedient manner and then can revisit it after the program is working. Working code can be iterated on, by multiple people. Non-functioning code cannot. Having a repo in state that can be shown functional by a CICD pipeline is the equivalent of achieving a chain reaction and freezing to death. Light that fire.
• stagger87 5y ago
  If the "right way" leads to - Wasting your time and your company's time. - More complex code which is harder to maintain in the long term. did it ever deserve to be called the "right way"?
  • dan-robertson 5y ago
    Well I think this code is harder to maintain in the long term than a proper parser. For example you may want foo"bar"baz to parse as two symbols and a string, or (a.b) as a cons. It also looks scary to me because I need to think a bit to be convinced that it works. But looking at code for a parser is surely hard too, and I don’t think any of those are strong enough arguments not to use the code above.
  • nomel 5y ago
    To me the "right way" is that it's not technical debt that will self destruct in the near future. If it will in the far future, a comment with an explanation for that future person is plenty.
• jcelerier 5y ago
  > I'll take a step back and realize that the performance of that code does not matter, or that the implementation was the wrong approach and I need to delete it all and do something else, or just that it wasn't very important and I should've done the easy solution and moved on. you're lucky, I don't remember a lot of times where following exactly this process did not lead to having to rewrite it 6 months later because it turns out that some user has some corner case which actually ends up requiring data going through that exact codepath in as little time as possible.
• kenniskrag 5y ago
  > those .replace() calls will allocate unnecessarily, All the time or only in Rust or Lisp? I would guess the compiler can optimize it because it is an XValue.
  • mastax 5y ago
    `str::replace(from, to)` replaces all instances of the pattern `from` in the string with `to` returning the result as a newly allocated String. In theory the compiler could notice that the spaces added to the newly allocated strings are only used to influence the behavior of `split_whitespace` and do not otherwise affect the output of the function. In practice, it does not: https://rust.godbolt.org/z/z573Mvjzz https://rust.godbolt.org/z/z573Mvjzz The compiler does not decide to inline the call to `replace` and therefore does not do any special reasoning about it.
  • lmkg 5y ago
    Those replace calls are growing the strings--one character is being replaced by several. That has the potential for re-allocation in any language, depending on how much spare space is present. Unless the optimizer is smart enough to figure out the space characters are never used because they're splitting points.
• sreque 5y ago
  I consider my self a bit of an optimization geek who sometimes errs on over-optimizing from the start. I think there are two major concerns. First, you want to write code whose style fits in with the team, project, and organization. If other people are writing high-level code chaining method calls like above and not worrying about performance, then you should write your code that way as well. It will fit in fine and no one will notice the performance impact because all the rest of the code is already equally slow. On the other hand, there are certain architectural decisions that affect performance that can be hard to undo after the fact. Those decisions can be a little harder but it usually boils down to meeting short-term requirements. Unless your boss or org appreciates optimizations and you can sell it on your quarterly review that you saved X dollars due to optimized code, then why bother? Your optimization may even cause conflict with people who don't like or appreciate it and don't like the extra complexity it might bring to the project. Or, if you ship with a fast-enough solution and the product's popularity grows to the point that scaling becomes hard, you can optimize it and look like a hero! Whereas, if you had optimized it in the first place pre-launch, your work could likely go un-appreciated. It's a bit cynical of a take, but a useful one to consider, I think.
• rtoway 5y ago
  In Rust, many times this kind of code will compile to the same code using plain old loops and if statements
  • mastax 5y ago
    Yes, however for this specific situation `replace()` is not an iterator adapter but a method of `str` which returns a newly allocated `String`. In practice, the compiler does not do something smarter than that: https://rust.godbolt.org/z/z573Mvjzz https://rust.godbolt.org/z/z573Mvjzz
    • rtoway 5y ago
      Ah you are right about that. Yes, it would be a nice place to avoid an allocation
• 6keZbCECT2uB 5y ago
  I 100% struggle with this, and it makes me a slower developer. As I said elsewhere, the dirty data / control flow of two passes to do a one character substitution is as painful to me as synchronizing two copies of the same function for different character substitutions. Probably the biggest help is to work somewhere that it's a productive use of your time to care about the details. Second, getting the details right can steal time from getting broad strokes right. Who cares if you reduce a predictable branch if you are sitting on top of a massive bucketed hash table the serializes all keys to strings by allocating? Fight the battles worth winning. One of the things that helps is to work in a language with an execution model that I don't understand, e.g. Python or JavaScript.
  • deleted 5y ago
    [deleted]
• lisper 5y ago
  The most important thing is not to get the code right but to get the interfaces right. If you do that, you can isolate the ugliness and the inefficiency and fix it all incrementally as needed and as the mood strikes you. Getting the interfaces right is not easy, but it's a hell of a lot easier than getting everything right.
• todd3834 5y ago
  Over the years I've just learned to recognize that the "right way" is really about weighing trade offs. Is this hot code that needs maximum efficiency? Is this code that is going to be touched a lot and readability and maintainability are more important? If the latter then the "right" way might be a less optimized version but given the trade offs it is the right way.
  • nomel 5y ago
    And, benchmarks always tell you if there's a problem and exactly where it is, and squash any unneeded discussion. I once had a new-to-the-group developer pointing out how things could be some much better if we optimized various code paths (I was them at one point in my career). I returned the next meeting with some benchmarks showing that we spent a total of 100ms in these code paths, in an application that ran for hours.
    • 6keZbCECT2uB 5y ago
      "benchmarks always tell you if there's a problem and exactly where it is, and squash any unneeded discussion." Unfortunately, always is too strong of a word. Cache eviction in one parts can cause memory stalls on another part, indirection in the caller can prevent speculation. Type erasure can prevent inlining resulting in the called function being blamed for problem in the caller. Your problem might not even be CPU, if it's contention related, or timing related, overloaded queues, not pushing back at the right places, io bound, the bottleneck is work which is queued and executed elsewhere... Causal profiling is a technique which is relevant specifically because profiling can miss the forest for the trees: https://github.com/plasma-umass/coz https://github.com/plasma-umass/coz It's really easy to write a benchmark which measures a different scenario from what your application is doing. A classic example might be benchmarking a hashmap in a loop when that hashmap is usually used when cold. I definitely agree about directing efforts to where you can make an impact and guiding that through measurement, but benchmarks can miss that there's a problem and blame the wrong part of the application. If the difference is large enough, ms vs hours, you'd have to really screw up methodology to get the wrong result (I've done it almost that badly before).
• dec0dedab0de 5y ago
  My problem is I'll catch myself playing code golf in the name of efficient expression, or writing mini frameworks when they'll never be used again. In general, I agree with your first draft idea. Get the thing working before you optimize for whatever is bothering you. Though, I also find that when I do waste time doing something that doesnt matter, I end up staying engaged longer than if i just stuck to the problem at hand. That is to say, if you're going to procrastinate it's better to procrastinate by writing code.
• zelphirkalt 5y ago
  I think most good engineers struggle with it, unless there is time given to plan ahead properly and properly think about where you are going with a project. In ones own projects one can decide to take that time and often it is helpful to spend a little time ahead of writing the code.