4 ms·

Again, I think we need a concrete example to really understand what you are trying to say. If two algorithms are functionally identical (same results, same spa

Again, I think we need a concrete example to really understand what you are trying to say. If two algorithms are functionally identical (same results, same space/time complexity, etc.), who cares why? What are you going to gain from knowing more about the choice? I'll give an example. Let's say you need to add 1 to each integer in an array. You could use a for loop, a map function, etc. We'll assume the compiler optimizes each choice to the exact same machine code. What would you like to have documented about the choice?

wpietri 4y ago
Exactly my view, too. If a particular algorithm was chosen for, say, performance reasons, then those performance needs should be expressed in a test.
- citrin_ru 4y ago
  An attempt to measure say 5% difference in a tests would likely produce a flaky test which frequent failures would waste team's time. I'm not even starting that to measure this difference in a test one would have to maintain two implementations - slow and faster one and run in a test suite both to have two times to compare. Looks like unnecessary work to me. Enough to run benchmark manually when a decision is being made.
  - still_grokking 4y ago
    Exactly. I wonder people don't recognize that upfront. I'm getting the impression, like it was stated by some people here, that this whole "no comments" / "everything should be a unit test" idea comes form inexperienced but idealistic developers mostly. People with more experience seem to know already OTOH that good comments can save your ass in the middle of the night.
    - randomdata 4y ago
      > this whole "no comments" / "everything should be a unit test" idea Nothing in this thread says "no comments" or "everything should be a unit test". But the reality is that most code doesn't matter. It is just a means to an end and if it no longer serves that means it's expendable. Explaining why something was chosen doesn't tell you anything meaningful most of the time. In fact, I expect the "why" 99% of the time is "I felt like it and it worked". When a specific approach is essential in meeting certain application requirements, absolutely you need to comment that. But you also need tests, else how are you going to ensure that you stay within those requirements? Of an application of any meaningful size, you're going to have a hard time manually ensuring that some externality didn't break your assumption on every deploy. Even if it weren't hard, why put in the man hours when a machine can do it for you?
      - still_grokking 4y ago
        > In fact, I expect the "why" 99% of the time is "I felt like it and it worked". Erm, we're talking about professional software development here. What someone is tinkering together in his hobby cellar is of no interest to me. Code created by the principle "I felt like that" is garbage by definition. It makes no sense to even discuss such stuff. > When a specific approach is essential in meeting certain application requirements, absolutely you need to comment that. But you also need tests, else how are you going to ensure that you stay within those requirements? Of an application of any meaningful size, you're going to have a hard time manually ensuring that some externality didn't break your assumption on every deploy. Even if it weren't hard, why put in the man hours when a machine can do it for you? Here I actually agree fully. What can be tested by the machine should be. That's for example a reason to use statically languages, as in contrast to such thing as Ruby, the machine can give much better correctness guaranties with static type checking. But the point is: Not everything can be expressed as meaningful test. For example you proposed something that would result in a flaky test case… Some other things can't be expressed as test either. And the experience shows that's this things are almost always related to the why question. In the end you put exactly this question front while discussing some imaginary implementation of something, which shows my point nicely. :-)
        4y ago
    - wpietri 4y ago
      One, I didn't say that. And two, I write under my real name. It's pretty easy to find out how much experience I have. As a person with "more" experience, I went through a phase of writing lots of comments. Obviously, given how much I comment here, I like writing. But as I said elsewhere in this discussion, comments have become for me a last resort. I'd rather put the information I'm trying to convey almost anywhere else. Variable names, method names, improved interfaces, better object relationships, doc strings, test code, test names, commit comments, or my colleagues' heads.
  - randomdata 4y ago
    If you can't consistently measure a 5% difference in a tightly controlled environment, how are your real world users going to notice? If your users don't notice, who cares? Furthermore, if that 5% optimization is truly critical to the application (despite not being measurable?), how are you going to ensure that some future change doesn't break that? Something as simple as a compiler update could change some optimization that breaks your assumptions. What do you do in the absence of tests to ensure that doesn't slip through?
    - citrin_ru 4y ago
      Even if user would not notice 5% difference in CPU usage can mean reduction in AWS bill significant enough to care about (but not significant enough to spend days to avoid adding a few lines of comments by embedding all knowledge into tests). > 5% difference in a tightly controlled environment, A test suite usually doesn't run in tightly controlled environment. To get meaningful performance testing results you'll need to run it on dedicated hardware (no noisy neighbors) and don't run different tests in parallel (so one test will not be a noisy neighbor for another test). It is something not hard to do once in a while manually but would be quite expensive to do as part of a test suite which typically is running on each change.
      - randomdata 4y ago
        > Even if user would not notice 5% difference in CPU usage can mean reduction in AWS bill significant enough to care about Users won't notice, but they will notice...? > A test suite usually doesn't run in tightly controlled environment. What don't you have control over? > but not significant enough to spend days to avoid adding a few lines of comments by embedding all knowledge into tests Why would you avoid comments? Why would your tests have to take longer to write than your comments? Days to write a test or two to clarify your intent?
  - wpietri 4y ago
    If the difference is so small it's hard to measure, then that sounds like something that's not worth testing to me. Meaning that it's also ok if somebody changes the implementation later on. On the other hand, if it's performance critical code in a way that's important to the project, then running a manual benchmark once and hoping that nobody accidentally breaks it strikes me as poor practice.
- still_grokking 4y ago
  How would such test look like?
still_grokking 4y ago
> We'll assume the compiler optimizes each choice to the exact same machine code. Where is this—possibly wrong—assumption documented? Or do you think it makes no sense to document random assumptions, which may or may not hold in the future? Another simple example: Almost all usable sorting algorithm have the same space / time complexity on paper. But as everybody knows they behave quite different in reality (otherwise we wouldn't have so many of them to choose form). Do you really think it makes no sense to document such design choice where it matters? Also in this case unit tests won't help you. (But you given up already on this line of argumentation anyway, I see ;-))
- randomdata 4y ago
  > Where is this—possibly wrong—assumption documented? Presumably in the compiler documentation. Although the assumption was made merely to ensure that we understood that each algorithm choice was functionally equivalent, to continue in the theme of discussion. > which may or may not hold in the future? Totally fair. Let's say your choice, which was fine at the time of choosing, with a later compiler update does get optimized in a new way that that longer satisfy your application's requirements. How do you plan to keep up with that if you don't have automated tests to prove whether or not your code still operates within your expectations? > But you given up already on this line of argumentation anyway, I see ;-) No, but I was still working on getting a concrete example in which we can build a test or two for to provide a demonstration. Unfortunately, the vague sorting suggestion still isn't concrete. What are you sorting? Which algorithm did you choose and why? I had hoped that if I rephrased the question you would feel more comfortable coming out of your shell, but I can see your worry remains. Back in my day education was something to get excited about so I struggle to understand, but recognizing your uncomfortableness now I'll not needle you further and apologize for not seeing it sooner.
  - still_grokking 4y ago
    You're distracting form the main topic, which is: > How does a test inform you why a function was implemented using algorithm A and not algorithm B which would also do the same? You made up than a straw man saying something something that you assume that both algos get optimized to the same machine code. Which is completely irrelevant to the actual question. I've only mentioned that both algos need to have the same big-O characteristic obviously as otherwise they wouldn't be interchangeable in the first place. > What are you sorting? Which algorithm did you choose and why? LOL, that's exactly the question a good code comments should answer. Because code can't do that! The example here is: Big-O is one thing, but "what are you sorting" matters, and some algo may be better suited for the task than the other. If this isn't obvious it needs to be documented. The best place to do that is right where the code is. As only this will make sure it will be read together with the code. (Otherwise someone could for example rewrite the code to something "more standard" and it's not a given that the people reviewing the change would know about the reasoning behind the original implementation; a comment would have prevented bugs and regressions in such a case). Thanks for proving my point… :-)

wpietri 4y ago

Exactly my view, too. If a particular algorithm was chosen for, say, performance reasons, then those performance needs should be expressed in a test.

citrin_ru 4y ago
An attempt to measure say 5% difference in a tests would likely produce a flaky test which frequent failures would waste team's time. I'm not even starting that to measure this difference in a test one would have to maintain two implementations - slow and faster one and run in a test suite both to have two times to compare. Looks like unnecessary work to me. Enough to run benchmark manually when a decision is being made.
- still_grokking 4y ago
  Exactly. I wonder people don't recognize that upfront. I'm getting the impression, like it was stated by some people here, that this whole "no comments" / "everything should be a unit test" idea comes form inexperienced but idealistic developers mostly. People with more experience seem to know already OTOH that good comments can save your ass in the middle of the night.
  - randomdata 4y ago
    > this whole "no comments" / "everything should be a unit test" idea Nothing in this thread says "no comments" or "everything should be a unit test". But the reality is that most code doesn't matter. It is just a means to an end and if it no longer serves that means it's expendable. Explaining why something was chosen doesn't tell you anything meaningful most of the time. In fact, I expect the "why" 99% of the time is "I felt like it and it worked". When a specific approach is essential in meeting certain application requirements, absolutely you need to comment that. But you also need tests, else how are you going to ensure that you stay within those requirements? Of an application of any meaningful size, you're going to have a hard time manually ensuring that some externality didn't break your assumption on every deploy. Even if it weren't hard, why put in the man hours when a machine can do it for you?
    - still_grokking 4y ago
      > In fact, I expect the "why" 99% of the time is "I felt like it and it worked". Erm, we're talking about professional software development here. What someone is tinkering together in his hobby cellar is of no interest to me. Code created by the principle "I felt like that" is garbage by definition. It makes no sense to even discuss such stuff. > When a specific approach is essential in meeting certain application requirements, absolutely you need to comment that. But you also need tests, else how are you going to ensure that you stay within those requirements? Of an application of any meaningful size, you're going to have a hard time manually ensuring that some externality didn't break your assumption on every deploy. Even if it weren't hard, why put in the man hours when a machine can do it for you? Here I actually agree fully. What can be tested by the machine should be. That's for example a reason to use statically languages, as in contrast to such thing as Ruby, the machine can give much better correctness guaranties with static type checking. But the point is: Not everything can be expressed as meaningful test. For example you proposed something that would result in a flaky test case… Some other things can't be expressed as test either. And the experience shows that's this things are almost always related to the why question. In the end you put exactly this question front while discussing some imaginary implementation of something, which shows my point nicely. :-)
      - 4y ago
  - wpietri 4y ago
    One, I didn't say that. And two, I write under my real name. It's pretty easy to find out how much experience I have. As a person with "more" experience, I went through a phase of writing lots of comments. Obviously, given how much I comment here, I like writing. But as I said elsewhere in this discussion, comments have become for me a last resort. I'd rather put the information I'm trying to convey almost anywhere else. Variable names, method names, improved interfaces, better object relationships, doc strings, test code, test names, commit comments, or my colleagues' heads.
- randomdata 4y ago
  If you can't consistently measure a 5% difference in a tightly controlled environment, how are your real world users going to notice? If your users don't notice, who cares? Furthermore, if that 5% optimization is truly critical to the application (despite not being measurable?), how are you going to ensure that some future change doesn't break that? Something as simple as a compiler update could change some optimization that breaks your assumptions. What do you do in the absence of tests to ensure that doesn't slip through?
  - citrin_ru 4y ago
    Even if user would not notice 5% difference in CPU usage can mean reduction in AWS bill significant enough to care about (but not significant enough to spend days to avoid adding a few lines of comments by embedding all knowledge into tests). > 5% difference in a tightly controlled environment, A test suite usually doesn't run in tightly controlled environment. To get meaningful performance testing results you'll need to run it on dedicated hardware (no noisy neighbors) and don't run different tests in parallel (so one test will not be a noisy neighbor for another test). It is something not hard to do once in a while manually but would be quite expensive to do as part of a test suite which typically is running on each change.
    - randomdata 4y ago
      > Even if user would not notice 5% difference in CPU usage can mean reduction in AWS bill significant enough to care about Users won't notice, but they will notice...? > A test suite usually doesn't run in tightly controlled environment. What don't you have control over? > but not significant enough to spend days to avoid adding a few lines of comments by embedding all knowledge into tests Why would you avoid comments? Why would your tests have to take longer to write than your comments? Days to write a test or two to clarify your intent?
- wpietri 4y ago
  If the difference is so small it's hard to measure, then that sounds like something that's not worth testing to me. Meaning that it's also ok if somebody changes the implementation later on. On the other hand, if it's performance critical code in a way that's important to the project, then running a manual benchmark once and hoping that nobody accidentally breaks it strikes me as poor practice.
still_grokking 4y ago
How would such test look like?

still_grokking 4y ago

> We'll assume the compiler optimizes each choice to the exact same machine code. Where is this—possibly wrong—assumption documented? Or do you think it makes no sense to document random assumptions, which may or may not hold in the future? Another simple example: Almost all usable sorting algorithm have the same space / time complexity on paper. But as everybody knows they behave quite different in reality (otherwise we wouldn't have so many of them to choose form). Do you really think it makes no sense to document such design choice where it matters? Also in this case unit tests won't help you. (But you given up already on this line of argumentation anyway, I see ;-))

randomdata 4y ago
> Where is this—possibly wrong—assumption documented? Presumably in the compiler documentation. Although the assumption was made merely to ensure that we understood that each algorithm choice was functionally equivalent, to continue in the theme of discussion. > which may or may not hold in the future? Totally fair. Let's say your choice, which was fine at the time of choosing, with a later compiler update does get optimized in a new way that that longer satisfy your application's requirements. How do you plan to keep up with that if you don't have automated tests to prove whether or not your code still operates within your expectations? > But you given up already on this line of argumentation anyway, I see ;-) No, but I was still working on getting a concrete example in which we can build a test or two for to provide a demonstration. Unfortunately, the vague sorting suggestion still isn't concrete. What are you sorting? Which algorithm did you choose and why? I had hoped that if I rephrased the question you would feel more comfortable coming out of your shell, but I can see your worry remains. Back in my day education was something to get excited about so I struggle to understand, but recognizing your uncomfortableness now I'll not needle you further and apologize for not seeing it sooner.
- still_grokking 4y ago
  You're distracting form the main topic, which is: > How does a test inform you why a function was implemented using algorithm A and not algorithm B which would also do the same? You made up than a straw man saying something something that you assume that both algos get optimized to the same machine code. Which is completely irrelevant to the actual question. I've only mentioned that both algos need to have the same big-O characteristic obviously as otherwise they wouldn't be interchangeable in the first place. > What are you sorting? Which algorithm did you choose and why? LOL, that's exactly the question a good code comments should answer. Because code can't do that! The example here is: Big-O is one thing, but "what are you sorting" matters, and some algo may be better suited for the task than the other. If this isn't obvious it needs to be documented. The best place to do that is right where the code is. As only this will make sure it will be read together with the code. (Otherwise someone could for example rewrite the code to something "more standard" and it's not a given that the people reviewing the change would know about the reasoning behind the original implementation; a comment would have prevented bugs and regressions in such a case). Thanks for proving my point… :-)