4 ms·

Thankfully there's a trend of excessive merge conflicts caused by reformatters. Reformat-on-every-commit really only works for highly-centralized, tightly-coup

Thankfully there's a trend of excessive merge conflicts caused by reformatters. Reformat-on-every-commit really only works for highly-centralized, tightly-coupled, monorepo-using monolithic organizations. Basically the exact opposite of kerneldev. For those folks reformat-on-every-commit works great.

em-bee 3y ago
actually, what we want is code management tools that work with tokenized code and do not depend on formatting. i want a diff tool that shows me exactly which tokens have changed, and which haven't, regardless of how they are laid out. when we get that, then we should get even less merge conflicts.
- jasomill 3y ago
  And I'd prefer a system with an AST as the canonical representation, allowing each programmer to set their own display formatting options entirely independently of the underlying codebase. Realistically, either of these options would only really work for tools and codebases designed with these workflows in mind, as they'd be useless for inputs to inherently text-based tools like the C preprocessor.
  - karma_pharmer 3y ago
    allowing each programmer to set their own display formatting options entirely independently of the underlying codebase. That's the easy case. The hard case is situations where the formatting (whitespace, etc) is not simply some deterministic function applied to the AST. For example: comment placement and tabular layout of repetitive constant definitions. Or situations where vertical alignment is used to call out parallel structure. Mathematicians have been using horizontal alignment to call out important structure in equations (both written and typeset) for hundreds of years. Here's are two examples if you don't know what I'm talking about: https://kagi.com/proxy/ILVap.jpg?c=085W0W0Vgv9kix8Uo-TAMYM_FT1u23UFqHfiq4zyoC7SNFxXinp9WqjiTM14lkaA https://kagi.com/proxy/ILVap.jpg?c=085W0W0Vgv9kix8Uo-TAMYM_F... https://kagi.com/proxy/Zr5re.png?c=085W0W0Vgv9kix8Uo-TAMb-ZJ9tlIOckKNxTWVDXFV8DwbMWKEFKbX9zuX2S1K7U https://kagi.com/proxy/Zr5re.png?c=085W0W0Vgv9kix8Uo-TAMb-ZJ... The people who stubbornly insist that programmers have no need for this sort of expressiveness seem like innumerate cavemen, frankly. They cheapen our field, reducing it from informatics to mere keyboard labor. The urge to run code formatters is a "language smell" indicating that your language has way, way, way too much odd and irregular syntax. See Haskell for a counterexample. Sure, there are a few code formatters for Haskell, but nobody really uses them. The language's syntax is so minimal and clean that there just aren't all that many preference-based choices to be made.
    - tome 2y ago
      > there are a few code formatters for Haskell, but nobody really uses them I haven't run a survey, but I get the impression that code formatters are widely used in the Haskell world, especially Ormolu and Fourmolu.
      - karma_pharmer 2y ago
        They are in fact not widely used. Ormolu and Fourmolu That's only one formatter; the latter is a fork of the former... which is the in-house formatting tool of one particular consulting firm. It has only existed since 2018, and is said to be mainly a response to golang/rust people trying to impose their culture on the Haskell world. Pointing at ormolu is kinda like saying that formatters are popular in the Java world because google-java-format exists. The fact that that project exists says more about the company that created it than the community around the language it operates on.
        tome 2y ago
        > They are in fact not widely used. Ah, as a fact it presumably has some form of evidence that demonstrates it. Would you mind sharing that evidence?
- karma_pharmer 3y ago
  i want a diff tool that shows me exactly which tokens have changed, and which haven't, regardless of how they are laid out. These already exist: https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic when we get that, then we should get even less merge conflicts. Counterintuitively, that is not the case. AST-merge is a much, much, much, much, much harder problem than AST-diff. https://github.com/Wilfred/difftastic?tab=readme-ov-file#can-difftastic-do-merges https://github.com/Wilfred/difftastic?tab=readme-ov-file#can... The fact that diffs can be used to drive a 3-way merge is in fact an accidental property that arises due to the sheer crudeness of the line-based diff format. As soon as you start using more-sophisticated diff formats, solutions to "the diff problem" no longer lead directly to solutions to "the merge problem".
quectophoton 3y ago
My bad, I should have added "/s" (because Cunningham's Law). It was a reference to Futurama, where a problem was not solved at all. But on a more serious note, in my experience I've not had any issues with Go or Rust codebases (for example). Not using their formatters is heavily frown upon, so I haven't really seen any reformat happen at all; not in my bubble at least[1]. Other languages, on the other hand? Yeah, good luck with trying to have consistent formatting. Even if a project has formatting rules "enforced", there's always (always) going to be an exception, bikeshedding, etc. [1]: Unless it's someone obviously very junior. The few times I've noticed badly formatted code in Go, has been in random repos from someone who clearly didn't have that much programming experience in general (looking at how code was written).
ghnws 3y ago
What do you mean by reformat? Any decent code formatter keeps a consistent style. Getting conflics only happens if you missconfigure your editor or don't have checks to catch invalid formatting before merging to remote.
- quectophoton 3y ago
  They probably mean because of stuff like: "Our Python codebase is old af and has inconsistent styling. Should we start using black/ruff to format it?" "Hmm that's maybe a bit too intrusive. Could autopep8 be enough? Since it follows the recommendations of the PSF, so it's a more 'official' way of formatting, and it should be more future-proof [famous last words]." "Hey, I heard this pyxkcd927 formatter just came out, and it also automatically fixes linter warnings [meaning, different output]. Any thoughts on migrating to this?"
  - hedora 3y ago
    “If you want to change the formatting, warn people to merge any big PRs that are ready. Then, you fix any merge conflicts that result in the outstanding feature branches. Do this twice; once as a dry run where you get the tests to keep passing, and once for real while the team pauses merges for a short window.” In general, any software engineering thing that causes multiplicative work for the rest of the org should be handled this way. For instance, at a prior job, it cost 100+ engineers a week or so of productivity to rename master to main because tooling and automation had hardcoded master all over the place. It cost the person that initiated the change an hour or so, and I’m sure it helped their promotion case. Of course, accounting didn’t compute the cost of this to the business ahead of time. If they had, I’d hope they would have insisted we spend the money on recruiting and hiring more diverse candidates instead.
GoblinSlayer 3y ago
It's much worse for monolithic organizations, because they develop complex software and reformatting scrambles code history and it becomes difficult to untangle business logic.
- ikekkdcjkfke 3y ago
  Your comment gave me a mental image of looking through file change history in 3d, each version of the document layered on top of another translucent in 3d and scrolling through them with the wheel

em-bee 3y ago

actually, what we want is code management tools that work with tokenized code and do not depend on formatting. i want a diff tool that shows me exactly which tokens have changed, and which haven't, regardless of how they are laid out. when we get that, then we should get even less merge conflicts.

jasomill 3y ago
And I'd prefer a system with an AST as the canonical representation, allowing each programmer to set their own display formatting options entirely independently of the underlying codebase. Realistically, either of these options would only really work for tools and codebases designed with these workflows in mind, as they'd be useless for inputs to inherently text-based tools like the C preprocessor.
- karma_pharmer 3y ago
  allowing each programmer to set their own display formatting options entirely independently of the underlying codebase. That's the easy case. The hard case is situations where the formatting (whitespace, etc) is not simply some deterministic function applied to the AST. For example: comment placement and tabular layout of repetitive constant definitions. Or situations where vertical alignment is used to call out parallel structure. Mathematicians have been using horizontal alignment to call out important structure in equations (both written and typeset) for hundreds of years. Here's are two examples if you don't know what I'm talking about: https://kagi.com/proxy/ILVap.jpg?c=085W0W0Vgv9kix8Uo-TAMYM_FT1u23UFqHfiq4zyoC7SNFxXinp9WqjiTM14lkaA https://kagi.com/proxy/ILVap.jpg?c=085W0W0Vgv9kix8Uo-TAMYM_F... https://kagi.com/proxy/Zr5re.png?c=085W0W0Vgv9kix8Uo-TAMb-ZJ9tlIOckKNxTWVDXFV8DwbMWKEFKbX9zuX2S1K7U https://kagi.com/proxy/Zr5re.png?c=085W0W0Vgv9kix8Uo-TAMb-ZJ... The people who stubbornly insist that programmers have no need for this sort of expressiveness seem like innumerate cavemen, frankly. They cheapen our field, reducing it from informatics to mere keyboard labor. The urge to run code formatters is a "language smell" indicating that your language has way, way, way too much odd and irregular syntax. See Haskell for a counterexample. Sure, there are a few code formatters for Haskell, but nobody really uses them. The language's syntax is so minimal and clean that there just aren't all that many preference-based choices to be made.
  - tome 2y ago
    > there are a few code formatters for Haskell, but nobody really uses them I haven't run a survey, but I get the impression that code formatters are widely used in the Haskell world, especially Ormolu and Fourmolu.
    - karma_pharmer 2y ago
      They are in fact not widely used. Ormolu and Fourmolu That's only one formatter; the latter is a fork of the former... which is the in-house formatting tool of one particular consulting firm. It has only existed since 2018, and is said to be mainly a response to golang/rust people trying to impose their culture on the Haskell world. Pointing at ormolu is kinda like saying that formatters are popular in the Java world because google-java-format exists. The fact that that project exists says more about the company that created it than the community around the language it operates on.
      - tome 2y ago
        > They are in fact not widely used. Ah, as a fact it presumably has some form of evidence that demonstrates it. Would you mind sharing that evidence?
karma_pharmer 3y ago
i want a diff tool that shows me exactly which tokens have changed, and which haven't, regardless of how they are laid out. These already exist: https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic when we get that, then we should get even less merge conflicts. Counterintuitively, that is not the case. AST-merge is a much, much, much, much, much harder problem than AST-diff. https://github.com/Wilfred/difftastic?tab=readme-ov-file#can-difftastic-do-merges https://github.com/Wilfred/difftastic?tab=readme-ov-file#can... The fact that diffs can be used to drive a 3-way merge is in fact an accidental property that arises due to the sheer crudeness of the line-based diff format. As soon as you start using more-sophisticated diff formats, solutions to "the diff problem" no longer lead directly to solutions to "the merge problem".

quectophoton 3y ago

My bad, I should have added "/s" (because Cunningham's Law). It was a reference to Futurama, where a problem was not solved at all. But on a more serious note, in my experience I've not had any issues with Go or Rust codebases (for example). Not using their formatters is heavily frown upon, so I haven't really seen any reformat happen at all; not in my bubble at least[1]. Other languages, on the other hand? Yeah, good luck with trying to have consistent formatting. Even if a project has formatting rules "enforced", there's always (always) going to be an exception, bikeshedding, etc. [1]: Unless it's someone obviously very junior. The few times I've noticed badly formatted code in Go, has been in random repos from someone who clearly didn't have that much programming experience in general (looking at how code was written).

ghnws 3y ago

What do you mean by reformat? Any decent code formatter keeps a consistent style. Getting conflics only happens if you missconfigure your editor or don't have checks to catch invalid formatting before merging to remote.

quectophoton 3y ago
They probably mean because of stuff like: "Our Python codebase is old af and has inconsistent styling. Should we start using black/ruff to format it?" "Hmm that's maybe a bit too intrusive. Could autopep8 be enough? Since it follows the recommendations of the PSF, so it's a more 'official' way of formatting, and it should be more future-proof [famous last words]." "Hey, I heard this pyxkcd927 formatter just came out, and it also automatically fixes linter warnings [meaning, different output]. Any thoughts on migrating to this?"
- hedora 3y ago
  “If you want to change the formatting, warn people to merge any big PRs that are ready. Then, you fix any merge conflicts that result in the outstanding feature branches. Do this twice; once as a dry run where you get the tests to keep passing, and once for real while the team pauses merges for a short window.” In general, any software engineering thing that causes multiplicative work for the rest of the org should be handled this way. For instance, at a prior job, it cost 100+ engineers a week or so of productivity to rename master to main because tooling and automation had hardcoded master all over the place. It cost the person that initiated the change an hour or so, and I’m sure it helped their promotion case. Of course, accounting didn’t compute the cost of this to the business ahead of time. If they had, I’d hope they would have insisted we spend the money on recruiting and hiring more diverse candidates instead.

GoblinSlayer 3y ago

It's much worse for monolithic organizations, because they develop complex software and reformatting scrambles code history and it becomes difficult to untangle business logic.

ikekkdcjkfke 3y ago
Your comment gave me a mental image of looking through file change history in 3d, each version of the document layered on top of another translucent in 3d and scrolling through them with the wheel