18 ms·

Django: Reformatted code with Black

284 points by tams 5y ago

VWWHFSfQ 5y ago
So now when you look at the annotated change history all you're going to see is a bunch of changes by the person that reformatted the code instead of the person that wrote it.
- justinmchase 5y ago
  You can see both of course. That's the beauty of history.
- deleted 5y ago
  [deleted]
- tempay 5y ago
  The `.git-blame-ignore-revs` file can be used to ignore that (and will be [1]). Unfortunatly GitHub doesn't support it but at least it's possible to have clients behave in a reasonable way. [1] https://github.com/django/django/pull/15387#issuecomment-1031848322 https://github.com/django/django/pull/15387#issuecomment-103...
  - sciurus 5y ago
    For anyone looking for more explanation of this feature: https://michaelheap.com/git-ignore-rev/ https://michaelheap.com/git-ignore-rev/
  - acidburnNSA 5y ago
    TIL about that git feature. Very nice.
  - terr-dav 5y ago
    You can automate setup for developers using this simple script: https://github.com/ipython/ipython/pull/12091/files https://github.com/ipython/ipython/pull/12091/files And here’s a GitLab issue requesting support for blame-ignore: https://gitlab.com/gitlab-org/gitlab/-/issues/31423 https://gitlab.com/gitlab-org/gitlab/-/issues/31423 I don’t think there’s a corresponding GitHub request, but maybe if GitLab adds this feature GitHub will have some incentive to follow suit.
- deleted 5y ago
  [deleted]
- Cthulhu_ 5y ago
  There's workarounds that others have mentioned, but indeed, the unfortunate side-effect of deciding to apply a formatter is a 'formatting' commit, causing a lot of code churn and issues if naively using git blame. But, it's a "rip the plaster off" kinda thing, because it should ensure a lot less churn, inconsistent code style, or arguments and reviews about formatting after this is merged. It frees up a lot of headspace and distractions in code reviews. I don't know about you, but when I did code reviews I'd always end up zooming in on code style issues - ' vs ", things on newlines or no, JS objects with stringed keys, etc.
- alecbz 5y ago
  Uh so is your take "don't do broad refactors ever?" Beyond `.git-blame-ignore-revs` (which is neat and TIL), in GitHub's web viewer, if you find the line you're interested in and see that the most recent PR is a reformat, you click the "view blame prior to this change" button. I think most blame viewers do (or at least should) have a feature like this.
justinmchase 5y ago
The output does look better but this also just looks like every PR for applying a linter / formatter I've ever seen. Not sure why this is news worthy.
- deleted 5y ago
  [deleted]
- owaislone 5y ago
  Using black is not about how the code looks but to eliminate an entire suite of review comments/discussions. Everyone simply runs black over all code before submitting and no one ever comments about how anything is formatted.
  - musingsole 5y ago
  - captainmuon 5y ago
    Naive question, but why is everybody so aggravated by formatting discussions? It seems to be a widespread opinion that these discussions are just 1) pointless and 2) difficult and time consuming. My personal experience is that 1) in many cases you do benefit from taking a moment, going through your code and thinking about presentation. And 2) I find it not at all difficult to settle. A change either doesn't matter, then you just don't discuss it at all, or it is important, then you quickly agree on the best solution. (In the worst case, "best" means what the project lead finds prettier.) If you don't have a social mechanism to agree on something as basic as coding style, then your team probably has bigger problems. I actually find robo-formated code annoying to read: Go code from a bloody beginner who doesn't know what they are doing looks exactly like carefully tended for, highly thought-out code. And in autoformatted Python, you for example cannot make formulas clearer by removing spaces around operators with higher precidence. Parentheses placement is dicated by how long words are and not by what logically belongs together, etc..
    - WesolyKubeczek 5y ago
samwillis 5y ago
I believe from memory Django decided to move to using Black back in 2019 [0] but delayed the change until Black exited Beta. Black became none beta at the end of January [1]. This was finally merged to the main branch today [2]. I suspect there are lots of other both open source and private projects that are also making the change now. This is a show of confidence in Black as the standard code formatter for Python. 0: https://github.com/django/deps/blob/main/accepted/0008-black.rst https://github.com/django/deps/blob/main/accepted/0008-black... 1: https://news.ycombinator.com/item?id=30130316 https://news.ycombinator.com/item?id=30130316 2: https://github.com/django/django/pull/15387 https://github.com/django/django/pull/15387
- dwightgunning 5y ago
  This is right. Black emerging from beta was discussed on the Django mailing list in the last week or so, and triggered the work.
supreme_berry 5y ago
“Black” developer refused for a long time to add option to format code with single quotes with very aggressive manners. Now Django devs didn’t see that option for single quotes and code looks unpleasant.
- vitorfs 5y ago
  I have always used single quotes for Python code since I start working with it. When I started to adopt Black on my projects it indeed felt weird and the code looked unpleasant. But after a while you get used to it. Some people make the case that it's easier to write single quotes (well, depending on the keyboard format anyway). For keyboards in the US standard you have to hold the Shift key to write a double quote. But the good thing about Black is that you can still write your code using single quote and when you run the command line utility it will fix/normalize the code to use double quotes. Nowadays I got so used to it that I even write my Python code using double quotes. And looking at Python code using single quotes looks weird/unpleasant for me.
  - valparaiso 5y ago
  - digisign 5y ago
    The repl still uses single quotes.
  - spc476 5y ago
    I use single quotes for items that, while technically a string, could be considered a value or symbol. For example: syslog('debug',"Just opened %s for output",filename) While there's no semantic difference between single and double quote, in my code base, there is. And if black becomes very popular, why even support single quotes anymore?
- INTPenis 5y ago
  I reacted to this too, in the changed files tab. Technically single or double quotes have the exact same meaning in Python. What makes people use single quotes is probably other languages like PHP, Perl and Bash. I know I've made it a habit to default to single quotes unless I know I need double quotes. So that might be where the habit comes from in the Django project. But it's not actually necessary in python so might as well use the most commonly used type of quote.
declnz 5y ago
Aside: I love a good linter, but as a long-time Python fan I find it sad that Black has so little configuration (yes, I know, but still) and moreover that it often produces code that no human Python dev I know would write... Python was always meant to look concise / beautiful... (MyPy has also made this trickier too)
- alecbz 5y ago
  OOC what are your grips with black's style? I generally find black pretty "beautiful" (concise maybe not as much).
  - declnz 5y ago
    I guess the closing parens irk me the most e.g. assert outputs.get("foo.bar.baz", "default") == pytest.approx( time_recorder.time_taken, abs=0.0001 ) I get why it's done that, but I just don't think it helps humans read. Part of the twisted beauty of PEP-008's narrow lines is that you're forced to extract (named) variables, or avoid overly indented code by extracting methods or applying higher level abstractions. In the last few years I find devs are happier to format and push to "sort that problem out", leaving the readability benefit of that thought process lost. TL;DR writing readable code isn't just about getting the spaces and brackets right...
    - alecbz 5y ago
      I tend to prefer the trailing paren on the following line. I'm not sure if there's a principled reason it helps (or hurts), but stuff like: assert outputs.get("foo.bar.baz", "default") == pytest.approx( time_recorder.time_taken, abs=0.0001) always feels a bit off and "unbalanced" to me. The opening paren doesn't have anything immediately following it, so it feels 'symmetric' that the closing paren shouldn't have anything preceding it. And also it feels like the open and closing parens should be on lines that start at the same indentation level. Honestly this I think does aid in readability a bit. > Part of the twisted beauty of PEP-008's narrow lines is that you're forced to extract (named) variables, or avoid overly indented code by extracting methods or applying higher level abstractions. This feels orthogonal? The line is wrapping either way, which might sufficiently annoy someone to extract things out a bit more. But IMO it feels like a bit of an anti-pattern to create abstractions on the basis of syntax as opposed to the structure of the program. > writing readable code isn't just about getting the spaces and brackets right... ? I mean of course not, but that's what we're talking about in the context of formatters right? I think the real, major way auto-formatters help with readability is by getting people to stop wasting mental cycles on things like spaces and brackets so that they can focus on more important code organization concerns.
glacials 5y ago
Black is slowly creeping into gofmt-level universality in the Python community and it’s great. The next big milestone is a first-party recommendation by python.org itself.
- VWWHFSfQ 5y ago
  I'm pretty sure it's a PSF project
- spc476 5y ago
  No, the next big milestone is embedding the format style as the syntax of the language. I'm curious as to why Go didn't even do this (they should have, in my opinion, but wimped out and left it to an external tool).
  - shpx 5y ago
    If they change print(repr('some string')) to print "some string" instead of 'some string' then that would remove the only hangup about Black that I have.
vitorfs 5y ago
This is such a great news. We've been using Black in the company that I work for the past 3 years or so and it was a game changer for code reviews. Hopefully other open source Python/Django projects will follow the lead.
daenz 5y ago
I'm so happy that languages are settling more and more on heavy reformatter usage. I'd like to think it was triggered by Go and gofmt. Working on a team where each engineer has their own personal syntax is not fun.
- belval 5y ago
  Indeed, I don't like Black's style, but I prefer working in a Black codebase than one where everyone has their own preference. Having style guidelines in a team is also a great way to remove pointless debates when reviewing PRs.
  - declnz 5y ago
    +1 ...which is why I wish Black allowed more configuration. A team can often agree on a set of styles. Every team on the Python planet agreeing... now that's much harder
    - heavenlyblue 5y ago
      > A team can often agree on a set of styles. What for? Just to be clear even Django itself isn't obviously using a configuration, what makes your team so special they need this?
    - harikb 5y ago
      > A team can often agree this usually just means new team members are stuck respecting the wishes of the old-timers
      - david422 5y ago
        Yea, but in this case the old timers have chosen Black so what's the difference?
    - belval 5y ago
      I disagree on that though. By sticking to vanilla Black (no pun intended) you ensure that people joining your time will probably already be familiar with the style, you prevent strongly opinionated employees from pushing for changes in the linter config. Black is opinionated, so it skips the debate entirely. We just use Black, not black with 120 characters lines, just Black. To each their own I guess, but to me it just seems like pandora's box. Once you show that you are open to changes in the linting configuration, it makes the rules mutable and pretty much guarantees that at some point someone will say "how about we just change this one parameter in the linter", which will probably be agreed by the rest of the team, not because they actually agree but because they don't want to argue.
mrtranscendence 5y ago
I've been using black at work for over a year now. I don't much care for some of the choices it makes, which can sometimes be quite ugly, but I've grown used to it and can (nearly) always anticipate how it will format code. One nice side effect of encouraging its use is how, at least where I work, it was very common to use the line continuation operator \ instead of encompassing an expression in parentheses. I always hated that and black does away with it. What I don't much care for is reorder-python-imports, which I think is related to black (but don't quote me). For the sake of reducing merge conflicts it turns the innocuous from typing import overload, List, Dict, Tuple, Option, Any into from typing import overload from typing import List from typing import Tuple from typing import Option from typing import Any Ugh. Gross. Maybe I'm just lucky but I've never had a merge conflict due to an import line so the cure seems worse than the disease. Edit: Just to be 100% clear: this is python-reorder-imports, not black. I thought they were related projects, though maybe I'm wrong. Regardless, black on its own won't reorder imports.
- magnusmundus 5y ago
  Really? I just put that exact line in a file I'm working on, and black didn't change anything. Maybe you mean in case it exceeds the line length limit, rather than that specific example. In any case, you can wrap those in parentheses, in which case black will just enforce its usual tuple formatting: single line if it fits; one line per item if not, with a trailing comma. edit: I tried it on a long line with a backslash break, and black wrapped the imports in parentheses like I suggested above. I wonder what causes the behaviour you see on your end.
  - mrtranscendence 5y ago
    No, sorry, I meant python-reorder-imports, not black. It's a separate project. I thought it was related but maybe I was wrong.
- heavenlyblue 5y ago
  Why do you even care? I never look at that part of the code. If PyCharm automatically removed/added imports without me managing them I would be a happier person.
wyuenho 5y ago
Every time I was tempted to do something like this, I hesitated because I didn't want every other line in every file with my name on a single commit, mostly to avoid making git blame harder than necessary. It would be nice if there was a kind of diffing algorithm that can diff code units *syntactically* across history.
- simonw 5y ago
  You can tell "git blame" to ignore specific commits which helps a lot here: https://www.moxio.com/blog/43/ignoring-bulk-change-commits-with-git-blame https://www.moxio.com/blog/43/ignoring-bulk-change-commits-w...
  - terr-dav 5y ago
    Here’s a script that automates the once-per-repository local setup of this feature: https://github.com/ipython/ipython/pull/12091/files https://github.com/ipython/ipython/pull/12091/files Unfortunately there isn’t support for it in GitHub or GitLab yet, but there’s at least a GitLab issue here requesting it: https://gitlab.com/gitlab-org/gitlab/-/issues/31423 https://gitlab.com/gitlab-org/gitlab/-/issues/31423
  - dmart 5y ago
    This is a nice feature, but I do wish that .git-blame-ignore-revs was automatically applied, similarly to .gitignore and .gitattributes. Hopefully there are plans to do so in a future Git release?
  - wyuenho 5y ago
    The problem with this approach is, the blame before and after the ignored wouldn’t make any sense to the viewer if he didn’t know about ignoring the formatting commit. Also, you will need to configure that for every clone. Since tree diffing algorithms are pretty well known these days, I don’t know why there hasn’t been any real effort to implement a git plugin that can chase syntax tree node changes instead of doing string diffing like it was the 70s. Syntax parsers are so easy write now and surely the tree node changes can be cached. Your usual diff/patch tooling wouldn’t work for this kind of diff, but that’s just an option away when you need them back.
ibejoeb 5y ago
In general, what are the strategies for large public codebases like this to mitigate supply chain attacks or other source-level attacks? For clarity, I'm hoping to open us discussion about how we're dealing with massive changesets like this that are difficult to review due chiefly to the breadth of it.
- sciurus 5y ago
  For a purely mechanical change like this, someone could run black against the same revision of Django and verify the changes they see locally match the changes in this PR.
  - ibejoeb 5y ago
    That's true as long as the results are predictable and reproducible. I don't happen to know if Black is, and it's not apparent from the documentation. Update: Found it: > How stable is Black’s style? > Starting in 2022, the formatting output will be stable for the releases made in the same year https://black.readthedocs.io/en/stable/faq.html https://black.readthedocs.io/en/stable/faq.html
    - Bedon292 5y ago
      The same version of black, with the same settings, will always produce the same results from the same input code. Definitely re-producible. That question is about how stable the formatting is from version to version. Which is now more stable, and why Django finally made the move.
- fritzo 5y ago
  Interesting! Can you help me imagine attack scenarios? All I can think of is: - The changeset is authored by a trusted committer but the committer's tools have been locally compromised. - The public tool itself (e.g. black) has been compromised to automatically create vulnerabilities in difficult-to-review bits of code (a Ken Thompson hack).
- jamessb 5y ago
tomp 5y ago
worst things about Black: - doesn't respect vertical space - sure, making the code fit on screen might be valuable (though the default width should be at least 120 characters, I mean we're in 2022 after all), but Black does it by blowing up the vertical space used by the code - spurious changes in commits - if you happen to indent a block, Black will cause lines to break - Black fails at its most basic premise - "avoiding manual code formatting" - because a trailing comma causes a list/function call to be split over lines regardless of width
- wodenokoto 5y ago
  > - Black fails at its most basic premise - "avoiding manual code formatting" - because a trailing comma causes a list/function call to be split over lines regardless of width Yeah, this one drives me nuts too.
  - epistasis 5y ago
    It's one of my favorite things about black, and I've started to use that formatting of function calls with long arguments for other languages too. But I also despise long lines with a passion, I hate having to go to the right, and would much much rather scroll up and down with a consistent width, so that I can put multiple views next to each other.
    - wodenokoto 5y ago
      I don’t mind the formatting, I mind that the formatting is done depending on wether the list ends with a comma or not. [ Item1, Item2 ] Is combined to one line, while [ Item1, Item2, ] Stays as multi line. Now I am once again in charge of formatting my code, by virtue of the comma. Does this stay multi line or is it short enough enough to combine? That should be for black to decide not me!
  - flightlevel180 5y ago
    If I'm understanding your problem correctly, it seems that you can avoid it by using the --skip-magic-trailing-comma option [0]. [0] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#the-magic-trailing-comma https://black.readthedocs.io/en/stable/the_black_code_style/...
SoylentOrange 5y ago
I’ve been using black for about a year and I’m generally a big fan. However my biggest gripe with it is bad VS Code integration.
- claytonjy 5y ago
  bad how? i use vscode, I save a file, it reformats on save, that's it.
euler_angles 5y ago
Had a great experience with black. Only thing I did was change its default line length limit to 120 characters (I was regularly dealing with signal names from source data that were about 90 chars).
umvi 5y ago
What's the point of putting linters into CI? Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? Or is the point to autoformat and autocommit the formatted code?
- selestify 5y ago
  > Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? It's this. Ensures that anything merged to master keeps the formatting conventions established in the project.
- seattle_spring 5y ago
  The former, in my case. Last thing I want is someone merging their own "creative interpretation" of proper formatting.
- bckr 5y ago
  > Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? It's this one > Or is the point to autoformat This one is done with pre-commit (which should probably be named pre-push?) hooks > and autocommit the formatted code? I don't think this one is done, and I think it's undesirable
  - mkesper 5y ago
    Pre-commit hooks really happen when you type 'git commit'. If you have failing checks in them, your commit will be aborted.
rowanseymour 5y ago
I love this except the use of the default black line length of 88. One of the things I appreciate about gofmt is being trusted with deciding on line breaks.
bwhmather 5y ago
Shameless plug: For people who like black, I've been working on ssort[0], a python source code sorter that will organize python statements into topological order based on their dependencies. It aims to resolve a similar source of bikeshedding and back and forth commits. 0: https://github.com/bwhmather/ssort https://github.com/bwhmather/ssort
- stjohnswarts 5y ago
  This sounds like a living hell if you use git diff a lot to compare for small changes that might introduce a bug? which is what happens at work all the time since our unit test and CI are a joke. Not dumping on your project but the idea of that much of a change up of the code scares the dickens out of me.
  - danuker 5y ago
    Once the code is initially migrated (which should not break it), the diffs won't be large, since the order should be consistent.
    - bwhmather 5y ago
      One thing worth mentioning is that the `git blame` ignore file trick doesn't work as well with ssort as it does with black because the changes ssort makes tend to be much less local.
  - drcongo 5y ago
    Use it at the editor level instead of in CI and I can't see how it can cause you any problems at all. I could easily be missing something though?
- BiteCode_dev 5y ago
  Very interesting, especially the method order part. I dislike the order you chose, and yet, I would be tempted to use it on my projects anyway, because being congruent is so important to me.
jnothing 5y ago
Why is it impossible to rebase? I didn’t understand the conversation around merging and rebasing
codingkev 5y ago
A little shoutout to a alternative Python formating tool https://github.com/google/yapf https://github.com/google/yapf (developed by Google). The built in "facebook style" formating felt by far the most natural to me with the out of the box settings and no extra config.
- timhh 5y ago
  I did a blind survey of YAPF vs Black at my work. The results came back as 70% in favour of Black. Black gives generally nicer output, and also more predictable output because its folding algorithm is simpler. YAPF uses a global optimisation which makes it make very strange decisions sometimes. Black does too, but much less often. There are also non-style problems with YAPF. It occasionally fails to produce stable output, i.e. yapf(yapf(x)) != yapf(x). In some cases it never stabilises - flip flopping between alternatives forever! Finally it seems to have very bad worst case performance. On some long files it takes so long that we have to exclude them from formatting. Black has no issue. In conclusion, don't use YAPF! Black is better in almost every way!
  - VectorLock 5y ago
    How did you perform the blind survey? Format some code with Black and YAPF and ask people which they liked better?
    - timhh 5y ago
      Yeah exactly. I had 20 samples from our codebase that showed some representative differences and you had to click on which one you liked more. The order (Black/YAPF or YAPF/Black) was randomised. I also had to turn off Black's quote normalisation otherwise it is really obvious which is which. Quote normalisation is another point in Black's favour. I could put the survey up somewhere if anyone is interested.
- BiteCode_dev 5y ago
  yapf is configurable, and that's why it never won.
SodiumMerchant0 5y ago
MahajanVardhan 5y ago
I am so sorry, but what is Black? I use django but I have never heard of Black
- rcv 5y ago
  Black is a tool that can reformat Python code. It's remarkable for it's lack of configuration. https://github.com/psf/black https://github.com/psf/black
VBprogrammer 5y ago
Reading some of the comments here it's become clear to me that the next stage in the development of auto-formatters is to have the formatter commit the code as a canonical format but to display the code to each individual contributor in the style of their choosing. Thus removing all kinds of arguments about whether 80 or 120 columns is the one true width.
- gfunk911 5y ago
  You brilliant lunatic
- dom111 5y ago
  I've been thinking about this for a while too. I think that making editors do this is within the realms of feasibility. Most support auto-formatting to your preferred style so it doesn't feel like a leap for it to format to your preferred style but keep the file on disk the project owner's preferred style. I haven't looked extensively to see if this already exists though but we chatted about this at work as I was advocating for use of prettier on a front-end project!
- michaelbarton 5y ago
  I think that’s already possible using git smudge. Example here: https://bignerdranch.com/blog/git-smudge-and-clean-filters-making-changes-so-you-dont-have-to/ https://bignerdranch.com/blog/git-smudge-and-clean-filters-m...
  - williamvds 5y ago
    Smudge & clean might do the trick, but it could be dirty. The smudge -> clean process might produce additional changes that aren't related to the purpose of your commit. Whitespace in particular could be a problem, especially where there's ambiguity in how it should be used. black isn't as bad because it has stricter rules on whitespace. Still, if you aren't checking style rules before every merge someone using smudge and clean could end up reformatting entire files. IMO the next next step is, as others have discussed on HN, getting your version control to store and abstract syntax tree. tree-sitter could make this easier nowadays, but I think it'd need more invasive changes in Git than just using the filters. See this HN thread https://news.ycombinator.com/item?id=28670372 https://news.ycombinator.com/item?id=28670372
phplovesong 5y ago
Good bye git history!
- Noumenon72 5y ago
  They used .git-blame-ignore-revs.
- yedpodtrzitko 5y ago
  hello .git-blame-ignore-revs
ReleaseCandidat 5y ago
I would really appreciate if there would exist exactly _one_ formatter (without any options) per language. It is way better to deal with ugly formatting as long as it is consistent than with discussions where to put a closing brace/bracket/paren.
NAHWheatCracker 5y ago
I suggested Black to a team I was on a year ago and one developer hemmed and hawed about how he likes to format arrays or something. I didn't win any friends by pointing out that disregarding those personal preferences is part of why I was recommending it. A year later and it seems to be the default on all projects I'm working on and I'm loving it.
- themeiguoren 5y ago
  Autoformatters are hell for 2d arrays of data where the columns have meaning and you want them to be aligned (time series, matrix math). It’s my only real gripe.
TheRealPomax 5y ago
The reason to use Black is the same as Prettier on the HTML/CSS/JS side: forever stop having an opinion on code style, it's wasted time and effort. Any "it's not exactly what we want" comment with an attempt to customize the style to be closer to "what we were already using" is exactly why these things exist: by all means have that opinion, but that's exactly the kind of opinion you shouldn't ever even need to have, tooling should style the code universally consistently "good enough". Which quotes to use, what indent to use, when to split args over multiple lines, it's all time wasted. Even if you worked on a project for 15 years, once you finally add autoformatting, buy in to it. It's going to give you a new code style, and you will never even actively have to follow it. You just need to be able to read it. Auto-formatting will do the rest.
- wraptile 5y ago
  Except Python is a general purpose programming language so it's hard to have 1 shoe fits all solution when style vary based on medium you're working with. Are you making an OOP GUI app? Django? Something that is using loads of long Xpaths?
  - yurishimo 5y ago
    I don't know if that applies. Ideally, a good code formatting tool would work with any project. If there is a specific flag you want to disable for some block to use your own format, then the tool should support that. As a couple of examples, PHP has had a unified formatting standard since 2013 and Elixir has a formatter built into the language. Both languages need the formatter to be enabled by your IDE/CI and that's also the case for Black.
  - pyuser583 5y ago
    Python throws exceptions if you don’t have the right number of indents.
wolverine876 5y ago
Do Black and other autoformatters enable significantly more reusable code and computer-generated code? Formatting is certainly not the only or greatest barrier, but if format is standardized across projects, it's easier to plug and play code from outside.

284 points by tams 5y ago

So now when you look at the annotated change history all you're going to see is a bunch of changes by the person that reformatted the code instead of the person that wrote it.

justinmchase 5y ago
You can see both of course. That's the beauty of history.
deleted 5y ago
[deleted]
tempay 5y ago
The `.git-blame-ignore-revs` file can be used to ignore that (and will be [1]). Unfortunatly GitHub doesn't support it but at least it's possible to have clients behave in a reasonable way. [1] https://github.com/django/django/pull/15387#issuecomment-1031848322 https://github.com/django/django/pull/15387#issuecomment-103...
- sciurus 5y ago
  For anyone looking for more explanation of this feature: https://michaelheap.com/git-ignore-rev/ https://michaelheap.com/git-ignore-rev/
- acidburnNSA 5y ago
  TIL about that git feature. Very nice.
- terr-dav 5y ago
  You can automate setup for developers using this simple script: https://github.com/ipython/ipython/pull/12091/files https://github.com/ipython/ipython/pull/12091/files And here’s a GitLab issue requesting support for blame-ignore: https://gitlab.com/gitlab-org/gitlab/-/issues/31423 https://gitlab.com/gitlab-org/gitlab/-/issues/31423 I don’t think there’s a corresponding GitHub request, but maybe if GitLab adds this feature GitHub will have some incentive to follow suit.
deleted 5y ago
[deleted]
Cthulhu_ 5y ago
There's workarounds that others have mentioned, but indeed, the unfortunate side-effect of deciding to apply a formatter is a 'formatting' commit, causing a lot of code churn and issues if naively using git blame. But, it's a "rip the plaster off" kinda thing, because it should ensure a lot less churn, inconsistent code style, or arguments and reviews about formatting after this is merged. It frees up a lot of headspace and distractions in code reviews. I don't know about you, but when I did code reviews I'd always end up zooming in on code style issues - ' vs ", things on newlines or no, JS objects with stringed keys, etc.
alecbz 5y ago
Uh so is your take "don't do broad refactors ever?" Beyond `.git-blame-ignore-revs` (which is neat and TIL), in GitHub's web viewer, if you find the line you're interested in and see that the most recent PR is a reformat, you click the "view blame prior to this change" button. I think most blame viewers do (or at least should) have a feature like this.

justinmchase 5y ago

The output does look better but this also just looks like every PR for applying a linter / formatter I've ever seen. Not sure why this is news worthy.

deleted 5y ago
[deleted]
owaislone 5y ago
Using black is not about how the code looks but to eliminate an entire suite of review comments/discussions. Everyone simply runs black over all code before submitting and no one ever comments about how anything is formatted.
- musingsole 5y ago
- captainmuon 5y ago
  Naive question, but why is everybody so aggravated by formatting discussions? It seems to be a widespread opinion that these discussions are just 1) pointless and 2) difficult and time consuming. My personal experience is that 1) in many cases you do benefit from taking a moment, going through your code and thinking about presentation. And 2) I find it not at all difficult to settle. A change either doesn't matter, then you just don't discuss it at all, or it is important, then you quickly agree on the best solution. (In the worst case, "best" means what the project lead finds prettier.) If you don't have a social mechanism to agree on something as basic as coding style, then your team probably has bigger problems. I actually find robo-formated code annoying to read: Go code from a bloody beginner who doesn't know what they are doing looks exactly like carefully tended for, highly thought-out code. And in autoformatted Python, you for example cannot make formulas clearer by removing spaces around operators with higher precidence. Parentheses placement is dicated by how long words are and not by what logically belongs together, etc..
  - WesolyKubeczek 5y ago

samwillis 5y ago

I believe from memory Django decided to move to using Black back in 2019 [0] but delayed the change until Black exited Beta. Black became none beta at the end of January [1]. This was finally merged to the main branch today [2]. I suspect there are lots of other both open source and private projects that are also making the change now. This is a show of confidence in Black as the standard code formatter for Python. 0: https://github.com/django/deps/blob/main/accepted/0008-black.rst https://github.com/django/deps/blob/main/accepted/0008-black... 1: https://news.ycombinator.com/item?id=30130316 https://news.ycombinator.com/item?id=30130316 2: https://github.com/django/django/pull/15387 https://github.com/django/django/pull/15387

dwightgunning 5y ago
This is right. Black emerging from beta was discussed on the Django mailing list in the last week or so, and triggered the work.

supreme_berry 5y ago

“Black” developer refused for a long time to add option to format code with single quotes with very aggressive manners. Now Django devs didn’t see that option for single quotes and code looks unpleasant.

vitorfs 5y ago
I have always used single quotes for Python code since I start working with it. When I started to adopt Black on my projects it indeed felt weird and the code looked unpleasant. But after a while you get used to it. Some people make the case that it's easier to write single quotes (well, depending on the keyboard format anyway). For keyboards in the US standard you have to hold the Shift key to write a double quote. But the good thing about Black is that you can still write your code using single quote and when you run the command line utility it will fix/normalize the code to use double quotes. Nowadays I got so used to it that I even write my Python code using double quotes. And looking at Python code using single quotes looks weird/unpleasant for me.
- valparaiso 5y ago
- digisign 5y ago
  The repl still uses single quotes.
- spc476 5y ago
  I use single quotes for items that, while technically a string, could be considered a value or symbol. For example: syslog('debug',"Just opened %s for output",filename) While there's no semantic difference between single and double quote, in my code base, there is. And if black becomes very popular, why even support single quotes anymore?
INTPenis 5y ago
I reacted to this too, in the changed files tab. Technically single or double quotes have the exact same meaning in Python. What makes people use single quotes is probably other languages like PHP, Perl and Bash. I know I've made it a habit to default to single quotes unless I know I need double quotes. So that might be where the habit comes from in the Django project. But it's not actually necessary in python so might as well use the most commonly used type of quote.

declnz 5y ago

Aside: I love a good linter, but as a long-time Python fan I find it sad that Black has so little configuration (yes, I know, but still) and moreover that it often produces code that no human Python dev I know would write... Python was always meant to look concise / beautiful... (MyPy has also made this trickier too)

alecbz 5y ago
OOC what are your grips with black's style? I generally find black pretty "beautiful" (concise maybe not as much).
- declnz 5y ago
  I guess the closing parens irk me the most e.g. assert outputs.get("foo.bar.baz", "default") == pytest.approx( time_recorder.time_taken, abs=0.0001 ) I get why it's done that, but I just don't think it helps humans read. Part of the twisted beauty of PEP-008's narrow lines is that you're forced to extract (named) variables, or avoid overly indented code by extracting methods or applying higher level abstractions. In the last few years I find devs are happier to format and push to "sort that problem out", leaving the readability benefit of that thought process lost. TL;DR writing readable code isn't just about getting the spaces and brackets right...
  - alecbz 5y ago
    I tend to prefer the trailing paren on the following line. I'm not sure if there's a principled reason it helps (or hurts), but stuff like: assert outputs.get("foo.bar.baz", "default") == pytest.approx( time_recorder.time_taken, abs=0.0001) always feels a bit off and "unbalanced" to me. The opening paren doesn't have anything immediately following it, so it feels 'symmetric' that the closing paren shouldn't have anything preceding it. And also it feels like the open and closing parens should be on lines that start at the same indentation level. Honestly this I think does aid in readability a bit. > Part of the twisted beauty of PEP-008's narrow lines is that you're forced to extract (named) variables, or avoid overly indented code by extracting methods or applying higher level abstractions. This feels orthogonal? The line is wrapping either way, which might sufficiently annoy someone to extract things out a bit more. But IMO it feels like a bit of an anti-pattern to create abstractions on the basis of syntax as opposed to the structure of the program. > writing readable code isn't just about getting the spaces and brackets right... ? I mean of course not, but that's what we're talking about in the context of formatters right? I think the real, major way auto-formatters help with readability is by getting people to stop wasting mental cycles on things like spaces and brackets so that they can focus on more important code organization concerns.

glacials 5y ago

Black is slowly creeping into gofmt-level universality in the Python community and it’s great. The next big milestone is a first-party recommendation by python.org itself.

VWWHFSfQ 5y ago
I'm pretty sure it's a PSF project
spc476 5y ago
No, the next big milestone is embedding the format style as the syntax of the language. I'm curious as to why Go didn't even do this (they should have, in my opinion, but wimped out and left it to an external tool).
- shpx 5y ago
  If they change print(repr('some string')) to print "some string" instead of 'some string' then that would remove the only hangup about Black that I have.

vitorfs 5y ago

This is such a great news. We've been using Black in the company that I work for the past 3 years or so and it was a game changer for code reviews. Hopefully other open source Python/Django projects will follow the lead.

daenz 5y ago

I'm so happy that languages are settling more and more on heavy reformatter usage. I'd like to think it was triggered by Go and gofmt. Working on a team where each engineer has their own personal syntax is not fun.

belval 5y ago
Indeed, I don't like Black's style, but I prefer working in a Black codebase than one where everyone has their own preference. Having style guidelines in a team is also a great way to remove pointless debates when reviewing PRs.
- declnz 5y ago
  +1 ...which is why I wish Black allowed more configuration. A team can often agree on a set of styles. Every team on the Python planet agreeing... now that's much harder
  - heavenlyblue 5y ago
    > A team can often agree on a set of styles. What for? Just to be clear even Django itself isn't obviously using a configuration, what makes your team so special they need this?
  - harikb 5y ago
    > A team can often agree this usually just means new team members are stuck respecting the wishes of the old-timers
    - david422 5y ago
      Yea, but in this case the old timers have chosen Black so what's the difference?
  - belval 5y ago
    I disagree on that though. By sticking to vanilla Black (no pun intended) you ensure that people joining your time will probably already be familiar with the style, you prevent strongly opinionated employees from pushing for changes in the linter config. Black is opinionated, so it skips the debate entirely. We just use Black, not black with 120 characters lines, just Black. To each their own I guess, but to me it just seems like pandora's box. Once you show that you are open to changes in the linting configuration, it makes the rules mutable and pretty much guarantees that at some point someone will say "how about we just change this one parameter in the linter", which will probably be agreed by the rest of the team, not because they actually agree but because they don't want to argue.

mrtranscendence 5y ago

I've been using black at work for over a year now. I don't much care for some of the choices it makes, which can sometimes be quite ugly, but I've grown used to it and can (nearly) always anticipate how it will format code. One nice side effect of encouraging its use is how, at least where I work, it was very common to use the line continuation operator \ instead of encompassing an expression in parentheses. I always hated that and black does away with it. What I don't much care for is reorder-python-imports, which I think is related to black (but don't quote me). For the sake of reducing merge conflicts it turns the innocuous from typing import overload, List, Dict, Tuple, Option, Any into from typing import overload from typing import List from typing import Tuple from typing import Option from typing import Any Ugh. Gross. Maybe I'm just lucky but I've never had a merge conflict due to an import line so the cure seems worse than the disease. Edit: Just to be 100% clear: this is python-reorder-imports, not black. I thought they were related projects, though maybe I'm wrong. Regardless, black on its own won't reorder imports.

magnusmundus 5y ago
Really? I just put that exact line in a file I'm working on, and black didn't change anything. Maybe you mean in case it exceeds the line length limit, rather than that specific example. In any case, you can wrap those in parentheses, in which case black will just enforce its usual tuple formatting: single line if it fits; one line per item if not, with a trailing comma. edit: I tried it on a long line with a backslash break, and black wrapped the imports in parentheses like I suggested above. I wonder what causes the behaviour you see on your end.
- mrtranscendence 5y ago
  No, sorry, I meant python-reorder-imports, not black. It's a separate project. I thought it was related but maybe I was wrong.
heavenlyblue 5y ago
Why do you even care? I never look at that part of the code. If PyCharm automatically removed/added imports without me managing them I would be a happier person.

wyuenho 5y ago

Every time I was tempted to do something like this, I hesitated because I didn't want every other line in every file with my name on a single commit, mostly to avoid making git blame harder than necessary. It would be nice if there was a kind of diffing algorithm that can diff code units *syntactically* across history.

simonw 5y ago
You can tell "git blame" to ignore specific commits which helps a lot here: https://www.moxio.com/blog/43/ignoring-bulk-change-commits-with-git-blame https://www.moxio.com/blog/43/ignoring-bulk-change-commits-w...
- terr-dav 5y ago
  Here’s a script that automates the once-per-repository local setup of this feature: https://github.com/ipython/ipython/pull/12091/files https://github.com/ipython/ipython/pull/12091/files Unfortunately there isn’t support for it in GitHub or GitLab yet, but there’s at least a GitLab issue here requesting it: https://gitlab.com/gitlab-org/gitlab/-/issues/31423 https://gitlab.com/gitlab-org/gitlab/-/issues/31423
- dmart 5y ago
  This is a nice feature, but I do wish that .git-blame-ignore-revs was automatically applied, similarly to .gitignore and .gitattributes. Hopefully there are plans to do so in a future Git release?
- wyuenho 5y ago
  The problem with this approach is, the blame before and after the ignored wouldn’t make any sense to the viewer if he didn’t know about ignoring the formatting commit. Also, you will need to configure that for every clone. Since tree diffing algorithms are pretty well known these days, I don’t know why there hasn’t been any real effort to implement a git plugin that can chase syntax tree node changes instead of doing string diffing like it was the 70s. Syntax parsers are so easy write now and surely the tree node changes can be cached. Your usual diff/patch tooling wouldn’t work for this kind of diff, but that’s just an option away when you need them back.

ibejoeb 5y ago

In general, what are the strategies for large public codebases like this to mitigate supply chain attacks or other source-level attacks? For clarity, I'm hoping to open us discussion about how we're dealing with massive changesets like this that are difficult to review due chiefly to the breadth of it.

sciurus 5y ago
For a purely mechanical change like this, someone could run black against the same revision of Django and verify the changes they see locally match the changes in this PR.
- ibejoeb 5y ago
  That's true as long as the results are predictable and reproducible. I don't happen to know if Black is, and it's not apparent from the documentation. Update: Found it: > How stable is Black’s style? > Starting in 2022, the formatting output will be stable for the releases made in the same year https://black.readthedocs.io/en/stable/faq.html https://black.readthedocs.io/en/stable/faq.html
  - Bedon292 5y ago
    The same version of black, with the same settings, will always produce the same results from the same input code. Definitely re-producible. That question is about how stable the formatting is from version to version. Which is now more stable, and why Django finally made the move.
fritzo 5y ago
Interesting! Can you help me imagine attack scenarios? All I can think of is: - The changeset is authored by a trusted committer but the committer's tools have been locally compromised. - The public tool itself (e.g. black) has been compromised to automatically create vulnerabilities in difficult-to-review bits of code (a Ken Thompson hack).
jamessb 5y ago

tomp 5y ago

worst things about Black: - doesn't respect vertical space - sure, making the code fit on screen might be valuable (though the default width should be at least 120 characters, I mean we're in 2022 after all), but Black does it by blowing up the vertical space used by the code - spurious changes in commits - if you happen to indent a block, Black will cause lines to break - Black fails at its most basic premise - "avoiding manual code formatting" - because a trailing comma causes a list/function call to be split over lines regardless of width

wodenokoto 5y ago
> - Black fails at its most basic premise - "avoiding manual code formatting" - because a trailing comma causes a list/function call to be split over lines regardless of width Yeah, this one drives me nuts too.
- epistasis 5y ago
  It's one of my favorite things about black, and I've started to use that formatting of function calls with long arguments for other languages too. But I also despise long lines with a passion, I hate having to go to the right, and would much much rather scroll up and down with a consistent width, so that I can put multiple views next to each other.
  - wodenokoto 5y ago
    I don’t mind the formatting, I mind that the formatting is done depending on wether the list ends with a comma or not. [ Item1, Item2 ] Is combined to one line, while [ Item1, Item2, ] Stays as multi line. Now I am once again in charge of formatting my code, by virtue of the comma. Does this stay multi line or is it short enough enough to combine? That should be for black to decide not me!
- flightlevel180 5y ago
  If I'm understanding your problem correctly, it seems that you can avoid it by using the --skip-magic-trailing-comma option [0]. [0] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#the-magic-trailing-comma https://black.readthedocs.io/en/stable/the_black_code_style/...

SoylentOrange 5y ago

I’ve been using black for about a year and I’m generally a big fan. However my biggest gripe with it is bad VS Code integration.

claytonjy 5y ago
bad how? i use vscode, I save a file, it reformats on save, that's it.

euler_angles 5y ago

Had a great experience with black. Only thing I did was change its default line length limit to 120 characters (I was regularly dealing with signal names from source data that were about 90 chars).

umvi 5y ago

What's the point of putting linters into CI? Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? Or is the point to autoformat and autocommit the formatted code?

selestify 5y ago
> Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? It's this. Ensures that anything merged to master keeps the formatting conventions established in the project.
seattle_spring 5y ago
The former, in my case. Last thing I want is someone merging their own "creative interpretation" of proper formatting.
bckr 5y ago
> Is the point to fail the build if the code wasn't pre-formatted with i.e. Black? It's this one > Or is the point to autoformat This one is done with pre-commit (which should probably be named pre-push?) hooks > and autocommit the formatted code? I don't think this one is done, and I think it's undesirable
- mkesper 5y ago
  Pre-commit hooks really happen when you type 'git commit'. If you have failing checks in them, your commit will be aborted.

rowanseymour 5y ago

I love this except the use of the default black line length of 88. One of the things I appreciate about gofmt is being trusted with deciding on line breaks.

bwhmather 5y ago

Shameless plug: For people who like black, I've been working on ssort[0], a python source code sorter that will organize python statements into topological order based on their dependencies. It aims to resolve a similar source of bikeshedding and back and forth commits. 0: https://github.com/bwhmather/ssort https://github.com/bwhmather/ssort

stjohnswarts 5y ago
This sounds like a living hell if you use git diff a lot to compare for small changes that might introduce a bug? which is what happens at work all the time since our unit test and CI are a joke. Not dumping on your project but the idea of that much of a change up of the code scares the dickens out of me.
- danuker 5y ago
  Once the code is initially migrated (which should not break it), the diffs won't be large, since the order should be consistent.
  - bwhmather 5y ago
    One thing worth mentioning is that the `git blame` ignore file trick doesn't work as well with ssort as it does with black because the changes ssort makes tend to be much less local.
- drcongo 5y ago
  Use it at the editor level instead of in CI and I can't see how it can cause you any problems at all. I could easily be missing something though?
BiteCode_dev 5y ago
Very interesting, especially the method order part. I dislike the order you chose, and yet, I would be tempted to use it on my projects anyway, because being congruent is so important to me.

jnothing 5y ago

Why is it impossible to rebase? I didn’t understand the conversation around merging and rebasing

codingkev 5y ago

A little shoutout to a alternative Python formating tool https://github.com/google/yapf https://github.com/google/yapf (developed by Google). The built in "facebook style" formating felt by far the most natural to me with the out of the box settings and no extra config.

timhh 5y ago
I did a blind survey of YAPF vs Black at my work. The results came back as 70% in favour of Black. Black gives generally nicer output, and also more predictable output because its folding algorithm is simpler. YAPF uses a global optimisation which makes it make very strange decisions sometimes. Black does too, but much less often. There are also non-style problems with YAPF. It occasionally fails to produce stable output, i.e. yapf(yapf(x)) != yapf(x). In some cases it never stabilises - flip flopping between alternatives forever! Finally it seems to have very bad worst case performance. On some long files it takes so long that we have to exclude them from formatting. Black has no issue. In conclusion, don't use YAPF! Black is better in almost every way!
- VectorLock 5y ago
  How did you perform the blind survey? Format some code with Black and YAPF and ask people which they liked better?
  - timhh 5y ago
    Yeah exactly. I had 20 samples from our codebase that showed some representative differences and you had to click on which one you liked more. The order (Black/YAPF or YAPF/Black) was randomised. I also had to turn off Black's quote normalisation otherwise it is really obvious which is which. Quote normalisation is another point in Black's favour. I could put the survey up somewhere if anyone is interested.
BiteCode_dev 5y ago
yapf is configurable, and that's why it never won.

SodiumMerchant0 5y ago

MahajanVardhan 5y ago

I am so sorry, but what is Black? I use django but I have never heard of Black

rcv 5y ago
Black is a tool that can reformat Python code. It's remarkable for it's lack of configuration. https://github.com/psf/black https://github.com/psf/black

VBprogrammer 5y ago

Reading some of the comments here it's become clear to me that the next stage in the development of auto-formatters is to have the formatter commit the code as a canonical format but to display the code to each individual contributor in the style of their choosing. Thus removing all kinds of arguments about whether 80 or 120 columns is the one true width.

gfunk911 5y ago
You brilliant lunatic
dom111 5y ago
I've been thinking about this for a while too. I think that making editors do this is within the realms of feasibility. Most support auto-formatting to your preferred style so it doesn't feel like a leap for it to format to your preferred style but keep the file on disk the project owner's preferred style. I haven't looked extensively to see if this already exists though but we chatted about this at work as I was advocating for use of prettier on a front-end project!
michaelbarton 5y ago
I think that’s already possible using git smudge. Example here: https://bignerdranch.com/blog/git-smudge-and-clean-filters-making-changes-so-you-dont-have-to/ https://bignerdranch.com/blog/git-smudge-and-clean-filters-m...
- williamvds 5y ago
  Smudge & clean might do the trick, but it could be dirty. The smudge -> clean process might produce additional changes that aren't related to the purpose of your commit. Whitespace in particular could be a problem, especially where there's ambiguity in how it should be used. black isn't as bad because it has stricter rules on whitespace. Still, if you aren't checking style rules before every merge someone using smudge and clean could end up reformatting entire files. IMO the next next step is, as others have discussed on HN, getting your version control to store and abstract syntax tree. tree-sitter could make this easier nowadays, but I think it'd need more invasive changes in Git than just using the filters. See this HN thread https://news.ycombinator.com/item?id=28670372 https://news.ycombinator.com/item?id=28670372

phplovesong 5y ago

Good bye git history!

Noumenon72 5y ago
They used .git-blame-ignore-revs.
yedpodtrzitko 5y ago
hello .git-blame-ignore-revs

ReleaseCandidat 5y ago

I would really appreciate if there would exist exactly _one_ formatter (without any options) per language. It is way better to deal with ugly formatting as long as it is consistent than with discussions where to put a closing brace/bracket/paren.

NAHWheatCracker 5y ago

I suggested Black to a team I was on a year ago and one developer hemmed and hawed about how he likes to format arrays or something. I didn't win any friends by pointing out that disregarding those personal preferences is part of why I was recommending it. A year later and it seems to be the default on all projects I'm working on and I'm loving it.

themeiguoren 5y ago
Autoformatters are hell for 2d arrays of data where the columns have meaning and you want them to be aligned (time series, matrix math). It’s my only real gripe.

TheRealPomax 5y ago

The reason to use Black is the same as Prettier on the HTML/CSS/JS side: forever stop having an opinion on code style, it's wasted time and effort. Any "it's not exactly what we want" comment with an attempt to customize the style to be closer to "what we were already using" is exactly why these things exist: by all means have that opinion, but that's exactly the kind of opinion you shouldn't ever even need to have, tooling should style the code universally consistently "good enough". Which quotes to use, what indent to use, when to split args over multiple lines, it's all time wasted. Even if you worked on a project for 15 years, once you finally add autoformatting, buy in to it. It's going to give you a new code style, and you will never even actively have to follow it. You just need to be able to read it. Auto-formatting will do the rest.

wraptile 5y ago
Except Python is a general purpose programming language so it's hard to have 1 shoe fits all solution when style vary based on medium you're working with. Are you making an OOP GUI app? Django? Something that is using loads of long Xpaths?
- yurishimo 5y ago
  I don't know if that applies. Ideally, a good code formatting tool would work with any project. If there is a specific flag you want to disable for some block to use your own format, then the tool should support that. As a couple of examples, PHP has had a unified formatting standard since 2013 and Elixir has a formatter built into the language. Both languages need the formatter to be enabled by your IDE/CI and that's also the case for Black.
- pyuser583 5y ago
  Python throws exceptions if you don’t have the right number of indents.

wolverine876 5y ago

Do Black and other autoformatters enable significantly more reusable code and computer-generated code? Formatting is certainly not the only or greatest barrier, but if format is standardized across projects, it's easier to plug and play code from outside.