13 ms·

I do kind of love the idea of Git using ASTs instead of source code. It makes a ton of sense. Even just in the immediate term I wish I could make Git(hub) tabs

• enriquto 5y ago
  [whispers] don't give up! There's quite a bunch of us. Our day will come! Long live glorious tabs!
  • silon42 5y ago
    I'm fine with using tabs, but my tab width will be set to 8... be sure to obey line length limits with that in mind.
    • jcelerier 5y ago
      anything beyond 2 is heresy, and some days I'm tempted to go down to 1
      • enriquto 5y ago
        Heretic! From the book of Linus [0], chapter one: > Tabs are 8 characters, and thus indentations are also 8 characters. There are heretic movements that try to make indentations 4 (or even 2!) characters deep, and that is akin to trying to define the value of PI to be 3. [0] https://www.kernel.org/doc/html/latest/process/coding-style.html https://www.kernel.org/doc/html/latest/process/coding-style....
        • jcelerier 5y ago
          of course PI isn't 3, it's 1 (from a distance)
          • a1369209993 5y ago
            Only if you're a cosmologist.[0] 0: http://xkcd.com/2205/ http://xkcd.com/2205/
        • klyrs 5y ago
          Cursed April fools update: tabs are now π spaces wide.
        • giomasce 5y ago
          Math trivia: there are cases on which it is sensible, in sufficiently advanced mathematics, to define pi as 3 (or whatever other number). I don't use tabs, but if I'd say that the biggest advantage of using tabs is that everybody can configure their own editor to make them as large as they wish.
      • mellavora 5y ago
        Three shall be the number of the counting and the number of the counting shall be three. Four shalt thou not count, neither shalt thou count two, excepting that thou then proceedeth to three. Five is right out.
        • deleted 5y ago
          [deleted]
    • encryptluks2 5y ago
      There are no line limits. That is what word wrapping is for.
• convolvatron 5y ago
  having presentation by flexible and different than the underlying model is a great idea for code but admit it, tabs are fragile and a pretty weak implementation
  • wutbrodo 5y ago
    > admit it, tabs are fragile and a pretty weak implementation Could you elaborate? I don't have a personal opinion here and have only worked in orgs that require spaces, but I'm not familiar with the criticisms of tabs.
    • gregmac 5y ago
      For me the problem happens as soon as tabs are used for alignment, instead of just indent. The benefit of tabs is custom tabstop. If anyone does anything that undermines that benefit, you might as well use spaces to avoid all the problems caused. Consider the following code: if (x) { SomeMethod(paramater1, paramater2, parameter3); } If done "properly", it is: if (x) { <tab>SomeMethod(paramater1, <tab><spaces...>paramater2, <tab><spaces...>parameter3); } What I often see, that totally breaks the entire point of tabs: if (x) { <tab>SomeMethod(paramater1, <tab><tab><tab><space><space>paramater2, <tab><tab><tab><space><space>parameter3); } The same thing happens if you are trying to align table-style code: var badMixedTypeArrayExample = [ [ "some", true, 128, x ], [ "long strings", true, 8, someLongVariable ], [ "and", false, 16384, x ], [ "short", true, 12345678, anotherVariable ], ]; If tabs are used between fields, it will look like a hot mess to anyone with a different tabstop than the author.
      • cool_scatter 5y ago
        Which is the reason for the very common stance "tabs for indentation, spaces for alignment".
        • 5y ago
• williamdclt 5y ago
  It's not that you're going too far, it's that you're not going far enough! It's not a Git question, it's a programming language question. There's no reason source code need to be stored as plain text[1]! Editors show it as text, we edit it as text, but why wouldn't it be _stored_ as an AST? Not only does formatting becomes an editor concern, but code could even be edited as a tree, as a graph, as whatever you want [1] - well, actually there's plenty of reasons: chiefly because plaintext is very interoperable
  • jerf 5y ago
    "but why wouldn't it be _stored_ as an AST?" It profoundly is. You can't store "an AST". You can only store a serialization of it. The official language grammar is a serialization of the AST custom crafted for that language. It is as much an "AST" as any other serialization would be; all such alternative representations would all produce isomorphic memory representations if parsed from a proper library. At a high level it may sound useful to try to then provide a cross-language AST representation, but it's one of those things that sounds great at a high level but as soon as you actually tried to implement it for, say, Python and C++, you'd rapidly discover that in practice there's not as much opportunity for "generic AST operations" as you may think. The problem isn't that it isn't "stored as an AST" but that $YOUR_LANGUAGE apparently doesn't have good libraries or mechanisms for getting at it. Go, for instance, ships with the relevant bits of the compiler exposed, and as a result there are tons of tools that operate on Go code as ASTs and not textually, because it's readily available and supported by the core language team. I use this only as an example I know personally, there are other languages that have similar sorts of support as well.
    • vlovich123 5y ago
      I feel like you're picking a strawman here. The AST serialization everyone is implying is one where you don't need to token/lex but can just load it directly & manipulate it (i.e. implying the on-disk version is a valid AST or one who's validity can be trivially validated without needing to have the entire language syntax & grammar). First, that makes the compiler much faster because tokenization/lexing is moved to the "save" phase which happens infrequently at human scale vs the compile/processing phase which happens in an automated fashion where the overhead can be notable. Additionally, if you mmap the AST from disk into memory, you can use finer-grained caching to memoize expensive analysis that happens for faster compiles of code that's only changed slightly (e.g. changing whitespace/comments wouldn't recompile anything). More importantly for advocates, it avoids needing to ship the deserialization library and makes tooling simpler. That's really why the idea of a simple AST format is so attractive. Typically compiler frontends are typically very tightly coupled to the underlying middle & back end. There's some work in some languages to decouple this (e.g. LSPs & Idea's failable parsing approach), but the efforts are still very immature & it's still not clear to me that it's worth it (see the last paragraph). The main underlying challenge with making sure the on-disk contents is well-formed according to the syntax rules is that frequently you want to pause work at an intermediate stage. This means you either have to make sure that whatever state the user saves is a valid AST via editor tricks (although I think this also typically means you have to design the language around it), you reject saves, every tooling library has to be capable of parsing malformed ASTs, or you save a dirty transformation to apply to the last known saved version so you can have the user resume editing but otherwise tooling uses the "last known good" version. That's the real challenge with having a serialized version that's amenable to 3p tooling for interop. Finally, all the "serialize the AST" solutions ignore the problem of wanting to grep the codebase. This means you need to change out several decades of line-oriented manipulation tools in favor of new ones that are AST-based & likely more complicated to write/maintain as compared with one-line regular expressions. At least I've yet to see any AST manipulation libraries that aren't drastically different from existing text manipulation tools if clang-tidy and Rust macros are any indication about what good solutions to the problem look like today. I think eventually we'll get AST serialization, but I think it will be packaged into an entirely new language (like Rust did with ownership) that also considers the tooling aspect end-to-end rather than as a retrofit into existing languages. Once that's successful, then I think we'll see retrofits because the space will have been better explored & other languages will benefit from the R&D into what a successful path would look like.
• BiteCode_dev 5y ago
  Yes, but only if it falls back to text diff as soon as there is the smallest doubt it can't provide a good AST diff.
• thrwyoilarticle 5y ago
  You can also write git hooks to turn their spaces into your tabs & vice versa.
  • OJFord 5y ago
    That's not a good solution - every commit with an author (well technically committer I suppose) whose opinion differs to the last will be horrendous.
    • thrwyoilarticle 5y ago
      There won't be any difference. OP will run the script when they checkout, work with their tabs, then run the script when they commit. Spaces in, spaces out.
      • OJFord 5y ago
        Oh ok, sure. But then that's just a weak version of what's being requested - an entirely neutral more agnostic, abstract format that stores the meaning without any formatting at all.
• fstrthnscnd 5y ago
  Tabs do work as long as they aren't fixed width (I don't know what you mean by "custom"). For instance, in many languages, one will sometimes have to split a function call to many lines, and in most languages function names aren't of fixed length, thus in order to get a correct alignment for parameters, the tab width at that point will have to match the function name length. #include<stdio.h> int main(int argc, char* argv[]) { printf("%s %s %s %s\n", __FILE__, __LINE__, __DATE__, __TIME__); return 0; } I agree with your idea of storing a normalized version of the code in the repo: it wouldn't then matter whether that version contains characters to align the code properly, it would just be inserted by the editor/linter as needed. The difficulty is that sometimes linting isn't enough, and some manual formatting is needed. Or perhaps the formatting rules are under specified? Another issue with AST diffing is when languages allow some form of syntactic sugar as preprocessing: the compiler might just see the simplified tree, not the one with the "sugary" forms. A tool capable of parsing such languages should also be able to handle these extensions.
  • Asraelite 5y ago
    > the tab width at that point will have to match the function name length. This is a non-issue. Use tabs for indentation and spaces for alignment.
    • njharman 5y ago
      That is the kind of problem solution that ends up with you now having 2 problems. New problem(s); having tabs and spaces, having to think when to use them, having to train/document everyone in usage, having to debate that usage, having to correct code and chastise people who get usage wrong. Use a automatic code formatter with minimal options. Automate either running code formatter on commit or denying commits that change when code formatter is run on them.
      • Asraelite 5y ago
        Absolutely, I wouldn't dream of doing any kind of fancy alignment by hand, only with an auto-formatter. If I had to break arguments onto multiple lines without an auto-formatter I would just keep it simple and use another level of indentation instead of aligning them with the function name.
• Anon_troll 5y ago
  The whitespace and formatting are not significant to the compiler, but they can provide a lot of information to the reader of the code. You can often see where the writer put the most effort and thought by just seeing how they wrote it. This can help analyzing a codebase considerably. If everything is normalized, you lose those valuable cues.
• geofft 5y ago
  One of the practical issues here is, if your code fails to compile in CI with an error like /home/ci/src/foo.c:123:45: error: use of undeclared identifier 'a' or /home/ci/src/bar.py:50: syntax error in type comment or crashes in production with an error like java.lang.NullPointerException at com.example.Baz.doThings(Baz.java:1337) you really want to be able to find line 123 column 45, line 50, or line 1337 in your editor, and have that be the same line as what your CI compiled and deployed. On its own, tabs vs. spaces only affects columns, and you can probably figure things out without columns (although it's a shame to lose it). But different tab sizes affect how long your lines are, and line wrapping is a thing that people care about at least as much as tabs vs. spaces (people with different size monitor or fonts will easily see too-long or too-short lines on their display; if your spaces are equivalent to the tab stop, the distinction is literally invisible). And once you start rewrapping lines, everyone's line numbers are different. I think it's possible to solve this by using some sort of AST-based index into the file and teaching IDEs to let you seek based on that, but it's suddenly a more complex problem.
  • ratww 5y ago
    This is already a very common problem with a solution: transpiled JS already needs source maps to display errors correctly.
    • geofft 5y ago
      No, I don't think that's the same problem / the same solution. A source map translates between a layout checked into the code and a format generated at build time. I'm talking about translating between a layout in a developer's local workspace and the layout checked into the code. Since the developer can choose whatever formatting options they want, there isn't a single source map that can be referenced in the compiled version of the code, so backtraces etc. So the transformation cannot be done at the point the error is displayed (compiler output or backtrace output), it has to be done in the context of the developer's local workspace. I think source maps could probably be inspiration for solving this problem, but I don't think they would work directly - and even if they did, the real problem here is not designing a solution, it's getting everyone's IDEs to work properly with it. Source maps work largely because the major browsers know how to deal with source maps in JS. You'd have to extend this to all the other ecosystems, at the very least.
• thefreeman 5y ago
  If you append `?w=1` to the diff view URL on a pull request it makes it whitespace agnostic just FYI
• pbiggar 5y ago
  fwiw, this is what we do in Dark [1]. We store (serialized) ASTs, then then we pretty print them in the editor. This converts the AST into tokens that you see on your screen, complete with configurable* indentation, line-length, etc. Code would be displayed according to your config* and the same code displayed differently to a different developer looking at the same code. [1] https://darklang.com https://darklang.com * I haven't actually enabled users to configure this, but it's just some variables called 'indent' and `lineLength` in the code