5 ms·

The tradeoff here is not being able to use a universal set of tooling to interact with source files. Anything but text makes grep, diff, sed, and version contro

• Avshalom 1y ago
  The entire OS was built around these source files. the unix philosophy on the other hand only "thrives" if every other tool is designed around (and contains code to parse) "plain text"
  • lmm 1y ago
    > The entire OS was built around these source files. And how did that work out for them? This seems like one of the many cases where unix won out by being a lowest common denominator. Every platform can handle plain text.
    • aleph_minus_one 1y ago
      > This seems like one of the many cases where unix won out by being a lowest common denominator. The lowest common denominator rather is binary blobs. :-)
      • thfuran 1y ago
        The conversion of which to text and back has historically proven rather fraught.
    • account42 1y ago
      Not all platforms come with powerful text handling tools out of the box - or at least they didn't used to until Unix-based systems forced them to catch up.
• jsharpe 1y ago
  Exactly. This idea comes up time and time again, but the cost/benefit just doesn't make sense at all. You're adding an unbelievable amount of complex tooling just to avoid running a simple formatter. The goal of having every developer viewing the code with their own preferences just isn't that important. On every team I've been on, we just use a standard style guide, enforced by formatter, and while not everyone agrees with every rule, it just doesn't matter. You get used to it. Arguing and obsessing about code formatting is simply useless bikeshedding.
  • scubbo 1y ago
    I disagree with almost every choice made by the Go language designers, but `Gofmt's style is no one's favorite, yet gofmt is everyone's favorite` is solid. Pick a not-unreasonable standard, enforce it, and move on to more important things.
    • spyspy 1y ago
      My only complaint about gofmt is that it’s not even stricter about some things.
      • duskwuff 1y ago
        Good news: there are tools like https://github.com/mvdan/gofumpt https://github.com/mvdan/gofumpt which fork gofmt and enforce stricter rules (while remaining invariant under gofmt).
  • rbits 1y ago
    Yeah it would probably be a waste of time. It's a nice idea to dream about though. It would be nice to be able to look at some C# code and not have opening curly brackets on a separate line.
    • mdaniel 1y ago
      I say this fully cognizant of the thread in which it's posted, but these people are sick https://astyle.sourceforge.net/astyle.html#_style=whitesmith https://astyle.sourceforge.net/astyle.html#_style=whitesmith And then someone said: oh yeah? Hold my beer https://astyle.sourceforge.net/astyle.html#_style=pico https://astyle.sourceforge.net/astyle.html#_style=pico
• cowsandmilk 1y ago
  How is diff less effective? I see the diff in the formatting I prefer? With sed, I can project the source into a formatting most convenient for what I’m trying to do with sed. And I have no idea what you’re on about version control. It ruins sending patch files that require a line number around, but most places don’t do that any more. What I would be curious on is tracing from errors back to the source code. Nearly every language I’ve used prints line number and offset on the line for the error. How that worked in the Diana world would be interesting to learn.
  • sublinear 1y ago
    You'd have to run diff and sed before the formatter which is harder for everyone.
    • peanball 1y ago
      I can only recommend difftastic[1], which is a language aware diff. Independent of linter that shows the logical diff, not an assortment of characters or lines that changed. [1]: https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic
• accelbred 1y ago
  What if the common intermediate encoding is text, not binary? Then grep/diff/sed all still work. If we had a formatting tool that operated solely on AST, checked in code could be in a canonical form for a given AST. Editors could then parse the AST and display the source with a different formatting of the users choice, and convert to canonical form when writing the file to disk.
  • sublinear 1y ago
    Nobody wants to have to run their own formatter rules in reverse in their head just to know what to grep for. That defeats the point of formatting at all.
    • pwdisswordfishz 1y ago
      That's why you grep for a syntactic structure, not undifferentiated text.
      • michaelmrose 1y ago
        Which grep doesn't do and you need to either use a new different tool or more likely several for little real benefit
        • jitl 1y ago
          comby is fantastic, give it a shot. It’s saved me huge amounts of time.
        • hnlmorg 1y ago
          grep is half a century old now. If we can’t progress our ecosystem because we are reliant on one very specific 50+ year old line parser, then that says more about the inflexibility of the industry to move forward than it does about the “new” ideas being presented.
          • komali2 1y ago
• charcircuit 1y ago
  In practice how many tools do you really need to handle the custom format? Probably single digits and they could all use a common library to handle the formatting aspect of things.
• bee_rider 1y ago
  Is it possible converted from the DIANA ir back to something that looks like source code? Then the result of the conversion backward could be grepped, etc…
  • teo_zero 1y ago
    From TFA: > Everyone had their own pretty-printing settings for viewing [DIANA] however they wanted.
    • bee_rider 1y ago
      > Back when he was working on Ada, they didn't store text sources at all — they used an IR called DIANA. Everyone had their own pretty-printing settings for viewing it however they wanted. I’m still confused because the specifically call the IR DIANA, and they talk about viewing the IR. It isn’t clear to me if the IR is more like a bytecode or something, or more like just the original source code with a little processing done to it. They also have a quote, > Grady Booch summarizes it well: R1000 was effectively a DIANA machine. We didn't store source code: source code was simply a pretty-printing of the DIANA tree. So maybe the other visualizations they could do by transforming the IR were so nice that nobody even cared to look at the original ADA that they’d written to generate it?
      • brabel 1y ago
        I imagine it’s like storing JVM bytecode, ie class files instead of Java files. So when you open it up the editor decompiles it , like IntelliJ does if you try to open a class file, but then it also applies your own style, like from .editorconfig, on the code it shows. It’s a really good idea and I can’t believe people here are complaining that it’s bad because they can’t use grep! But that’s a good thing!! Who the hell is grepping code as if code had no structure and that’s the best you can do? So you also grep JSON instead of using jq? Just don’t!
• froh 1y ago
  yes, contemporary editors and tools like treesitter have decided this debate in favor of plain text file representation, exactly for the reasons you give: universal accessibility by general purpose tools. xslt was a Diana like pre-parsed representation of dsssl. oh how I miss dsssl (a scheme based sgml transformation language) but no. dsssl was a lisp! with hygienic macros! "ikes" they went and invented XSLT. the "logic" escapes me to this day. no. plain text it is. human readable. and grep/sed/diff able.
• danielheath 1y ago
  If you’re going to store the source in a canonical format and unpack that to suit each developer… why should the canonical format just be regular source code? All the same tools can exist with a text backend, and you get grep/sed support for free too!
  • giveita 1y ago
    My grep may not work on your settings for the same code. This becomes an issue with say CI where maybe I add a gate to check something with grep. But whose format do I assume? My local (that I used to test it locally) or the canonical (which means I need to switch local format to test it)?
    • treadmill 1y ago
      You're misunderstanding the idea I think. You would use the format on disk for the grep. "Your format" only exists displayed in your editor.
      • giveita 1y ago
        Aha
    • brabel 1y ago
      You really rely on grep on CI? How fragile is that ?! This is a good argument for storing non-text. Grepping code is laughably unreliable. The only way to write things like that reliably is by actually parsing the code and working in its AST. Working in text is like writing code in a completely untyped language. It can be done, but it’s beyond stupid for anything where accuracy matters.
  • psychoslave 1y ago
    That’s seems like a genious remark actually. If you store the abstract objects and have the mechanism to transform to whatever the desired output form is, it’s almost trivial to expose a version as files and text rendering for tools that are thus oriented, isn’t it?
• eviks 1y ago
  > If initial space width was configurable in their editor of choice, would those who prefer tabs have any other arguments? Yes, of course, because tab width is * dynamically* flexible, so initial space width isn't enough
  • pasc1878 1y ago
    Yes because if you want to deindent with tabs it is just delete one character whilst spaces requires you top delete x characters where x is the number of spaces you indent by.
    • eviks 1y ago
      For "clean-fixed-width" unambiguous indent (eg, at the beginning of lines) you can make delete also delete X=indent_width spaces. But for "dirty-width" indents, eg, after some text that can vary in size (proportional fonts or some special chars even in fixed fonts) you can't align with spaces while a tab width can be auto-adjusted to match the other line
• MyOutfitIsVague 1y ago
  The way I envision this working is with something like git filters. Checking out from version control converts it all into text in your preferred formatting, which you then work with as expected. Staging it converts it into the stored representation. In git, this would be done with smudge and clean filters, like how git LFS works. You'd also have viewers for forges and the like that are built to interpret all the stored representations as needed. You still work with text, the text just isn't the canonical stored representation. You get diffs to resolve only when structure is changed. You get most of the same benefit with a pre-commit linter hook, though.
  • bapak 1y ago
    This is it, unfortunately git is "too dumb" for this. In order to merge code, it would have to either understand the AST. What happens when you stage the line `} else return {`? git doesn't allow to stage specific AST nodes. It would also mean that you can't stage partial code (that produces syntax errors)
    • Hendrikto 1y ago
      Smudge and clean filters work on text, git would not need to change at all. You would still store text, and still check out text, just transformed text. You could still check in anything you want, including partial code, syntax errors, or any other arbitrary text. Diffs would work the same way they do now.
    • zokier 1y ago
      Git can use arbitrary merge (and diff) tools. Something like https://mergiraf.org/introduction.html https://mergiraf.org/introduction.html works with git and gets you ast aware merging. Do not underestimate gits flexibility.
  • account42 1y ago
    Please no, git trying to automatically "correct" \n vs \r\n line endings is already horrible enough. At least you can turn that off.
  • 1y ago
• rendaw 1y ago
  Grep, diff, sed, and line-based non-semantic merge are all terrible tools for manipulating code... rather than dig ourselves in either further with those maybe a reason to come up with something better would be good.
• aleph_minus_one 1y ago
  > Anything but text makes grep, diff, sed, and version control less effective. Perhaps this is rather a design mistake in how UNIX handles things and is so focused on text.
• gr__or 1y ago
  Text surely is a hill, but I believe it's a local one, we got stuck on due to our short-sighted inability to go into a valley for a few miles until we find the (projectional) mountain. All of your examples work better for code with structural knowledge: - grep: symbol search (I use it about 100x as often as a text grep) or https://github.com/ast-grep/ast-grep https://github.com/ast-grep/ast-grep - diff: https://semanticdiff.com https://semanticdiff.com (and others), i.e.: hide noisy syntax only changes, attempt to capture moved code. I say attempt, because with projectional programming we could have a more expressive notion of code being moved - sed: https://npmjs.com/package/@codemod/cli https://npmjs.com/package/@codemod/cli - version control: I'd look towards languages like Unison to see what funky things we could do here, especially for libraries. A general example: no conflicts due to non-semantic changes (re-orderings, irrelevant whitespaces, etc.)
  • zokier 1y ago
    But as the tools you link demonstrate, having "text" as the on-disk format does not preclude AST based (or even smarter) tools. So there is little benefit in having non-text format. Ultimately it's all just bytes on disk
    • gr__or 1y ago
      Even that is not without its cost. Most of these tools are written in different languages, which all have to maintain their own parsers, which have to keep up with language changes. And there are abilities we lose completely by making text the source of truth, like a reliable version control for "this function moved to a new file".
      • theamk 1y ago
        At least the parsers are optional now - you can still grep, diff, etc.. even if your tools have no idea about language's semantics. But if you store ASTs, you _have_ to have the support of each of the language for each of the tools (because each language has its own AST). This basically means a major chicken-and-egg problem - a new language won't be compatible with any of the tools, so the adoption will be very low until the editor, diff, sed etc.. are all updated.. and those tools won't be updated until the language is popular. And you still don't get any advantages over text! For example, if you really cared about "this function moved to new file" functionality, you could have unique id after each function ("def myfunc{f8fa2bdd}..."), and insert/hide them in your editor. This way the IDE can show nice definition, but grep/git etc.. still work but with extra noise. In fact, I bet that any technology that people claim requires non-readable AST files, can be implemented as text for many extra upsides and no major downsides (with the obvious exception of truly graphical things - naive diffs on auto-generated images, graphs or schematics files are not going to be very useful, no matter what kind of text format is used) Want to have each person see it's own formatting style? Reformat to person's style on load and format back to project style on save. Modern formatters are so fast, people won't even notice this. Want fast semantic search? Maintain the binary cache files, but use text as source-of-truth. Want better diff output? Same deal, parse and cache. Want to have no files, but instead have function list and edit each one directly, a la Smalltalk? Maintain files transparently with text code - maybe one file per function, or one file per class, or one per project... The reason people keep source code as text as it's really a global maximum. The non-text format gives you a modest speedup, but at the expense of imposing incredible version compatibility pain.
• Ygg2 1y ago
  > would those who prefer tabs have any other arguments? Yes. Because Yaml exists. And mixing tabs and spaces is horrible in it. And the rules are very finnicky. Optimal tab usage is emit 2-4 spaces.