4 ms·
I’ve wondered before whether the world would be well served by a programming language (or source control system, I suppose) that just stores ASTs in files rathe
by afavour 2y ago
I’ve wondered before whether the world would be well served by a programming language (or source control system, I suppose) that just stores ASTs in files rather than text code. When users open the file the editor formats to whatever their personal preference is, then saves edits back to the AST.
It really is dumb to be arguing over tabs vs spaces, after all.
- rsc 2y agoAt that point you could even use different languages. Maybe you like programs that look like Lisp and I don't. There was a project at Microsoft Research in the late 1990s/early 2000s that did exactly this - storing ASTs in source control instead of code - but the name escapes me at the moment.
- whstl 2y agoWas it Intentional programming? I remember it being described as something similar to what you say, but the Wikipedia shows something slightly different. https://en.wikipedia.org/wiki/Intentional_programming https://en.wikipedia.org/wiki/Intentional_programming
- refulgentis 2y agoQuick 10 min into Googling, def. double check me: You're right, ex. https://dev.to/jillesvangurp/comment/6gnb/ https://dev.to/jillesvangurp/comment/6gnb/ Sadly there's enough bitrot that ex. intentsoft.com is offline. More here, but article is v opinionated/judgement oriented, comments are useful, but again, bitrot :( https://wiki.c2.com/?IntentionalProgramming https://wiki.c2.com/?IntentionalProgramming It strikes me that what I am describing as "bitrot" may also be "never really shipped, so the vagueness isn't accidental"
- throwaway2037 2y agoHat tip for c2.com link! The follow-up is savage: https://wiki.c2.com/?CritiqueOfIntentionalProgramming https://wiki.c2.com/?CritiqueOfIntentionalProgramming
- freedomben 2y agoThat's a fascinating idea, but I wonder if code written in Go or similar language would make for some gargantuan and awful-to-read Lisp, and possibly vice-versa (Go with a ton of functions that are all only a few lines long. Sounds ok to me, but very different from any Go code I've worked on that I didn't write :-D).
- turtleyacht 2y agoOne (naive) approach I keep thinking on is using ed(1) to write transforms of code, so chunks of Java are later sewn together to create the app. This always sounds more difficult on paper than just wrestling dependencies till dawn, upgrading from JDK 11 to JDK 17, for example. So I usually give up the mental exercise there. Plus, following a file of transforms is mind-bending: someone may follow a method definition with a pattern seek, and then start appending some more code. Context is lost. It would be literate programming only with enough empathy for comments. Which is all to say, would it be easier to move between Spring versions if the app's commit history were a series of transforms instead of changes to static files? Suppose a commit establishes a framework version, and then follow a bunch of commits for domain objects, a skeleton controller, and so on. If we could play those decisions forward, but edit the transform instead of the source, would it be easier to dissect which next dependency to manage? This loops back to ASTs: we would still edit and change files, but the history would be ed(1) macros (or something better, like ASTs). Somehow, it feels like there could be reconciliation between source control and "manipulating a timeline of changes." Git may already have this, or a simple while loop with some decisions about how far to play the changes, like editing a cassette tape. A list of patches to apply, with pre- and post- hooks for rules scripts.
- striking 2y agoA sufficiently-motivated engineer could do this today by setting a different formatter in their editor than in their pre-commit hook. Then they need only activate the formatter in their editor as they edit, and ensure pre-commit hooks run as they commit. This saves you the trouble of authoring a separate programming language, or finding a way to preserve all of the niceties of the original syntax and formatting that wouldn't directly translate to an AST (like how many newlines are after a particular stanza or function). Case in point: Recast (https://github.com/benjamn/recast https://github.com/benjamn/recast) is of particular interest with regards to JS/TS in this vein, because it does preserve a lot of the spirit of the source in its conception of the AST. But also last time I used it (couple years ago now) it would explode on any code with an emoji in it. It's genuinely not an easy problem.
- shawn_w 2y agoI think that's what InterLisp did.
- eru 2y ago> It really is dumb to be arguing over tabs vs spaces, after all. In an in-house dialect of Haskell I used to work with, we solved this problem by just making tabs a syntax error. Never had any problems. (I think tabs might be have been allowed inside strings.)
- kccqzy 2y agoSo I'm guessing you mean Mu, the dialect by Standard Chartered.
- eru 2y agoYes, indeed. In it's circa 2014 incarnation, when I last worked there. (I don't really know what happened to it since.)
- afavour 2y agoArguably it would be a problem for a coworker that wanted to use tabs. “We’ll just force everyone to do things one way” kind of ignores the point I was making. It shouldn’t be necessary for you to care how anyone else formats their code, same as you don’t care what font their code displays in or what text editor they use. It feels like a vestigial aspect of programming that we have to concern ourselves with it in 2024.
- refulgentis 2y ago100%, way this felt in practice at Google was I could have whatever I wanted in my IDE, and it'd be transformed upon check-in into the house style, which I don't need to care about FWIW, just happy to have a chance to unload this thought finally: it had surprisingly little impact on code reviews, in that the "personal preference I need to enforce" just ascended abstraction levels.
- ukuina 2y agoDoes the IDE transform the existing code in the repo back into your preferred style on the next checkout?
- python_pele 2y agoUnison has something similar to what you're talking about: https://www.unison-lang.org/ https://www.unison-lang.org/
- coolcoder613 2y agoMany BASIC dialects did this.
- tengbretson 2y agoDoes that mean that a file with a syntax error is not able to be saved?
- SR2Z 2y agoWell, not as an AST at least. Presumably it would still be ok as text.
- __s 2y agolanguage servers have pushed things to inspecting source code while programmer has partially written code, so having a kind of (invalid-span content="garbage") node in AST helps
- duped 2y agoLanguage servers usually are designed around full/concrete syntax trees instead of ASTs for exactly this reason. Adding error nodes to the AST is a hack that hurts more than helps. More technically, language servers usually have a CST that they use to build the AST incrementally, and the AST contains references back to the CST that generated it. This is what allows you to handle incremental text edits and compile small deltas to the AST instead of the typical batch compiler design that attempts to parse everything all at once.
- jenadine 2y agoWhat's wrong with error node in the ast? I've seen language server that completely ignores the parts with error, and I much prefer error nodes because then I still know there is something and these error node can still have children
- duped 2y agoThere are a few problems with errors-as-nodes in a syntax tree. An abstract syntax tree is a hierarchical representation of the program in the language's grammar - and errors are not members of the grammar (they're everything else!) There is also the problem that an error returned by a language server is a class of a "diagnostic" that includes syntax errors, semantic errors, warnings, lints, etc, associated with a span in the source code. It's much easier to think of diagnostics as a separate data structure that gets filled up during lexical/semantic analysis and associated with spans in the full syntax tree (you can even store them there as fields, but they don't necessarily have children). Then it's obvious how the structure gets created and fed back to the user. And finally, the whole point of an AST is to be a valid canonical representation of a program so the compiler query it drives doesn't have to do additional input validation. So it just makes the queries/compiler passes easier to write.
- duped 2y agoOne very nice property of an AST is that it's an IR that is allowed be instable even if the syntax it represents is stabilized. If the AST becomes the source of truth on the file system you lose that property. On top of that, now you need to write a parser and compiler for your AST file. It's probably very simple and does rudimentary validation, but that defeats the point of the AST - it's a valid, canonical representation of a program by construction. All in all, it seems like a good idea, and people have done it. But there's also good reasons to be apprehensive. And at the end of the day, you need to ingest text as input, and you need to do it as fast as possible. There's not a ton of benefit to keeping an AST around on disk and in sync with the text that generated it when you are already able to compute it faster than you can read and deserialize it.
- lmm 2y agoNo, because there are no good tools for this format. It's a classic "worse is better" problem - a programming language cannot succeed unless its source code format is unix text files.
- arp242 2y agoWell-formatted code is more than just an AST. Even in Go, a language with probably one of the lowest style divergences out there (for better or worse) there are tons of style choices that aren't in an AST. Blank lines is an obvious one: where do you insert them to "group" sections of a 30 line function? Or are there no blank lines at all? Line length: just "wrap at columns X" (or never wrap) is not enough, because people can and do wrap at specific locations for specific reasons, because that makes more semantic sense or looks nicer than cramming as much as possible.
- icholy 2y agoI prefer no blank lines. If you feel like you need one, write a line comment instead describing the next section.
- arp242 2y agoWell okay, but a great many people disagree and this sort of thing isn't captured in an AST, so clearly it's not a suitable format for storing code.
- josephg 2y agoI think that's a terrible choice. Its like saying "I don't like whitespace in webpage design. If they need whitespace, fill it with content - like maybe some text." Whitespace gives readers subtle information about the structure of a function before they read any of it. Its a powerful tool. Dismissing or - worse - deleting whitespace wholesale sounds profoundly misguided to me. Why make code harder to read? Where's the benefit to your approach? I can't see any.
- icholy 2y agoWhitespace is useful in normal prose because it has no inherent visual structure aside from punctuation. A programing language is inherently structured into logical functions and blocks. If your function is so complex that you need to start adding whitespace to make it visually parseable then that's its own problem.