6 ms·

To me, the most interesting aspect of S-expression programming languages is that they are very well suited for other editors than the text editor. I know of a n

To me, the most interesting aspect of S-expression programming languages is that they are very well suited for other editors than the text editor. I know of a number of experiments in this regard but I have yet to see anything quite as ground breaking as what I can imagine is possible. How come we don't have Google Maps-like zooming with LOD mapping to abstractions at different levels?

deleted 4y ago
[deleted]
jerf 4y ago
Many have tried. The upshot is, it is either not as cool as it is in our heads if you get it, or it isn't as easy as it is in our heads. You don't even really need s-expressions to do what you're talking about. You can do it off of any grammar, which all current programming languages have[1]. S-expressions are slightly easier, but they aren't easier in the place where the real problem lies. The real problem lies in the fact that the code isn't shaped the way you think it is and it isn't as nice as you think it is and it turns out nearly every attempt to project code into something other than our current plain text representation flounders on the pathological nature of source code. Personally I think this is a fundamental problem, but there is an avenue to where I would be wrong. If somebody who respected text and understood why it is currently so dominant, in a positive sense, also took on the task of trying to transcend it based on that understanding, perhaps they could find something, revealing the current lack of such editors to be accidental after all. But as long as the only rebels trying it also labor under the crippling error that text is a weakness that needs to be discarded as quickly as possible, rather than the only methodology currently known to be even remotely strong enough to support our programming endeavors [2], they're never going to get anywhere in trying to deploy a system that for anyone to be even remotely interested in it pretty much has to immediately be able to scale up to some reasonably fraction of what text editors already do. I find it telling that the stickiest such effort I've seen to date is literally to simply provide the programmer with a completely zoomed out graphic of the text file in the scrollbar area. [1] Since nobody has yet had the resources to try to figure out how to take a language model and turn it into a programming language. Patience. [2] Or, to put it another way... make a list of all the weaknesses text putatively has for programming languages. Then sit and stare at it for a while until you grok that that is still the shortest list of weaknesses we know of, and all other efforts have even worse weaknesses at scale. It isn't that I deny the weaknesses; they are manifest and obvious to pretty much everybody. It is that it's still the shortest such known list at scale.
- mav88 4y ago
  Fascinating insights, thank you. Is it not perhaps that text as is is perfect for programming languages but we could do a lot better in terms of displaying and presenting it?
  - jerf 4y ago
    I'm sure there's improvements around the edges to be made. I'm not sure if there's a massive paradigm shift that can be done. At least not in terms of current technology; we can all imagine the Star Trek future where you can just tell the computer that you want an analysis of all the currently-interesting factors related to this diplomatic situation and a collection of AIs turn that into something effective. At that point "programmers" aren't staring at large blocks of text. Though don't be fooled; someone is going in deeper than the initial instruction. "Programming" may instead be a matter of working with an AI almost like you'd work with a coworker, but someone's going to be dedicated to the task of maximizing output. You've seen we've recently taken a big step in this direction, but we've still got a number of steps to go, and even this big step is still being processed by the world at large. But in terms of current tech and the world of programming as we know it (which I actually believe will always be with us; while it has its manifest limitations, the ability to take concrete instructions and chew through them at light speed will always be useful; even if you've got an AI that can use teraflops and petaflops to have a real-time conversation with you, even that AI will have uses for conventional programming techniques to use those teraflops & petaflops just as we'd use them today, rather than for high-level AI functionality), I don't know if there's a huge paradigm shift that can scale to, say, the size of the Linux kernel. But if anyone wants to create it, I think they need to start from an understanding of the positives of our current paradigm, rather than the false assumption that it's all just some sort of big mistake and it's so obvious what the better answer would have been.
- runlaszlorun 4y ago
  I’m looking at the task of making programming more visual and structured but am at a very early stage (still getting my head around OpenGL/WebGL). I do realize text us still used for its own reasons. I’d say they are: 1) Lack of need for a proprietary tool to view/edit. 2) Density of information. 3) Nearly every existing attempt I’ve seen at visual programming is insultingly bad. So I get it why folks would be hesitant to waste time on some schmuck’s new invention. Am I missing anything on the advantages of text? This is an honest attempt to see where I’m missing perspective, not an attempt to convince. I don’t think the target audience is developers with an existing working workflow- the potential upside prob wouldn’t justify someone spending the time/mental effort to grok a new model. But I think the audience is those who have some aptitude and could be coding and aren’t yet. In my mind, primarily so that maybe individuals can have a little more influence over the code and algorithms in their own life more so than industry roles. Given that the ways you can convey information graphically are a super set of what text provides, I can’t help but think there’s some room for improvement- especially with concepts like literate programming around. Overall, I’d like to give your average personal a little more agency. Maybe that’s a graphical approach. Maybe its getting folks to do basic Rails/Django/Flask/PHP whatever. But I am curious if I’m missing obvious rationales for text…
Jach 4y ago
Some IDEs can auto-inline something temporarily if that's an approximation of what you want, it's the same sort of trick as intellisense popping up documentation when you hover or whatever, it's extra info that's there when you want it and not there when you don't. But for the broader idea I think you can also just as well ask why Bret Victor's vision has gone nowhere (and probably never will). I'm satisfied myself with an answer like this: "Linux supports the notion of a command line or a shell for the same reason that only children read books with only pictures in them. Language, be it English or something else, is the only tool flexible enough to accomplish a sufficiently broad range of tasks." -- Bill Garrett But I also think that for many abstractions, the unabstracted form is often just fine too, and if it's common enough it becomes a 'pattern' that is read quickly in the same way that adults read words or chunks of words and not letter by letter. rpg's free book "Patterns of Software" gets into this near the beginning. For a simple example, in Common Lisp you might at some point write something like (let ((result (mapcar f list)) (length (length list))) ...) This is clean and abstract, you want the result of applying some function to some input list, and also get the list's length. It's too bad that this involves traversing the list twice, it'd be nice if mapcar could return the length as a second value or something. You probably don't want to modify f itself to mutate a counter, that goes beyond f's scope. But the unabstracted form of mutating a local counter and just traversing the list once with mapcar is fine, and a common enough pattern (again copying from the book): (let ((length 0)) (let ((result (mapcar (lambda (x) (incf length) (f x)) list))) ...)) You might call the pattern "accumulating a result" but naming things is overrated. You see it in other languages too. It's just: Initialize some local variable result Loop Within the loop, update the result (in the above, it's the "length" variable) Outside the loop, use the result Anyway, perhaps a clever compiler could turn the first into the second, or a clever IDE could turn the second into the first, and a suitable environment lets you toggle between them at will. It'd make a cool demo maybe. Perhaps s-expressions make one or both a lot easier compared to not having them? But I doubt it because the most advanced IDEs today focus on non-s-expression languages and do fancy things just fine, some fancy things not done by any environment currently available for Common Lisp. But for such a common pattern, with straightforward code in both cases (even if the second is a bit more wordy), it'd just be unnecessary to "zoom in and out", an experienced programmer reads the second as easily as the first because they can read the "pattern". How many non-pattern abstractions do we really have in a typical software project that would benefit, and situations that aren't already good enough with simple tools like inlining, jump-to-def, and intellisense?
pritambaral 4y ago
> How come we don't have Google Maps-like zooming with LOD mapping to abstractions at different levels? With macroexpand-* family of emacs+slime, which I use all the time, I do have access to this. I haven't done any Aspect-Oriented Programming, but suppose it wouldn't be impossible to get similar functionality with it too. A lesser form I use often with non-Lisp languages is narrow-to-defun and narrow-to-region, especially when I need to look at more than one area of code simultaneously.
faitswulff 4y ago
The closest editor I've seen to what you're describing is probably a web app called Workflowy: https://workflowy.com/ https://workflowy.com/
- eternalban 4y ago
  That reminded me a bit of UserLand's Frontier.
seabass-labrax 4y ago
You might be interested in Dion[1], which is the most impressive demonstration of such a feature that I have seen so far. However, it does not appear to be in active development any longer. [1]: https://dion.systems/gallery.html https://dion.systems/gallery.html
- eternalban 4y ago
  "Our ultimate goal is to build a better way to develop software. Last year we developed a prototype of some of our ideas and presented it at Handmade Seattle 2020. Some of the results were exciting, but now we think we shouldn't start by building a single integrated tool. Our new priority is the design of a new file format for source code. It's still a challenging task, but it's well scoped and achievable. Here we explain our roadmap for the development of this new format. ... "Our goal is to build a format that compares to text files in the way that text files compare to bitmaps." https://dion.systems/blog_0002_roadmap.html https://dion.systems/blog_0002_roadmap.html
akkartik 4y ago
I work on this. Check out the final demo at http://akkartik.name/post/roundup22 http://akkartik.name/post/roundup22 Another demo since, showing how I write tests: https://merveilles.town/@akkartik/109742488349122478 https://merveilles.town/@akkartik/109742488349122478
adamddev1 4y ago
As Hal Abelson said back in 1986 in the original SICP lectures [1] while drawing `(+ 3 (* 5 6) 8 2)` and the corresponding syntax tree on the board: >"We're writing these trees ... and parentheses are just a way to write this two dimensional structure as a linear character string. Of course, when LISP first started and people had teletypes or punchcards or whatever and (the parentheses character structure) was more convinient. But maybe if LISP started today the syntax would look more like (a 2D syntax tree)." [1]: https://youtu.be/-J_xL4IGhJA?t=2101 https://youtu.be/-J_xL4IGhJA?t=2101
bmitc 4y ago
What is LOD mapping?
- kaveh808 4y ago
  Level of detail.
  - bmitc 4y ago
    Thank you!

deleted 4y ago

[deleted]

jerf 4y ago

Many have tried. The upshot is, it is either not as cool as it is in our heads if you get it, or it isn't as easy as it is in our heads. You don't even really need s-expressions to do what you're talking about. You can do it off of any grammar, which all current programming languages have[1]. S-expressions are slightly easier, but they aren't easier in the place where the real problem lies. The real problem lies in the fact that the code isn't shaped the way you think it is and it isn't as nice as you think it is and it turns out nearly every attempt to project code into something other than our current plain text representation flounders on the pathological nature of source code. Personally I think this is a fundamental problem, but there is an avenue to where I would be wrong. If somebody who respected text and understood why it is currently so dominant, in a positive sense, also took on the task of trying to transcend it based on that understanding, perhaps they could find something, revealing the current lack of such editors to be accidental after all. But as long as the only rebels trying it also labor under the crippling error that text is a weakness that needs to be discarded as quickly as possible, rather than the only methodology currently known to be even remotely strong enough to support our programming endeavors [2], they're never going to get anywhere in trying to deploy a system that for anyone to be even remotely interested in it pretty much has to immediately be able to scale up to some reasonably fraction of what text editors already do. I find it telling that the stickiest such effort I've seen to date is literally to simply provide the programmer with a completely zoomed out graphic of the text file in the scrollbar area. [1] Since nobody has yet had the resources to try to figure out how to take a language model and turn it into a programming language. Patience. [2] Or, to put it another way... make a list of all the weaknesses text putatively has for programming languages. Then sit and stare at it for a while until you grok that that is still the shortest list of weaknesses we know of, and all other efforts have even worse weaknesses at scale. It isn't that I deny the weaknesses; they are manifest and obvious to pretty much everybody. It is that it's still the shortest such known list at scale.

mav88 4y ago
Fascinating insights, thank you. Is it not perhaps that text as is is perfect for programming languages but we could do a lot better in terms of displaying and presenting it?
- jerf 4y ago
  I'm sure there's improvements around the edges to be made. I'm not sure if there's a massive paradigm shift that can be done. At least not in terms of current technology; we can all imagine the Star Trek future where you can just tell the computer that you want an analysis of all the currently-interesting factors related to this diplomatic situation and a collection of AIs turn that into something effective. At that point "programmers" aren't staring at large blocks of text. Though don't be fooled; someone is going in deeper than the initial instruction. "Programming" may instead be a matter of working with an AI almost like you'd work with a coworker, but someone's going to be dedicated to the task of maximizing output. You've seen we've recently taken a big step in this direction, but we've still got a number of steps to go, and even this big step is still being processed by the world at large. But in terms of current tech and the world of programming as we know it (which I actually believe will always be with us; while it has its manifest limitations, the ability to take concrete instructions and chew through them at light speed will always be useful; even if you've got an AI that can use teraflops and petaflops to have a real-time conversation with you, even that AI will have uses for conventional programming techniques to use those teraflops & petaflops just as we'd use them today, rather than for high-level AI functionality), I don't know if there's a huge paradigm shift that can scale to, say, the size of the Linux kernel. But if anyone wants to create it, I think they need to start from an understanding of the positives of our current paradigm, rather than the false assumption that it's all just some sort of big mistake and it's so obvious what the better answer would have been.
runlaszlorun 4y ago
I’m looking at the task of making programming more visual and structured but am at a very early stage (still getting my head around OpenGL/WebGL). I do realize text us still used for its own reasons. I’d say they are: 1) Lack of need for a proprietary tool to view/edit. 2) Density of information. 3) Nearly every existing attempt I’ve seen at visual programming is insultingly bad. So I get it why folks would be hesitant to waste time on some schmuck’s new invention. Am I missing anything on the advantages of text? This is an honest attempt to see where I’m missing perspective, not an attempt to convince. I don’t think the target audience is developers with an existing working workflow- the potential upside prob wouldn’t justify someone spending the time/mental effort to grok a new model. But I think the audience is those who have some aptitude and could be coding and aren’t yet. In my mind, primarily so that maybe individuals can have a little more influence over the code and algorithms in their own life more so than industry roles. Given that the ways you can convey information graphically are a super set of what text provides, I can’t help but think there’s some room for improvement- especially with concepts like literate programming around. Overall, I’d like to give your average personal a little more agency. Maybe that’s a graphical approach. Maybe its getting folks to do basic Rails/Django/Flask/PHP whatever. But I am curious if I’m missing obvious rationales for text…

Jach 4y ago

Some IDEs can auto-inline something temporarily if that's an approximation of what you want, it's the same sort of trick as intellisense popping up documentation when you hover or whatever, it's extra info that's there when you want it and not there when you don't. But for the broader idea I think you can also just as well ask why Bret Victor's vision has gone nowhere (and probably never will). I'm satisfied myself with an answer like this: "Linux supports the notion of a command line or a shell for the same reason that only children read books with only pictures in them. Language, be it English or something else, is the only tool flexible enough to accomplish a sufficiently broad range of tasks." -- Bill Garrett But I also think that for many abstractions, the unabstracted form is often just fine too, and if it's common enough it becomes a 'pattern' that is read quickly in the same way that adults read words or chunks of words and not letter by letter. rpg's free book "Patterns of Software" gets into this near the beginning. For a simple example, in Common Lisp you might at some point write something like (let ((result (mapcar f list)) (length (length list))) ...) This is clean and abstract, you want the result of applying some function to some input list, and also get the list's length. It's too bad that this involves traversing the list twice, it'd be nice if mapcar could return the length as a second value or something. You probably don't want to modify f itself to mutate a counter, that goes beyond f's scope. But the unabstracted form of mutating a local counter and just traversing the list once with mapcar is fine, and a common enough pattern (again copying from the book): (let ((length 0)) (let ((result (mapcar (lambda (x) (incf length) (f x)) list))) ...)) You might call the pattern "accumulating a result" but naming things is overrated. You see it in other languages too. It's just: Initialize some local variable result Loop Within the loop, update the result (in the above, it's the "length" variable) Outside the loop, use the result Anyway, perhaps a clever compiler could turn the first into the second, or a clever IDE could turn the second into the first, and a suitable environment lets you toggle between them at will. It'd make a cool demo maybe. Perhaps s-expressions make one or both a lot easier compared to not having them? But I doubt it because the most advanced IDEs today focus on non-s-expression languages and do fancy things just fine, some fancy things not done by any environment currently available for Common Lisp. But for such a common pattern, with straightforward code in both cases (even if the second is a bit more wordy), it'd just be unnecessary to "zoom in and out", an experienced programmer reads the second as easily as the first because they can read the "pattern". How many non-pattern abstractions do we really have in a typical software project that would benefit, and situations that aren't already good enough with simple tools like inlining, jump-to-def, and intellisense?

pritambaral 4y ago

> How come we don't have Google Maps-like zooming with LOD mapping to abstractions at different levels? With macroexpand-* family of emacs+slime, which I use all the time, I do have access to this. I haven't done any Aspect-Oriented Programming, but suppose it wouldn't be impossible to get similar functionality with it too. A lesser form I use often with non-Lisp languages is narrow-to-defun and narrow-to-region, especially when I need to look at more than one area of code simultaneously.

faitswulff 4y ago

The closest editor I've seen to what you're describing is probably a web app called Workflowy: https://workflowy.com/ https://workflowy.com/

eternalban 4y ago
That reminded me a bit of UserLand's Frontier.

seabass-labrax 4y ago

You might be interested in Dion[1], which is the most impressive demonstration of such a feature that I have seen so far. However, it does not appear to be in active development any longer. [1]: https://dion.systems/gallery.html https://dion.systems/gallery.html

eternalban 4y ago
"Our ultimate goal is to build a better way to develop software. Last year we developed a prototype of some of our ideas and presented it at Handmade Seattle 2020. Some of the results were exciting, but now we think we shouldn't start by building a single integrated tool. Our new priority is the design of a new file format for source code. It's still a challenging task, but it's well scoped and achievable. Here we explain our roadmap for the development of this new format. ... "Our goal is to build a format that compares to text files in the way that text files compare to bitmaps." https://dion.systems/blog_0002_roadmap.html https://dion.systems/blog_0002_roadmap.html

akkartik 4y ago

I work on this. Check out the final demo at http://akkartik.name/post/roundup22 http://akkartik.name/post/roundup22 Another demo since, showing how I write tests: https://merveilles.town/@akkartik/109742488349122478 https://merveilles.town/@akkartik/109742488349122478

adamddev1 4y ago

As Hal Abelson said back in 1986 in the original SICP lectures [1] while drawing `(+ 3 (* 5 6) 8 2)` and the corresponding syntax tree on the board: >"We're writing these trees ... and parentheses are just a way to write this two dimensional structure as a linear character string. Of course, when LISP first started and people had teletypes or punchcards or whatever and (the parentheses character structure) was more convinient. But maybe if LISP started today the syntax would look more like (a 2D syntax tree)." [1]: https://youtu.be/-J_xL4IGhJA?t=2101 https://youtu.be/-J_xL4IGhJA?t=2101

bmitc 4y ago

What is LOD mapping?

kaveh808 4y ago
Level of detail.
- bmitc 4y ago
  Thank you!