8 ms·

Lamdu - towards the next generation IDE

200 points by mjn 13y ago

solomatov 13y ago
There are other efforts to create structural editing: * JetBrains MPS : http://www.jetbrains.com/mps/ http://www.jetbrains.com/mps/ * The project I currently work on at JetBrains jetpad-projectional. https://github.com/JetBrains/jetpad-projectional https://github.com/JetBrains/jetpad-projectional It allows you to have structural editing on the web with the help of GWT.
- hrjet 13y ago
  An example structured editor based on MPS: http://mbeddr.com/ http://mbeddr.com/ It allows state-machines to be coded in an enhanced C like language with nice visual representations.
- mike_hearn 13y ago
  MPS is really amazing (or crazy, depending on your perspective). It's been in development for years and is perhaps best described an IDE for building languages and IDEs. So there's a structural/projectionally edited language for defining structurally/projectionally edited languages (yes really), another one for type systems, another one for converting such languages into other languages. Eventually things compile down to Java (textually) so you can use all the standard libraries and so on. There are even languages for defining IntelliSense style IDE features. The main problem with these kinds of things is the incredibly sharp learning curve. It's more of a learning wall, really. It's a lot of effort to learn, and then it turns out that it's really hard to get in the mindset of defining new languages to solve problems, even when it's been made a lot easier than using yacc/lex/writing your own compiler backend/etc. I never really found a use for it, but I keep it filed away in the back of my mind in case one day I do.
andrewflnr 13y ago
A true HN Christmas present. If nothing else it reminded me to take time to watch the Subtext videos. :) I noticed that the (single) argument to sum in most of the screenshots doesn't have a name? Is that special-cased in the language or the editor, or am missing something bigger?
- agumonkey 13y ago
  Subtext website, for the other curious http://www.subtext-lang.org/ http://www.subtext-lang.org/ .
- Ixiaus 13y ago
  I might be interpreting your question wrong but that is specific to Haskell (which he uses as the foundation of lambdu). It looks like sum is being composed with filter which is given the iterable and a predicate.
  - andrewflnr 13y ago
    They mention that they modified haskell to use explicitly named parameters. You can see that in the filter call but not the sum call.
    - Peaker 13y ago
      Our current thought is that single-parameter functions do not need an argument name. We might change a lot of our decisions later as we tune it for working with more realistic code.
      - judk 13y ago
        I hope you expose the color coding as a config file, so folks with an eye for design can contribute.
        Peaker 13y ago
        https://github.com/Peaker/lamdu/blob/master/config.json https://github.com/Peaker/lamdu/blob/master/config.json You can change it while Lamdu is running for immediate effect :-)
bsaul 13y ago
Looks cool ! The screenshots weren't really explicit to someone not familiar with haskell. Is there any video available ?
ibdknox 13y ago
FWIW, the main goal of Light Table is to provide a platform for people to experiment with and build these kinds of experiences without having to worry about all the other standard editory stuff. By using a browser for the UI, we have the opportunity to very easily play with all sorts of really interesting display and editing paradigms. Unfortunately, the notion of editing an AST is by definition language specific, so it's unlikely that someone could create the structural editor and just have it work for everybody. Moreover, as they mention here for Lamdu, it often requires looking at the language slightly differently and enforcing some rules that wouldn't normally exist in the code. But, with a decent foundation we can at least make writing the necessarily language specific parts relatively straightforward. For those new to this world and wondering why this stuff doesn't seem to exist, one of the biggest problems with projectional editors is handling the translation problem. How do you reconcile changes in your representation with changes in the underlying code? Can you reliably parse handwritten code into your AST representation write it back and so on without any loss? What happens with handwritten styles that maybe don't fit into the projection's way of viewing the world? What if I know a better way to output the AST than you? It's also particularly difficult dealing with change over time since there are no true unique identifiers for bits of code. The approach they're taking is likely the correct approach for the "future" - we should be designing languages and approaches to coincide with the ability to tool them. Even better would be to never have a handwritten format at all: the cannonical representation is always the AST. And though you might be able to edit a projection of it as text, you're never at a loss for how to get back to your "good" representation. This is the world I think we ultimately need to get to and you'll be seeing some really cool stuff from us in that vein early in 2014.
- seanmcdirmid 13y ago
  The main problem with structured editing is that it requires programmers to write code in a certain order that is often goes against their "flow;" enforcing a rigid flow is a huge usability negative. Even without structured editing, we see cases where increased flexibility improves flow; e.g. in dynamically typed languages. Unfortunately, improvements in flow often lead to reductions in feedback (e.g. via static typing). There are things we can do to have our cake (flow) and eat it (feedback) also. Code completion was one of the great boons of structured editing (introduced in Alice Pascal circa 1985), and by the late 90s we learned how to create "language aware" editors that could leverage this feature without flow disrupting structured editing. The same goes for static typing: we can, through some heavy type inference, infer semantic information responsively while the user is typing, and use that to provide responsive feedback. I'm in the camp where the programming experience should be considered holistically. The IDE is a part of that experience, and so language design should occur concurrent with IDE design. With some smart incremental compilation magic along with language-specific rendering in the IDE, we can build programming experiences that provide the benefits of structured (and projectional) editing without the flow costs. Or at least, that is the premise of my research :)
partycoder 13y ago
Also similar to this: http://sclang.sourceforge.net/index.php?p=Media http://sclang.sourceforge.net/index.php?p=Media
seanmcdirmid 13y ago
If the author is reading, consider throwing in this link into the Similar efforts section: http://research.microsoft.com/en-us/people/smcdirm/liveprogramming.aspx http://research.microsoft.com/en-us/people/smcdirm/liveprogr...
- judk 13y ago
  The author is "Peaker" on HN, Reddit, and maybe also SO.
Edmond 13y ago
Very cool but I am afraid this is geek candy that would not fly for mere mortals. When you say next-gen IDE do you mean for a certain class of developers? The problem with these type of development tools is that it moves your brain from thinking in terms that of a human to thinking in a very structured way more attuned to machines. This is a problem with functional programming in general, it is fundamentally anti-human, people don't think functionally but rather procedurally. Full disclosure here, IDE maker so I have skin in this game :)
- seanmcdirmid 13y ago
  What IDE are you working on? I agree with the argument against functional programming, though I would qualify that with pureness (doing everything functionally) rather than just the use of functions in general: functions are sometimes the most natural way to do something. Haskell is good for what it is, an experiment in pure functional programming, which has taught us a lot about programming in general.
  - Edmond 13y ago
    I agree, that's why I was asking which class of developers they are targeting... My company is building an IDE that is purely about productivity, specifically for building business web applications. Check my profile for link.
    - seanmcdirmid 13y ago
      Cool. I checked out your page, it seems your tool logo is very similar to ours (MS) :). I'm not sure if it is appropriate to think about specific groups of developers in a general purpose IDE. You really can't guess how they will use your tool, which applies to PL design in general. Personally speaking, there are functional programming enthusiasts who think wildly different from the way I do, and I guess they would sort of select out given my lack of understanding of their psychology as reflected in my design.
      - Edmond 13y ago
grondilu 13y ago
« The project is not open source and is planned to be a commercial IDE. » Not saying there's anything wrong with that, but it's worth emphasizing, imho.
- S4M 13y ago
  You're talking about Projucer, not Lamdu, right?
- aristidb 13y ago
  That refers to Projucer, not Lamdu. Lamdu is open source.
- Peaker 13y ago
  That was a misleading sentence in the "Projucer" section, referring to Projucer. Fixed. Lamdu is open source.
ilaksh 13y ago
Reminds me of Intentional Programming http://en.wikipedia.org/wiki/Intentional_programming http://en.wikipedia.org/wiki/Intentional_programming which I think is the furthest along in this area.
JadeNB 13y ago
There is also a text editor called [Yi](http://www.haskell.org/haskellwiki/Yi http://www.haskell.org/haskellwiki/Yi) written in Haskell, that seems similarly to pursue the idea of continually parsing a program as it is written. Is there any similarity or relationship?
kamaal 13y ago
One of the things that worries me about IDE's these days is how its making learning syntax obsolete and promotes an attitude to avoid reading documentation. You still need to have some idea of the syntax though, but you don't have to know every little syntax detail of a language. This is largely true with Eclipse + Java combination. I've recently had problems during interviews. The interviewer asks questions related to Java syntax, now though I spend a good deal of time coding in Java- My mind is totally broken to think outside the IDE. The interviewer thinks I might simply be lying about the Java experience I claim to have. The same is not true for C and Perl, where I use simple editors. I can think problems out on the paper, or even build a abstract version of a whole complex app with just pen and paper. The best kind of IDE's is what I've seen in the embedded domain. Which assist you during work, you still have to read and internalize the documentation well. But the IDE will suggest you improvements, let you probe ports, registers and let you put break points to see if things are actually going around as you think they should. Above all you still have to learn the best practices the hard way, the IDE simply assists you to do that along the way.
- yohanatan 13y ago
  It sounds like you have a lot of experience. Why do you waste time with interviewers who are interested in whether you know a particular syntax or not?
- undoware 13y ago
  I've had exactly the experience you describe, although it's not from IDE use in my case -- it's from coffeescript and livescript. I bombed a JS interview about six months ago because they wanted to sit in the room and watch my screen while I took their test (already problematic, in my opinion, but let's pretend I would have taken the job.) What clinched the deal I'm sure was when I couldn't remember if it was .length or .length(), started to Google it, realized I looked like an idiot, wasting time because none of my bookmarks were on the laptop I'd been handed, etc. etc. You see, I usually write in the functional style, so my code seldom has to ask arrays about their sizes, and when it does, it uses '[*]'. When I was back in for-loop land, I spoke like a for-eigner...
vinodkd 13y ago
The real gem in lamdu compared to LT, Subtext,MPS etc is the integrated versioning, IMO. Far too long have we been doing snapshot-based development. I hope this is a language feature and not just an IDE feature, however, so that the evolution of the program is expressed within itself.
efnx 13y ago
This looks awesome and interesting. On another note it looks like the rendering is all done in OpenGL, which I have always wondered about - how often are IDEs written from scratch using OpenGL? Is it commonplace?
- seanmcdirmid 13y ago
  Why do you think its being rendered in OpenGL? If you render to a 3D device directly, you'll at least need to work with an appropriate 2D graphics layer on top of it (e.g. Quartz, Cairo, or Direct2D) to help you with font rendering or even basic line rendering! Bezier curves, font vectors, and anti-aliasing ain't easy.
  - yairchu 13y ago
    It is using OpenGL indirectly via the graphics-drawingcombinators library (http://hackage.haskell.org/package/graphics-drawingcombinators http://hackage.haskell.org/package/graphics-drawingcombinato...). Indeed the fonts may not be as pretty as with Cairo etc but on the other hand there are very nice animations when edits make stuff move around.
    - seanmcdirmid 13y ago
      Ah! I hand rolled my own code editor using WPF (mostly canvas and font rendering) for my live programming work. Its amazing what can be done when not using inflexible high level components.
  - efnx 13y ago
    I just looked at the cabal file, which lists the build dependencies. Also, there's a lot of OpenGL code in the repo :)
ardz 13y ago
There are 3 things that will kill a project almost instantly at the beginning: bad design, bad implementation, bad license. I don't know about the first two but GPL is not a good choice.
- cordite 13y ago
  I think GPL is fine for a full application, as long as the reusable parts are made into a library under a different permissive license.
- mortyseinfeld 13y ago
  Not a big GPL fan, but why would that matter in this case...plugins?
  - ardz 13y ago
    Because this IDE is built on top of Haskell ecosystem I have a serious problem to see the bright future of it regardless of its license. I mean: - IDE should have an integrated compiler. - IDE should download only necessary libaries or functions needed by a program(from central repository, local or remote). - IDE should catch a bug and allow me to fix it while program is running. - IDE where I am able to fix a bug on my clients phone or his wrist watch remotely without the need to download whole Haskell Platform there. No JS/HTML5 BS.
- Peaker 13y ago
  We're probably going to switch to a BSD license.
e12e 13y ago
Interesting project. Still, if you end up with a document that is this rich - wouldn't the smalltalk/self approach be better? As I understand it, if you work in the "lamdu" realm, you've already departed the notion of programs as text files that are "lifted" into code and data (objects) by the compiler/interpreter system - and you still don't have real interaction with your smart data (you cant drag the scales of a graph, or have a live preview of animation to see what framerate would be best etc - saving the changes back into your vm image/document)?

200 points by mjn 13y ago

There are other efforts to create structural editing: * JetBrains MPS : http://www.jetbrains.com/mps/ http://www.jetbrains.com/mps/ * The project I currently work on at JetBrains jetpad-projectional. https://github.com/JetBrains/jetpad-projectional https://github.com/JetBrains/jetpad-projectional It allows you to have structural editing on the web with the help of GWT.

hrjet 13y ago
An example structured editor based on MPS: http://mbeddr.com/ http://mbeddr.com/ It allows state-machines to be coded in an enhanced C like language with nice visual representations.
mike_hearn 13y ago
MPS is really amazing (or crazy, depending on your perspective). It's been in development for years and is perhaps best described an IDE for building languages and IDEs. So there's a structural/projectionally edited language for defining structurally/projectionally edited languages (yes really), another one for type systems, another one for converting such languages into other languages. Eventually things compile down to Java (textually) so you can use all the standard libraries and so on. There are even languages for defining IntelliSense style IDE features. The main problem with these kinds of things is the incredibly sharp learning curve. It's more of a learning wall, really. It's a lot of effort to learn, and then it turns out that it's really hard to get in the mindset of defining new languages to solve problems, even when it's been made a lot easier than using yacc/lex/writing your own compiler backend/etc. I never really found a use for it, but I keep it filed away in the back of my mind in case one day I do.

andrewflnr 13y ago

A true HN Christmas present. If nothing else it reminded me to take time to watch the Subtext videos. :) I noticed that the (single) argument to sum in most of the screenshots doesn't have a name? Is that special-cased in the language or the editor, or am missing something bigger?

agumonkey 13y ago
Subtext website, for the other curious http://www.subtext-lang.org/ http://www.subtext-lang.org/ .
Ixiaus 13y ago
I might be interpreting your question wrong but that is specific to Haskell (which he uses as the foundation of lambdu). It looks like sum is being composed with filter which is given the iterable and a predicate.
- andrewflnr 13y ago
  They mention that they modified haskell to use explicitly named parameters. You can see that in the filter call but not the sum call.
  - Peaker 13y ago
    Our current thought is that single-parameter functions do not need an argument name. We might change a lot of our decisions later as we tune it for working with more realistic code.
    - judk 13y ago
      I hope you expose the color coding as a config file, so folks with an eye for design can contribute.
      - Peaker 13y ago
        https://github.com/Peaker/lamdu/blob/master/config.json https://github.com/Peaker/lamdu/blob/master/config.json You can change it while Lamdu is running for immediate effect :-)

bsaul 13y ago

Looks cool ! The screenshots weren't really explicit to someone not familiar with haskell. Is there any video available ?

ibdknox 13y ago

FWIW, the main goal of Light Table is to provide a platform for people to experiment with and build these kinds of experiences without having to worry about all the other standard editory stuff. By using a browser for the UI, we have the opportunity to very easily play with all sorts of really interesting display and editing paradigms. Unfortunately, the notion of editing an AST is by definition language specific, so it's unlikely that someone could create the structural editor and just have it work for everybody. Moreover, as they mention here for Lamdu, it often requires looking at the language slightly differently and enforcing some rules that wouldn't normally exist in the code. But, with a decent foundation we can at least make writing the necessarily language specific parts relatively straightforward. For those new to this world and wondering why this stuff doesn't seem to exist, one of the biggest problems with projectional editors is handling the translation problem. How do you reconcile changes in your representation with changes in the underlying code? Can you reliably parse handwritten code into your AST representation write it back and so on without any loss? What happens with handwritten styles that maybe don't fit into the projection's way of viewing the world? What if I know a better way to output the AST than you? It's also particularly difficult dealing with change over time since there are no true unique identifiers for bits of code. The approach they're taking is likely the correct approach for the "future" - we should be designing languages and approaches to coincide with the ability to tool them. Even better would be to never have a handwritten format at all: the cannonical representation is always the AST. And though you might be able to edit a projection of it as text, you're never at a loss for how to get back to your "good" representation. This is the world I think we ultimately need to get to and you'll be seeing some really cool stuff from us in that vein early in 2014.

seanmcdirmid 13y ago
The main problem with structured editing is that it requires programmers to write code in a certain order that is often goes against their "flow;" enforcing a rigid flow is a huge usability negative. Even without structured editing, we see cases where increased flexibility improves flow; e.g. in dynamically typed languages. Unfortunately, improvements in flow often lead to reductions in feedback (e.g. via static typing). There are things we can do to have our cake (flow) and eat it (feedback) also. Code completion was one of the great boons of structured editing (introduced in Alice Pascal circa 1985), and by the late 90s we learned how to create "language aware" editors that could leverage this feature without flow disrupting structured editing. The same goes for static typing: we can, through some heavy type inference, infer semantic information responsively while the user is typing, and use that to provide responsive feedback. I'm in the camp where the programming experience should be considered holistically. The IDE is a part of that experience, and so language design should occur concurrent with IDE design. With some smart incremental compilation magic along with language-specific rendering in the IDE, we can build programming experiences that provide the benefits of structured (and projectional) editing without the flow costs. Or at least, that is the premise of my research :)

partycoder 13y ago

Also similar to this: http://sclang.sourceforge.net/index.php?p=Media http://sclang.sourceforge.net/index.php?p=Media

seanmcdirmid 13y ago

If the author is reading, consider throwing in this link into the Similar efforts section: http://research.microsoft.com/en-us/people/smcdirm/liveprogramming.aspx http://research.microsoft.com/en-us/people/smcdirm/liveprogr...

judk 13y ago
The author is "Peaker" on HN, Reddit, and maybe also SO.

Edmond 13y ago

Very cool but I am afraid this is geek candy that would not fly for mere mortals. When you say next-gen IDE do you mean for a certain class of developers? The problem with these type of development tools is that it moves your brain from thinking in terms that of a human to thinking in a very structured way more attuned to machines. This is a problem with functional programming in general, it is fundamentally anti-human, people don't think functionally but rather procedurally. Full disclosure here, IDE maker so I have skin in this game :)

seanmcdirmid 13y ago
What IDE are you working on? I agree with the argument against functional programming, though I would qualify that with pureness (doing everything functionally) rather than just the use of functions in general: functions are sometimes the most natural way to do something. Haskell is good for what it is, an experiment in pure functional programming, which has taught us a lot about programming in general.
- Edmond 13y ago
  I agree, that's why I was asking which class of developers they are targeting... My company is building an IDE that is purely about productivity, specifically for building business web applications. Check my profile for link.
  - seanmcdirmid 13y ago
    Cool. I checked out your page, it seems your tool logo is very similar to ours (MS) :). I'm not sure if it is appropriate to think about specific groups of developers in a general purpose IDE. You really can't guess how they will use your tool, which applies to PL design in general. Personally speaking, there are functional programming enthusiasts who think wildly different from the way I do, and I guess they would sort of select out given my lack of understanding of their psychology as reflected in my design.
    - Edmond 13y ago

grondilu 13y ago

« The project is not open source and is planned to be a commercial IDE. » Not saying there's anything wrong with that, but it's worth emphasizing, imho.

S4M 13y ago
You're talking about Projucer, not Lamdu, right?
aristidb 13y ago
That refers to Projucer, not Lamdu. Lamdu is open source.
Peaker 13y ago
That was a misleading sentence in the "Projucer" section, referring to Projucer. Fixed. Lamdu is open source.

ilaksh 13y ago

Reminds me of Intentional Programming http://en.wikipedia.org/wiki/Intentional_programming http://en.wikipedia.org/wiki/Intentional_programming which I think is the furthest along in this area.

JadeNB 13y ago

There is also a text editor called [Yi](http://www.haskell.org/haskellwiki/Yi http://www.haskell.org/haskellwiki/Yi) written in Haskell, that seems similarly to pursue the idea of continually parsing a program as it is written. Is there any similarity or relationship?

kamaal 13y ago

One of the things that worries me about IDE's these days is how its making learning syntax obsolete and promotes an attitude to avoid reading documentation. You still need to have some idea of the syntax though, but you don't have to know every little syntax detail of a language. This is largely true with Eclipse + Java combination. I've recently had problems during interviews. The interviewer asks questions related to Java syntax, now though I spend a good deal of time coding in Java- My mind is totally broken to think outside the IDE. The interviewer thinks I might simply be lying about the Java experience I claim to have. The same is not true for C and Perl, where I use simple editors. I can think problems out on the paper, or even build a abstract version of a whole complex app with just pen and paper. The best kind of IDE's is what I've seen in the embedded domain. Which assist you during work, you still have to read and internalize the documentation well. But the IDE will suggest you improvements, let you probe ports, registers and let you put break points to see if things are actually going around as you think they should. Above all you still have to learn the best practices the hard way, the IDE simply assists you to do that along the way.

yohanatan 13y ago
It sounds like you have a lot of experience. Why do you waste time with interviewers who are interested in whether you know a particular syntax or not?
undoware 13y ago
I've had exactly the experience you describe, although it's not from IDE use in my case -- it's from coffeescript and livescript. I bombed a JS interview about six months ago because they wanted to sit in the room and watch my screen while I took their test (already problematic, in my opinion, but let's pretend I would have taken the job.) What clinched the deal I'm sure was when I couldn't remember if it was .length or .length(), started to Google it, realized I looked like an idiot, wasting time because none of my bookmarks were on the laptop I'd been handed, etc. etc. You see, I usually write in the functional style, so my code seldom has to ask arrays about their sizes, and when it does, it uses '[*]'. When I was back in for-loop land, I spoke like a for-eigner...

vinodkd 13y ago

The real gem in lamdu compared to LT, Subtext,MPS etc is the integrated versioning, IMO. Far too long have we been doing snapshot-based development. I hope this is a language feature and not just an IDE feature, however, so that the evolution of the program is expressed within itself.

efnx 13y ago

This looks awesome and interesting. On another note it looks like the rendering is all done in OpenGL, which I have always wondered about - how often are IDEs written from scratch using OpenGL? Is it commonplace?

seanmcdirmid 13y ago
Why do you think its being rendered in OpenGL? If you render to a 3D device directly, you'll at least need to work with an appropriate 2D graphics layer on top of it (e.g. Quartz, Cairo, or Direct2D) to help you with font rendering or even basic line rendering! Bezier curves, font vectors, and anti-aliasing ain't easy.
- yairchu 13y ago
  It is using OpenGL indirectly via the graphics-drawingcombinators library (http://hackage.haskell.org/package/graphics-drawingcombinators http://hackage.haskell.org/package/graphics-drawingcombinato...). Indeed the fonts may not be as pretty as with Cairo etc but on the other hand there are very nice animations when edits make stuff move around.
  - seanmcdirmid 13y ago
    Ah! I hand rolled my own code editor using WPF (mostly canvas and font rendering) for my live programming work. Its amazing what can be done when not using inflexible high level components.
- efnx 13y ago
  I just looked at the cabal file, which lists the build dependencies. Also, there's a lot of OpenGL code in the repo :)

ardz 13y ago

There are 3 things that will kill a project almost instantly at the beginning: bad design, bad implementation, bad license. I don't know about the first two but GPL is not a good choice.

cordite 13y ago
I think GPL is fine for a full application, as long as the reusable parts are made into a library under a different permissive license.
mortyseinfeld 13y ago
Not a big GPL fan, but why would that matter in this case...plugins?
- ardz 13y ago
  Because this IDE is built on top of Haskell ecosystem I have a serious problem to see the bright future of it regardless of its license. I mean: - IDE should have an integrated compiler. - IDE should download only necessary libaries or functions needed by a program(from central repository, local or remote). - IDE should catch a bug and allow me to fix it while program is running. - IDE where I am able to fix a bug on my clients phone or his wrist watch remotely without the need to download whole Haskell Platform there. No JS/HTML5 BS.
Peaker 13y ago
We're probably going to switch to a BSD license.

e12e 13y ago

Interesting project. Still, if you end up with a document that is this rich - wouldn't the smalltalk/self approach be better? As I understand it, if you work in the "lamdu" realm, you've already departed the notion of programs as text files that are "lifted" into code and data (objects) by the compiler/interpreter system - and you still don't have real interaction with your smart data (you cant drag the scales of a graph, or have a live preview of animation to see what framerate would be best etc - saving the changes back into your vm image/document)?