5 ms·

Eliminating Errors with Little Languages

39 points by fogus 16y ago

gfodor 16y ago
Giving parsers and lexers to the average man, or removing the need for them altogether, will be the thing that pushes the bar forward next. I don't know if Perl 6 is going to be the medium for this change, but it might be. Arguably, lisp should have given us a world filled with little languages, but it hasn't taken hold. People like syntax, and people like compile time checks. I have yet to dig into the Perl 6 features that make this possible, but having a low-barrier-to-entry way to create DSLs that are compile-time checked and usable opens the door to something big. Domain experts who are not programmers may finally have a lever in the process stronger than the "spec" and weaker than the code. Give them a DSL, with a good compiler, and let them wreak havoc. The big question here I still have is: text? Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Solve that problem, and you've got something big.
- loup-vaillant 16y ago
  For the ASCII problem, you might want to study structured editors, like http://www.subtextual.org/ http://www.subtextual.org/ Here is an example video: http://www.subtextual.org/subtext2.html http://www.subtextual.org/subtext2.html (It's in Flash or something)
  - gfodor 16y ago
    This is what I'm getting at. I don't quite understand his implementation, but the core problem is well articulated here. Good notation is a prequisite to clear thinking. The kinds of notation we have available to use is severely limited by the medium we write software in.
- coliveira 16y ago
  This is an idea that was pioneered by Smalltalk, where you don't need files to organize your code. But I don't see why it can't be used in other languages, such as Java, using a class-based editor (instead of a file-based one).
  - larsberg 16y ago
    Fundamentally, the problem comes down to the other 90% of the toolstack. Even if you get people to use your language, are you really going to get them to change their source code control, every person's text editor, build system, scripts that run over the code, the external localization service, the third party code escrow service, automated performance and security assessment tools, etc.? Anything that is more than a toy project has real requirements, often with tools that are outside of the control of any one vendor or even the company -- they are sometimes requirements of the industry. This code-stored-not-as-text-thing came up so often (it was a recurring Serious Question about once a year while I worked on Visual Studio) that it was simply referred to as the Emacs/grep barrier. Any tool solution that prevents the use of Emacs and grep out of box is just dead unless you have a from-scratch codebase and team. The old Smalltalk hands (some of whom worked on original Smalltalk systems) also like to relate the biggest single issue they had. At one point, they needed to reconstruct the basis library for a new implementation... but there was no code. How do you do a review of a system where the code was (in this case) completely gone, as it had long since become something interpretable as a jump point in a heap image but not as code? (Punchline: they rewrote most of it from scratch, recreating decades-old bugs)
    - shaunxcode 16y ago
      Was this before the concept of "filing out"?
  - kristiandupont 16y ago
    I am going to plug one of my favourite editors, Code Browser: http://code-browser.sf.net/ http://code-browser.sf.net/ - it is a folding editor that I have been using for outlining (I have my task list and stuff in it and it is just always open), but it is meant for programming and does it well. It is nice to structure a file into classes and functions etc. and be able to concentrate on one at a time.
    - coliveira 16y ago
      This is a great program. Thanks for sharing.
- chromatic 16y ago
  Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Latin-1 in truth, but a fair point. Even so, what's wrong with text that alternate input forms (graphics, speech) can fix? You can be inspecific with speech, and you can organize volumes of text with far more efficiency than you can images.
- epochwolf 16y ago
  > Arguably, lisp should have given us a world filled with little languages, but it hasn't taken hold. Arguably, ruby does this. Look at RSpec, Sinatra, and Rails (Rails 3 especially).
  - chromatic 16y ago
    Arguably, ruby does this. If so, you could make the argument that an ORM library does the same thing. I don't make either argument.
- dkarl 16y ago
  The big question here I still have is: text? Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Solve that problem, and you've got something big. A text editor is just one way of viewing the bytes that represent source code. Theoretically, any language supports other ways of viewing and editing code. It's just the tools that are missing.
- brlewis 16y ago
  "People like compile time checks" cannot be the reason interpreted, non-interactive languages like PHP have taken off while compiled, interactive Lisps have not. "People like syntax" has not been tested. I think it's safe to say that people like syntax that's like syntax they've already learned. We don't really know why Lisp hasn't taken hold, and we don't know that it won't take hold at some point in the future.
- Goladus 16y ago
  I have yet to dig into the Perl 6 features that make this possible, but having a low-barrier-to-entry way to create DSLs that are compile-time checked and usable opens the door to something big. I agree but am also going to say-- I'll believe it when I see it. The article was very light on details. What they're claiming is already facilitated by libraries in many existing languages (including Perl 5) Generally, the barrier to entry for designing and implementing your own language is not the difficulty of understanding how lex and yacc work, it's the need for a thorough and precise understanding of the DSL and every step of the compilation process. Error checking, for example... It's understanding what the compiler knows about your program, and what you can do to figure out things about the program, and how expensive that figuring is going to be. The details here are going to be different for every DSL.
- JadeNB 16y ago
  It's not clear to me that Perl 6 should be regarded as a prime mover in this: in addition to the old standby Lisp, which you mentioned, there are more recent little-language builders like XMF (http://itcentre.tvu.ac.uk/~clark/xmf.html http://itcentre.tvu.ac.uk/~clark/xmf.html), XLR (http://xlr.sourceforge.net http://xlr.sourceforge.net), Katahdin (http://www.chrisseaton.com/katahdin http://www.chrisseaton.com/katahdin), and Converge (http://convergepl.org http://convergepl.org) (in reverse alphabetical order, because that's how I happened to remember them). Actually, although it's fashionable to sneer at it, I think that TeX (http://www.tug.org http://www.tug.org) provides perhaps the most compelling example of a little-language maker. The languages that one can make on top of it are perhaps rather specialised to text markup, but I think it's telling that LaTeX (http://www.latex-project.org http://www.latex-project.org) was designed, not by fiddling with the source, but rather as a macro layer on top of TeX. LaTeX's not little by any means, but I think it would be hard to find many other examples of such successful languages implemented literally on top of existing languages. Finally, I'm not really familiar with Perl 6, but I'm pretty sure it's still dynamically typed. (After all, it targets Parrot, which is specifically advertised as being a low-level target for dynamically typed languages.) As such, I don't see it solving people's problems with lack of compile-time guarantees from Lisp. (Of course, there's no denying that Perl, in every version, does offer gobs and gobs of syntax. :-) )
  - chromatic 16y ago
    I'm not really familiar with Perl 6, but I'm pretty sure it's still dynamically typed. Perl 6 allows optional typing. After all, it targets Parrot, which is specifically advertised as being a low-level target for dynamically typed languages. Parrot allows type checking. Very few languages of which I am aware perform it to any degree, but nothing about Parrot rules it out.
agentultra 16y ago
Why not take the SQLAlchemy approach with its lower-level expression language? Avoid string parsing, implementing specific dialects of SQL, etc. I love perl, but why strings? Writing SQL is not that fun. The cognitive leaps one has to make is annoying. Why not write queries in natural perl statements and translate them to SQL? That being said though, modernperl++ and yay for perl 6. :) </fanboyism>
- chromatic 16y ago
  Why not take the SQLAlchemy approach with its lower-level expression language? You can indeed do that. I prefer SQL to ORM; de gustibus non est disputandum. I love perl, but why strings? Even structured data, such as a nested data structure serialized to JSON, is a string until you parse it. I don't understand the question.
epochwolf 16y ago
I can't help but think the specific problem mentioned has a specific solution. It's only three letters: ORM.
- chromatic 16y ago
  For some purposes, ORM works fine. I find ORM frustrating and unpleasant far more than I find it useful.
  - wazoox 16y ago
    That. And the fact that ORM usually don't allow to structure your database precisely the way you want, in the good ol' fashioned way.
scotty79 16y ago
I have a much simpler idea that does not provide so much functionality but it prevents sql injections. We should have datatype for SQL, and special literal for it. Functions that execute queries should take only parameters of type SQL (never strings). There should be almost no way to construct SQL from string. There should be 'formatting' operator that allows you to put values in SQL and it should do default escaping on them. As for XSS attack I'd suggest HTML datatype with its own literals. Output functions should only accept HTML (not strings). Similarly there should be almost no way to convert string (or any value) to HTML without escaping it.
- chromatic 16y ago
  That's essentially the same idea. See also "working with Unicode strings in different encodings" or "working with values of different (but compatible units)".

39 points by fogus 16y ago

Giving parsers and lexers to the average man, or removing the need for them altogether, will be the thing that pushes the bar forward next. I don't know if Perl 6 is going to be the medium for this change, but it might be. Arguably, lisp should have given us a world filled with little languages, but it hasn't taken hold. People like syntax, and people like compile time checks. I have yet to dig into the Perl 6 features that make this possible, but having a low-barrier-to-entry way to create DSLs that are compile-time checked and usable opens the door to something big. Domain experts who are not programmers may finally have a lever in the process stronger than the "spec" and weaker than the code. Give them a DSL, with a good compiler, and let them wreak havoc. The big question here I still have is: text? Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Solve that problem, and you've got something big.

loup-vaillant 16y ago
For the ASCII problem, you might want to study structured editors, like http://www.subtextual.org/ http://www.subtextual.org/ Here is an example video: http://www.subtextual.org/subtext2.html http://www.subtextual.org/subtext2.html (It's in Flash or something)
- gfodor 16y ago
  This is what I'm getting at. I don't quite understand his implementation, but the core problem is well articulated here. Good notation is a prequisite to clear thinking. The kinds of notation we have available to use is severely limited by the medium we write software in.
coliveira 16y ago
This is an idea that was pioneered by Smalltalk, where you don't need files to organize your code. But I don't see why it can't be used in other languages, such as Java, using a class-based editor (instead of a file-based one).
- larsberg 16y ago
  Fundamentally, the problem comes down to the other 90% of the toolstack. Even if you get people to use your language, are you really going to get them to change their source code control, every person's text editor, build system, scripts that run over the code, the external localization service, the third party code escrow service, automated performance and security assessment tools, etc.? Anything that is more than a toy project has real requirements, often with tools that are outside of the control of any one vendor or even the company -- they are sometimes requirements of the industry. This code-stored-not-as-text-thing came up so often (it was a recurring Serious Question about once a year while I worked on Visual Studio) that it was simply referred to as the Emacs/grep barrier. Any tool solution that prevents the use of Emacs and grep out of box is just dead unless you have a from-scratch codebase and team. The old Smalltalk hands (some of whom worked on original Smalltalk systems) also like to relate the biggest single issue they had. At one point, they needed to reconstruct the basis library for a new implementation... but there was no code. How do you do a review of a system where the code was (in this case) completely gone, as it had long since become something interpretable as a jump point in a heap image but not as code? (Punchline: they rewrote most of it from scratch, recreating decades-old bugs)
  - shaunxcode 16y ago
    Was this before the concept of "filing out"?
- kristiandupont 16y ago
  I am going to plug one of my favourite editors, Code Browser: http://code-browser.sf.net/ http://code-browser.sf.net/ - it is a folding editor that I have been using for outlining (I have my task list and stuff in it and it is just always open), but it is meant for programming and does it well. It is nice to structure a file into classes and functions etc. and be able to concentrate on one at a time.
  - coliveira 16y ago
    This is a great program. Thanks for sharing.
chromatic 16y ago
Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Latin-1 in truth, but a fair point. Even so, what's wrong with text that alternate input forms (graphics, speech) can fix? You can be inspecific with speech, and you can organize volumes of text with far more efficiency than you can images.
epochwolf 16y ago
> Arguably, lisp should have given us a world filled with little languages, but it hasn't taken hold. Arguably, ruby does this. Look at RSpec, Sinatra, and Rails (Rails 3 especially).
- chromatic 16y ago
  Arguably, ruby does this. If so, you could make the argument that an ORM library does the same thing. I don't make either argument.
dkarl 16y ago
The big question here I still have is: text? Perl 6 gives us little languages, but they are still shackled by ASCII-based notation. Solve that problem, and you've got something big. A text editor is just one way of viewing the bytes that represent source code. Theoretically, any language supports other ways of viewing and editing code. It's just the tools that are missing.
brlewis 16y ago
"People like compile time checks" cannot be the reason interpreted, non-interactive languages like PHP have taken off while compiled, interactive Lisps have not. "People like syntax" has not been tested. I think it's safe to say that people like syntax that's like syntax they've already learned. We don't really know why Lisp hasn't taken hold, and we don't know that it won't take hold at some point in the future.
Goladus 16y ago
I have yet to dig into the Perl 6 features that make this possible, but having a low-barrier-to-entry way to create DSLs that are compile-time checked and usable opens the door to something big. I agree but am also going to say-- I'll believe it when I see it. The article was very light on details. What they're claiming is already facilitated by libraries in many existing languages (including Perl 5) Generally, the barrier to entry for designing and implementing your own language is not the difficulty of understanding how lex and yacc work, it's the need for a thorough and precise understanding of the DSL and every step of the compilation process. Error checking, for example... It's understanding what the compiler knows about your program, and what you can do to figure out things about the program, and how expensive that figuring is going to be. The details here are going to be different for every DSL.
JadeNB 16y ago
It's not clear to me that Perl 6 should be regarded as a prime mover in this: in addition to the old standby Lisp, which you mentioned, there are more recent little-language builders like XMF (http://itcentre.tvu.ac.uk/~clark/xmf.html http://itcentre.tvu.ac.uk/~clark/xmf.html), XLR (http://xlr.sourceforge.net http://xlr.sourceforge.net), Katahdin (http://www.chrisseaton.com/katahdin http://www.chrisseaton.com/katahdin), and Converge (http://convergepl.org http://convergepl.org) (in reverse alphabetical order, because that's how I happened to remember them). Actually, although it's fashionable to sneer at it, I think that TeX (http://www.tug.org http://www.tug.org) provides perhaps the most compelling example of a little-language maker. The languages that one can make on top of it are perhaps rather specialised to text markup, but I think it's telling that LaTeX (http://www.latex-project.org http://www.latex-project.org) was designed, not by fiddling with the source, but rather as a macro layer on top of TeX. LaTeX's not little by any means, but I think it would be hard to find many other examples of such successful languages implemented literally on top of existing languages. Finally, I'm not really familiar with Perl 6, but I'm pretty sure it's still dynamically typed. (After all, it targets Parrot, which is specifically advertised as being a low-level target for dynamically typed languages.) As such, I don't see it solving people's problems with lack of compile-time guarantees from Lisp. (Of course, there's no denying that Perl, in every version, does offer gobs and gobs of syntax. :-) )
- chromatic 16y ago
  I'm not really familiar with Perl 6, but I'm pretty sure it's still dynamically typed. Perl 6 allows optional typing. After all, it targets Parrot, which is specifically advertised as being a low-level target for dynamically typed languages. Parrot allows type checking. Very few languages of which I am aware perform it to any degree, but nothing about Parrot rules it out.

agentultra 16y ago

Why not take the SQLAlchemy approach with its lower-level expression language? Avoid string parsing, implementing specific dialects of SQL, etc. I love perl, but why strings? Writing SQL is not that fun. The cognitive leaps one has to make is annoying. Why not write queries in natural perl statements and translate them to SQL? That being said though, modernperl++ and yay for perl 6. :) </fanboyism>

chromatic 16y ago
Why not take the SQLAlchemy approach with its lower-level expression language? You can indeed do that. I prefer SQL to ORM; de gustibus non est disputandum. I love perl, but why strings? Even structured data, such as a nested data structure serialized to JSON, is a string until you parse it. I don't understand the question.

epochwolf 16y ago

I can't help but think the specific problem mentioned has a specific solution. It's only three letters: ORM.

chromatic 16y ago
For some purposes, ORM works fine. I find ORM frustrating and unpleasant far more than I find it useful.
- wazoox 16y ago
  That. And the fact that ORM usually don't allow to structure your database precisely the way you want, in the good ol' fashioned way.

scotty79 16y ago

I have a much simpler idea that does not provide so much functionality but it prevents sql injections. We should have datatype for SQL, and special literal for it. Functions that execute queries should take only parameters of type SQL (never strings). There should be almost no way to construct SQL from string. There should be 'formatting' operator that allows you to put values in SQL and it should do default escaping on them. As for XSS attack I'd suggest HTML datatype with its own literals. Output functions should only accept HTML (not strings). Similarly there should be almost no way to convert string (or any value) to HTML without escaping it.

chromatic 16y ago
That's essentially the same idea. See also "working with Unicode strings in different encodings" or "working with values of different (but compatible units)".