6 ms·

One of the best parts of Go is that the grammar is (almost) entirely context-free - it is one of only two languages I know of in this regard - Lisp is the other

One of the best parts of Go is that the grammar is (almost) entirely context-free - it is one of only two languages I know of in this regard - Lisp is the other[0]. This comes in handy because it is very easy to parse Go and make syntactic transformations while preserving semantic meaning. The "go fix" tool (which was used to port pre-1.0 code to Go 1.0) meant that Go never had a "py3k" moment[1]. This is not like python's "2to3" tool, which is reasonably good, but still doesn't handle every edge case. My favorite part about writing Go is the "go fmt" tool. Because all code in the standard lib (and most third-party code by convention) is formatted using the exact same tool, it is very easy to read any Go source code I find online. I don't need to worry about stylistic differences or bikeshedding. And the "go fmt" tool exists (and is so simple to implement) in part because Go's grammar is so simple. Go is emphatically not a Lisp (it's not even a functional language[2]). However, this one trait - the ability to make deterministic and reliable syntactic transformations - are at the core of what make Lisp's macro's powerful[3]. Like Lisp, Go's beauty lies not in the syntax but in the semantics. While I would appreciate a Go (or a Lisp) that had both more beautiful syntax and beautiful semantics, I would not want to compromise one bit on the latter. [0] (EDIT) For the PLT nerds & pedants among us, I think mehrdada is right - in actuality, the grammar (IIRC) is fully context free, but in other languages, many of the rules which define a valid program cannot be encapsulated by the language grammar (whereas a higher proportion can). So it's less that the grammar is "(almost) entirely context-free" and more that the language is "(almost) entirely described by its grammar". That said, it has zero impact on the rest of my comment. :) [1] Or, in the case of Python, not so much a "moment" as 5+ years and counting. [2] It supports first-class functions, but that's about it. [3] It does not mean that Go has macros, or even that Go can provide the same things that Lisp macros provide. It simply means that Go derives some of its power from the same place that Lisp macros derive their power.

nailer 13y ago
You can still make syntactic transformations in the same way you would for any other language: use an AST. No syntax compromise required.
fenollp 13y ago
> One of the best parts of Go is that the grammar is (almost) entirely context-free - it is one of only two languages I know of in this regard[0] There is also Erlang [1] and my personal rethought version [2] featuring no ,.; and more… [1] https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_parse.yrl https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_... [2] https://github.com/fenollp/kju https://github.com/fenollp/kju
mehrdada 13y ago
> One of the best parts of Go is that the grammar is (almost) entirely context-free You mean almost regular. Almost all programming language grammars are context-free. Regularity is a much higher bar to reach.
- Crito 13y ago
  Pretty sure he means 'almost' context free (as opposed to 'very' context sensitive. C is context sensitive. A language like C++ is heavily context sensitive (actually, C++'s templates make it recursively enumerable iirc, so even worse..)). Being regular would mean that you could decide it with regular expressions. Regular expressions are insufficiently powerful for matched parentheses of arbitrary depth, which is a good hint that Lisp and Go are not regular (or 'near' regular).
  - mehrdada 13y ago
    > Being regular would mean that you could decide it with regular expressions. Hence almost regular. Of course, there is no concrete definition for "almost regular", but what I mean by it (Rob Pike himself also characterizes it as such too[1]), is that you get to decide which way to parse a construct quite definitely early in the token stream and you don't have to read arbitrary following tokens to figure out how to construct a parse tree branch. C++ is the exception here. Most other block oriented high level programming language (at the syntax level) are indeed fully context-free, so just being context-free does not deserve a high prize. [1]: http://www.infoq.com/presentations/Go-Google http://www.infoq.com/presentations/Go-Google
    - Crito 13y ago
      Do you have the timestamp of when in this talk Rob Pike mentions this? If he says that it is "almost regular", then that settles it in my book, but I am pretty certain that you are wrong about most block oriented high level languages being context free. C++ is an extreme case, as it's Turing complete, but C is definitely context sensitive (http://eli.thegreenplace.net/2007/11/24/the-context-sensitivity-of-cs-grammar/ http://eli.thegreenplace.net/2007/11/24/the-context-sensitiv...), Perl is just as bad as C++ (http://www.perlmonks.org/?node_id=663393 http://www.perlmonks.org/?node_id=663393), and I can't find a link for it right now but IIRC the Java is context sensitive for some reason too.
derefr 13y ago
It'd certainly be possible to have a tool -- let's say "go edit" -- which would read a set of user preferences, and apply transformations (e.g. setting spaces or tabs, variables-in-camel-case, comment-styles , etc.) to Go code. It'd be fine as long as each transformation was bijective in a way "go fmt" could revert. Interestingly, the code existing between the "go edit" and "go fmt" steps wouldn't necessarily need to be executable as plain Go. If the Go compiler's parser+lexer shared their canonicalization logic with "go fmt", then feeding "go edit"-munged Go to the compiler would implicitly re-canonicalize it before compiling it. So you could have transformations involving things like significant whitespace. In fact, the pair of filters "go edit" and "go fmt" could even be made into a read/write filter-pair for a FUSE filesystem: in effect, the code on-disk would always be represented in the canonical "go fmt"ed manner, while the code you'd see in your editor would always be in your own personal "go edit" style. And then, 56 years after McCarthy's LISP, we'd finally have our M-expressions. ;) (And the serialized representation could finally be stored as a binary AST tree, and we could embrace visual-symbolic hybrid languages, and entertain divergent syntaxes with unified AST-level semantics, and have one universal runtime library, and...)
- deleted 13y ago
  [deleted]
- DAddYE 13y ago
  This is mostly the aim of this project.
masklinn 13y ago
> One of the best parts of Go is that the grammar is (almost) entirely context-free If the threshold is almost-context-free there's a host of matching languages: C (IIRC it's context-free if you remove typedefs), Rust, Pascal and descendants (Oberon), Python, java, … If the threshold is actual context-freedom, I don't recall anything outside lisps.
stcredzero 13y ago
Go should be in an excellent position to have Refactoring Editors. Smalltalk had a Refactoring Browser that worked well enough that it changed the economics/trade offs around refactoring. (Literally an order of magnitude easier refactoring combined with immediate/really fast responsiveness.) Such tools, positioned such that they are defacto standards can cause qualitative changes in how programming happens in a language.

nailer 13y ago

You can still make syntactic transformations in the same way you would for any other language: use an AST. No syntax compromise required.

fenollp 13y ago

> One of the best parts of Go is that the grammar is (almost) entirely context-free - it is one of only two languages I know of in this regard[0] There is also Erlang [1] and my personal rethought version [2] featuring no ,.; and more… [1] https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_parse.yrl https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_... [2] https://github.com/fenollp/kju https://github.com/fenollp/kju

mehrdada 13y ago

> One of the best parts of Go is that the grammar is (almost) entirely context-free You mean almost regular. Almost all programming language grammars are context-free. Regularity is a much higher bar to reach.

Crito 13y ago
Pretty sure he means 'almost' context free (as opposed to 'very' context sensitive. C is context sensitive. A language like C++ is heavily context sensitive (actually, C++'s templates make it recursively enumerable iirc, so even worse..)). Being regular would mean that you could decide it with regular expressions. Regular expressions are insufficiently powerful for matched parentheses of arbitrary depth, which is a good hint that Lisp and Go are not regular (or 'near' regular).
- mehrdada 13y ago
  > Being regular would mean that you could decide it with regular expressions. Hence almost regular. Of course, there is no concrete definition for "almost regular", but what I mean by it (Rob Pike himself also characterizes it as such too[1]), is that you get to decide which way to parse a construct quite definitely early in the token stream and you don't have to read arbitrary following tokens to figure out how to construct a parse tree branch. C++ is the exception here. Most other block oriented high level programming language (at the syntax level) are indeed fully context-free, so just being context-free does not deserve a high prize. [1]: http://www.infoq.com/presentations/Go-Google http://www.infoq.com/presentations/Go-Google
  - Crito 13y ago
    Do you have the timestamp of when in this talk Rob Pike mentions this? If he says that it is "almost regular", then that settles it in my book, but I am pretty certain that you are wrong about most block oriented high level languages being context free. C++ is an extreme case, as it's Turing complete, but C is definitely context sensitive (http://eli.thegreenplace.net/2007/11/24/the-context-sensitivity-of-cs-grammar/ http://eli.thegreenplace.net/2007/11/24/the-context-sensitiv...), Perl is just as bad as C++ (http://www.perlmonks.org/?node_id=663393 http://www.perlmonks.org/?node_id=663393), and I can't find a link for it right now but IIRC the Java is context sensitive for some reason too.

derefr 13y ago

It'd certainly be possible to have a tool -- let's say "go edit" -- which would read a set of user preferences, and apply transformations (e.g. setting spaces or tabs, variables-in-camel-case, comment-styles , etc.) to Go code. It'd be fine as long as each transformation was bijective in a way "go fmt" could revert. Interestingly, the code existing between the "go edit" and "go fmt" steps wouldn't necessarily need to be executable as plain Go. If the Go compiler's parser+lexer shared their canonicalization logic with "go fmt", then feeding "go edit"-munged Go to the compiler would implicitly re-canonicalize it before compiling it. So you could have transformations involving things like significant whitespace. In fact, the pair of filters "go edit" and "go fmt" could even be made into a read/write filter-pair for a FUSE filesystem: in effect, the code on-disk would always be represented in the canonical "go fmt"ed manner, while the code you'd see in your editor would always be in your own personal "go edit" style. And then, 56 years after McCarthy's LISP, we'd finally have our M-expressions. ;) (And the serialized representation could finally be stored as a binary AST tree, and we could embrace visual-symbolic hybrid languages, and entertain divergent syntaxes with unified AST-level semantics, and have one universal runtime library, and...)

deleted 13y ago
[deleted]
DAddYE 13y ago
This is mostly the aim of this project.

masklinn 13y ago

> One of the best parts of Go is that the grammar is (almost) entirely context-free If the threshold is almost-context-free there's a host of matching languages: C (IIRC it's context-free if you remove typedefs), Rust, Pascal and descendants (Oberon), Python, java, … If the threshold is actual context-freedom, I don't recall anything outside lisps.

stcredzero 13y ago

Go should be in an excellent position to have Refactoring Editors. Smalltalk had a Refactoring Browser that worked well enough that it changed the economics/trade offs around refactoring. (Literally an order of magnitude easier refactoring combined with immediate/really fast responsiveness.) Such tools, positioned such that they are defacto standards can cause qualitative changes in how programming happens in a language.