4 ms·

Could Dart use this solution, which Python uses and Oil shell now uses? 1. Newlines are significant tokens and they behave like semicolons (this is how shell i

by chubot 7y ago

Could Dart use this solution, which Python uses and Oil shell now uses? 1. Newlines are significant tokens and they behave like semicolons (this is how shell is) 2. Within () [] {}, newlines are suppressed. The lexer has to recognize the matching pairs. (Oil added an expression language to shell which does this) This means that var x = 1 + 2 is not valid because there's a statement terminator after +, but var x = f(1 + 2) is valid. I feel like this captures the fast majority of cases where you want to break a statement over lines. I don't know Dart, but since it seems to have JavaScript-like syntax, I don't see why that wouldn't work? JavaScript has some weird rules, the common gotcha seems to be is: return { } // oops this does not return a dictionary like I thought With those rules you would write: return { } and hopefully the former is a syntax error, not a mis-parsed program. ---- edit: OK my guess is that Dart probably doesn't want to enforce a brace style like Go (and Oil) do? That is in Go only if (x) { } is valid, rather than if (x) { } which is pretty much like the 'return' example. I have been using that brace style for so long that I forgot other people don't :)

pansa2 7y ago
> Could Dart use this solution, which Python uses and Oil shell now uses? This works well for Python, because the following causes IndentationError: x = 1 + 2 In a language without significant indentation, would this be two statements, an assignment and a separate unary `+` expression? Is that OK? If not, one possible solution would be to disallow arbitrary expression statements. AFAIK this is what Go does. > JavaScript has some weird rules, the common gotcha seems to be `return \n {...}` Disallowing expression statements also fixes this particular example, but it doesn't solve the `return` problem in general. It's not practical to disallow standalone function calls, so `return \n f()` still has the same issue. Lua solves this by enforcing that a `return` must be the last statement in a block.
- dragonwriter 7y ago
  > It's not practical to disallow standalone function calls If your language structurally differentiated procedures (i.e., void-returning functions) from (other) functions, it would be practical to disallow standalone functions (not procedures), though you might have a modifier to explicitly discard the result of a function that allows it to be standalone.
  - zem 7y ago
    > you might have a modifier to explicitly discard the result of a function that allows it to be standalone. ocaml does this and i find it very annoying, even though i know it's theoretically saving me from accidentally not consuming a return value.
munificent 7y ago
> Within () [] {}, newlines are suppressed. Unfortunately, no. That doesn't work. Consider: var map = { key: long - value } // block { long - value } The newlines should be ignored inside the map literal, but not inside the block. A correct semicolon insertion for this should be: var map = { key: long // <-- no semicolon here - value }; // block { long; // <-- but there is one here - value; } But the lexer doesn't know when curlies are maps and when they are blocks. The parser does, which means you could theoretically define the semicolon insertion rules in the grammar, which is what we'd likely have to do, but that makes it much more complex.
- chubot 7y ago
  Ah OK, that makes sense. So Python doesn't have that issue because it doesn't overload braces for hashes and blocks. Oil does overload them, but it has a separate lexing mode for statements and expressions. It switches when it sees say the 'var' keyword, so the right of var x = {a: 3} is lexed as an expression rather than a statement. This sort of fell out of compatibility constraints with shell, but ended up being pretty convenient and powerful. The lexing mode relies pretty strongly on having the "first word", e.g. 'var' or 'func', Pascal-style. So yeah I can see how it would be more complex with Java-style syntax.
  - munificent 7y ago
    > So Python doesn't have that issue because it doesn't overload braces for hashes and blocks. Yes, and also because lambdas in Python can only have expression bodies, not statements. That means you can never have a statement nested inside an expression. This is important because Python's rule to ignore all newlines between parentheses would fall apart if you could stuff a statement-bodied function inside a parenthesized expression.
    - chubot 7y ago
      Yes the lambda issue is something I ran into for Oil. Although this part of the language is deferred, the parser is implemented: # Ruby/Rust style lambda, with the Python constraint that the body can only be an expression var inc = |x| x + 1 # NOT possible in Oil because | isn't a distinct enough token to change the lexer mode var inc = |x| { return x + 1 } # This is possible but I didn't implement it. # "func" can change the lexer mode so { is known to start a block. # In other words, { and } each have two distinct token IDs. var inc = func(x) { return x + 1 } I think this is a decent tradeoff, but it's indeed tricky and something I probably spent too much time thinking about ... the perils of "familiar" syntax :-/

by chubot 7y ago

pansa2 7y ago

> Could Dart use this solution, which Python uses and Oil shell now uses? This works well for Python, because the following causes IndentationError: x = 1 + 2 In a language without significant indentation, would this be two statements, an assignment and a separate unary `+` expression? Is that OK? If not, one possible solution would be to disallow arbitrary expression statements. AFAIK this is what Go does. > JavaScript has some weird rules, the common gotcha seems to be `return \n {...}` Disallowing expression statements also fixes this particular example, but it doesn't solve the `return` problem in general. It's not practical to disallow standalone function calls, so `return \n f()` still has the same issue. Lua solves this by enforcing that a `return` must be the last statement in a block.

dragonwriter 7y ago
> It's not practical to disallow standalone function calls If your language structurally differentiated procedures (i.e., void-returning functions) from (other) functions, it would be practical to disallow standalone functions (not procedures), though you might have a modifier to explicitly discard the result of a function that allows it to be standalone.
- zem 7y ago
  > you might have a modifier to explicitly discard the result of a function that allows it to be standalone. ocaml does this and i find it very annoying, even though i know it's theoretically saving me from accidentally not consuming a return value.

munificent 7y ago

> Within () [] {}, newlines are suppressed. Unfortunately, no. That doesn't work. Consider: var map = { key: long - value } // block { long - value } The newlines should be ignored inside the map literal, but not inside the block. A correct semicolon insertion for this should be: var map = { key: long // <-- no semicolon here - value }; // block { long; // <-- but there is one here - value; } But the lexer doesn't know when curlies are maps and when they are blocks. The parser does, which means you could theoretically define the semicolon insertion rules in the grammar, which is what we'd likely have to do, but that makes it much more complex.

chubot 7y ago
Ah OK, that makes sense. So Python doesn't have that issue because it doesn't overload braces for hashes and blocks. Oil does overload them, but it has a separate lexing mode for statements and expressions. It switches when it sees say the 'var' keyword, so the right of var x = {a: 3} is lexed as an expression rather than a statement. This sort of fell out of compatibility constraints with shell, but ended up being pretty convenient and powerful. The lexing mode relies pretty strongly on having the "first word", e.g. 'var' or 'func', Pascal-style. So yeah I can see how it would be more complex with Java-style syntax.
- munificent 7y ago
  > So Python doesn't have that issue because it doesn't overload braces for hashes and blocks. Yes, and also because lambdas in Python can only have expression bodies, not statements. That means you can never have a statement nested inside an expression. This is important because Python's rule to ignore all newlines between parentheses would fall apart if you could stuff a statement-bodied function inside a parenthesized expression.
  - chubot 7y ago
    Yes the lambda issue is something I ran into for Oil. Although this part of the language is deferred, the parser is implemented: # Ruby/Rust style lambda, with the Python constraint that the body can only be an expression var inc = |x| x + 1 # NOT possible in Oil because | isn't a distinct enough token to change the lexer mode var inc = |x| { return x + 1 } # This is possible but I didn't implement it. # "func" can change the lexer mode so { is known to start a block. # In other words, { and } each have two distinct token IDs. var inc = func(x) { return x + 1 } I think this is a decent tradeoff, but it's indeed tricky and something I probably spent too much time thinking about ... the perils of "familiar" syntax :-/