4 ms·

DSL processing in Scala

40 points by dpapathanasiou 11y ago

tlarkworthy 11y ago
For DSL implementation see string interpolation, http://stackoverflow.com/questions/10708136/how-to-create-dsl-in-scala-for-command-lines-with-minimum-extra-boilerplate http://stackoverflow.com/questions/10708136/how-to-create-ds...
dpapathanasiou 11y ago
I'm still a scala newbie, so any comments/suggestions are appreciated. The full source code is here: https://gist.github.com/dpapathanasiou/b9d85685a0381f1deea0 https://gist.github.com/dpapathanasiou/b9d85685a0381f1deea0
- eranation 11y ago
  Well done on posting this! it's a great way to learn and share your experience of learning new languages and get feedback. (just take the comments by face value without any sentiments, you should learn from them, not make them discourage you, ignore "my eyes are bleeding" type of comments or "Scala is sooo complexxx" unless they have some really constructive criticism in them...) One tiny cosmetic suggestion that popped to my eyes since you asked for comments :) def parseInputRange (s: String): Array[Int] = { val ab = s.split(":").map(_.toInt) (ab(0) to ab(1)).toArray } the `to` method is inclusive on the end of the range, so it is the same as until x + 1 Good luck!
anon3_ 11y ago
To anyone here who works on a large, scala codebase as a team: how do you manage compilation times and narrowing code standards to a manageable set of idioms?
- ecopoesis 11y ago
  The "Scala is slow to compile" meme needs to die in a fire. At my last job we had a large Scala codebase and it compiles faster then our similarly sized C# project, and much faster then my current gig's much smaller Gradle based Java 8 project. We didn't do anything special: just a standard SBT project and a few logical modules. As for feature set, we did code reviews across the whole team to keep everyone on the same page. That said, we used a very large part of Scala, and never really made an attempt to keep out of parts of the language. Our biggest battle was always with Scalaz, which we were always on the verge of adding, but never got enough team buyin.
  - anon3_ 11y ago
    > At my last job we had a large Scala codebase and it compiles faster then our similarly sized C# project, and much faster then my current gig's much smaller Gradle based Java 8 project I never looked at that way before. We were a shop that used scala and Ruby. So many of us come from interpretted languages and are used to code and tests reloading with no downtime. How does scalac match up to C# and Java compilation? I'd be interested in seeing the numbers. Thank you!
  - incepted 11y ago
    > and much faster then my current gig's much smaller Gradle based Java 8 project Yeah I call BS on that. Scala's slow compilation doesn't really bother me that much but Java compiles at least ten times faster.
    - vorg 11y ago
      > much faster then my current gig's much smaller Gradle based Java 8 project > I call BS on that [...] Java compiles at least ten times faster Then it must be Gradle that's slow, probably the Groovy in it.
  - AdieuToLogic 11y ago
    > The "Scala is slow to compile" meme needs to die in a fire. Well said. Additionally, I think people often forget that the same JVM tuning parameters available for production deployments are available for Scala compilations. These can make a huge difference in build times, especially since the default JVM parameters are understandably conservative.
    - incepted 11y ago
      These tunings are for the runtime, what compiler settings are you referring to?
      - desdiv 11y ago
        1. Optimizing JVM parameters can speed up Scala programs. 2. The Scala compiler is itself a Scala program. 1 + 2 => Optimizing JVM parameters can speed up the Scala compiler.
- joostdevries 11y ago
  I use the sbt command ~testQuick. That continuously recompiles scala files as soon as they're saved and reruns only the unittest(s) that are affected by the change. For running my app I use the revolver plugin that offers the same thing: as soon as I save a file it is compiled and the app is restarted. When I use the Play framework this is supported out of the box: I save my file and refresh in the browser and see the changes. This works very well.
  - saryant 11y ago
    scalac is definitely slower than javac, but I do wonder if half the complaints about speed are from people who don't realize you can leave SBT running but rather do an "sbt compile" every time and end up waiting for JVM initialization.
rco8786 11y ago
You should check out the Parser package in the scala standard lib. I maintain a few DSLs using this and it's an absolute dream. https://wiki.scala-lang.org/display/SW/Parser+Combinators--Getting+Started https://wiki.scala-lang.org/display/SW/Parser+Combinators--G... Code looks solid otherwise...at least where I work the style would be to favor functional idioms over for loops but I think that's just personal preference.
- joostdevries 11y ago
  The Parboiled2 framework is often used for building parsers.
- kodablah 11y ago
  That has been removed from the Scala standard lib and is now distributed separately. I am unsure how well it will be maintained by the community. As the other commenter said, parboiled2 is a good choice using macros and it is lightning fast. A lighter weight, runtime choice that was released recently is fastparse[0]. 0 - https://github.com/lihaoyi/fastparse https://github.com/lihaoyi/fastparse
  - rco8786 11y ago
    Oh wow very good to know. We're just now looking to get to 2.11(very large codebase, upgrades are hard) so will definitely need to keep this in mind.
_pmf_ 11y ago
Yuck. No wonder Lispers look down on us.
birdsbolt 11y ago
This doesn't seem to solve the puzzle. As much as DSL processing part is done, the whole point of the puzzle was to make the operations sublinear in time complexity.
sklogic 11y ago
Just for fun and to illustrate what's in my opinion is wrong with Scala approach to DSLs, implemented the same DSL using a specialised meta-language: http://bit.ly/1U2PGlV http://bit.ly/1U2PGlV The meta-language is available here: https://github.com/combinatorylogic/mbase https://github.com/combinatorylogic/mbase

For DSL implementation see string interpolation, http://stackoverflow.com/questions/10708136/how-to-create-dsl-in-scala-for-command-lines-with-minimum-extra-boilerplate http://stackoverflow.com/questions/10708136/how-to-create-ds...

dpapathanasiou 11y ago

I'm still a scala newbie, so any comments/suggestions are appreciated. The full source code is here: https://gist.github.com/dpapathanasiou/b9d85685a0381f1deea0 https://gist.github.com/dpapathanasiou/b9d85685a0381f1deea0

eranation 11y ago
Well done on posting this! it's a great way to learn and share your experience of learning new languages and get feedback. (just take the comments by face value without any sentiments, you should learn from them, not make them discourage you, ignore "my eyes are bleeding" type of comments or "Scala is sooo complexxx" unless they have some really constructive criticism in them...) One tiny cosmetic suggestion that popped to my eyes since you asked for comments :) def parseInputRange (s: String): Array[Int] = { val ab = s.split(":").map(_.toInt) (ab(0) to ab(1)).toArray } the `to` method is inclusive on the end of the range, so it is the same as until x + 1 Good luck!

anon3_ 11y ago

To anyone here who works on a large, scala codebase as a team: how do you manage compilation times and narrowing code standards to a manageable set of idioms?

ecopoesis 11y ago
The "Scala is slow to compile" meme needs to die in a fire. At my last job we had a large Scala codebase and it compiles faster then our similarly sized C# project, and much faster then my current gig's much smaller Gradle based Java 8 project. We didn't do anything special: just a standard SBT project and a few logical modules. As for feature set, we did code reviews across the whole team to keep everyone on the same page. That said, we used a very large part of Scala, and never really made an attempt to keep out of parts of the language. Our biggest battle was always with Scalaz, which we were always on the verge of adding, but never got enough team buyin.
- anon3_ 11y ago
  > At my last job we had a large Scala codebase and it compiles faster then our similarly sized C# project, and much faster then my current gig's much smaller Gradle based Java 8 project I never looked at that way before. We were a shop that used scala and Ruby. So many of us come from interpretted languages and are used to code and tests reloading with no downtime. How does scalac match up to C# and Java compilation? I'd be interested in seeing the numbers. Thank you!
- incepted 11y ago
  > and much faster then my current gig's much smaller Gradle based Java 8 project Yeah I call BS on that. Scala's slow compilation doesn't really bother me that much but Java compiles at least ten times faster.
  - vorg 11y ago
    > much faster then my current gig's much smaller Gradle based Java 8 project > I call BS on that [...] Java compiles at least ten times faster Then it must be Gradle that's slow, probably the Groovy in it.
- AdieuToLogic 11y ago
  > The "Scala is slow to compile" meme needs to die in a fire. Well said. Additionally, I think people often forget that the same JVM tuning parameters available for production deployments are available for Scala compilations. These can make a huge difference in build times, especially since the default JVM parameters are understandably conservative.
  - incepted 11y ago
    These tunings are for the runtime, what compiler settings are you referring to?
    - desdiv 11y ago
      1. Optimizing JVM parameters can speed up Scala programs. 2. The Scala compiler is itself a Scala program. 1 + 2 => Optimizing JVM parameters can speed up the Scala compiler.
joostdevries 11y ago
I use the sbt command ~testQuick. That continuously recompiles scala files as soon as they're saved and reruns only the unittest(s) that are affected by the change. For running my app I use the revolver plugin that offers the same thing: as soon as I save a file it is compiled and the app is restarted. When I use the Play framework this is supported out of the box: I save my file and refresh in the browser and see the changes. This works very well.
- saryant 11y ago
  scalac is definitely slower than javac, but I do wonder if half the complaints about speed are from people who don't realize you can leave SBT running but rather do an "sbt compile" every time and end up waiting for JVM initialization.

rco8786 11y ago

You should check out the Parser package in the scala standard lib. I maintain a few DSLs using this and it's an absolute dream. https://wiki.scala-lang.org/display/SW/Parser+Combinators--Getting+Started https://wiki.scala-lang.org/display/SW/Parser+Combinators--G... Code looks solid otherwise...at least where I work the style would be to favor functional idioms over for loops but I think that's just personal preference.

joostdevries 11y ago
The Parboiled2 framework is often used for building parsers.
kodablah 11y ago
That has been removed from the Scala standard lib and is now distributed separately. I am unsure how well it will be maintained by the community. As the other commenter said, parboiled2 is a good choice using macros and it is lightning fast. A lighter weight, runtime choice that was released recently is fastparse[0]. 0 - https://github.com/lihaoyi/fastparse https://github.com/lihaoyi/fastparse
- rco8786 11y ago
  Oh wow very good to know. We're just now looking to get to 2.11(very large codebase, upgrades are hard) so will definitely need to keep this in mind.

_pmf_ 11y ago

Yuck. No wonder Lispers look down on us.

birdsbolt 11y ago

This doesn't seem to solve the puzzle. As much as DSL processing part is done, the whole point of the puzzle was to make the operations sublinear in time complexity.

sklogic 11y ago

Just for fun and to illustrate what's in my opinion is wrong with Scala approach to DSLs, implemented the same DSL using a specialised meta-language: http://bit.ly/1U2PGlV http://bit.ly/1U2PGlV The meta-language is available here: https://github.com/combinatorylogic/mbase https://github.com/combinatorylogic/mbase