9 ms·

Typelevel Scala and the future of the Scala ecosystem

135 points by LiveTheDream 12y ago

pkinsky 12y ago
I'm conflicted. Type constructor partial application would be great (the current type lambda syntax is just awful), but I really don't want to see Scala fragmented.
- mhax 12y ago
  I think it's fantastic news - even if the net result is just forcing typesafe to address concerns about scala that have been festering for some time.
- jghn 12y ago
  Agreed on conflicted-ness. The thing that concerns me the most is that from observation it feels like many of the high muckity mucks of the Scala world would probably be heading in this direction. There's nothing inherently wrong with that, but IMO that increases the chance of serious fragmentation.
robert_tweed 12y ago
What's the history behind this fork and what is it likely to mean in terms of fragmentation? In particular what are the irreconcilable beliefs on each side that have led to this? I haven't tried Scala yet, but my gut reaction is I should stick with Clojure until the dust settles. A fragmented community around a relatively niche language is never good.
- pkinsky 12y ago
  It's not a niche language. Household names including Twitter, Meetup, Foursquare, and Coursera run on Scala.
  - willismichael 12y ago
    According to Cedric Beust, it's still a language that "pretty much hardly anyone uses". http://beust.com/weblog/2014/05/11/more-about-language-popularity/#comments http://beust.com/weblog/2014/05/11/more-about-language-popul...
    - frowaway001 12y ago
      Yes, let's cite this guy who is "so uninterested in Scala" that he spends an considerable amount of time maintaining multiple user accounts to spread FUD about the langauge in various places and makes up stories about how "his team/group/company/... was trying to use Scala but abandoned it due to all the horrible issues with it". You know that he is really persistent (despite claiming that he doesn't care about Scala) when the community starts maintaining a list of his sockpuppet accounts.
    - jroesch 12y ago
      Anyone who tries to make the claim that "pretty much hardly anyone uses" is standing on shaky ground. It definitely has not surpassed Java in popularity but it seems to be quickly gaining traction at lots of large companies and is being used more and more. I was recently at Scala by the Bay in San Fransisco and it seems each year more and more companies are on board. If LinkedIn, Twitter, PayPal, Cisco, Verizon, and more are all using Scala I would hardly call it a niche language anymore.
    - pkolaczk 12y ago
      There are a few really big and impactful things already implemented in Scala, and some companies may be using it without even actually writing any Scala code (for they provide Java APIs): Play, Kafka, Spark, Akka.
- ssmoot 12y ago
  As someone who's been using Scala for the better part of a couple years: This seems more like a more ambitious version of the Shapeless and ScalaZ libraries (with some of the same crowd/authors). I (and I suspect most people who program in Scala for a living) don't use those libraries. Not because they're in any way bad. But there are trade-offs using them IME. The biggest being compile-time. So I get by fine without. I think this is probably generally good news. In the same way that (I believe) JRuby pushed MRI forward and popularized the case for a multi-threading VM, more advanced type features in this fork probably benefit the Scala community as a whole. I certainly don't see how it could be harmful. Typesafe isn't going away. The vast majority of businesses will still use their compiler. If you didn't use the advanced-type-based-libraries before, this news probably isn't going to impact you anyway. So from where I'm standing, having made a late 2.9 to 2.10 transition, and now having made several 2.10 to 2.11 transitions, Scala seems a very mature language/platform to me. Libraries are generally upgraded fairly quickly, and source wise, those releases broke very little (2.11 being the biggest culprit just because of the modularization effort, but that's pretty trivial to resolve). I dunno. It's not Java. But it's not Ruby either. It's gotten new features since 2.9, but it's broken or changed very little. I find working with it, even working through upgrades, very stable and mostly frustration free.
  - runT1ME 12y ago
    As someone involved in the scala community (and scalaz) community, I'd say about 60% of the companies I talk to at the various meetups use at the very least #scalaz. I was pleasantly surprised.
    - jghn 12y ago
      Could this be a biased sample though? Not saying this is the case but it seems logical that the people at the scala meetups are cut from a cloth that would make them more likely to be using things such as scalaz. I know that my local scala meetup seems to be more heavily oriented towards the functional end of things than the bulk of the scala developers I interact with. OTOH, perhaps the bias is with my sample.
- Shamanmuni 12y ago
  The fork means nothing dramatic. I'll make a short summary of both positions: Typesafe: we need to offer our clients a good guarantee on stability and compatibility with Java 8 or they won't take Scala seriously. So, for the moment we won't introduce new features or make big changes in the compiler. Typelevel: we are OK with compatibility and stability, but there are some features that we really want which either don't exist or are a bit quirky with the current compiler. So we will make a fork in which we can add these things while staying binary and merge compatible with yours. If they make it work, this is great news. There wouldn't be fragmentation as "bleeding edge Scala" would be binary and merge compatible with "stable Scala". And nobody should have hurt feelings, both positions are completely sensible.
- VLM 12y ago
  By analogy its not terribly far off from Debian issuing a "testing" branch off "unstable" a couple years or decade ago. Its a little more extreme than that, but not much. Usually forks are more dramatic. This isn't MariaDB or something like that. For PR reasons they probably should have stolen the Debian "testing" vs "unstable" names instead of getting all forky which is scarier sounding.
- lmm 12y ago
  > What's the history behind this fork and what is it likely to mean in terms of fragmentation? In particular what are the irreconcilable beliefs on each side that have led to this? What I've managed to put together, corrections welcome: Typesafe has been moving to stabilize Scala, primarily to make it more attractive to enterprise clients. Prior to 2.10, even point releases broke binary compatibility; now compatibility is maintained for those and they're making efforts to make minor releases binary compatible. The tooling they've produced and the testing framework that builds all the major libraries in the ecosystem and checks for binary compatibility means they now have quite possibly the best binary compatibility regime of any language. Typesafe recently published a roadmap for the next few releases, saying that the next release would focus exclusively on Java 8 support and remain compatible with the current release. This means work to merge any new features won't even start until January 2016, and those features are unlikely to be in an official release until 2017. Less frequent and more compatible releases are what a lot of people have been asking for, and it would be very awkward to maintain Java 8 and pre-Java 8 releases of Scala. So the typesafe position that there can be no new features in the release that introduces Java 8 (so that people will definitely be able to switch to Java 8 without making any code changes) makes a certain amount of sense. At the same time it's meant Scala's getting "frozen" in a bit of a weird state; macros were a recently-introduced experimental feature and the official version is kind of half-finished (there's a branch with a more complete version). And there are a couple of features that are quite small and simple but would be a tremendous help for the kind of code the typelevel guys work on (e.g. partial application of types would be a very simple syntactic rewrite, and there's already an implementation as a compiler plugin), and there's long been a feeling that they were coming as soon as we figured out the little details - so to then be told that we won't see these features until 2017 at the earliest was a bit upsetting. So there was a bit of consternation among the typelevel folks - a group who maintain some quite abstract and powerful libraries that tend to push the type system a bit further than most user code. Some of them started seriously talking about porting their libraries to one of the smaller, more experimental languages with a more elegant type system (Idris, Ermine, or I forget the other option). At the same time, the Scala community now has a lot of expertise in maintaining binary compatibility across quite complex changes, so at some point Miles came up with this idea. I don't think a fork would have happened otherwise - most of the goals can be achieved by compiler plugins (at the cost of being harder to maintain), and if they were going to break compatibility they might as well have jumped all the way to another language. But if the typelevel guys can pull off this "conservative fork", where they'll use their own compiler to compile their libraries but those libraries will then be binary compatible with "vanilla" scala and usable by ordinary scala users, then they get the best of both worlds. It's ambitious, but I hope it works out. > I haven't tried Scala yet, but my gut reaction is I should stick with Clojure until the dust settles. A fragmented community around a relatively niche language is never good. If they're maintaining binary compatibility then that should keep fragmentation to a minimum. Honestly I don't think this changes much compared to how it was with compiler plugins - if you're building some super-duper-typey project with these advanced features then maybe you'll muck around and use this forked compiler, but if you're a normal user then you can ignore it and carry on using their libraries without noticing any different. Plus even if the Scala community were to split in two, I think either half would be bigger than the Clojure community.
- kamaal 12y ago
  >>In particular what are the irreconcilable beliefs on each side that have led to this? I'm not familiar with this one. But in general most of these forks are generally out of personality clashes where one person just refuses to take another as the boss. Scala is still not among the top languages used in the industry. This kind of my way or the highway attitude will only fragment the already fragile ecosystem. Imagine any serious software shop wanting to use Scala reading this. If I want to be writing an enterprise application which I plan to keep running for the next decade, should I be choosing a language which isn't just not widely adopted, but the existing ecosystem itself has forks because a few members don't give two hoots about stability(Which is far more important to 99% of the software shops out there, than adding new features). Instead of doing something like this, and putting your broader community of users in perpetual confusion as to which ecosystem to use, you could rather contribute to something meaningful.
  - frowaway001 12y ago
    Sometimes it's useful to read the blog post before commenting.
    - kamaal 12y ago
      I did, and its the same template story always. We don't agree with technical decisions made by Set A people to be in best interests of the community, therefore we plan to fork Y, with intelligent Set B people. The actual story is always about control and ego.
      - frowaway001 12y ago
        You really didn't read it.
dkarapetyan 12y ago
So the elitist community component of Scala that is constantly trying to turn it into Haskell has decided they need an island of their own. Great, they can have it. Now I'll know where to go when I need monadic and monoidal libraries for serving HTTP requests using nothing but type level hacks. In all seriousness though I don't really know what this is going to accomplish. The article mentions the academic and non-academic components of Scala and it is clear the author falls on the academic side but if that's the case then why not just go and hack on something like Haskell (or Coq), a perfectly unapologetic academic language that has tried to avoid success at all costs. If push comes to shove I'm with team Odersky.
- pkinsky 12y ago
  >Now I'll know where to go when I need monadic and monoidal libraries for serving HTTP requests using nothing but type level hacks. Monadic libraries with classes like Option, Future, and List? Used in elitist features like for comprehensions? Anyone who writes Scala on a regular basis deals with monads constantly. Why not take 15 minutes and learn how they work?
  - dkarapetyan 12y ago
    Oh no that is all a little too pedestrian for me. I've moved my sights much higher now. These days I use template haskell and applicatives to generate HTML forms: https://github.com/yesodweb/yesod/wiki/Multi-select-field-populated-from-database-%28using-esqueleto%29 https://github.com/yesodweb/yesod/wiki/Multi-select-field-po.... I'm all for applying theory to enrich the practice of programming but lets be reasonable here. Lists and Options existed way before anyone added monadic sugar on top of it as did mapping and filtering collections. I like category theory as much as the next guy but lets just pull back a little bit. Not everything needs to be point-free, functorial, and shapeless.
    - pkinsky 12y ago
      >I'm all for applying theory to enrich the practice of programming but lets be reasonable here. Lists and Options existed way before anyone added monadic sugar on top of it. I like category theory as much as the next guy but lets just pull back a little bit. Not having needs to be point-free and functorial. I agree that Spire/Shapeless/Scalaz tend to go far beyond anything I would use in a production app, but Options/Futures/etc are barely usable without for comprehensions, which are just syntactic sugar for flatMap. I see the ability to write for { x <- f1() // returns Option[T1] y <- f2(x) // returns Option[T2] z <- f3(y, x) // returns Option[T3] } yield foo(x, y, z) // for expression returns Option[T4] as a huge part of Scala's success
saosebastiao 12y ago
My biggest frustration with Scala is the exact opposite of their frustration with Typesafe's Scala: it moves too fast. Scala's backwards compatibility problems are obnoxious for the people who don't feel the need to reinvent everything. I can find a version of Foo that works with 2.11 but the only version of Bar I can use only has artifacts for <=2.10. The play framework exemplifies these problems. Documentation doesn't keep up with releases, blog posts and tutorials don't either. Sometimes I'm looking for example code for some plugin or library, and the code only works with some other version of another dependency that I can't use. I totally understand that people prefer Agile development, but you can't combine Agile development with non-Agile documentation, unless you want to create a framework that only experts can use.
- ffreire 12y ago
  We've been bitten by this as well. We started several projects within a few months of each that also used what we believed to be similar enough versions of Play but there are just enough inconsistencies between versions of play (and Scala language versions as well) to make it exceedingly frustrating to work with.
- pea 12y ago
  Yeah I've wasted so much time dealing with code for the wrong version of Play. It's horrendously organised: case in point, the JSON parsing (WTF?)
  - Rezo 12y ago
    The documentation for JSON parsing in Play is hilarious. You read the "The Play JSON library Basics" document, it looks like a lot of code for something that should be simple, but you go ahead and implement as explained. After a while, the verbosity starts bugging you, so you read the "JSON Reads/Writes/Format Combinators", followed by "JSON transformers" and finally the "JSON Macro Inception" pages. The punchline in the last section can be summarized as "forget all that other stuff, just do "implicit val personFmt = Json.format[Person]". I was happy to be able to delete hundreds of lines of code from my project after learning this, but couldn't you just have told me this upfront since it probably covers 99% of all cases? The documentation also has an aura of "Isn't this all very clever?" feeling to it. It's also quite hard to search for anything, as outdated versions of the docs keep popping up in Google as the first results.
saryant 12y ago
Background: I saw Miles' tweet on this last week but didn't follow the apparent brouhaha that followed. All I know of him was his (hilarious) talk at Scala Days in June. As someone who uses Scala daily, I don't think this is a good thing. I fall squarely into the Typesafe side of the camp—we don't use Scalaz or Shapeless or much of anything from the Typelevel project but Akka and Play are core technologies for us. We don't have a Typesafe subscription, we just use Akka and Play, a lot. > As I observed earlier, there are already several variants of the language in existence and it has been clear for a long time that different sections of the community have different interests. I don't think the author's examples really hold water. Nobody is using the Dotty compiler commercially, Scala.js is still in its infancy and while I know large organizations have at times run on minor Scala forks (I think Twitter used to but is now on 2.10.4), I'm not sure I believe that this is a major constituency. The official Scala is the Scala. If this fork gains steam, it's going to be a huge opportunity for FUD around Scala and will hurt its adoptability, especially if existing Scala teams now have to choose sides. That said, if the Haskell side of Scalaland wants to hang out somewhere else, that might be good too. Too many Scala questions on SO are answered (and upvoted) with obscure Scalaz solutions that make no sense to someone coming in from Ruby or Java. I like functional programming. I like how monads can simplify things. But I also have to write code that someone without a lot of Scala (or Haskell) experience can maintain.
- lmm 12y ago
  > I like functional programming. I like how monads can simplify things. But I also have to write code that someone without a lot of Scala (or Haskell) experience can maintain. I agree with this, but I think a same replacement for the type lambda syntax goes a long way towards making those scalaz solutions maintainable.
- stuhood 12y ago
  Twitter has not used a fork of the scala compiler/library since the pre-2.8 days (and maybe not even then, since we have always had a fair fraction of open source code.)
noelwelsh 12y ago
To people worried about fragmentation, retaining binary compatibility means it really doesn't matter. Scalaz can be written in the Typelevel fork and you can use it without issue from code compiled with the standard Scala compiler. Also, from the announcement: "As a conservative fork, our aim is to be merge compatible from the Typesafe Scala compiler, and a key objective is to feed pull requests (many of which will resolve long standing Scala bugs or feature requests) back to the Typesafe Scala compiler." So we aren't looking at radical changes, and hopefully these changes will get into Typesafe's compiler. So basically it's an opt-in better Scala compiler with no effect on compatibility. What's not to like?
zak_mc_kracken 12y ago
Regardless of the outcome of this initiative (which has little chance of succeeding, but best luck to them), this is pretty bad for the Scala ecosystem overall. There's a general lack of confidence from the community toward Scala and Typesafe and this certainly contributes greatly to the general uncertainty about Scala's chances of success.
- frowaway001 12y ago
  I guess it was only a matter of time until the "Scala will surely die!!! Really!!! This time it will go down!!!" crowd would show up ...
cryptos 12y ago
I'm worried about this fork. Scala is far away from being a wildly used industry language. Yes, there are some well known companies using Scala, but how many of them develop a really big part of their software with Scala? Not so much. If you look around you'll see Java, C#, C, C++ (depending on the business) and actually not so much Scala. The ones who think forking Scala does help the community are in a reality distortion field. Take a look and Linux desktops to see that diversity is not always good, especially if you're in a niche.
- propensive 12y ago
  I think there is a lot of Scala usage you perhaps don't see. As an example from the corporate world, at least six major international banks (probably more) are doing a significant amount of their development in Scala, some with teams in the hundreds. That's a solid market for Typesafe right there, which isn't going to go away overnight, and won't be stirred by the existence of the Typelevel fork of Scala. Furthermore, they should benefit from its existence because it represents a means for more people to be able to contribute to and enhance the language, without negatively impacting the stable, reliable releases from Typesafe.
benjaminjackman 12y ago
I don't understand what features they want to add beyond type lambda's, singleton types, and literals for bytes / shorts. 1. I understand if you spend all day making shapeless / scalaz the lack of sugar for type lambda's is annoying. Why not make a SIP to add some sugar there? 2. Singleton types would have use in ScalaJs as well for encoding. There is already a SIP for this. 3. Byte / Short literals, seem like again, a SIP could be made for the next release. So all this boils down to getting SIPs into the language faster. So I think the real feature is not having to wait 3+ years for language improvements that significantly improve the QOL of some library writers. That is a feature I would like as well. I really miss being able to open the mailing list and scan for exciting new features, Instead I know even if someone has a great idea, it'll be years before it sees the light of day. This sucks and I've moved onto to getting my neat idea -> wow I can use this in my language fix to ScalaJS[1]. I think (thought?) about Scala as ycombinator for language features. I see this enterprise mindset coming in now trying to tie down what Scala is and ultimately I fear that it will kill all the things that has made Scala popular in the first place. Pushing to a more infrequent release model might help enterprise customers but it absolutely crushes library developers in the vanguard working to make the Scala of tomorrow by fighting to get the language features of today. They need turn around on the improvements listed above and can't wait 4+ years to get them. So does Scala take all the strength it has built up from acting as a feature incubator for the last 10 years and switch and lock down what works, it is in a prime position to do so? It has a virtual monopoly on it's space. Or does Scala continue to act as a language feature incubator giving its coders early access to cutting edge CS idioms in a mainstream language. There are a lot of CS ideas waiting to get their shot. Some people would see this situation and say Scala is facing a mutually exclusive set of outcomes, it must decide now. Nonsense I say! When confronted with tough choices I think of the famous Yogi Berra quote "When you see a fork in the road, take it." I ask myself is there some way we can have our cake and eat it too? Can we go on trying out new features while still oferring stability to those willing to wait for it? What if we moved to model of more frequently releases + less frequent releases? A controlled fork within the ecosystem, similar to what Ubuntu / Fedora does. Publish 2 major short term releases a year that cutting edge library writers can target and build their libraries against. Vend these are properly tagged releases in maven central. Stick any old SIP in there, the enterprise overlords don't have to know about it. Rapid iteration, with rapid fall-off in support, a fun Scala for those of us that love using new features, bugs and all, for us the language is about growth. Then have an LTS version released every other year that picks up the features that worked and everyone seems to love and that will be supported for years into the future. Relatively slow iteration with slow fall off in support, vend this Scala to enterprise users with a coherent multi-year support policy that just oozes with that 'this is rock solid and will work for eons to come' professional vibe that makes them happy. 1: I am using ScalaJS and man it feels great, it's like Scala of 6 years ago, I love opening the mailing list to see if there is a new release with some awesome new features / improvements. There is a lot of green territory over there because it's targeting a new platform and that will settle with time, but already it beats the socks off all the other competitors in the statically typed compiles to JS camp, just no one knows it yet.
vorg 12y ago
When I see all the various Scala compilers out there (Typesafe Scala compiler, LAMP/EPFL Dotty compiler, Scala.js, Scala Virtualized compiler, Scala.Meta system, and several research and private variants of the Scala compiler, IDEs own Scala variants) compared to the lone version of the Groovy one, I realize just how detrimental to the Groovy ecosystem was Codehaus Groovy's practise of ignoring creator James Strachan's JSR-241 standardization effort and redefining "Groovy" to mean the org.codehaus.groovy implementation only. At times it seemed they were scuttling others attempts to build a variant compiler, and even cannabalizing the annotation-based addon market like when they duplicated the Groovy++ static compilation "speed booster".
_gkossakowski 12y ago
I'm somewhat surprised that everybody missed the fact that Scala 2.12 has more to it than just Java 8 compatibility and stability. We're working on replacing the whole backend in scalac. We'll ship faster and easier to maintain optimizer that fixes lots of long-standing bugs in scalac. In fact, it will allows us to fix the second most voted issue ever: https://issues.scala-lang.org/browse/SI-4633?jql=project%20%3D%20SI%20AND%20status%20%3D%20Open%20ORDER%20BY%20votes%20DESC https://issues.scala-lang.org/browse/SI-4633?jql=project%20%... More details: http://magarciaepfl.github.io/scala/ http://magarciaepfl.github.io/scala/ -- Grzegorz, member of Scala team at Typesafe

135 points by LiveTheDream 12y ago

I'm conflicted. Type constructor partial application would be great (the current type lambda syntax is just awful), but I really don't want to see Scala fragmented.

mhax 12y ago
I think it's fantastic news - even if the net result is just forcing typesafe to address concerns about scala that have been festering for some time.
jghn 12y ago
Agreed on conflicted-ness. The thing that concerns me the most is that from observation it feels like many of the high muckity mucks of the Scala world would probably be heading in this direction. There's nothing inherently wrong with that, but IMO that increases the chance of serious fragmentation.

robert_tweed 12y ago

What's the history behind this fork and what is it likely to mean in terms of fragmentation? In particular what are the irreconcilable beliefs on each side that have led to this? I haven't tried Scala yet, but my gut reaction is I should stick with Clojure until the dust settles. A fragmented community around a relatively niche language is never good.

pkinsky 12y ago
It's not a niche language. Household names including Twitter, Meetup, Foursquare, and Coursera run on Scala.
- willismichael 12y ago
  According to Cedric Beust, it's still a language that "pretty much hardly anyone uses". http://beust.com/weblog/2014/05/11/more-about-language-popularity/#comments http://beust.com/weblog/2014/05/11/more-about-language-popul...
  - frowaway001 12y ago
    Yes, let's cite this guy who is "so uninterested in Scala" that he spends an considerable amount of time maintaining multiple user accounts to spread FUD about the langauge in various places and makes up stories about how "his team/group/company/... was trying to use Scala but abandoned it due to all the horrible issues with it". You know that he is really persistent (despite claiming that he doesn't care about Scala) when the community starts maintaining a list of his sockpuppet accounts.
  - jroesch 12y ago
    Anyone who tries to make the claim that "pretty much hardly anyone uses" is standing on shaky ground. It definitely has not surpassed Java in popularity but it seems to be quickly gaining traction at lots of large companies and is being used more and more. I was recently at Scala by the Bay in San Fransisco and it seems each year more and more companies are on board. If LinkedIn, Twitter, PayPal, Cisco, Verizon, and more are all using Scala I would hardly call it a niche language anymore.
  - pkolaczk 12y ago
    There are a few really big and impactful things already implemented in Scala, and some companies may be using it without even actually writing any Scala code (for they provide Java APIs): Play, Kafka, Spark, Akka.
ssmoot 12y ago
As someone who's been using Scala for the better part of a couple years: This seems more like a more ambitious version of the Shapeless and ScalaZ libraries (with some of the same crowd/authors). I (and I suspect most people who program in Scala for a living) don't use those libraries. Not because they're in any way bad. But there are trade-offs using them IME. The biggest being compile-time. So I get by fine without. I think this is probably generally good news. In the same way that (I believe) JRuby pushed MRI forward and popularized the case for a multi-threading VM, more advanced type features in this fork probably benefit the Scala community as a whole. I certainly don't see how it could be harmful. Typesafe isn't going away. The vast majority of businesses will still use their compiler. If you didn't use the advanced-type-based-libraries before, this news probably isn't going to impact you anyway. So from where I'm standing, having made a late 2.9 to 2.10 transition, and now having made several 2.10 to 2.11 transitions, Scala seems a very mature language/platform to me. Libraries are generally upgraded fairly quickly, and source wise, those releases broke very little (2.11 being the biggest culprit just because of the modularization effort, but that's pretty trivial to resolve). I dunno. It's not Java. But it's not Ruby either. It's gotten new features since 2.9, but it's broken or changed very little. I find working with it, even working through upgrades, very stable and mostly frustration free.
- runT1ME 12y ago
  As someone involved in the scala community (and scalaz) community, I'd say about 60% of the companies I talk to at the various meetups use at the very least #scalaz. I was pleasantly surprised.
  - jghn 12y ago
    Could this be a biased sample though? Not saying this is the case but it seems logical that the people at the scala meetups are cut from a cloth that would make them more likely to be using things such as scalaz. I know that my local scala meetup seems to be more heavily oriented towards the functional end of things than the bulk of the scala developers I interact with. OTOH, perhaps the bias is with my sample.
Shamanmuni 12y ago
The fork means nothing dramatic. I'll make a short summary of both positions: Typesafe: we need to offer our clients a good guarantee on stability and compatibility with Java 8 or they won't take Scala seriously. So, for the moment we won't introduce new features or make big changes in the compiler. Typelevel: we are OK with compatibility and stability, but there are some features that we really want which either don't exist or are a bit quirky with the current compiler. So we will make a fork in which we can add these things while staying binary and merge compatible with yours. If they make it work, this is great news. There wouldn't be fragmentation as "bleeding edge Scala" would be binary and merge compatible with "stable Scala". And nobody should have hurt feelings, both positions are completely sensible.
VLM 12y ago
By analogy its not terribly far off from Debian issuing a "testing" branch off "unstable" a couple years or decade ago. Its a little more extreme than that, but not much. Usually forks are more dramatic. This isn't MariaDB or something like that. For PR reasons they probably should have stolen the Debian "testing" vs "unstable" names instead of getting all forky which is scarier sounding.
lmm 12y ago
> What's the history behind this fork and what is it likely to mean in terms of fragmentation? In particular what are the irreconcilable beliefs on each side that have led to this? What I've managed to put together, corrections welcome: Typesafe has been moving to stabilize Scala, primarily to make it more attractive to enterprise clients. Prior to 2.10, even point releases broke binary compatibility; now compatibility is maintained for those and they're making efforts to make minor releases binary compatible. The tooling they've produced and the testing framework that builds all the major libraries in the ecosystem and checks for binary compatibility means they now have quite possibly the best binary compatibility regime of any language. Typesafe recently published a roadmap for the next few releases, saying that the next release would focus exclusively on Java 8 support and remain compatible with the current release. This means work to merge any new features won't even start until January 2016, and those features are unlikely to be in an official release until 2017. Less frequent and more compatible releases are what a lot of people have been asking for, and it would be very awkward to maintain Java 8 and pre-Java 8 releases of Scala. So the typesafe position that there can be no new features in the release that introduces Java 8 (so that people will definitely be able to switch to Java 8 without making any code changes) makes a certain amount of sense. At the same time it's meant Scala's getting "frozen" in a bit of a weird state; macros were a recently-introduced experimental feature and the official version is kind of half-finished (there's a branch with a more complete version). And there are a couple of features that are quite small and simple but would be a tremendous help for the kind of code the typelevel guys work on (e.g. partial application of types would be a very simple syntactic rewrite, and there's already an implementation as a compiler plugin), and there's long been a feeling that they were coming as soon as we figured out the little details - so to then be told that we won't see these features until 2017 at the earliest was a bit upsetting. So there was a bit of consternation among the typelevel folks - a group who maintain some quite abstract and powerful libraries that tend to push the type system a bit further than most user code. Some of them started seriously talking about porting their libraries to one of the smaller, more experimental languages with a more elegant type system (Idris, Ermine, or I forget the other option). At the same time, the Scala community now has a lot of expertise in maintaining binary compatibility across quite complex changes, so at some point Miles came up with this idea. I don't think a fork would have happened otherwise - most of the goals can be achieved by compiler plugins (at the cost of being harder to maintain), and if they were going to break compatibility they might as well have jumped all the way to another language. But if the typelevel guys can pull off this "conservative fork", where they'll use their own compiler to compile their libraries but those libraries will then be binary compatible with "vanilla" scala and usable by ordinary scala users, then they get the best of both worlds. It's ambitious, but I hope it works out. > I haven't tried Scala yet, but my gut reaction is I should stick with Clojure until the dust settles. A fragmented community around a relatively niche language is never good. If they're maintaining binary compatibility then that should keep fragmentation to a minimum. Honestly I don't think this changes much compared to how it was with compiler plugins - if you're building some super-duper-typey project with these advanced features then maybe you'll muck around and use this forked compiler, but if you're a normal user then you can ignore it and carry on using their libraries without noticing any different. Plus even if the Scala community were to split in two, I think either half would be bigger than the Clojure community.
kamaal 12y ago
>>In particular what are the irreconcilable beliefs on each side that have led to this? I'm not familiar with this one. But in general most of these forks are generally out of personality clashes where one person just refuses to take another as the boss. Scala is still not among the top languages used in the industry. This kind of my way or the highway attitude will only fragment the already fragile ecosystem. Imagine any serious software shop wanting to use Scala reading this. If I want to be writing an enterprise application which I plan to keep running for the next decade, should I be choosing a language which isn't just not widely adopted, but the existing ecosystem itself has forks because a few members don't give two hoots about stability(Which is far more important to 99% of the software shops out there, than adding new features). Instead of doing something like this, and putting your broader community of users in perpetual confusion as to which ecosystem to use, you could rather contribute to something meaningful.
- frowaway001 12y ago
  Sometimes it's useful to read the blog post before commenting.
  - kamaal 12y ago
    I did, and its the same template story always. We don't agree with technical decisions made by Set A people to be in best interests of the community, therefore we plan to fork Y, with intelligent Set B people. The actual story is always about control and ego.
    - frowaway001 12y ago
      You really didn't read it.

dkarapetyan 12y ago

So the elitist community component of Scala that is constantly trying to turn it into Haskell has decided they need an island of their own. Great, they can have it. Now I'll know where to go when I need monadic and monoidal libraries for serving HTTP requests using nothing but type level hacks. In all seriousness though I don't really know what this is going to accomplish. The article mentions the academic and non-academic components of Scala and it is clear the author falls on the academic side but if that's the case then why not just go and hack on something like Haskell (or Coq), a perfectly unapologetic academic language that has tried to avoid success at all costs. If push comes to shove I'm with team Odersky.

pkinsky 12y ago
>Now I'll know where to go when I need monadic and monoidal libraries for serving HTTP requests using nothing but type level hacks. Monadic libraries with classes like Option, Future, and List? Used in elitist features like for comprehensions? Anyone who writes Scala on a regular basis deals with monads constantly. Why not take 15 minutes and learn how they work?
- dkarapetyan 12y ago
  Oh no that is all a little too pedestrian for me. I've moved my sights much higher now. These days I use template haskell and applicatives to generate HTML forms: https://github.com/yesodweb/yesod/wiki/Multi-select-field-populated-from-database-%28using-esqueleto%29 https://github.com/yesodweb/yesod/wiki/Multi-select-field-po.... I'm all for applying theory to enrich the practice of programming but lets be reasonable here. Lists and Options existed way before anyone added monadic sugar on top of it as did mapping and filtering collections. I like category theory as much as the next guy but lets just pull back a little bit. Not everything needs to be point-free, functorial, and shapeless.
  - pkinsky 12y ago
    >I'm all for applying theory to enrich the practice of programming but lets be reasonable here. Lists and Options existed way before anyone added monadic sugar on top of it. I like category theory as much as the next guy but lets just pull back a little bit. Not having needs to be point-free and functorial. I agree that Spire/Shapeless/Scalaz tend to go far beyond anything I would use in a production app, but Options/Futures/etc are barely usable without for comprehensions, which are just syntactic sugar for flatMap. I see the ability to write for { x <- f1() // returns Option[T1] y <- f2(x) // returns Option[T2] z <- f3(y, x) // returns Option[T3] } yield foo(x, y, z) // for expression returns Option[T4] as a huge part of Scala's success

saosebastiao 12y ago

My biggest frustration with Scala is the exact opposite of their frustration with Typesafe's Scala: it moves too fast. Scala's backwards compatibility problems are obnoxious for the people who don't feel the need to reinvent everything. I can find a version of Foo that works with 2.11 but the only version of Bar I can use only has artifacts for <=2.10. The play framework exemplifies these problems. Documentation doesn't keep up with releases, blog posts and tutorials don't either. Sometimes I'm looking for example code for some plugin or library, and the code only works with some other version of another dependency that I can't use. I totally understand that people prefer Agile development, but you can't combine Agile development with non-Agile documentation, unless you want to create a framework that only experts can use.

ffreire 12y ago
We've been bitten by this as well. We started several projects within a few months of each that also used what we believed to be similar enough versions of Play but there are just enough inconsistencies between versions of play (and Scala language versions as well) to make it exceedingly frustrating to work with.
pea 12y ago
Yeah I've wasted so much time dealing with code for the wrong version of Play. It's horrendously organised: case in point, the JSON parsing (WTF?)
- Rezo 12y ago
  The documentation for JSON parsing in Play is hilarious. You read the "The Play JSON library Basics" document, it looks like a lot of code for something that should be simple, but you go ahead and implement as explained. After a while, the verbosity starts bugging you, so you read the "JSON Reads/Writes/Format Combinators", followed by "JSON transformers" and finally the "JSON Macro Inception" pages. The punchline in the last section can be summarized as "forget all that other stuff, just do "implicit val personFmt = Json.format[Person]". I was happy to be able to delete hundreds of lines of code from my project after learning this, but couldn't you just have told me this upfront since it probably covers 99% of all cases? The documentation also has an aura of "Isn't this all very clever?" feeling to it. It's also quite hard to search for anything, as outdated versions of the docs keep popping up in Google as the first results.

saryant 12y ago

Background: I saw Miles' tweet on this last week but didn't follow the apparent brouhaha that followed. All I know of him was his (hilarious) talk at Scala Days in June. As someone who uses Scala daily, I don't think this is a good thing. I fall squarely into the Typesafe side of the camp—we don't use Scalaz or Shapeless or much of anything from the Typelevel project but Akka and Play are core technologies for us. We don't have a Typesafe subscription, we just use Akka and Play, a lot. > As I observed earlier, there are already several variants of the language in existence and it has been clear for a long time that different sections of the community have different interests. I don't think the author's examples really hold water. Nobody is using the Dotty compiler commercially, Scala.js is still in its infancy and while I know large organizations have at times run on minor Scala forks (I think Twitter used to but is now on 2.10.4), I'm not sure I believe that this is a major constituency. The official Scala is the Scala. If this fork gains steam, it's going to be a huge opportunity for FUD around Scala and will hurt its adoptability, especially if existing Scala teams now have to choose sides. That said, if the Haskell side of Scalaland wants to hang out somewhere else, that might be good too. Too many Scala questions on SO are answered (and upvoted) with obscure Scalaz solutions that make no sense to someone coming in from Ruby or Java. I like functional programming. I like how monads can simplify things. But I also have to write code that someone without a lot of Scala (or Haskell) experience can maintain.

lmm 12y ago
> I like functional programming. I like how monads can simplify things. But I also have to write code that someone without a lot of Scala (or Haskell) experience can maintain. I agree with this, but I think a same replacement for the type lambda syntax goes a long way towards making those scalaz solutions maintainable.
stuhood 12y ago
Twitter has not used a fork of the scala compiler/library since the pre-2.8 days (and maybe not even then, since we have always had a fair fraction of open source code.)

noelwelsh 12y ago

To people worried about fragmentation, retaining binary compatibility means it really doesn't matter. Scalaz can be written in the Typelevel fork and you can use it without issue from code compiled with the standard Scala compiler. Also, from the announcement: "As a conservative fork, our aim is to be merge compatible from the Typesafe Scala compiler, and a key objective is to feed pull requests (many of which will resolve long standing Scala bugs or feature requests) back to the Typesafe Scala compiler." So we aren't looking at radical changes, and hopefully these changes will get into Typesafe's compiler. So basically it's an opt-in better Scala compiler with no effect on compatibility. What's not to like?

zak_mc_kracken 12y ago

Regardless of the outcome of this initiative (which has little chance of succeeding, but best luck to them), this is pretty bad for the Scala ecosystem overall. There's a general lack of confidence from the community toward Scala and Typesafe and this certainly contributes greatly to the general uncertainty about Scala's chances of success.

frowaway001 12y ago
I guess it was only a matter of time until the "Scala will surely die!!! Really!!! This time it will go down!!!" crowd would show up ...

cryptos 12y ago

I'm worried about this fork. Scala is far away from being a wildly used industry language. Yes, there are some well known companies using Scala, but how many of them develop a really big part of their software with Scala? Not so much. If you look around you'll see Java, C#, C, C++ (depending on the business) and actually not so much Scala. The ones who think forking Scala does help the community are in a reality distortion field. Take a look and Linux desktops to see that diversity is not always good, especially if you're in a niche.

propensive 12y ago
I think there is a lot of Scala usage you perhaps don't see. As an example from the corporate world, at least six major international banks (probably more) are doing a significant amount of their development in Scala, some with teams in the hundreds. That's a solid market for Typesafe right there, which isn't going to go away overnight, and won't be stirred by the existence of the Typelevel fork of Scala. Furthermore, they should benefit from its existence because it represents a means for more people to be able to contribute to and enhance the language, without negatively impacting the stable, reliable releases from Typesafe.

benjaminjackman 12y ago

I don't understand what features they want to add beyond type lambda's, singleton types, and literals for bytes / shorts. 1. I understand if you spend all day making shapeless / scalaz the lack of sugar for type lambda's is annoying. Why not make a SIP to add some sugar there? 2. Singleton types would have use in ScalaJs as well for encoding. There is already a SIP for this. 3. Byte / Short literals, seem like again, a SIP could be made for the next release. So all this boils down to getting SIPs into the language faster. So I think the real feature is not having to wait 3+ years for language improvements that significantly improve the QOL of some library writers. That is a feature I would like as well. I really miss being able to open the mailing list and scan for exciting new features, Instead I know even if someone has a great idea, it'll be years before it sees the light of day. This sucks and I've moved onto to getting my neat idea -> wow I can use this in my language fix to ScalaJS[1]. I think (thought?) about Scala as ycombinator for language features. I see this enterprise mindset coming in now trying to tie down what Scala is and ultimately I fear that it will kill all the things that has made Scala popular in the first place. Pushing to a more infrequent release model might help enterprise customers but it absolutely crushes library developers in the vanguard working to make the Scala of tomorrow by fighting to get the language features of today. They need turn around on the improvements listed above and can't wait 4+ years to get them. So does Scala take all the strength it has built up from acting as a feature incubator for the last 10 years and switch and lock down what works, it is in a prime position to do so? It has a virtual monopoly on it's space. Or does Scala continue to act as a language feature incubator giving its coders early access to cutting edge CS idioms in a mainstream language. There are a lot of CS ideas waiting to get their shot. Some people would see this situation and say Scala is facing a mutually exclusive set of outcomes, it must decide now. Nonsense I say! When confronted with tough choices I think of the famous Yogi Berra quote "When you see a fork in the road, take it." I ask myself is there some way we can have our cake and eat it too? Can we go on trying out new features while still oferring stability to those willing to wait for it? What if we moved to model of more frequently releases + less frequent releases? A controlled fork within the ecosystem, similar to what Ubuntu / Fedora does. Publish 2 major short term releases a year that cutting edge library writers can target and build their libraries against. Vend these are properly tagged releases in maven central. Stick any old SIP in there, the enterprise overlords don't have to know about it. Rapid iteration, with rapid fall-off in support, a fun Scala for those of us that love using new features, bugs and all, for us the language is about growth. Then have an LTS version released every other year that picks up the features that worked and everyone seems to love and that will be supported for years into the future. Relatively slow iteration with slow fall off in support, vend this Scala to enterprise users with a coherent multi-year support policy that just oozes with that 'this is rock solid and will work for eons to come' professional vibe that makes them happy. 1: I am using ScalaJS and man it feels great, it's like Scala of 6 years ago, I love opening the mailing list to see if there is a new release with some awesome new features / improvements. There is a lot of green territory over there because it's targeting a new platform and that will settle with time, but already it beats the socks off all the other competitors in the statically typed compiles to JS camp, just no one knows it yet.

vorg 12y ago

When I see all the various Scala compilers out there (Typesafe Scala compiler, LAMP/EPFL Dotty compiler, Scala.js, Scala Virtualized compiler, Scala.Meta system, and several research and private variants of the Scala compiler, IDEs own Scala variants) compared to the lone version of the Groovy one, I realize just how detrimental to the Groovy ecosystem was Codehaus Groovy's practise of ignoring creator James Strachan's JSR-241 standardization effort and redefining "Groovy" to mean the org.codehaus.groovy implementation only. At times it seemed they were scuttling others attempts to build a variant compiler, and even cannabalizing the annotation-based addon market like when they duplicated the Groovy++ static compilation "speed booster".

_gkossakowski 12y ago

I'm somewhat surprised that everybody missed the fact that Scala 2.12 has more to it than just Java 8 compatibility and stability. We're working on replacing the whole backend in scalac. We'll ship faster and easier to maintain optimizer that fixes lots of long-standing bugs in scalac. In fact, it will allows us to fix the second most voted issue ever: https://issues.scala-lang.org/browse/SI-4633?jql=project%20%3D%20SI%20AND%20status%20%3D%20Open%20ORDER%20BY%20votes%20DESC https://issues.scala-lang.org/browse/SI-4633?jql=project%20%... More details: http://magarciaepfl.github.io/scala/ http://magarciaepfl.github.io/scala/ -- Grzegorz, member of Scala team at Typesafe