9 ms·
Metals with Scala 3
- lihaoyi 6y agoScala gets a lot of hate, but honestly it's a great language. As fast as Java with all the great tooling and ecosystem and compiler/IDE help, as concise as Python and just as usable for fast prototyping and rapid development. How many Python developers would like a 10-20x faster version of the language with better parallelism and concurrency, or Java developers a more concise language better suited for rapid development on top of the huge ecosystem they built their systems on? Scala provides both. There are a bunch of confusing frameworks from Scala's early days that leant it a bad rap, or niche tools that do not fit everyone's use case, but you don't need to use them. I've personally never used Shapeless/Akka/Scalaz/Cats in my entire career and I've gotten by OK. I haven't used SBT for years now. You can use these things if you want, and some people do, but only if you really want to. The Scala community is a Big Tent. Really, every language is converging on Scala: a concise, hybrid OO/FP language with a rich, inferred static type system it uses for bug-catching, tooling support, and performance. Python has got static types and case classes, is getting pattern matching and type-driven compilation. Ruby is getting static types. C# is getting pattern matching and case classes. Java got lambdas, is getting type inference and pattern matching and case classes. Haskell is getting OO-style dot-syntax for records. Go is getting generics. This is a spot in the language design space everyone is trying to get to, and Scala is already there and works great. I use it professionally to implement programming languages, distributed systems, websites, etc. and it really works as well as you'd expect "Superfast Python with static types" or "Java without all the Java-verbosity problems" to work. A true general-purpose high-level language
- throwawaygh 6y agoThe one place where Scala really doesn't shine is ML. There's nothing even close to equivalent to PyTorch or Tensorflow. Actually, languages with rich type systems in general seem to really struggle in this space.
- lihaoyi 6y agoI'd argue it's just "languages that are not Python struggle in this space". Python has an undoubtably unique ecosystem and gravity around ML, but that's more a unique advantage of Python than a weakness of every other language design
- MrPowers 6y agoScala does have Spark's ML capabilities. Spark's ML capabilities might not be the best, but they can be run on huge datasets. I'm guessing Dask has better model support, but isn't as scalable as Spark ML.
- currymj 6y agoi don't know if it's the type system... Julia has a very rich type system. It's not exactly popular (yet?) but it's certainly very usable and good for numerical work and ML. but they started out with the goal of building a numerical language from day one. I can imagine if you have a completely general purpose language, and want to provide an library that's an interface to low-level numerical libraries, NumPy style, that this task might be more difficult to get right in a language with more complicated types. There seem to be a lot of half-assed unfinished numerical libraries in both Scala and Rust.
- dwohnitmok 6y agoJulia has dynamic types. I think the parent is referring to statically typed languages by "rich type systems" (because dynamic types can often have arbitrary richness since they're basically just another form of runtime errors/exceptions).
- eigenspace 6y agoThis depends a lot on what one means by richness I suppose. I'd say that Python, Javascript, mosts lisps that don't have CLOS, etc. are all dynamic languages with very non-rich type systems. Julia's type system (and to a lesser extent Common Lisp's) are very different. They can do a lot of extremely powerful things with their type systems, just not (usually) static type checking (though even that's changing in julia land with JET.jl: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl)
- reader_mode 6y agoLast time I tried scala it took forever to build a relatively small project and the compiler was a heavyweight ram gobbler, which translated to shitty IDE performance as well. Now granted last time I tried Scala was probably 6 years ago, but I honestly doubt they got that language to compile as fast as Go - so I don't really see the Python comparison.
- lihaoyi 6y agoNot as fast as Go, but straight compilation is about ~3x faster than 2015, with many improvements around incrementality and IDE speed that make things a lot better. Compilation still isn't "as fast as Go", but has probably approached "acceptable" now in 2021. As one data point, "slow Scala compiles" was the #1 biggest developer complaint at my company when I joined in 2017, and nowadays that complaint doesn't even break the top 10. Scala compiles are not great, but not terrible. A lot about Scala sucked in 2015, and I wouldn't recommend 2015 Scala to anyone, but it's not 2015 any more. 2021 Scala is what I'm talking about
- harryh 6y agoThis is super interesting. I was a pretty early adopter of Scala, but our org grew fairly frustrated with in in 2013/14 due to the slow compiler. We invested a ton of effort into building tools to mitigate this problem (effort we'd rather have spent on our actual business). I haven't really touched it beyond tiny toy projects since 2015 though. Really glad to hear that compilation speed has gotten so much better.
- lihaoyi 6y agoYeah the effort to speed up compilation around Scala 2.12 really took off. You can see it in the compiler benchmarks: https://scala-ci.typesafe.com/grafana/dashboard/db/scala-benchmark?orgId=1&from=now-5y&to=now https://scala-ci.typesafe.com/grafana/dashboard/db/scala-ben... Where the better-files library went from taking ~700ms to compile in 2016 to ~230ms today. Definitely a huge improvement that our developers appreciate every day!
- rizpanjwani 6y agowhich framework would you recommend instead of Akka?
- lihaoyi 6y agoFor most use cases you probably don't need Actors at all. How many systems do you see using Actors in Java/Python/Ruby/PHP/C#? Not many, and they get by just fine. A plain old web server handling HTTP/Thrift/GRPC works for 90% of people, maybe add Futures for parallelism/concurrency and you satisfy 90% of those remaining I use my own Castor actor library for some of my own projects where Actors were really becoming necessary, but at 2 files and 200 lines of code it would be a stretch to call it a framework. If you really think the "Akka-way" is what fits your thinking, go ahead and use it, but it's by no means necessary, and I think most people can get by just fine without it.
- cheriot 6y agoIt depends on what you're doing. Do you need a framework at all?
- dwohnitmok 6y ago> Haskell is getting OO-style dot-syntax for records. Go is getting generics. I would dispute the former means Haskell is getting OO features (more the case that it's finally realized that having proper records a la Purescript or Elm is pretty important for day-to-day code; records and objects are quite different beasts) and Go getting generics is Go getting functional (or OO features, I don't find Go particularly OO to begin with). > Really, every language is converging on Scala I think this is more symptomatic of Scala trying to do so much to begin with, such that any change any language makes can be tied to some feature Scala has. That being said, I'm really happy to see Scala 3 reining in a lot of those choices (except a new parallel set of whitespace-based syntax... that seemed like a gratuitous cost to one's novelty budget, but time will tell; the community already seems to be warming to it). To those starting out with Scala, lihaoyi's book is a great place to begin! I'll offer one piece of advice (to those beginners, not lihaoyi, who I'm sure needs none of my advice). These days when I talk to people about Scala who've never used it before, I don't like talking about it as a hybrid FP/OO language. I prefer to describe it as an OO language with a lot of syntax sugar and standard library choices to enable emulation of FP on top when desired. It is useful to think of Scala the base language, i.e. what everything desugars to, as an OO language. This is neither good, nor bad. However, I think this realization helps illuminate a lot of strategies around coding in Scala. If your having performance issues, drop down to the desugared OO layer and you'll probably find ways to squeeze out more performance. If you're having a hard time understanding how some of the FP constructs work, step through their desugared OO equivalents (e.g. there aren't really higher-order functions, just an automatic instantiation of FunctionN classes wrapping methods at call sites). Scala 3 has smoothed out a lot of the mismatch so that its most obvious issues are gone (monomorphic functions vs polymorphic methods, weird eta expansion issues, weird subtyping requirements for implicit prioritization, etc.), but I've still found treating OO as the core of Scala to be a general rule of thumb which is useful for new Scala programmers when they come across something they think should work but doesn't. As you work more and more in Scala and that conversion of FP to OO primitives becomes extremely fluid and natural for you, a lot of choices and quirks that Scala has become far easier to explain and a lot easier to deal with.
- lihaoyi 6y agoYeah the line between FP and OO can be argued over. At my work it's pretty OO, with FP used where convenient. People join and start reading/writing Scala with zero experience or training, and it works out OK. At Stripe, I understand their Scala is pure FP, with the first onboarding docs discussing "What is a Monad". I assume they need more training for people who don't have a pure-FP background, but if it works for them who am I to judge. My own personal style is somewhere in between, certainly more FP than at my work, but nowhere near as FP as those who are really into it.
- crystaln 6y agoI used Scala and loved it for a period, back in the Scala Lift days. It’s a beautiful language in many ways, that is also just too bloated with complex features that enable a small amount of brevity at the expense of a huge amount of cognitive overhead. I do kind of miss it though. Right now I’m pretty happy with Typescript...
- lihaoyi 6y agoThe community as a whole has definitely dialled back use of complex features. If you're used to Lift-era Scala, that was probably the high water mark in terms of "people having fun with confusing features and syntax", and things have gotten steadily more "normal" in the intervening half decade (perhaps approaching a full decade!) To be clear, these complex features are still there, it's just that people tend to not use/overuse/abuse them as much. Just like how Python has metaclasses and import-hook metaprogramming, but it's not something that a typical developer has to deal with on a day-to-day or even year-to-year basis
- cmollis 6y agolove your book.. and your blog. Thanks!!
- dwohnitmok 6y agoTo add to lihaoyi's sibling comment, some good examples of things people realized were overly complex pits of pain in Scala but were all the rage in the Lift days are the cake pattern and gratuitous use of implicit conversions. Those thankfully have largely disappeared. The design space of programs in Scala is still large (detractors might say too large), but it's way better explored now and there's a much clearer idea of the various tradeoffs among the various approaches as well as how to harmonize one with another.
- cutler 6y agoI think Scala will always suffer from its choice to make implicits so fundamental. Whether abused or not they impede compiler efficiency and complicate code. Implicits were the main reason I dropped Scala a few years ago.
- fmakunbound 6y ago> 10-20x faster version It’s also at least 10-20x more complex a language. Can’t see the appeal to Python users.
- FowlSoft2013 6y agoI agree, I picked up Scala a bit over a year ago and have loved it. I believe there is a happy medium between the functional and OO aspects. Finding this sweet spot is where the language outshines others. Using classes as decomposition components and then using Scala's functional aspects such as immutability and the large offerings of classic functions (map, reduce, fold) give you a powerful and productive language.
- MrPowers 6y agoThe new libs make Scala way more enjoyable: * os-lib: allows for nice filesystem operations * upickle: allows for sane JSON * utest / munit: sane testing (rather than Scalatest) * scalafmt: great go-like automatic code formatting As long as Scala keeps replicating all the features in other programming languages, it'll have a bright future. It's a really powerful programming language.
- cutler 6y agoI think the space you describe is better served by Kotlin now ie. better Java compatibility, decent FP (even better with Arrow), first-class Spring integration and dominance of the Android platform. That's a lot of value.
- lihaoyi 6y agoYeah I don't disagree. Kotlin is great. I personally think Scala is a better Kotlin than Kotlin is for most use cases, with the exception of Android, but there's a big enough market to support multiple such languages
- killtimeatwork 6y ago> I've personally never used Shapeless/Akka/Scalaz/Cats in my entire career and I've gotten by OK. Everything's great until you join a team which uses it...
- cmollis 6y agocompletely agree..scala is a fantastic language. Scala has gotten a lot of hate, for sure Some for good reasons, but much of it is, I think, doesn't really apply anymore. I've been using daily for about 2 years now and it would be difficult to go back to Java at this point, for all of the reasons you stated. For me, it feels most like Swift.. concise but expressive, but with more mature collection and concurrency primitives. I think developers are looking at it more closely now that they're more experienced with functional capabilities in other languages and now can understand all of the type constructions. Granted, most of what I've done is in Spark, but I use it now for other mundane things like os scripting (something I would have used Python for previously). There are some great libraries for it and can't recommend it enough.
- rcgorton 6y agoNO, NO, NO. Scala is NOT a 'great' language. It is implemented in Java, so suffers ALL of the performance issues of Java (and then some). I would be THRILLED to see a new, straightforward language (vs. C++) which is strongly typed. From my personal perspective, GOLang is the best improvement so far, but suffers from a worse case of 'rpm hell' equivalant than general linux rpm packages do. Secondly, "Functional Programming" is best described as "DisFunctional Programming". The primary precept of functional programming is that there are ZERO side effects. Well, welcome to the real world of hardware/network failures. Please stop pitching garbage ideas to the rest of us realists.
- brokencode 6y agoDismissing new ideas out of hand does not make you a realist, it makes you ignorant. You seem to be confusing functional programming with purely functional programming for one, and for two, if you think purely functional programming doesn’t deal with side effects or failures in any way, then you don’t know enough about it to pass judgement.
- type0 6y agoSounds like you never tried it, just because it runs on JVM doesn't make it bad, on the contrary actually. Go is a completely different thing. Scala is one of the most pleasant languages to write. > Please stop pitching garbage ideas to the rest of us realists. trolling much!?
- RussianCow 6y ago> The primary precept of functional programming is that there are ZERO side effects. Well, welcome to the real world of hardware/network failures. You clearly have never actually learned a functional programming language, because functional programming tends to make it easier to handle errors and side effects precisely because they are more contained, instead of allowing any arbitrary code to throw an exception, or having to remember to handle error codes, etc. I strongly suggest that you build a small toy program in a language like Haskell—it really does change how you approach problems, even in more mainstream, imperative languages.
- RyanDeLap 6y ago
- k0dede 6y agoso what is "metals"? is it the apple graphic api, or a scala library, or a cli tool?
- philipkglass 6y ago"Scala language server with rich IDE features" https://scalameta.org/metals/ https://scalameta.org/metals/
- zdragnar 6y agoThe answer is in the first section titled "What is metals"- it is a language server for editors and IDEs to base their scala support on.
- type0 6y agoA language server that helps you write Scala code
- worace 6y agoMetals is such an awesome tool. Definitely one of the most polished LSP implementations I have used, and a great demonstration of what the protocol is capable of. I know it's all old hat to the IntelliJ crowd, but I just don't like using an industrial IDE if I can help it. Metals lets me stick with a lightweight editor like Emacs or VSCode while keeping the great tooling that a powerful type system like Scala's enables. I still fire up IntelliJ from time to time for certain types of mechanical refactoring, but as Metals has improved I find those cases come up less and less. Big props to the Metals team, and thanks for all the hard work.
- sireat 6y agoWhat advantages would switching from IntelliJ to Metals backed editor bring? I've been using Scala for a few years and IntelliJ with Scala plugin has been pretty amazing for Scala use. 90% of time the suggestions it makes to fix everyday blunders (like missing imports, basic typos) are right on the mark. Have not hit any realy pain points with sbt either. I do not want to spend time messing with configuration. Compared to say using VS Code with Python plugin Scala development feels so much nicer that I am considering switching to PyCharm.
- worace 6y agoWell if you're already happy using IntelliJ, I'm not sure there's much reason to switch. It's more that you're not forced to switch in the other direction. I'm happy with my Emacs setup, and in particular I spend a lot of time editing code in other languages as well as text docs like markdown or org. In the past, while this workflow was great for most things, I'd usually have to switch over to IntelliJ for serious scala editing. But now I don't have to any more!