7 ms·

On the future of Akka and Lightbend

89 points by yiksanchan 5y ago

AheadOfTime295 5y ago
The recent focus on Akka is in line with Lightbend securing a 25MM funding round last year, and the ensuing layoffs of Scala engineers https://news.ycombinator.com/item?id=22842166 https://news.ycombinator.com/item?id=22842166
bostonsre 5y ago
Has anyone had great success with akka? It seems like it could definitely solve some problems elegantly. My team has had trouble with maintainability due to complexity (not clear if that's inherent in akka or just with how we implemented it). It can be somewhat managed by senior engineers but has seemed to be difficult for junior engineers to effectively maintain akka code.
- edejong 5y ago
  In the past I have designed, implemented and brought into production many Akka based systems with medior and senior teams. Most of these were akka-streams based. The Akka actor model requires a different mode of thinking which can cause problems for experienced engineers. Ideas such as 'let it crash' and at-most-once-delivery are not trivial. Basically systems design based on resilience instead of robustness. When Actor based systems are well designed, they can be extremely powerful, scalable, resilient and adaptable. Small nitpick. I found that akka-streams missed many useful primitives. For our toolkit, I designed flows with retry-logic, flows with cache lines and flows with metric measurements (sending data to Datadog).
- coreyoconnor 5y ago
  I've had good success with akka at multiple companies. At Protenus the core ETL is based on akka streams. Which has worked great. No notable issues. Tho we don't run in clustered mode. At glngn we didn't use akka streams (outside of akka http) but did use event sourced, persistent entities powered by akka typed in clustered mode. Deployed to k8s. Which definitely took effort to set up nicely and enable smooth deployments. Our clusters were not huge so many problems did not show up (split brain). I haven't used other systems that provided an equivalent to akka typed persistent entities. I can't compare akka in that sense. However the event sourced persistent entities model is really, really effective. Definitely a different paradigm: some stuff is trivial that would otherwise be a trial. I suspect many of the benefits could be achieved by using something like Kafka feeding non clustered nodes. But never tried.
  - drewhk 5y ago
    I used to work on the Akka Team, mostly on streams, but I was doing different stuff elsewhere in the last years, not touching Akka much. To be honest, I am quite emotionally distanced nowadays from Akka... That said, it still feels good reading success stories from people who used it. Thanks, you made my day!
    - coreyoconnor 5y ago
      You're welcome! I think akka has been making great strides in polishing the rough edges. (Along with the rest of the Scala ecosystem TBH). Unfortunately that doesn't seem to generate the hype I'd argue akka deserves shrug
- lvice 5y ago
  I've now worked for about six months on a Akka.NET codebase. I find it a very elegant high-performance framework that gives you a lot of flexibility in how you want to solve a wide range problems. This being said, I've been burned already several times by the complexity and its raw power. The codebase tends to become verbose and difficult to navigate (everything being an ActorRef). Debugging is difficult and coding is challenging for junior engineers, as you said. I found it very unforgiving to mistakes, and it's easy to shoot yourself in the foot if you use the abstractions without knowing very well what's going on under the hood. Edge cases can be very tricky to manage. I'm still very conflicted about it. One one side the services powered by Akka are quite stable and performant, but I'm not sure the complexity justifies the means.
  - emodendroket 5y ago
    Doesn't .NET now have a very similar core library anyway?
    - vips7L 5y ago
      Only thing I can think of is the Task Parallel Library but I’m not sure how that compares to Actors.
      - aliswe 5y ago
        Not at all, what I know. well it runs stuff in parallel but i believe thats where the similarities end.
        emodendroket 5y ago
        That jogged my memory; what I was thinking of was TPL Dataflow. https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library https://docs.microsoft.com/en-us/dotnet/standard/parallel-pr... > The Task Parallel Library (TPL) provides dataflow components to help increase the robustness of concurrency-enabled applications. These dataflow components are collectively referred to as the TPL Dataflow Library. This dataflow model promotes actor-based programming by providing in-process message passing for coarse-grained dataflow and pipelining tasks. The dataflow components build on the types and scheduling infrastructure of the TPL and integrate with the C#, Visual Basic, and F# language support for asynchronous programming.
- AnotherGoodName 5y ago
  No. In fact the Play Framework 1.x (before it went to Akka) was simple perfection. Like Ruby on Rails but with so many improvements. Super simple but fast, performant and not really lacking anything a web framework needs (coordinating background periodic jobs between servers required setting a config file but it wasn't a big deal). In play framework 1 a new API method could be the following >@Check("administrator") // Authentication scopes allowed for this method >public static String returnHeading(int id) { > return createQuery("select heading from Article where id=%", id).getResult(); >} See how little boilerplate i have? A junior dev can work with this without shooting their own foot. I used to have clients come into the office and ask for a new API method and I'd type out the API in Play Framework 1.x, type the tests (the framework had a great foundation for unit and integration tests) check that the SQL was sane and deploy in half a day. There's nothing more i want. It scaled well horizontally too. You just had haproxy/nginx in front of multiple servers. Now you might say "But what if you had a web request that took days and needed all cores on all servers to communicate with each other as they concurrently worked on this mammoth task" to which I'd say shut the fuck up. Play Framework 2.x is when the Play framework adopted Akka. I'm going to be mean and takes Lightbends hello world using Akka as a taste for those unaware of what this beast is: https://youtu.be/rIFqJxMJ1MM?t=428 https://youtu.be/rIFqJxMJ1MM?t=428 I'm just going to copy paste some stuff from the Akka wiki to rub it in. Akka supports multiple programming models for concurrency, but it emphasizes actor-based concurrency and the eventsourced library provides event-driven architecture (see also domain-driven design) support for Akka actors. Akka has a modular structure, with a core module providing actors. Other modules are available to add features such as network distribution of actors, cluster support, Command and Event Sourcing, integration with various third-party systems (e.g. Apache Camel, ZeroMQ), and even support for other concurrency models such as Futures and Agents blah blah blah blah blah. Sorry I'm being rude but surely even the people making Akka can see the issues? There might be a use case for Akka. But a webserver for a startup is not a use case. Play Framework was built as a web framework and Play Framework 2.x seemed to focus in on a limitation that practically no one would encounter and make everything about that one use case. Play Framework 2.x overcomplicated the entire stack and gave no benefit. There's a reason Play Framework 1.x is being maintained and patched still. The complication from 2.x and Akka is not worth it at all.
  - 5y ago
- dastbe 5y ago
  i’ve seen at least two cloud services using akka for a component that required long-lived stateful connection management. while i think both ended up being successful because of akka, it required retrofitting a lot of libraries that already existed to fit the actor paradigm, as well as requiring at least one person who really understood akka. id also say some decisions around akka monitoring and how that has related to lightbends monetization makes it more difficult than it should be to build observability around the internals of akka, ex. emitting metrics on actor queue depth.
- amdelamar 5y ago
  My team had migrated about 35 codebases from a mix of Play, Scalatra, Ruby, and Python, to Akka HTTP. The unification definitely made it easier for juniors to start work on unfamiliar codebases, but the same could be said if we unified on something else. That being said, a portion of the services did make use of Akka streams & actors and we’ve found a lot of success there that otherwise would’ve been complex code with locks or synchronized blocks. Stuff that I’d rather not have to onboard engineers with or review their PRs and miss an unlock(). It’s probably a steeper learning curve by teaching them about actors and futures, but it means they’ll be writing safer code by design, and reviewing their PRs becomes easier for me. Just make sure the actors are small and have one focus. When they get too large and do too much is when they become unwieldy and difficult to maintain.
- lmm 5y ago
  I found that akka (akka-core) was not just bad, but bad enough that it dragged down the reputation of Scala. Most (maybe all) of the time simple future-based code accomplishes the same thing in a much clearer way, and without sacrificing the type safety that's one of the biggest strengths of Scala. (Some higher-level things under the akka umbrella e.g. akka-http (formerly spray-http) and akka-streams might be useful; not coincidentally they also tend to be more type-safe)
nekitamo 5y ago
It’s a shame that the Play framework is being abandoned. v1 had a lot of great ideas, but v2 kind of jumped off the cliff into Scala lala-land, where it got lost in the weeds. It never really recovered. If you’re looking for a straightforward Java framework similar to Play v1 check out Ninja: https://www.ninjaframework.org/ https://www.ninjaframework.org/ For me at least, it hits a sweet spot of enabling fast productive development and maintainable code.
- mountainriver 5y ago
  You could say that again, play v1 was so simple and productive. Then v2 was pure insanity. I switched from v2 to golang and it felt like I had been punishing myself. This is what really sold me in Go as a language and I’ve used it liberally since then. Nothing beats just being able to get your head around things.
- yumraj 5y ago
  I can concur. Many years ago, I had used Play v1 to solo develop a reasonably functioning website, in part time. But then came Play v2, and for some reason it never clicked for me. Haven't looked at Ninja. Currently I'm looking at Phoenix as my go-to framework for side projects.
- eropple 5y ago
  I really enjoyed Play V1. I got some stuff done with V2, but (and I say this as somebody who really liked Scala at the time) I think it absolutely disappeared into the morass and never came back. Ninja is pretty cool. It works well with Kotlin, which is what I reach for today with the JVM, but tbh the easy path today is pretty much Spring Boot so I have some trouble rationalizing using anything else.
  - mwcampbell 5y ago
    > tbh the easy path today is pretty much Spring Boot so I have some trouble rationalizing using anything else. My main reservation with Spring is the heavy dependence on runtime reflection. I'd like to see a Java web framework designed for conventional server-rendered web applications (as opposed to API backends), with authentication, CSRF protection, form validation, and so forth, but without heavy use of runtime reflection for wiring up objects. Edit: Also, especially given that Project Loom is on its way, I think a blocking API like Ninja or Spring MVC is best.
zz865 5y ago
Biggest problem I had with Akka was that it was that it worked better with Scala than Java. As Java recently has re-taken most of Scala's momentum its a bit awkward fit.
- vips7L 5y ago
  Same story with Lightbends Play Framework. Using it from Java is abysmal.
  - KptMarchewa 5y ago
    Using anything build in Scala from Java is abysmal. Have fun with no default parameters, creating Seq$.MODULE$ everywhere etc. I'm not from US, so I've never have seen so much dollars anywhere.
    - hardlianotion 5y ago
      I agree, it is a mess. I had a little success by simplifying the interop layer. If you can wrap your scala code in simple classes that expose only Java-compatible structure, then you can remove additional complexity of java into scala interop.
      - KptMarchewa 5y ago
        I agree - unfortunately I work with 3rd party scala code in this case.
        lmm 5y ago
        You might be able to write a shim layer in Scala. Java interfaces, Scala classes that implement those interfaces and call the third-party code. That's the nicest way to do it IME.
        KptMarchewa 5y ago
        I think that's what we plan to do. Still, it introduces some additional problems: we were able to target all the supported Scala versions with one package, now we'll have to do separate ones for 2.11, 2.12, and 2.13.

89 points by yiksanchan 5y ago

The recent focus on Akka is in line with Lightbend securing a 25MM funding round last year, and the ensuing layoffs of Scala engineers https://news.ycombinator.com/item?id=22842166 https://news.ycombinator.com/item?id=22842166

bostonsre 5y ago

Has anyone had great success with akka? It seems like it could definitely solve some problems elegantly. My team has had trouble with maintainability due to complexity (not clear if that's inherent in akka or just with how we implemented it). It can be somewhat managed by senior engineers but has seemed to be difficult for junior engineers to effectively maintain akka code.

edejong 5y ago
In the past I have designed, implemented and brought into production many Akka based systems with medior and senior teams. Most of these were akka-streams based. The Akka actor model requires a different mode of thinking which can cause problems for experienced engineers. Ideas such as 'let it crash' and at-most-once-delivery are not trivial. Basically systems design based on resilience instead of robustness. When Actor based systems are well designed, they can be extremely powerful, scalable, resilient and adaptable. Small nitpick. I found that akka-streams missed many useful primitives. For our toolkit, I designed flows with retry-logic, flows with cache lines and flows with metric measurements (sending data to Datadog).
coreyoconnor 5y ago
I've had good success with akka at multiple companies. At Protenus the core ETL is based on akka streams. Which has worked great. No notable issues. Tho we don't run in clustered mode. At glngn we didn't use akka streams (outside of akka http) but did use event sourced, persistent entities powered by akka typed in clustered mode. Deployed to k8s. Which definitely took effort to set up nicely and enable smooth deployments. Our clusters were not huge so many problems did not show up (split brain). I haven't used other systems that provided an equivalent to akka typed persistent entities. I can't compare akka in that sense. However the event sourced persistent entities model is really, really effective. Definitely a different paradigm: some stuff is trivial that would otherwise be a trial. I suspect many of the benefits could be achieved by using something like Kafka feeding non clustered nodes. But never tried.
- drewhk 5y ago
  I used to work on the Akka Team, mostly on streams, but I was doing different stuff elsewhere in the last years, not touching Akka much. To be honest, I am quite emotionally distanced nowadays from Akka... That said, it still feels good reading success stories from people who used it. Thanks, you made my day!
  - coreyoconnor 5y ago
    You're welcome! I think akka has been making great strides in polishing the rough edges. (Along with the rest of the Scala ecosystem TBH). Unfortunately that doesn't seem to generate the hype I'd argue akka deserves shrug
lvice 5y ago
I've now worked for about six months on a Akka.NET codebase. I find it a very elegant high-performance framework that gives you a lot of flexibility in how you want to solve a wide range problems. This being said, I've been burned already several times by the complexity and its raw power. The codebase tends to become verbose and difficult to navigate (everything being an ActorRef). Debugging is difficult and coding is challenging for junior engineers, as you said. I found it very unforgiving to mistakes, and it's easy to shoot yourself in the foot if you use the abstractions without knowing very well what's going on under the hood. Edge cases can be very tricky to manage. I'm still very conflicted about it. One one side the services powered by Akka are quite stable and performant, but I'm not sure the complexity justifies the means.
- emodendroket 5y ago
  Doesn't .NET now have a very similar core library anyway?
  - vips7L 5y ago
    Only thing I can think of is the Task Parallel Library but I’m not sure how that compares to Actors.
    - aliswe 5y ago
      Not at all, what I know. well it runs stuff in parallel but i believe thats where the similarities end.
      - emodendroket 5y ago
        That jogged my memory; what I was thinking of was TPL Dataflow. https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library https://docs.microsoft.com/en-us/dotnet/standard/parallel-pr... > The Task Parallel Library (TPL) provides dataflow components to help increase the robustness of concurrency-enabled applications. These dataflow components are collectively referred to as the TPL Dataflow Library. This dataflow model promotes actor-based programming by providing in-process message passing for coarse-grained dataflow and pipelining tasks. The dataflow components build on the types and scheduling infrastructure of the TPL and integrate with the C#, Visual Basic, and F# language support for asynchronous programming.
AnotherGoodName 5y ago
No. In fact the Play Framework 1.x (before it went to Akka) was simple perfection. Like Ruby on Rails but with so many improvements. Super simple but fast, performant and not really lacking anything a web framework needs (coordinating background periodic jobs between servers required setting a config file but it wasn't a big deal). In play framework 1 a new API method could be the following >@Check("administrator") // Authentication scopes allowed for this method >public static String returnHeading(int id) { > return createQuery("select heading from Article where id=%", id).getResult(); >} See how little boilerplate i have? A junior dev can work with this without shooting their own foot. I used to have clients come into the office and ask for a new API method and I'd type out the API in Play Framework 1.x, type the tests (the framework had a great foundation for unit and integration tests) check that the SQL was sane and deploy in half a day. There's nothing more i want. It scaled well horizontally too. You just had haproxy/nginx in front of multiple servers. Now you might say "But what if you had a web request that took days and needed all cores on all servers to communicate with each other as they concurrently worked on this mammoth task" to which I'd say shut the fuck up. Play Framework 2.x is when the Play framework adopted Akka. I'm going to be mean and takes Lightbends hello world using Akka as a taste for those unaware of what this beast is: https://youtu.be/rIFqJxMJ1MM?t=428 https://youtu.be/rIFqJxMJ1MM?t=428 I'm just going to copy paste some stuff from the Akka wiki to rub it in. Akka supports multiple programming models for concurrency, but it emphasizes actor-based concurrency and the eventsourced library provides event-driven architecture (see also domain-driven design) support for Akka actors. Akka has a modular structure, with a core module providing actors. Other modules are available to add features such as network distribution of actors, cluster support, Command and Event Sourcing, integration with various third-party systems (e.g. Apache Camel, ZeroMQ), and even support for other concurrency models such as Futures and Agents blah blah blah blah blah. Sorry I'm being rude but surely even the people making Akka can see the issues? There might be a use case for Akka. But a webserver for a startup is not a use case. Play Framework was built as a web framework and Play Framework 2.x seemed to focus in on a limitation that practically no one would encounter and make everything about that one use case. Play Framework 2.x overcomplicated the entire stack and gave no benefit. There's a reason Play Framework 1.x is being maintained and patched still. The complication from 2.x and Akka is not worth it at all.
- 5y ago
dastbe 5y ago
i’ve seen at least two cloud services using akka for a component that required long-lived stateful connection management. while i think both ended up being successful because of akka, it required retrofitting a lot of libraries that already existed to fit the actor paradigm, as well as requiring at least one person who really understood akka. id also say some decisions around akka monitoring and how that has related to lightbends monetization makes it more difficult than it should be to build observability around the internals of akka, ex. emitting metrics on actor queue depth.
amdelamar 5y ago
My team had migrated about 35 codebases from a mix of Play, Scalatra, Ruby, and Python, to Akka HTTP. The unification definitely made it easier for juniors to start work on unfamiliar codebases, but the same could be said if we unified on something else. That being said, a portion of the services did make use of Akka streams & actors and we’ve found a lot of success there that otherwise would’ve been complex code with locks or synchronized blocks. Stuff that I’d rather not have to onboard engineers with or review their PRs and miss an unlock(). It’s probably a steeper learning curve by teaching them about actors and futures, but it means they’ll be writing safer code by design, and reviewing their PRs becomes easier for me. Just make sure the actors are small and have one focus. When they get too large and do too much is when they become unwieldy and difficult to maintain.
lmm 5y ago
I found that akka (akka-core) was not just bad, but bad enough that it dragged down the reputation of Scala. Most (maybe all) of the time simple future-based code accomplishes the same thing in a much clearer way, and without sacrificing the type safety that's one of the biggest strengths of Scala. (Some higher-level things under the akka umbrella e.g. akka-http (formerly spray-http) and akka-streams might be useful; not coincidentally they also tend to be more type-safe)

nekitamo 5y ago

It’s a shame that the Play framework is being abandoned. v1 had a lot of great ideas, but v2 kind of jumped off the cliff into Scala lala-land, where it got lost in the weeds. It never really recovered. If you’re looking for a straightforward Java framework similar to Play v1 check out Ninja: https://www.ninjaframework.org/ https://www.ninjaframework.org/ For me at least, it hits a sweet spot of enabling fast productive development and maintainable code.

mountainriver 5y ago
You could say that again, play v1 was so simple and productive. Then v2 was pure insanity. I switched from v2 to golang and it felt like I had been punishing myself. This is what really sold me in Go as a language and I’ve used it liberally since then. Nothing beats just being able to get your head around things.
yumraj 5y ago
I can concur. Many years ago, I had used Play v1 to solo develop a reasonably functioning website, in part time. But then came Play v2, and for some reason it never clicked for me. Haven't looked at Ninja. Currently I'm looking at Phoenix as my go-to framework for side projects.
eropple 5y ago
I really enjoyed Play V1. I got some stuff done with V2, but (and I say this as somebody who really liked Scala at the time) I think it absolutely disappeared into the morass and never came back. Ninja is pretty cool. It works well with Kotlin, which is what I reach for today with the JVM, but tbh the easy path today is pretty much Spring Boot so I have some trouble rationalizing using anything else.
- mwcampbell 5y ago
  > tbh the easy path today is pretty much Spring Boot so I have some trouble rationalizing using anything else. My main reservation with Spring is the heavy dependence on runtime reflection. I'd like to see a Java web framework designed for conventional server-rendered web applications (as opposed to API backends), with authentication, CSRF protection, form validation, and so forth, but without heavy use of runtime reflection for wiring up objects. Edit: Also, especially given that Project Loom is on its way, I think a blocking API like Ninja or Spring MVC is best.

zz865 5y ago

Biggest problem I had with Akka was that it was that it worked better with Scala than Java. As Java recently has re-taken most of Scala's momentum its a bit awkward fit.

vips7L 5y ago
Same story with Lightbends Play Framework. Using it from Java is abysmal.
- KptMarchewa 5y ago
  Using anything build in Scala from Java is abysmal. Have fun with no default parameters, creating Seq$.MODULE$ everywhere etc. I'm not from US, so I've never have seen so much dollars anywhere.
  - hardlianotion 5y ago
    I agree, it is a mess. I had a little success by simplifying the interop layer. If you can wrap your scala code in simple classes that expose only Java-compatible structure, then you can remove additional complexity of java into scala interop.
    - KptMarchewa 5y ago
      I agree - unfortunately I work with 3rd party scala code in this case.
      - lmm 5y ago
        You might be able to write a shim layer in Scala. Java interfaces, Scala classes that implement those interfaces and call the third-party code. That's the nicest way to do it IME.
        KptMarchewa 5y ago
        I think that's what we plan to do. Still, it introduces some additional problems: we were able to target all the supported Scala versions with one package, now we'll have to do separate ones for 2.11, 2.12, and 2.13.