5 ms·

I have a simple question. In the article they mentioned they had a concurrency issue with a timed buffer that they later neatly solved with go channels and gor

by unoti 10y ago

I have a simple question. In the article they mentioned they had a concurrency issue with a timed buffer that they later neatly solved with go channels and goroutines. They said that they solved the problem in Scala by moving to the actor model, but that required importing Akka into their project and training everyone how to use Akka. My simple question is: couldn't they have achieved the heart and soul of the actor model by just making an object on its own thread, and talking to that object on a simple synchronized message queue? It's a handful of easy to understand lines of code, and nobody needs to delve into the sea of madness that is learning and configuring Akka and its actor model. In more general terms, it's possible to use Scala as Java that plays well with immutability and functional programming techniques without turning your codebase into an overly complex difficult-to-understand mess. But for some reason people just can't stop themselves. For what it's worth, Elixir hits a real sweet spot of functional goodness, combined with awesome concurrency without getting too deep into bizarre complexity.

joshlemer 10y ago
I think it's becoming a more commonly held opinion in the Scala community that people often tend to go off the deep end with Akka and I tend to agree with that. In particular, I think that most of what people use Actors for can be done with Futures, and what can't be done with Futures can most of the time be done with Akka Agents (http://doc.akka.io/docs/akka/current/scala/agents.html http://doc.akka.io/docs/akka/current/scala/agents.html). And when I use Actors, I tend to want o wall them off in their own place in the codebase, instead of letting the actor-ness touch multiple parts of the code.
digitalzombie 10y ago
Actors should be in a process no? If it's in a thread then it'll take the main process down with it. Erlang's BEAM VM every actor is in it's own process. So if something goes bad then it can be restarted via supervisor. Scala is on JVM and JVM isn't built with concurrency in mind and I think Erlang's BEAM is too good at this. Akka is gimped too, you have to write actor a certain way iirc other wise it takes over the scheduler. BEAM is preemptive, it doesn't matter if you have a for(1) loop, your process/actor can only take some much of the cpu time. I think hands down Erlang is a really really beautiful language for concurrency. It's syntax is ugly but it's such a small language that does everything you need for concurrency. Scala is just big and there are so many way to shoot yourself in the foot and tons of compromises. I also think implicit type is too magical and shot myself in the foot many time using libraries that use implicit type.
- dalailambda 10y ago
  Erlang's processes are green threads, they're just called "processes". Neither Erlang, nor Akka run 1 thread or process per actor, they just schedule multiple actors over N native threads inside the single process (ignoring multi-node situations).
  - sbalea 10y ago
    Two notable differences: Akka Actors concurrency is done at library level and an actor can block a JVM thread if not coded carefully. Erlang processes concurrency is support at VM level and there's no way an Erlang process can block a VM scheduler (native code aside, but with native code all bets are off)
- zigzigzag 10y ago
  The JVM has tons of concurrency tools, it's absolutely designed with concurrency in mind. You can write code in the channels style if you want too.
- xutopia 10y ago
  Elixir makes Erlang look beautiful! You should check it out!
  - digitalzombie 10y ago
    Yeah I'm actually learning Elixir and eventually Phoenix. Erlang's syntax took a while to get used to but the community wasn't for me. There were no momentum really, it was really hard to convince anybody that Erlang needed some killer framework that people can get behind. Or hell anything to get excited about other than BEAM and that's behind the scene. Elixir is beautiful but some of the syntax is meh for me.
lazyjones 10y ago
I found the Go solution for that issue a bit odd - channels aren't for data processing, they're synchronisation primitives. Using them the way they did in the article ruined the piece for me, since it reads like a rather uninformed decision now.
- lentil 10y ago
  Are you sure they aren't intended for data processing? I've never heard that claim before, and the Go documentation seems to have a lot of examples of using channels in that way. (https://blog.golang.org/pipelines https://blog.golang.org/pipelines, for example; or https://tour.golang.org/concurrency/2 https://tour.golang.org/concurrency/2).
  - lazyjones 10y ago
    Let me rephrase: not suitable for large amounts of data that need high throughput. It's easy to see that the overhead is prohibitive for applications such as in the article, if you just benchmark it. https://groups.google.com/d/msg/golang-nuts/LM648yrPpck/tv40LGEfAwAJ https://groups.google.com/d/msg/golang-nuts/LM648yrPpck/tv40...
- cube2222 10y ago
  That's just not true. You often use channels for messages of various kinds, or for returning values in async code.
ktoso 10y ago
(disclaimer: Akka team here). The post, somewhat surprisingly, omits to mention Akka Streams (which is a perfect fit, since, as the post mentions 'The data came through a stream [...]'), and Reactive Kafka (which is an official Akka project https://github.com/akka/reactive-kafka https://github.com/akka/reactive-kafka ) which solve this exact use case. These projects/modules have been around since 2014, and we've been presenting / sharing information about them a lot since then. Perhaps this post can be interpreted that we need to put even more effort into the discoverability of them (in fact, we are right now reworking documentation for this and more). Using Akka Streams, a complete re-implementation of the example use-case mentioned in the post would look like this: `Consumer.plainSource(kafkaConsumerSettings, kafkaSubscription).groupedWithin(maxItemsInBatch, 100.milliseconds).runForeach(batch => persist(batch))` Too bad the team missed these... I would have loved to point the team to the right (existing) abstraction/library in one of our many communities (github, gitter chat and the mailing lists - we're pretty nice people, come around some time), rather than posting like this. What we've learnt here though is that we need to work even harder on the discoverability of those libraries - and indeed it is one of the things we focus on nowadays (with docs re-designs, cross links, better search and more). Anyway, just wanted to let you all know what Akka has in store for such use cases, Streaming is definitely a first class citizen in the toolkit.

by unoti 10y ago

joshlemer 10y ago

I think it's becoming a more commonly held opinion in the Scala community that people often tend to go off the deep end with Akka and I tend to agree with that. In particular, I think that most of what people use Actors for can be done with Futures, and what can't be done with Futures can most of the time be done with Akka Agents (http://doc.akka.io/docs/akka/current/scala/agents.html http://doc.akka.io/docs/akka/current/scala/agents.html). And when I use Actors, I tend to want o wall them off in their own place in the codebase, instead of letting the actor-ness touch multiple parts of the code.

digitalzombie 10y ago

Actors should be in a process no? If it's in a thread then it'll take the main process down with it. Erlang's BEAM VM every actor is in it's own process. So if something goes bad then it can be restarted via supervisor. Scala is on JVM and JVM isn't built with concurrency in mind and I think Erlang's BEAM is too good at this. Akka is gimped too, you have to write actor a certain way iirc other wise it takes over the scheduler. BEAM is preemptive, it doesn't matter if you have a for(1) loop, your process/actor can only take some much of the cpu time. I think hands down Erlang is a really really beautiful language for concurrency. It's syntax is ugly but it's such a small language that does everything you need for concurrency. Scala is just big and there are so many way to shoot yourself in the foot and tons of compromises. I also think implicit type is too magical and shot myself in the foot many time using libraries that use implicit type.

dalailambda 10y ago
Erlang's processes are green threads, they're just called "processes". Neither Erlang, nor Akka run 1 thread or process per actor, they just schedule multiple actors over N native threads inside the single process (ignoring multi-node situations).
- sbalea 10y ago
  Two notable differences: Akka Actors concurrency is done at library level and an actor can block a JVM thread if not coded carefully. Erlang processes concurrency is support at VM level and there's no way an Erlang process can block a VM scheduler (native code aside, but with native code all bets are off)
zigzigzag 10y ago
The JVM has tons of concurrency tools, it's absolutely designed with concurrency in mind. You can write code in the channels style if you want too.
xutopia 10y ago
Elixir makes Erlang look beautiful! You should check it out!
- digitalzombie 10y ago
  Yeah I'm actually learning Elixir and eventually Phoenix. Erlang's syntax took a while to get used to but the community wasn't for me. There were no momentum really, it was really hard to convince anybody that Erlang needed some killer framework that people can get behind. Or hell anything to get excited about other than BEAM and that's behind the scene. Elixir is beautiful but some of the syntax is meh for me.

lazyjones 10y ago

I found the Go solution for that issue a bit odd - channels aren't for data processing, they're synchronisation primitives. Using them the way they did in the article ruined the piece for me, since it reads like a rather uninformed decision now.

lentil 10y ago
Are you sure they aren't intended for data processing? I've never heard that claim before, and the Go documentation seems to have a lot of examples of using channels in that way. (https://blog.golang.org/pipelines https://blog.golang.org/pipelines, for example; or https://tour.golang.org/concurrency/2 https://tour.golang.org/concurrency/2).
- lazyjones 10y ago
  Let me rephrase: not suitable for large amounts of data that need high throughput. It's easy to see that the overhead is prohibitive for applications such as in the article, if you just benchmark it. https://groups.google.com/d/msg/golang-nuts/LM648yrPpck/tv40LGEfAwAJ https://groups.google.com/d/msg/golang-nuts/LM648yrPpck/tv40...
cube2222 10y ago
That's just not true. You often use channels for messages of various kinds, or for returning values in async code.

ktoso 10y ago

(disclaimer: Akka team here). The post, somewhat surprisingly, omits to mention Akka Streams (which is a perfect fit, since, as the post mentions 'The data came through a stream [...]'), and Reactive Kafka (which is an official Akka project https://github.com/akka/reactive-kafka https://github.com/akka/reactive-kafka ) which solve this exact use case. These projects/modules have been around since 2014, and we've been presenting / sharing information about them a lot since then. Perhaps this post can be interpreted that we need to put even more effort into the discoverability of them (in fact, we are right now reworking documentation for this and more). Using Akka Streams, a complete re-implementation of the example use-case mentioned in the post would look like this: `Consumer.plainSource(kafkaConsumerSettings, kafkaSubscription).groupedWithin(maxItemsInBatch, 100.milliseconds).runForeach(batch => persist(batch))` Too bad the team missed these... I would have loved to point the team to the right (existing) abstraction/library in one of our many communities (github, gitter chat and the mailing lists - we're pretty nice people, come around some time), rather than posting like this. What we've learnt here though is that we need to work even harder on the discoverability of those libraries - and indeed it is one of the things we focus on nowadays (with docs re-designs, cross links, better search and more). Anyway, just wanted to let you all know what Akka has in store for such use cases, Streaming is definitely a first class citizen in the toolkit.