3 ms·

What metric do you use for popularity? I'm more involved in the systems side so I'm definitely over exposed to Go projects. I would have considered Scala to be

by mhio 5y ago

What metric do you use for popularity? I'm more involved in the systems side so I'm definitely over exposed to Go projects. I would have considered Scala to be at least an order of magnitude less "popular" than Go. Kafka is the only open source project in Scala I can recall running into but there's probably a whole enterprise world I don't see.

yashap 5y ago
For open source projects, it’s very popular in the “big data” world. Kafka, as you mentioned, but also Spark, Flink, Summingbird, Scalding, etc. And yeah, it’s a pretty popular backend language at some large companies. Twitter, LinkedIn, Netflix, the Guardian, Starbucks, AirBnB, Coursera, AT&T, etc. all make significant use of Scala. Go is certainly bigger, but Scala has plenty of adoption too.
- kitd 5y ago
  Kafka is currently undergoing a rewrite into java.
  - geodel 5y ago
    Indeed. To add further it is also facing competition from Apache Pulsar and NATS. Not to forget Redpanda a Kafka rewrite in C++.
    - agallego 5y ago
      a small note on redpanda. Is not a `rewrite`. It is an entire new storage engine, different consensus protocol, different on-disk format, different internal RPC, no external dependencies, ...., in fact when i started it it was using flatbuffers as the API. What we did later to make money is add a kafka-API translation layer, but in terms of code size, is a very small part. The way i'd recommend thinking about it is a new storage engine that is compatible w/ kafka clients. https://github.com/redpanda-data/redpanda/tree/dev/src/v/kafka https://github.com/redpanda-data/redpanda/tree/dev/src/v/kaf... - this is basically all the kafka-related code, the rest is net-new design.
  - wfox 5y ago
    Source?
    - kitd 5y ago
      https://www.quora.com/Kafka-is-written-in-scala-Why-does-the-API-described-in-the-official-documentation-is-the-Javas-one https://www.quora.com/Kafka-is-written-in-scala-Why-does-the...
      - wfox 5y ago
        But the guy answering there is talking about the Java clients.

by mhio 5y ago

yashap 5y ago

For open source projects, it’s very popular in the “big data” world. Kafka, as you mentioned, but also Spark, Flink, Summingbird, Scalding, etc. And yeah, it’s a pretty popular backend language at some large companies. Twitter, LinkedIn, Netflix, the Guardian, Starbucks, AirBnB, Coursera, AT&T, etc. all make significant use of Scala. Go is certainly bigger, but Scala has plenty of adoption too.

kitd 5y ago
Kafka is currently undergoing a rewrite into java.
- geodel 5y ago
  Indeed. To add further it is also facing competition from Apache Pulsar and NATS. Not to forget Redpanda a Kafka rewrite in C++.
  - agallego 5y ago
    a small note on redpanda. Is not a `rewrite`. It is an entire new storage engine, different consensus protocol, different on-disk format, different internal RPC, no external dependencies, ...., in fact when i started it it was using flatbuffers as the API. What we did later to make money is add a kafka-API translation layer, but in terms of code size, is a very small part. The way i'd recommend thinking about it is a new storage engine that is compatible w/ kafka clients. https://github.com/redpanda-data/redpanda/tree/dev/src/v/kafka https://github.com/redpanda-data/redpanda/tree/dev/src/v/kaf... - this is basically all the kafka-related code, the rest is net-new design.
- wfox 5y ago
  Source?
  - kitd 5y ago
    https://www.quora.com/Kafka-is-written-in-scala-Why-does-the-API-described-in-the-official-documentation-is-the-Javas-one https://www.quora.com/Kafka-is-written-in-scala-Why-does-the...
    - wfox 5y ago
      But the guy answering there is talking about the Java clients.