5 ms·

I have a feeling that a naive implementation in Java would be a lot worse than a naive implementation in Go so optimizing matters more there. Had to parse csvs

I have a feeling that a naive implementation in Java would be a lot worse than a naive implementation in Go so optimizing matters more there. Had to parse csvs in Java on a very memory constrained system once... we ended up cutting out a feature because it wasn't worth it.

cangeroo 3y ago
Regarding Java, It probably could be done with arrays and object reuse (arenas). But it's slightly less ergonomic. And the ecosystem isn't designed for it, so you'd have to implement your own memory-efficient parser.
- nottorp 3y ago
  Yep, but it wasn't a critical feature and we were in a rush, so the feature was killed instead. > Depends on which Java implementation is used. ... if you have a choice. It was a port of AOSP, so we didn't. In any case it wasn't the jvm's fault, the device just had very little ram.
  - syspec 3y ago
    Sounds like a skill issue
pjmlp 3y ago
Depends on which Java implementation is used. People keep forgetting Java is like C and C++, plenty of implementations to choose from, each with its own approach to JIT, AOT, GC and escape analysis.
masklinn 3y ago
Depends what you call “naive”, but the “idiomatic Java solution” from last week’s post (https://questdb.io/blog/billion-row-challenge-step-by-step/ https://questdb.io/blog/billion-row-challenge-step-by-step/) clocked in at 71 seconds, or 1:11. And just running it on graal was down to 66. “Very memory constrained” would be a massive factor here, 1BRC is not really constrained (let alone very much so), it has 1 billion rows on a 32GB machine.
- nottorp 3y ago
  Gigabytes? It was a while ago and i had megabytes for the whole OS :) Anyway, it's just a fun memory now.
speedgoose 3y ago
Golang is actually not as efficient as Java in quite a few benchmarks. ~~Using LLVM isn’t a magic solution to perform better than something relying on the JVM.~~ Here is a source: https://sites.google.com/view/energy-efficiency-languages https://sites.google.com/view/energy-efficiency-languages
- TwentyPosts 3y ago
  Huh? Go doesn't use LLVM though, where did you get the idea that it does? That's part of why its compile times are so fast.
  - speedgoose 3y ago
    You are right, it doesn’t use LLVM. No idea where I got the idea, I was confidently wrong.
benhoyt 3y ago
For what it's worth, on my machine the simple/idiomatic "baseline" Java solution (https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_baseline.java https://github.com/gunnarmorling/1brc/blob/main/src/main/jav...) takes 2m0s, compared to 1m45s for my simple Go version. So Go is a bit better for the naive version.

cangeroo 3y ago

Regarding Java, It probably could be done with arrays and object reuse (arenas). But it's slightly less ergonomic. And the ecosystem isn't designed for it, so you'd have to implement your own memory-efficient parser.

nottorp 3y ago
Yep, but it wasn't a critical feature and we were in a rush, so the feature was killed instead. > Depends on which Java implementation is used. ... if you have a choice. It was a port of AOSP, so we didn't. In any case it wasn't the jvm's fault, the device just had very little ram.
- syspec 3y ago
  Sounds like a skill issue

pjmlp 3y ago

Depends on which Java implementation is used. People keep forgetting Java is like C and C++, plenty of implementations to choose from, each with its own approach to JIT, AOT, GC and escape analysis.

masklinn 3y ago

Depends what you call “naive”, but the “idiomatic Java solution” from last week’s post (https://questdb.io/blog/billion-row-challenge-step-by-step/ https://questdb.io/blog/billion-row-challenge-step-by-step/) clocked in at 71 seconds, or 1:11. And just running it on graal was down to 66. “Very memory constrained” would be a massive factor here, 1BRC is not really constrained (let alone very much so), it has 1 billion rows on a 32GB machine.

nottorp 3y ago
Gigabytes? It was a while ago and i had megabytes for the whole OS :) Anyway, it's just a fun memory now.

speedgoose 3y ago

Golang is actually not as efficient as Java in quite a few benchmarks. ~~Using LLVM isn’t a magic solution to perform better than something relying on the JVM.~~ Here is a source: https://sites.google.com/view/energy-efficiency-languages https://sites.google.com/view/energy-efficiency-languages

TwentyPosts 3y ago
Huh? Go doesn't use LLVM though, where did you get the idea that it does? That's part of why its compile times are so fast.
- speedgoose 3y ago
  You are right, it doesn’t use LLVM. No idea where I got the idea, I was confidently wrong.

benhoyt 3y ago

For what it's worth, on my machine the simple/idiomatic "baseline" Java solution (https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_baseline.java https://github.com/gunnarmorling/1brc/blob/main/src/main/jav...) takes 2m0s, compared to 1m45s for my simple Go version. So Go is a bit better for the naive version.