6 ms·

For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to opt

For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.

tester756 4y ago
C library for parsing? isn't it dangerous from security perspective?
- xyzzy4747 4y ago
  It's just an example. The options are really Rust (what I'd prefer), C++, C, or perhaps something like Nim that compiles to C. If you’re trying to make an unoptimized parser, then use whatever you want.
  - deleted 4y ago
    [deleted]
pharmakom 4y ago
For some reason we insist that language parsers are implemented in the language itself, even when the language isn’t great for parsers.
37ef_ced3 4y ago
You're in for a big surprise. Try using the language. Spend some time using Go, and you will be impressed by its performance. You'll wonder, "Were all those haters on Hacker News misinformed?"
- xyzzy4747 4y ago
  If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
  - dymk 4y ago
    This is a premature optimization, and keeping everything in the same language has benefits like greatly simplified tooling and building
    - xyzzy4747 4y ago
      It’s not a premature optimization - it’s deciding the maximum that the parser can be optimized in the future. Choosing Go sets a lower ceiling. > Keeping everything in the same language has benefits like greatly simplified tooling and building Surely there are other Go libraries that incorporate C, C++, or Rust? Also if both parsers existed and were equally easy to set up, and you were planning on doing a ton of parsing, it would make sense to go with the faster one.
      - dymk 4y ago
        It absolutely is a premature optimization. If it's fast enough, then it's fast enough. The author hasn't indicated that the current Go implementation is hitting a ceiling imposed by the language yet. If you'd like to, you can provide some real-world examples - or even microbenchmarks - showing that Go is so much slower than <your choice here> that it's going to make a difference. > Also if both parsers existed and were equally easy to set up They're not equally easy to set up. Language interop is a pain in the pass.
      - Jtsummers 4y ago
        Look at the current Makefile: https://github.com/aretext/aretext/blob/main/Makefile https://github.com/aretext/aretext/blob/main/Makefile Build is literally a `go build ...` and install is `go install`. Adding any other language to the mix would make this a polyglot project and not be "equally easy to set up". The other question is, do both parsers exist? In this write-up they point to tree-sitter as a possibility which is a JS program that produces C code. This would be viable, but here's the author's take: > I considered integrating tree-sitter, an incremental parsing library with parsers for many existing languages. However, running JavaScript to generate parsers and linking to a C library would have greatly complicated the build process. Today, aretext can be built on almost any platform using a single go install command. I’ve had users install aretext on ARM laptops, FreeBSD servers, Chromebooks, and Android phones. To maintain portability, I wanted a pure Go implementation. So this wasn't some casual decision, but something they at least considered long enough to describe here. And the parsing library itself is only around 1200 lines total (comments, blanks, and code). The parsers for each language add a lot more, of course, but should be roughly equivalent given the same library and interface. I imagine that if this project really takes off and performance becomes a real problem they can do the rewrite at that point. Right now, the code works, seems to work fast enough for its author and primary users, and it's trivial to install on any platform supported by Go. So yes, it would have been a premature optimization to complicate the build process, probably reduce the number of supported platforms (or greatly increase the effort to support the same number of platforms), just to have a slightly faster parser.
- bugfix-66 4y ago
  Correct, Go is fast, very close to C. And just like in C, if you want to avoid memory management overhead you can use a slice of structs, integers instead of pointers, and a freelist (if needed). For example, here is a pointerless sparse bit vector: https://bugfix-66.com/7256e0772dc3b02d72abf15b171731c933fd44d67de074d679f1e4cb7bb20f79 https://bugfix-66.com/7256e0772dc3b02d72abf15b171731c933fd44... The article is storing parses in a balanced binary tree, like a packrat memoizing parser. Here is the fastest balanced search tree in Go. It allocates (and uses Go's garbage collector) but you can easily use a slice of structs with integer index pointers and a freelist instead: https://bugfix-66.com/c93e950965804eba90a34e0055985b1c42d5a1f122b88d1e9c02c68f801388b3 https://bugfix-66.com/c93e950965804eba90a34e0055985b1c42d5a1... The above code will perform very similarly to C.
rollcat 4y ago
One of Go's primary goals has always been compilation speed. Go started out in C, and was later (post-1.0) incrementally rewritten to be self-hosting. One of the authors (Ken Thompson) is also one of the co-creators of C. I would argue these guys know what they are doing.
- kaba0 4y ago
  I don’t know, not implementing generics when it was pretty obviously needed was a huge oversight, so I’m not sure. Also, the reason for compiler bootstrapping is more of a “beauty thing”, then practicality. It would definitely be faster in a low-level language, but I doubt it would matter as an end user.
  - fredrikholm 4y ago
    You aren't sure if Ken Thompson knows what he's doing?
    - kaba0 4y ago
      As a software architect? Absolutely. Programming language designer? Not sure, neither C or Go are good languages in my personal opinion. EDIT: I meant to write that I think very highly of him as an architect/developer.
      - sjansen 4y ago
        Experience has shown that often “worse is better”. Go does an amazing job of balancing complexity and power. I haven’t seen a ”better” language that isn’t either slower, harder to become productive, or both. https://en.wikipedia.org/wiki/Worse_is_better https://en.wikipedia.org/wiki/Worse_is_better
  - kcartlidge 4y ago
    > not implementing generics when it was pretty obviously needed was a huge oversight I get the desire for generics. I do a lot of C# and have used generics for a very many years. Yet I've been writing Go for around 6 or 7 years and other than in the beginning (when I was new to it) I haven't found myself missing them at all. In other words, for many people the lack of generics comes across as an oversight. For others, including myself (again, a heavy generics user in C#) that really isn't the case. I write Go in the style of Go and it just hasn't been an issue. Blanket statements are rarely true. YMMV.
deleted 4y ago
[deleted]
Thaxll 4y ago
I've seen some real world example where Go was as fast or faster than Rust for CPU / io intensive task. Go is a fast language even with a GC. https://github.com/boyter/scc/#performance https://github.com/boyter/scc/#performance
akira2501 4y ago
For maximum return on investment, wouldn't it better to focus on something other than raw speed? I personally don't see the usefulness of trying to make everything in Rust itself too much as it's handicapped by it's compiler and lack of specification.
frou_dh 4y ago
None of your ideas in this thread mention involving a profiler, so I take it you're from the Wild-Ass-Guess school of optimization?

tester756 4y ago

C library for parsing? isn't it dangerous from security perspective?

xyzzy4747 4y ago
It's just an example. The options are really Rust (what I'd prefer), C++, C, or perhaps something like Nim that compiles to C. If you’re trying to make an unoptimized parser, then use whatever you want.
- deleted 4y ago
  [deleted]

pharmakom 4y ago

For some reason we insist that language parsers are implemented in the language itself, even when the language isn’t great for parsers.

37ef_ced3 4y ago

You're in for a big surprise. Try using the language. Spend some time using Go, and you will be impressed by its performance. You'll wonder, "Were all those haters on Hacker News misinformed?"

xyzzy4747 4y ago
If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
- dymk 4y ago
  This is a premature optimization, and keeping everything in the same language has benefits like greatly simplified tooling and building
  - xyzzy4747 4y ago
    It’s not a premature optimization - it’s deciding the maximum that the parser can be optimized in the future. Choosing Go sets a lower ceiling. > Keeping everything in the same language has benefits like greatly simplified tooling and building Surely there are other Go libraries that incorporate C, C++, or Rust? Also if both parsers existed and were equally easy to set up, and you were planning on doing a ton of parsing, it would make sense to go with the faster one.
    - dymk 4y ago
      It absolutely is a premature optimization. If it's fast enough, then it's fast enough. The author hasn't indicated that the current Go implementation is hitting a ceiling imposed by the language yet. If you'd like to, you can provide some real-world examples - or even microbenchmarks - showing that Go is so much slower than <your choice here> that it's going to make a difference. > Also if both parsers existed and were equally easy to set up They're not equally easy to set up. Language interop is a pain in the pass.
    - Jtsummers 4y ago
      Look at the current Makefile: https://github.com/aretext/aretext/blob/main/Makefile https://github.com/aretext/aretext/blob/main/Makefile Build is literally a `go build ...` and install is `go install`. Adding any other language to the mix would make this a polyglot project and not be "equally easy to set up". The other question is, do both parsers exist? In this write-up they point to tree-sitter as a possibility which is a JS program that produces C code. This would be viable, but here's the author's take: > I considered integrating tree-sitter, an incremental parsing library with parsers for many existing languages. However, running JavaScript to generate parsers and linking to a C library would have greatly complicated the build process. Today, aretext can be built on almost any platform using a single go install command. I’ve had users install aretext on ARM laptops, FreeBSD servers, Chromebooks, and Android phones. To maintain portability, I wanted a pure Go implementation. So this wasn't some casual decision, but something they at least considered long enough to describe here. And the parsing library itself is only around 1200 lines total (comments, blanks, and code). The parsers for each language add a lot more, of course, but should be roughly equivalent given the same library and interface. I imagine that if this project really takes off and performance becomes a real problem they can do the rewrite at that point. Right now, the code works, seems to work fast enough for its author and primary users, and it's trivial to install on any platform supported by Go. So yes, it would have been a premature optimization to complicate the build process, probably reduce the number of supported platforms (or greatly increase the effort to support the same number of platforms), just to have a slightly faster parser.
bugfix-66 4y ago
Correct, Go is fast, very close to C. And just like in C, if you want to avoid memory management overhead you can use a slice of structs, integers instead of pointers, and a freelist (if needed). For example, here is a pointerless sparse bit vector: https://bugfix-66.com/7256e0772dc3b02d72abf15b171731c933fd44d67de074d679f1e4cb7bb20f79 https://bugfix-66.com/7256e0772dc3b02d72abf15b171731c933fd44... The article is storing parses in a balanced binary tree, like a packrat memoizing parser. Here is the fastest balanced search tree in Go. It allocates (and uses Go's garbage collector) but you can easily use a slice of structs with integer index pointers and a freelist instead: https://bugfix-66.com/c93e950965804eba90a34e0055985b1c42d5a1f122b88d1e9c02c68f801388b3 https://bugfix-66.com/c93e950965804eba90a34e0055985b1c42d5a1... The above code will perform very similarly to C.

rollcat 4y ago

One of Go's primary goals has always been compilation speed. Go started out in C, and was later (post-1.0) incrementally rewritten to be self-hosting. One of the authors (Ken Thompson) is also one of the co-creators of C. I would argue these guys know what they are doing.

kaba0 4y ago
I don’t know, not implementing generics when it was pretty obviously needed was a huge oversight, so I’m not sure. Also, the reason for compiler bootstrapping is more of a “beauty thing”, then practicality. It would definitely be faster in a low-level language, but I doubt it would matter as an end user.
- fredrikholm 4y ago
  You aren't sure if Ken Thompson knows what he's doing?
  - kaba0 4y ago
    As a software architect? Absolutely. Programming language designer? Not sure, neither C or Go are good languages in my personal opinion. EDIT: I meant to write that I think very highly of him as an architect/developer.
    - sjansen 4y ago
      Experience has shown that often “worse is better”. Go does an amazing job of balancing complexity and power. I haven’t seen a ”better” language that isn’t either slower, harder to become productive, or both. https://en.wikipedia.org/wiki/Worse_is_better https://en.wikipedia.org/wiki/Worse_is_better
- kcartlidge 4y ago
  > not implementing generics when it was pretty obviously needed was a huge oversight I get the desire for generics. I do a lot of C# and have used generics for a very many years. Yet I've been writing Go for around 6 or 7 years and other than in the beginning (when I was new to it) I haven't found myself missing them at all. In other words, for many people the lack of generics comes across as an oversight. For others, including myself (again, a heavy generics user in C#) that really isn't the case. I write Go in the style of Go and it just hasn't been an issue. Blanket statements are rarely true. YMMV.

deleted 4y ago

[deleted]

Thaxll 4y ago

I've seen some real world example where Go was as fast or faster than Rust for CPU / io intensive task. Go is a fast language even with a GC. https://github.com/boyter/scc/#performance https://github.com/boyter/scc/#performance

akira2501 4y ago

For maximum return on investment, wouldn't it better to focus on something other than raw speed? I personally don't see the usefulness of trying to make everything in Rust itself too much as it's handicapped by it's compiler and lack of specification.

frou_dh 4y ago

None of your ideas in this thread mention involving a profiler, so I take it you're from the Wild-Ass-Guess school of optimization?