5 ms·

How to Use AVX512 in Golang

98 points by signa11 4y ago

carbocation 4y ago
I thought the /r/golang comments on this post were pretty useful[1]. They also introduced me to avo[2], a tool for generating x86 assembly from go that I hadn't seen before. There are some examples listed on the avo github page for generating AVX512 instructions with avo. 1 = https://www.reddit.com/r/golang/comments/10hmh07/how_to_use_avx512_in_golang/ https://www.reddit.com/r/golang/comments/10hmh07/how_to_use_... 2 = https://github.com/mmcloughlin/avo https://github.com/mmcloughlin/avo
- zhenghaoz 4y ago
  For someone experts assembly, avo is a better tool to write assembly in hand.
mappu 4y ago
Cgo is a real option here, or at least would be an interesting comparison. The function is likely called with large enough data to amortize the (few ns) overhead and it's certainly more maintainable. Another option is GOAMD64=v3 / v4 which will enable AVX2 / AVX512 on the regular Go output, although the compiler does almost no autovectorization compared to gccgo/gollvm.
- zhenghaoz 4y ago
  It is a pity that CGO isn't in the final benchmark. >_<
gnabgib 4y ago
Granted I don't know what will happen between now and March 1st, but there are some inaccuracies in this article... is this a GPT product? - Encoding Machine Code[0] Not sure what "Go assembly provides three instructions for representing binary machine code" is intended to mean, but go supports register moves/actions for Byte(1)/Word(2)/Long(4)/Quad(8)/Octo(16), both aligned and unaligned (depending on architecture/processor). The next section literally references a quad action (`movq`). Note O/Octo and DQ/Double Quad mean the same thing depending on instruction/arch (sigh) - When you're using specialized features you have other commands for larger register features (eg in AVX512 on AMD64 asm:`MULPS`, goasm:`VMULPS`) - Redirect Jump Instruction[1], what is the value of this conversion recommendation if this article is recommending an auto converter that should do this for you? Go-asm can support jumping to addresses (routine scoped), or labels.. labels can be in any case, perhaps goat requires only upper case? - Generate Go Function Definitions[2].. err isn't this the value prop of goat[3], the point of this article, that this can be done for you? Why are `cznic`, and `cc` being introduced here? - Function Calls[4] The "complete wrapper" won't compile (typos), I think it's supposed to be closer to [5] (in the repo) - Function Calls[4] the "Note" makes no sense given the code is go only (missing go-asm paste?) - The summary benchmark[6], demonstrates AVX2 (far more common) is often faster, and doesn't explain what was measured, nor on what architecture, whether it was aligned, whether any steps were taken to improve performance on the architecture. Based on the repo [7], there's no attempt at alignment, if you can't do AVX512, or AVX2, you can still do 64bit/quad operations rather than going straight to 8bit/byte ops. - The assembly (.s) files seemingly produced by GOAT[8] are far from human readable go-asm files (the comments in the file could be go-asm source code), don't follow the convention of <file>_<arch>.s, and don't support all platforms that C/golang do (although ARM64 & AMD64 is a large market share) [0]: https://gorse.io/posts/avx512-in-golang.html#encoding-machine-code https://gorse.io/posts/avx512-in-golang.html#encoding-machin... [1]: https://gorse.io/posts/avx512-in-golang.html#redirect-jump-instruction https://gorse.io/posts/avx512-in-golang.html#redirect-jump-i... [2]: https://gorse.io/posts/avx512-in-golang.html#generate-go-function-definitions https://gorse.io/posts/avx512-in-golang.html#generate-go-fun... [3]: https://github.com/gorse-io/gorse/tree/master/cmd/goat https://github.com/gorse-io/gorse/tree/master/cmd/goat [4]: https://gorse.io/posts/avx512-in-golang.html#function-calls https://gorse.io/posts/avx512-in-golang.html#function-calls [5]: https://github.com/gorse-io/gorse/blob/master/base/floats/floats_amd64.go https://github.com/gorse-io/gorse/blob/master/base/floats/fl... [6]: https://gorse.io/posts/avx512-in-golang.html#summary https://gorse.io/posts/avx512-in-golang.html#summary [7]: https://github.com/gorse-io/gorse/tree/master/base/floats https://github.com/gorse-io/gorse/tree/master/base/floats [8]: https://github.com/gorse-io/gorse/blob/master/base/floats/floats_avx512.s https://github.com/gorse-io/gorse/blob/master/base/floats/fl...
- zhenghaoz 4y ago
  Thanks, @gnabgib. Your comment is very insightful and reminds me of my mentor correcting my academic paper. The post introduces the basic idea of using AVX512 in Go by writing C codes. There are mistakes and many details are omitted. A complete example is https://github.com/gorse-io/gorse/tree/master/base/floats https://github.com/gorse-io/gorse/tree/master/base/floats
zhenghaoz 4y ago
I am the author and I am really regret about this title. I think a better title should be "How to Use AVX512 in Golang via the C Compiler" :O
- viraptor 4y ago
  The graph at the end is not readable on the dark theme background. The labels are almost the same colour.
- fulafel 4y ago
  Another suggestion for the title: Go, not Golang.
  - jjgreen 4y ago
    If faced with a co-worker who uses 'Golang', try pronouncing it 'Gol-ang' at regular intervals, liable to annoy them sufficiently to persuade them to desist ...
    - Kon-Peki 4y ago
      They’ll more likely ask if your code is still compiling, since you have so much free time.
      - jjgreen 4y ago
        For that sort of problem I'd generally use C, so laugh in their face :-)
  - jrockway 4y ago
    This is a hill I am also willing to die on. Last time this annoyed me, I did a thorough look at the Go source code to figure out how accurate calling the language "Go" is. Basically, aside from the mailing lists and bug tracker URLs, "golang" only appears in one place; the ppc64 port contributed by IBM. Everywhere else in the code, it refers to itself as "Go".
atomlib 4y ago
> March 1, 2023 What? Did a timetraveller write this?
- arp242 4y ago
  Or their timezone is +50 days.
  - viraptor 4y ago
    Or it's a US/normal date format confusion?
    - arp242 4y ago
      Doesn't seem like it: https://raw.githubusercontent.com/gorse-io/docs/main/src/posts/avx512-in-golang.md https://raw.githubusercontent.com/gorse-io/docs/main/src/pos... Sometimes I write a future date when I expect something to be finished. Probably just something like that and forgot to edit it before publishing (or, maybe they expected it wouldn't be published until that date).
  - sgt 4y ago
    Somewhere in the Oort cloud, perhaps.
- jbirer 4y ago
  Aliens are really sloppy these days. A real human would never do AVX512 in Golang and travel in time.
- jackmott42 4y ago
  I've been trying to make this clear to every one who whines about "oh no the clock rate goes down a little if you use this": AVX-512 is VERY fast, time even gets ripped asunder.
  - deleted 4y ago
    [deleted]
sylware 4y ago
Not really related to golang but: AVX512 is THE sweet spot to go for that for vector instructions as 512bits=64bytes=x86_64 cache line size.
- zamadatix 4y ago
  Dumb question but do you have to do anything manually to ensure it’s actually aligned to a single cache line or does it just happen because of the way things work?
  - stagger87 4y ago
    Not a dumb question. In theory, yes you want it to be aligned for performance reasons, and you can do this with one of the _aligned_malloc()/aligned_alloc() variants. In reality though, for any non-trivial algorithm, you may not be able to always enforce this.
    - sylware 4y ago
      Usually you alloc some slack memory to be able to re-align on a cache line. But yeah, it seems better if the allocator can cleanely deal with that.
neonsunset 4y ago
Upcoming .NET 8 will get native support for Vector512 (AVX512) in runtime, and you can write platform-independent code today with Vector<T> for length-agnostic algorithms that will automatically take advantage of it. No need to ever write assembly, or some other special syntax - just pure C#.
- Kon-Peki 4y ago
  That is fantastic. Swift in the Mac has something similar thanks to the Accelerate framework. What I’ve seen, however, is that if you are targeting a very specific platform (for example, the M1 family), you get better performance by directly using the intrinsics supported by that platform. In the real world, you are unlikely to actually have such a specific target, so this may not be so important. I look forward to seeing some benchmarks with .NET - Microsoft needs to support a pretty wide variety of platforms. It will be interesting to see if their implementation is better!
  - neonsunset 4y ago
    Accelerate is a bit different in being both overall more high level API and specific to Apple (and abstracts away the usage of AMX and ANE too). On the other hand, Vector<T>/Vector128/256/512<T> in .NET is what is 'portable-simd' to Rust except it is not in preview and widely used across standard library where applicable. As of now, Vector<T> automatically targets AVX2, SSE4.2 and AdvSimd (NEON). Vector256<T> targets AVX2 (for the lack of ARM counterpart) and Vector128<T> targets SSE / AdvSimd respectively.
deleted 4y ago
[deleted]

98 points by signa11 4y ago

I thought the /r/golang comments on this post were pretty useful[1]. They also introduced me to avo[2], a tool for generating x86 assembly from go that I hadn't seen before. There are some examples listed on the avo github page for generating AVX512 instructions with avo. 1 = https://www.reddit.com/r/golang/comments/10hmh07/how_to_use_avx512_in_golang/ https://www.reddit.com/r/golang/comments/10hmh07/how_to_use_... 2 = https://github.com/mmcloughlin/avo https://github.com/mmcloughlin/avo

zhenghaoz 4y ago
For someone experts assembly, avo is a better tool to write assembly in hand.

mappu 4y ago

Cgo is a real option here, or at least would be an interesting comparison. The function is likely called with large enough data to amortize the (few ns) overhead and it's certainly more maintainable. Another option is GOAMD64=v3 / v4 which will enable AVX2 / AVX512 on the regular Go output, although the compiler does almost no autovectorization compared to gccgo/gollvm.

zhenghaoz 4y ago
It is a pity that CGO isn't in the final benchmark. >_<

gnabgib 4y ago

Granted I don't know what will happen between now and March 1st, but there are some inaccuracies in this article... is this a GPT product? - Encoding Machine Code[0] Not sure what "Go assembly provides three instructions for representing binary machine code" is intended to mean, but go supports register moves/actions for Byte(1)/Word(2)/Long(4)/Quad(8)/Octo(16), both aligned and unaligned (depending on architecture/processor). The next section literally references a quad action (`movq`). Note O/Octo and DQ/Double Quad mean the same thing depending on instruction/arch (sigh) - When you're using specialized features you have other commands for larger register features (eg in AVX512 on AMD64 asm:`MULPS`, goasm:`VMULPS`) - Redirect Jump Instruction[1], what is the value of this conversion recommendation if this article is recommending an auto converter that should do this for you? Go-asm can support jumping to addresses (routine scoped), or labels.. labels can be in any case, perhaps goat requires only upper case? - Generate Go Function Definitions[2].. err isn't this the value prop of goat[3], the point of this article, that this can be done for you? Why are `cznic`, and `cc` being introduced here? - Function Calls[4] The "complete wrapper" won't compile (typos), I think it's supposed to be closer to [5] (in the repo) - Function Calls[4] the "Note" makes no sense given the code is go only (missing go-asm paste?) - The summary benchmark[6], demonstrates AVX2 (far more common) is often faster, and doesn't explain what was measured, nor on what architecture, whether it was aligned, whether any steps were taken to improve performance on the architecture. Based on the repo [7], there's no attempt at alignment, if you can't do AVX512, or AVX2, you can still do 64bit/quad operations rather than going straight to 8bit/byte ops. - The assembly (.s) files seemingly produced by GOAT[8] are far from human readable go-asm files (the comments in the file could be go-asm source code), don't follow the convention of <file>_<arch>.s, and don't support all platforms that C/golang do (although ARM64 & AMD64 is a large market share) [0]: https://gorse.io/posts/avx512-in-golang.html#encoding-machine-code https://gorse.io/posts/avx512-in-golang.html#encoding-machin... [1]: https://gorse.io/posts/avx512-in-golang.html#redirect-jump-instruction https://gorse.io/posts/avx512-in-golang.html#redirect-jump-i... [2]: https://gorse.io/posts/avx512-in-golang.html#generate-go-function-definitions https://gorse.io/posts/avx512-in-golang.html#generate-go-fun... [3]: https://github.com/gorse-io/gorse/tree/master/cmd/goat https://github.com/gorse-io/gorse/tree/master/cmd/goat [4]: https://gorse.io/posts/avx512-in-golang.html#function-calls https://gorse.io/posts/avx512-in-golang.html#function-calls [5]: https://github.com/gorse-io/gorse/blob/master/base/floats/floats_amd64.go https://github.com/gorse-io/gorse/blob/master/base/floats/fl... [6]: https://gorse.io/posts/avx512-in-golang.html#summary https://gorse.io/posts/avx512-in-golang.html#summary [7]: https://github.com/gorse-io/gorse/tree/master/base/floats https://github.com/gorse-io/gorse/tree/master/base/floats [8]: https://github.com/gorse-io/gorse/blob/master/base/floats/floats_avx512.s https://github.com/gorse-io/gorse/blob/master/base/floats/fl...

zhenghaoz 4y ago
Thanks, @gnabgib. Your comment is very insightful and reminds me of my mentor correcting my academic paper. The post introduces the basic idea of using AVX512 in Go by writing C codes. There are mistakes and many details are omitted. A complete example is https://github.com/gorse-io/gorse/tree/master/base/floats https://github.com/gorse-io/gorse/tree/master/base/floats

zhenghaoz 4y ago

I am the author and I am really regret about this title. I think a better title should be "How to Use AVX512 in Golang via the C Compiler" :O

viraptor 4y ago
The graph at the end is not readable on the dark theme background. The labels are almost the same colour.
fulafel 4y ago
Another suggestion for the title: Go, not Golang.
- jjgreen 4y ago
  If faced with a co-worker who uses 'Golang', try pronouncing it 'Gol-ang' at regular intervals, liable to annoy them sufficiently to persuade them to desist ...
  - Kon-Peki 4y ago
    They’ll more likely ask if your code is still compiling, since you have so much free time.
    - jjgreen 4y ago
      For that sort of problem I'd generally use C, so laugh in their face :-)
- jrockway 4y ago
  This is a hill I am also willing to die on. Last time this annoyed me, I did a thorough look at the Go source code to figure out how accurate calling the language "Go" is. Basically, aside from the mailing lists and bug tracker URLs, "golang" only appears in one place; the ppc64 port contributed by IBM. Everywhere else in the code, it refers to itself as "Go".

atomlib 4y ago

> March 1, 2023 What? Did a timetraveller write this?

arp242 4y ago
Or their timezone is +50 days.
- viraptor 4y ago
  Or it's a US/normal date format confusion?
  - arp242 4y ago
    Doesn't seem like it: https://raw.githubusercontent.com/gorse-io/docs/main/src/posts/avx512-in-golang.md https://raw.githubusercontent.com/gorse-io/docs/main/src/pos... Sometimes I write a future date when I expect something to be finished. Probably just something like that and forgot to edit it before publishing (or, maybe they expected it wouldn't be published until that date).
- sgt 4y ago
  Somewhere in the Oort cloud, perhaps.
jbirer 4y ago
Aliens are really sloppy these days. A real human would never do AVX512 in Golang and travel in time.
jackmott42 4y ago
I've been trying to make this clear to every one who whines about "oh no the clock rate goes down a little if you use this": AVX-512 is VERY fast, time even gets ripped asunder.
- deleted 4y ago
  [deleted]

sylware 4y ago

Not really related to golang but: AVX512 is THE sweet spot to go for that for vector instructions as 512bits=64bytes=x86_64 cache line size.

zamadatix 4y ago
Dumb question but do you have to do anything manually to ensure it’s actually aligned to a single cache line or does it just happen because of the way things work?
- stagger87 4y ago
  Not a dumb question. In theory, yes you want it to be aligned for performance reasons, and you can do this with one of the _aligned_malloc()/aligned_alloc() variants. In reality though, for any non-trivial algorithm, you may not be able to always enforce this.
  - sylware 4y ago
    Usually you alloc some slack memory to be able to re-align on a cache line. But yeah, it seems better if the allocator can cleanely deal with that.

neonsunset 4y ago

Upcoming .NET 8 will get native support for Vector512 (AVX512) in runtime, and you can write platform-independent code today with Vector<T> for length-agnostic algorithms that will automatically take advantage of it. No need to ever write assembly, or some other special syntax - just pure C#.

Kon-Peki 4y ago
That is fantastic. Swift in the Mac has something similar thanks to the Accelerate framework. What I’ve seen, however, is that if you are targeting a very specific platform (for example, the M1 family), you get better performance by directly using the intrinsics supported by that platform. In the real world, you are unlikely to actually have such a specific target, so this may not be so important. I look forward to seeing some benchmarks with .NET - Microsoft needs to support a pretty wide variety of platforms. It will be interesting to see if their implementation is better!
- neonsunset 4y ago
  Accelerate is a bit different in being both overall more high level API and specific to Apple (and abstracts away the usage of AMX and ANE too). On the other hand, Vector<T>/Vector128/256/512<T> in .NET is what is 'portable-simd' to Rust except it is not in preview and widely used across standard library where applicable. As of now, Vector<T> automatically targets AVX2, SSE4.2 and AdvSimd (NEON). Vector256<T> targets AVX2 (for the lack of ARM counterpart) and Vector128<T> targets SSE / AdvSimd respectively.

deleted 4y ago

[deleted]