3 ms·

Nice, my push for actually using the sha256 instructions on amd64 finally got released. 3x-4x increase in hash speed on most x86 which is really nice for conte

Nice, my push for actually using the sha256 instructions on amd64 finally got released. 3x-4x increase in hash speed on most x86 which is really nice for content addressable storage use cases like handling container images.

candiddevmike 3y ago
Got a link to the PR? Curious to see how this is implemented.
- jeffbee 3y ago
  It looks like https://go-review.googlesource.com/c/go/+/408795 https://go-review.googlesource.com/c/go/+/408795
  - xyst 3y ago
    Seems like these changes only benefit you if you have an Intel processor. If you have an AMD or Arm processor, you won’t see any difference. Also, interesting to see assembly again after many years. Haven’t touched that since college during a compilers and assembly course. Edit: never mind, amd has implemented these “sha-ni” instructions since “Zen” [1] [1] https://en.wikipedia.org/wiki/Intel_SHA_extensions https://en.wikipedia.org/wiki/Intel_SHA_extensions
    - monocasa 3y ago
      And adding that they already had support for ARM sha256 instructions https://github.com/golang/go/blob/master/src/crypto/sha256/sha256block_arm64.s https://github.com/golang/go/blob/master/src/crypto/sha256/s...
jeffbee 3y ago
Huh, that is interesting how they do that. They are enabling SHA instruction support based on CPUID and without respect to the value of GOAMD64. I did not realize Go was doing that.
- monocasa 3y ago
  AFAIK, the sha256 extension isn't a part of any of the x86_64 microarchitecture levels, so a cpuid check is most appropriate here at the moment.
  - jeffbee 3y ago
    Fair point. But what surprised me was the way HasAVX2 is getting set. It is set on the hardware that has AVX2, even if you set GOAMD64=v1.
    - als0 3y ago
      In case anyone else was wondering what this is about, here's some useful background https://github.com/golang/go/issues/45453 https://github.com/golang/go/issues/45453
    - burntsushi 3y ago
      Yup, that's standard, including in other ecosystems. It's what I do in ripgrep for example when your target is just standard `x86_64` (v1). GNU libc does it as well. And I believe Go has been doing it for quite some time. (The specific one I'm aware of is bytes.Index.) This was especially important back before the days of v1/v2/v3/etc of x86_64, since Linux distros distributed binaries compiled for the lowest common denominator. So the only way you got fast SIMD instructions (beyond SSE2) was with a CPUID check and some compiler features that let you build target specific functions. (And I'm not sure what the status is of Linux distros shipping v1/v2/v3/etc binaries.)

candiddevmike 3y ago

Got a link to the PR? Curious to see how this is implemented.

jeffbee 3y ago
It looks like https://go-review.googlesource.com/c/go/+/408795 https://go-review.googlesource.com/c/go/+/408795
- xyst 3y ago
  Seems like these changes only benefit you if you have an Intel processor. If you have an AMD or Arm processor, you won’t see any difference. Also, interesting to see assembly again after many years. Haven’t touched that since college during a compilers and assembly course. Edit: never mind, amd has implemented these “sha-ni” instructions since “Zen” [1] [1] https://en.wikipedia.org/wiki/Intel_SHA_extensions https://en.wikipedia.org/wiki/Intel_SHA_extensions
  - monocasa 3y ago
    And adding that they already had support for ARM sha256 instructions https://github.com/golang/go/blob/master/src/crypto/sha256/sha256block_arm64.s https://github.com/golang/go/blob/master/src/crypto/sha256/s...

jeffbee 3y ago

Huh, that is interesting how they do that. They are enabling SHA instruction support based on CPUID and without respect to the value of GOAMD64. I did not realize Go was doing that.

monocasa 3y ago
AFAIK, the sha256 extension isn't a part of any of the x86_64 microarchitecture levels, so a cpuid check is most appropriate here at the moment.
- jeffbee 3y ago
  Fair point. But what surprised me was the way HasAVX2 is getting set. It is set on the hardware that has AVX2, even if you set GOAMD64=v1.
  - als0 3y ago
    In case anyone else was wondering what this is about, here's some useful background https://github.com/golang/go/issues/45453 https://github.com/golang/go/issues/45453
  - burntsushi 3y ago
    Yup, that's standard, including in other ecosystems. It's what I do in ripgrep for example when your target is just standard `x86_64` (v1). GNU libc does it as well. And I believe Go has been doing it for quite some time. (The specific one I'm aware of is bytes.Index.) This was especially important back before the days of v1/v2/v3/etc of x86_64, since Linux distros distributed binaries compiled for the lowest common denominator. So the only way you got fast SIMD instructions (beyond SSE2) was with a CPUID check and some compiler features that let you build target specific functions. (And I'm not sure what the status is of Linux distros shipping v1/v2/v3/etc binaries.)