5 ms·

The amount of assembly in the Go ecosystem is crazy, ~1% of the standard library is assembly. There are better ways.

giovannibajo1 10y ago
Most of it is written to take advantage of special hardware instructions in CPUs. If you take OpenSSL (a C library), it's got many assembly code paths as well. I'm not sure why this should be a problem, it just shows attention to performance in my opinion.
- pcwalton 10y ago
  Opaque assembly blocks that the compiler doesn't understand the effects of are actually suboptimal for performance.
  - drchickensalad 10y ago
    Generally yes, specialized no.
  - dlsniper 10y ago
    Is this a general thing or a Go specific one? This generic statement seems to be false given how many optimizations that actually need assembly exist in Go and not only.
    - pjmlp 10y ago
      It is a general thing and quite common. Either you keep adding intrisics to the compiler, or outsource it to an external Assembler. Even managed languages have bytecode Assemblers available.
  - dlsniper 10y ago
    Is this a general thing or a Go specific one? This generic statement seems to be false given how many optimizations that actually need assembly exist in Go and not only.
  - Cthulhu_ 10y ago
    Only if the compiler is aware of and can translate code into e.g. ARM-specific sha instructions. If not, then the developer is still smarter than the compiler and should use assembly to use those specific instructions.
  - shinigami 10y ago
    Rarely true for low level crypto code.
bradleyjg 10y ago
What's the alternative in this case?
- white-flame 10y ago
  From a bird's eye view, adjust the compiler until it outputs the assembly you would have written. Hopefully this is done in such a way that such adjustments can be cross-beneficial across platforms, and can be easily created.
  - giovannibajo1 10y ago
    That's wishful thinking. GCC isn't able to do that with C language, a 20 years old compiler with a 40 years language. How would a compiler know that you can use a SHA2 instruction? Yes, there are intrinsics but then what about specialized vectorized instructions which don't match any primitive types or operations in the language (like, you can't express "add with carry" in C or Go)? And how can you explain a compiler that there is no data dependency in memory access between different buffers? Yes it might be able to prove it by itself in some cases, but history has shown that things like autovectorization are too fragile and can't be relied upon. Optimized C libraries like compressors or security have assembly implementations as well. I'm not sure why using assembler should be considered a signal of a defect rather than a proof of optimization. There's really nothing wrong in using assembly to write a bytes.Index version optimized for AVX2.
    - DannyBee 10y ago
      "That's wishful thinking. GCC isn't able to do that with C language, a 20 years old compiler with a 40 years language. How would a compiler know that you can use a SHA2 instruction?" Well, actually, they could, it's just not worth pattern matching because it occurs so infrequently. "but history has shown that things like autovectorization are too fragile and can't be relied upon." Errr, i'd say the opposite. History has shown that good autovectorizing compilers can come pretty damn close to whatever you want. Usually the only issue is that they may insert too many runtime checks, and that's easily solvable.
sgmansfield 10y ago
Do you have a source for the 1% number? Also I'd like to know what you would consider an appropriate amount of assembly code.

Most of it is written to take advantage of special hardware instructions in CPUs. If you take OpenSSL (a C library), it's got many assembly code paths as well. I'm not sure why this should be a problem, it just shows attention to performance in my opinion.

pcwalton 10y ago
Opaque assembly blocks that the compiler doesn't understand the effects of are actually suboptimal for performance.
- drchickensalad 10y ago
  Generally yes, specialized no.
- dlsniper 10y ago
  Is this a general thing or a Go specific one? This generic statement seems to be false given how many optimizations that actually need assembly exist in Go and not only.
  - pjmlp 10y ago
    It is a general thing and quite common. Either you keep adding intrisics to the compiler, or outsource it to an external Assembler. Even managed languages have bytecode Assemblers available.
- dlsniper 10y ago
  Is this a general thing or a Go specific one? This generic statement seems to be false given how many optimizations that actually need assembly exist in Go and not only.
- Cthulhu_ 10y ago
  Only if the compiler is aware of and can translate code into e.g. ARM-specific sha instructions. If not, then the developer is still smarter than the compiler and should use assembly to use those specific instructions.
- shinigami 10y ago
  Rarely true for low level crypto code.

bradleyjg 10y ago

What's the alternative in this case?

white-flame 10y ago
From a bird's eye view, adjust the compiler until it outputs the assembly you would have written. Hopefully this is done in such a way that such adjustments can be cross-beneficial across platforms, and can be easily created.
- giovannibajo1 10y ago
  That's wishful thinking. GCC isn't able to do that with C language, a 20 years old compiler with a 40 years language. How would a compiler know that you can use a SHA2 instruction? Yes, there are intrinsics but then what about specialized vectorized instructions which don't match any primitive types or operations in the language (like, you can't express "add with carry" in C or Go)? And how can you explain a compiler that there is no data dependency in memory access between different buffers? Yes it might be able to prove it by itself in some cases, but history has shown that things like autovectorization are too fragile and can't be relied upon. Optimized C libraries like compressors or security have assembly implementations as well. I'm not sure why using assembler should be considered a signal of a defect rather than a proof of optimization. There's really nothing wrong in using assembly to write a bytes.Index version optimized for AVX2.
  - DannyBee 10y ago
    "That's wishful thinking. GCC isn't able to do that with C language, a 20 years old compiler with a 40 years language. How would a compiler know that you can use a SHA2 instruction?" Well, actually, they could, it's just not worth pattern matching because it occurs so infrequently. "but history has shown that things like autovectorization are too fragile and can't be relied upon." Errr, i'd say the opposite. History has shown that good autovectorizing compilers can come pretty damn close to whatever you want. Usually the only issue is that they may insert too many runtime checks, and that's easily solvable.

sgmansfield 10y ago

Do you have a source for the 1% number? Also I'd like to know what you would consider an appropriate amount of assembly code.