7 ms·

>I'm curious about the statement that SHA-3 is slow; [...] I wonder how much relative attention the SHA-3 winner (Keccak) gets compared to other alternatives, l

by jasode 9y ago

>I'm curious about the statement that SHA-3 is slow; [...] I wonder how much relative attention the SHA-3 winner (Keccak) gets compared to other alternatives, like BLAKE? Coincidentally, I ran a bunch of hash performance benchmarks last week. These were my findings: test: hash a 500MB block of memory. hardware: Intel Core i7-5820K Haswell-E 6-Core 3.3GHz compiler: MSVC2017 (19.10.25019), 32-bit exe: blake2sp - official reference code[1] 153MB/sec SHA3 - Keccak official reference code[2] 12MB/sec SHA3 - rhash sha3[3] 45MB/sec SHA3 - Crypto++ library v5.6.5[4] 57MB/sec SHA256 - Crypto++ 181MB/sec SHA256 - MS Crypto API[5] 113MB/sec SHA1 - MS Crypto API 338MB/sec MD5 - Crypto++ 345MB/sec CRC32 - Crypto++ 323MB/sec The conclusion is that the fastest SHA3 implementation (Crypto++ lib with its assembly language optimizations) is more than 2x-3x slower than SHA256. I can't speak for SHA3 implemented in FPGA/ASIC but as far as C++ compilation targeting x86, it's slow. I've been meaning to try the Intel Compiler to see if it yields different results but haven't gotten around to it yet. Blake2sp is fast. The official reference code is not quite as fast as Crypto++ implementation of SHA256 but it's faster than Microsoft's Crypto API of SHA256. (There are several variants of BLAKE and I chose blake2sp because that's the algorithm WinRAR uses. I think the specific variant of BLAKE that directly competed with Keccack for NIST standardization is slower.) [1] https://github.com/BLAKE2/BLAKE2 https://github.com/BLAKE2/BLAKE2 [2] http://keccak.noekeon.org/files.html http://keccak.noekeon.org/files.html [3] https://github.com/rhash/RHash/blob/master/librhash/sha3.c https://github.com/rhash/RHash/blob/master/librhash/sha3.c [4] https://www.cryptopp.com/ https://www.cryptopp.com/ [5] https://msdn.microsoft.com/en-us/library/ms867086.aspx https://msdn.microsoft.com/en-us/library/ms867086.aspx

briansmith 9y ago
I guess 32-bit x86 performance is maybe not the best benchmark. I think people aren't optimizing for that ISA to the same extent as they are optimizing for x86-64, 64-bit ARMv8, or 32-bit ARM. If you care about performance and you don't have dedicated SHA-256 instructions then on a 64-bit platform you should evaluate SHA-512 as it is much faster. If you only have 256 bits of storage available then truncate its output to 256 bits. IIRC, it's about 1GB/sec on my Haswell laptop.
- jasode 9y ago
  >I guess 32-bit x86 performance is maybe not the best benchmark. I compiled for 32bit instead of 64bit because I wanted the same executable to also run on a 32bit Macbook. When Thomas Pornin ran benchmarks[1] in 2010 for both 32bit & 64bit, the SHA256 hash performance didn't change as much as the SHA512. I'll recompile for 64bit and report back if there was a massive difference. [1] https://stackoverflow.com/questions/2722943/is-calculating-an-md5-hash-less-cpu-intensive-than-sha-family-functions/2723941#2723941 https://stackoverflow.com/questions/2722943/is-calculating-a...
  - baby 9y ago
    SHA-3 lanes are 64 bits. In 64 bits arch they can use full registers for operations. I'd bet it would be way faster on 64bit
    - npscalar 9y ago
      Mismatches can show an interesting property. It is likely that SHA-256 is slower than Keccak on 64-bit platforms, and that SHA-512 is slower than Keccak on 32-bit platform.
gigatexal 9y ago
It'd be interesting to see what SHA2-512 would do since I think modern intel CPUs prefer that size to 256. (I could be wrong though as i have no source to cite just something I remember reading.)
- dchest 9y ago
  You're not wrong. Type this into Terminal: openssl speed sha512 sha256
  - gigatexal 9y ago
    tl;dr sha512 outperforms sha256 as message size increases. pastebin output for lack of decent formatting: https://pastebin.com/zqgtTKhm https://pastebin.com/zqgtTKhm on a linode system with 2 xeon E5-2680v3 cores @ 2.5ghz and 4GB of ram on this version and config of openssl on ubuntu server 17.04: OpenSSL 1.0.2g 1 Mar 2016 built on: reproducible build, date unspecified options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) compiler: cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fdebug-prefix-map=/build/openssl-p_sOry/openssl-1.0.2g=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
dchest 9y ago
Hmm, something is wrong with your benchmark: the absolute values in MB/s are very low for i7, and relative too — blake2sp should be much faster than SHA256.
- jasode 9y ago
  >blake2sp should be much faster than SHA256. blake2sp is indeed faster than Microsoft's builtin Crypto API for SHA256. However, it is not as fast as Wei Dai's Crypto++ library implementation of SHA256 that has lots of hand tuned assembly language code. The official C source code for blake2sp does not have assembly language primitives in it. It's very possible that if an assembly language expert wrote optimizations for blake2sp, it would beat Crypto++ SHA256 performance. The code I used is really simple. Used files "blake2sp-ref.c" and "blake2s-ref.c" from the BLAKE website. The hash code (no loops) is: blake2sp_state S[1]; // BLAKE 1 element array of the struct for state blake2sp_init(S, BLAKE2S_OUTBYTES); blake2sp_update(S, BufInput, iBuffersize); blake2sp_final(S, hashval_blake2sp_bytes, BLAKE2S_OUTBYTES); ... where iBuffersize is 500MB but I got the same results at larger buffer sizes 1GB+. With that code, I'm guessing anyone can whip up a C++ project to benchmark blake2sp in 10 minutes. It would be interseting to see what MB/sec that others achieve.
  - mappu 9y ago
    There is AVX/AVX2 assembly for BLAKE2 at https://github.com/minio/blake2b-simd https://github.com/minio/blake2b-simd , benchmarked at a 1.8x-3.9x speedup.
  - dchest 9y ago
    My pure JavaScript implementation of blake2s (which is approximately half the speed of parallelized sp variant) on 2.6 GHz Core i5 hashes at 170 MiB/s. JavaScript! Whatever you do with your benchmark, you're doing it wrong. Also, there is no reason to use such large buffer sizes, I suspect this only makes benchmarks more unreliable. For real numbers on many platforms, see https://bench.cr.yp.to/results-hash.html https://bench.cr.yp.to/results-hash.html (warning: very large page) For long messages on 2015 Intel Core i5-6600; 4 x 3310MHz: blake2b: 3.33 cycles/byte blake2s (remember, half speed of sp): 4.87 cycles/byte sha512: 5.06 cycles/byte sha256: 7.63 cycles/byte This means BLAKE2s is close to 700 MB/s, and BLAKE2sp will hash data at more than 1 GB/s. Here are the SHA-256 implementations measured: https://bench.cr.yp.to/impl-hash/sha256.html https://bench.cr.yp.to/impl-hash/sha256.html As you can see, the winner varies by platform, but in most cases OpenSSL wins, and Wei Dai's implementation is close.
snakeanus 9y ago
I am quite surprised that CRC32 is slower than MD5. > I think the specific variant of BLAKE that directly competed with Keccack is slower BLAKE is slower than BLAKE2 indeed. BLAKE also has more rounds than BLAKE2.
- dfox 9y ago
  CRC32 is slower because implementing it in software a way that exploits modern CPU's instruction level paralelism is non-trivial, also the obvious way how to implement CRC32 on 32bit byte oriented CPU isn't exactly cache friendly.
  - tjalfi 9y ago
    Are you referring to Intel's Slicing by 8 algorithm[0]. [0] https://static.aminer.org/pdf/PDF/000/432/446/a_systematic_approach_to_building_high_performance_software_based_crc.pdf https://static.aminer.org/pdf/PDF/000/432/446/a_systematic_a...
baby 9y ago
What about cShake or better KangarooTwelve? (K12 is to SHA-3 what blake2 is to blake).
optimiz3 9y ago
I'll just say this - there are private implementations of Keccak that blow the pants off of those numbers. Keccak is one of the primary functions in Ethash (Ethereum mining). It is heavily researched and completely destroys SHA2 performance-wise if you have the right implementation. Also, Keccak doesn't require the construction of a key-schedule, and can be implemented much more elegantly in parallel (SIMD software) and in hardware than SHA2.
ktta 9y ago
You might also want to test out BLAKE2bp, which is optimized for 64-bit platforms, while BLAKE2sp is for 32-bit platforms. So the numbers for BLAKE2 will be even higher

by jasode 9y ago

briansmith 9y ago

I guess 32-bit x86 performance is maybe not the best benchmark. I think people aren't optimizing for that ISA to the same extent as they are optimizing for x86-64, 64-bit ARMv8, or 32-bit ARM. If you care about performance and you don't have dedicated SHA-256 instructions then on a 64-bit platform you should evaluate SHA-512 as it is much faster. If you only have 256 bits of storage available then truncate its output to 256 bits. IIRC, it's about 1GB/sec on my Haswell laptop.

jasode 9y ago
>I guess 32-bit x86 performance is maybe not the best benchmark. I compiled for 32bit instead of 64bit because I wanted the same executable to also run on a 32bit Macbook. When Thomas Pornin ran benchmarks[1] in 2010 for both 32bit & 64bit, the SHA256 hash performance didn't change as much as the SHA512. I'll recompile for 64bit and report back if there was a massive difference. [1] https://stackoverflow.com/questions/2722943/is-calculating-an-md5-hash-less-cpu-intensive-than-sha-family-functions/2723941#2723941 https://stackoverflow.com/questions/2722943/is-calculating-a...
- baby 9y ago
  SHA-3 lanes are 64 bits. In 64 bits arch they can use full registers for operations. I'd bet it would be way faster on 64bit
  - npscalar 9y ago
    Mismatches can show an interesting property. It is likely that SHA-256 is slower than Keccak on 64-bit platforms, and that SHA-512 is slower than Keccak on 32-bit platform.

gigatexal 9y ago

It'd be interesting to see what SHA2-512 would do since I think modern intel CPUs prefer that size to 256. (I could be wrong though as i have no source to cite just something I remember reading.)

dchest 9y ago
You're not wrong. Type this into Terminal: openssl speed sha512 sha256
- gigatexal 9y ago
  tl;dr sha512 outperforms sha256 as message size increases. pastebin output for lack of decent formatting: https://pastebin.com/zqgtTKhm https://pastebin.com/zqgtTKhm on a linode system with 2 xeon E5-2680v3 cores @ 2.5ghz and 4GB of ram on this version and config of openssl on ubuntu server 17.04: OpenSSL 1.0.2g 1 Mar 2016 built on: reproducible build, date unspecified options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) compiler: cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fdebug-prefix-map=/build/openssl-p_sOry/openssl-1.0.2g=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM

dchest 9y ago

Hmm, something is wrong with your benchmark: the absolute values in MB/s are very low for i7, and relative too — blake2sp should be much faster than SHA256.

jasode 9y ago
>blake2sp should be much faster than SHA256. blake2sp is indeed faster than Microsoft's builtin Crypto API for SHA256. However, it is not as fast as Wei Dai's Crypto++ library implementation of SHA256 that has lots of hand tuned assembly language code. The official C source code for blake2sp does not have assembly language primitives in it. It's very possible that if an assembly language expert wrote optimizations for blake2sp, it would beat Crypto++ SHA256 performance. The code I used is really simple. Used files "blake2sp-ref.c" and "blake2s-ref.c" from the BLAKE website. The hash code (no loops) is: blake2sp_state S[1]; // BLAKE 1 element array of the struct for state blake2sp_init(S, BLAKE2S_OUTBYTES); blake2sp_update(S, BufInput, iBuffersize); blake2sp_final(S, hashval_blake2sp_bytes, BLAKE2S_OUTBYTES); ... where iBuffersize is 500MB but I got the same results at larger buffer sizes 1GB+. With that code, I'm guessing anyone can whip up a C++ project to benchmark blake2sp in 10 minutes. It would be interseting to see what MB/sec that others achieve.
- mappu 9y ago
  There is AVX/AVX2 assembly for BLAKE2 at https://github.com/minio/blake2b-simd https://github.com/minio/blake2b-simd , benchmarked at a 1.8x-3.9x speedup.
- dchest 9y ago
  My pure JavaScript implementation of blake2s (which is approximately half the speed of parallelized sp variant) on 2.6 GHz Core i5 hashes at 170 MiB/s. JavaScript! Whatever you do with your benchmark, you're doing it wrong. Also, there is no reason to use such large buffer sizes, I suspect this only makes benchmarks more unreliable. For real numbers on many platforms, see https://bench.cr.yp.to/results-hash.html https://bench.cr.yp.to/results-hash.html (warning: very large page) For long messages on 2015 Intel Core i5-6600; 4 x 3310MHz: blake2b: 3.33 cycles/byte blake2s (remember, half speed of sp): 4.87 cycles/byte sha512: 5.06 cycles/byte sha256: 7.63 cycles/byte This means BLAKE2s is close to 700 MB/s, and BLAKE2sp will hash data at more than 1 GB/s. Here are the SHA-256 implementations measured: https://bench.cr.yp.to/impl-hash/sha256.html https://bench.cr.yp.to/impl-hash/sha256.html As you can see, the winner varies by platform, but in most cases OpenSSL wins, and Wei Dai's implementation is close.

snakeanus 9y ago

I am quite surprised that CRC32 is slower than MD5. > I think the specific variant of BLAKE that directly competed with Keccack is slower BLAKE is slower than BLAKE2 indeed. BLAKE also has more rounds than BLAKE2.

dfox 9y ago
CRC32 is slower because implementing it in software a way that exploits modern CPU's instruction level paralelism is non-trivial, also the obvious way how to implement CRC32 on 32bit byte oriented CPU isn't exactly cache friendly.
- tjalfi 9y ago
  Are you referring to Intel's Slicing by 8 algorithm[0]. [0] https://static.aminer.org/pdf/PDF/000/432/446/a_systematic_approach_to_building_high_performance_software_based_crc.pdf https://static.aminer.org/pdf/PDF/000/432/446/a_systematic_a...

baby 9y ago

What about cShake or better KangarooTwelve? (K12 is to SHA-3 what blake2 is to blake).

optimiz3 9y ago

I'll just say this - there are private implementations of Keccak that blow the pants off of those numbers. Keccak is one of the primary functions in Ethash (Ethereum mining). It is heavily researched and completely destroys SHA2 performance-wise if you have the right implementation. Also, Keccak doesn't require the construction of a key-schedule, and can be implemented much more elegantly in parallel (SIMD software) and in hardware than SHA2.

ktta 9y ago

You might also want to test out BLAKE2bp, which is optimized for 64-bit platforms, while BLAKE2sp is for 32-bit platforms. So the numbers for BLAKE2 will be even higher