5 ms·
Chorba: A novel CRC32 implementation (2024)
- fnands 7mo agoNews to me, but a guy named Sam Russell came up with a new software only CRC32 algorithm that is competitive with hardware accelerated implementations. It's a surprisingly elegant solution.
- david-gpu 7mo agoIn Spain, "chorba" is very informal slang for "gal" [0]. Not vulgar, just very informal vernacular. [0] https://dle.rae.es/chorbo https://dle.rae.es/chorbo
- SalimoS 7mo agoand is a traditional Tunisian soup /Edit: actually in all North Africa https://en.wikipedia.org/wiki/Chorba https://en.wikipedia.org/wiki/Chorba
- woadwarrior01 7mo agoChorba is also soup in Eastern European languages like Bulgarian and Romanian. https://en.wikipedia.org/wiki/Chorba https://en.wikipedia.org/wiki/Chorba
- nz 7mo agoAnd also in Turkey. It (the word, if not the stew itself) arrived to the Balkans by way of the Ottomans. (And having just now clicked through to the link, it seems to have arrived to Turkey by way of the Persians).
- fnands 7mo agoApparently that's kinda where the name comes from. It's named after a Serbian musician who was known as Bora Čorba, who played for a band called Riblja Čorba (fish stew).
- nzeid 7mo agoAll this talk of soup making me wonder if these are Arabic/ME derivatives. Then again there are like 10 different ways to refer to soup in the various dialects.
- wongarsu 7mo agoThe wikipedia article traces it to Persian, which formed it as a compound of words from different East Iranian languages. So you are on the money with Middle Eastern. From there it spread to the Balkan via Ottoman Turkish, and also from Persian to dialectal Arabic, which would explain the occurrences in Northern Africa, and maybe even Spain
- rurban 7mo agoIn Europe this guy stands for vulgar and racist neofascism
- david-gpu 7mo agoIf you have a problem with my comments, there is a link in my profile to a convenient way to hide comments from people you don't like. You are welcome!
- Retr0id 7mo agoThis repo's readme gives a great overview of the previous-best approaches (as of ~6 years ago): https://github.com/komrad36/CRC https://github.com/komrad36/CRC
- xyzzy_plugh 7mo agoWarning: the README is 134KB... It's good but, like me, you might be here a while.
- ranger_danger 7mo ago> Dedication > This implementation is named after the Serbian singer Bora Đorđević (also known as Bora Čorba) who was born in 1952 and died in 2024. His birth year matches the number of the GZIP standard RFC 1952 that describes a common CRC32 implementation, and the original proof of concept for this method used the polynomial x21 +x15 + x14 + x11 + x10 + x7 + x3 which is x1952×8 mod G(x). That is indeed dedication.
- omoikane 7mo agoWhat are the units on the vertical axes for figures 1 and 2? I might have guessed seconds per TiB but the braiding line doesn't seem to match what's in figure 3.
- samrussellbg 7mo agoI forget, these were the outputs of my test scripts and I wasn't exactly fastidious here, at least the axes all start from zero :) IIRC they're off by maybe a factor of 10 or 100, the test scripts just generate a bunch of (seeded) random data and then execute the CRC of X bits Y times and that's where the number comes from, it's consistent across the different tests even if the units are wrong
- garganzol 7mo agoAnyone can replicate the results? In any case, works like this give me moments of epiphany when I start to believe the humanity is not totally lost.
- Retr0id 7mo agozlib-ng implements it so I assume it is practical at the very least
- samrussellbg 7mo agothe zlib-ng guys are super welcoming and helpful, working with them was a very pleasant experience
- fnands 7mo agoOh hey, the man himself! I was looking at the zlib-ng crc32 implementation which is where I saw that it was recently updated to include your algorithm. Good work, it's a surprisingly elegant solution when compared to the braiding approaches!
- samrussellbg 7mo agothanks :) the braiding approach is super clever too, this was one of those weird moments where you find something and then have to triple check your results because how could i accidentally find something better than the algorithm that hasn't been touched in decades... the part i really like is that it gives us small improvement on the pclmul too, as the non-accelerated algorithm doesn't really stand a chance against the accelerated opcode on newer hardware so it probably isn't going to see much use in practice. however... i think hardware solutions could possibly benefit (e.g. ethernet cards)