5 ms·

Does anyone know any systems where generating CRC32s is a bottleneck?

• rbranson 16y ago
  Perhaps not a bottleneck, but for very high network throughput systems, offloading the TCP checksumming can make more CPU available for other tasks. Often the offload engines built into network cards are either slow or have very poorly written drivers.
  • almost 16y ago
    I forgot about TCPs use of CRC32. Still, I wonder if the time spent in CRC32 is a noticeable fraction of the total time. I guess it must be in some situations, otherwise why would they put it in the chip...
    • deleted 16y ago
      [deleted]
    • dfox 16y ago
      That might be caused by the fact, that TCP does not use CRC, but it's own checksum algorithm (faster, less relaible).
      • vilda 16y ago
        ...and very often is calculated by network card itself and left blank by OS. (See checksum errors in wireshark.)
  • moxiemk1 16y ago
    I feel like hardware TCP/IP stacks would probably massively improve performance by removing all details of the communications from the OS and processor. In saber OSes, this would just mean that one of the higher levels of abstraction would essentially deal directly with the hardware instead of a few more layers of abstraction. Does this make sense? These communications seem so central to modern computing that they deserve specialized hardware support.
    • pmjordan 16y ago
      Many network cards offload some of the low-level protocols, typically ethernet and IP checksums, but some do deeper inspection and prepare packets conveniently for the driver if the OS supports it.
• mfukar 16y ago
  Although I can imagine a significant number of CRC32 operations in high-capacity gateways (LTE gateways, IP core routers and so forth), Intel processors are not so prevalent in those areas (edit: anecdotal, of course, it's just my perception of the area. I'm a bit bored to seek concrete numbers on this. MIPS-based multicore processors seem to be more commonly used.). Still, there may be a significant benefit from hardware implementations of those operations (checksumming, hashing, Triple-DES/AES for IPsec, etc).
• dadkins 16y ago
  Yes, they all use Adler32 instead because it's faster.
• xpaulbettsx 16y ago
  iSCSI with Data/Header validation is the main motivation for including CRC32