4 ms·
Yeah per packet processing at 40Gbps and higher is problematic on regular kernels, OS stack and CPUs. A lot of really can be cache hits -- hundreds of nanosecon
by rdtsc 11y ago
Yeah per packet processing at 40Gbps and higher is problematic on regular kernels, OS stack and CPUs. A lot of really can be cache hits -- hundreds of nanoseconds. Article mentions that too:
---
To put these numbers in context, acquiring a single uncontested lock on today's systems takes approximately 20ns, while a non-blocking cache invalidation can cost up to 100ns, only 25x less than an I/O operation.
---
It also depends if workload is throughput sensitive or latency sensitive. If it is latency, can do things like tie processes and interrupts to cores, isolate those cores, etc. For throughput can processes more than one packet at a time perhaps.
Then there is dpdk and even unikernels.
> CPU has responsibilities beyond simply servicing a device—at the very least, it must process a request and act as either a source or a sink for the data linked to it. In the case of data parallel frameworks such as Hadoop and Spark,7,17 the CPU
That's why you get more CPUs and explicitly isolate them if you can. But now depending on how they share data with other CPUs there will be invalidated cache lines so will pay that way as well.
In general if you run on RHEL / CentOS ( a lot banks, military and enterprise deployments do ), there is this helpful guide as an overview:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Performance_Tuning_Guide/sect-Red_Hat_Enterprise_Linux-Performance_Tuning_Guide-CPU-Configuration_suggestions.html https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...
- sgt101 11y agoI have just bought a machine with one of these drives and a bunch of GPU's with the intent of running tensor-flow on it. Do you think that Fedora will allow for the kind of tuning you describe or should I stick with Centos? I was thinking that I would need Fedora because these new devices need new kernels and drivers and so on, but maybe I am just confused?
- worried_citizen 11y agoFedora and centos are effectively the same OS with different release goals. You should be able to do anything on either unless you need truly cutting edge.
- rdtsc 11y agoI would guess give Fedora a try or whatever GPU drivers support better. CentOS / RHEL as a rule is behind as far as package versions in order to be more stable. But they do bring in new drivers and back-port many fixes and packages. Most of the stuff that applies to CentOS will apply to Fedora as well as a rule.
- sgt101 11y agothank you.
- vvanders 11y agoCache miss latency was the first thing that popped into my mind as well when I saw the title. It seems like they don't make a clear distinction between latency and bandwidth. From the little I know on SSDs(don't claim to be an expert here) the sequential reads are below or on-par with high spindle speed disks. What seems to be a better take-away would be that sequencing of your reads isn't nearly as important as it used to be. Back in games we'd duplicate data across a DVD so that we could do "seekfree" loading where duplicating 5-10MB of data would mean just a single big call to read() and gain massive load time performance.
- creshal 11y ago> From the little I know on SSDs(don't claim to be an expert here) the sequential reads are below or on-par with high spindle speed disks. Not quite, but they're "only" a 2-5 times faster, not orders of magnitude faster, as with random accesses.
- frankchn 11y agoSSDs are still faster than hard disks even with sequential reads. 15,000 rpm enterprise spinning disks read at about 260 MB/s [1] while NVMe SSDs (like those in a 2015 MacBook Pro) reads at >1300 MB/s [2]. [1]: http://www.tomshardware.com/charts/enterprise-hdd-charts/-02-Read-Throughput-Maximum-h2benchw-3.16,3372.html http://www.tomshardware.com/charts/enterprise-hdd-charts/-02... [2]: http://www.computerworld.com/article/2900330/apple-mac/holy-smoke-the-new-macbook-literally-is-twice-as-fast.html http://www.computerworld.com/article/2900330/apple-mac/holy-...
- rdtsc 11y ago> NVMe SSDs (like those in a 2015 MacBook Pro) reads at >1300 MB/s [2]. No joke. Just got an new MBP for work. Before had a spinning disk (well hybrid). Was running some silly benchmarks that I ran before and clocked my disk throughput at about 100MB/s. On MBP got 800MB/s. I thought something broke (was hitting page cache or some trickery like that) or didn't compile things right. But no, tried other tools, looked online and it seemed correct. It really surprised me.