5 ms·
Yeah, upvoted. 30x slower on 48 core (?) system sounds suspiciously like excessive lock contention (or some other shared resource). Non-NUMA aware allocator (or
by vardump 7y ago
Yeah, upvoted. 30x slower on 48 core (?) system sounds suspiciously like excessive lock contention (or some other shared resource). Non-NUMA aware allocator (or other code) might also contribute to the issue.
Should be fairly easy to investigate.
- Animats 7y agoHe has so many layers in there it's going to be tough to find the problem. "Ballista is an experimental distributed compute platform, powered by Apache Arrow, with support for Rust and JVM (Java, Kotlin, and Scala)." Plus he's got Docker, the Rust library, musl, and jemalloc sometimes. There's no application. All this is just infrastructure. Musl doesn't do much on its own. But it does do stdio buffering. Could it be that the buffering system is making too many I/O calls, like flushing on every write?
- vardump 7y agoExcessive I/O is indeed another common issue. The number of times I've seen a slow system and discovered excessive number of flushes, often in something like logging system to be the root cause... Or gazillion of unbuffered 1 byte writes or reads.