3 ms·

Regarding performance: For my first foray into rust, I recently tried reimplementing a simple C++ tool in rust. It reads pcap data from stdin, finds the vlan ta

Regarding performance: For my first foray into rust, I recently tried reimplementing a simple C++ tool in rust. It reads pcap data from stdin, finds the vlan tag for each packet and writes packets to one of a few different files (or stdout) depending on the vlan id. I know C++ quite well, and I'm confident that the C++ version is reasonably well optimized, though it probably has a little room for improvement. In rust, I think I'm doing things in a reasonable fashion but so far the performance is only half of the C++ version. So, not bad, but I was hoping it would be closer. I'd like to know if anyone has any suggestions for resources related to rust optimization.

steveklabnik 8y ago
You can use your usual tools, like perf, on Rust. What’s the situation with the IO? Are you buffering? Are you holding the lock for a long time or rapidly locking and unlocking? If you can share the code I can take a look.
- usefulcat 8y ago
  I realized that stdout was acquiring and releasing a mutex on every write operation; fixing that improved things. The good news is that when writing to /dev/null, it's now faster than the C++ version, but writing to stdout is still ~25% slower. I suppose there's something else suboptimal about the way I'm using stdout, but I haven't figured it out yet. Code is here: https://gist.github.com/usefulcat/56f334bc58c97edb073b457b683a9f93 https://gist.github.com/usefulcat/56f334bc58c97edb073b457b68... There is actually one other file but it only contains a couple of struct definitions for the libpcap file and packet headers. Thanks for having a look!
  - usefulcat 8y ago
    Update: I wrapped the locked stdout handle in a BufWriter and now the rust version is >25% faster than the C++ version in all cases. I didn't think there was room for that much improvement; I'm really impressed.
    - steveklabnik 8y ago
      Awesome! Glad you got it sorted. Those two things are always footguns...
    - gnuvince 8y ago
      If you have a blog, that would make for an amazing entry, going from 50% slower to 25% faster. Edit: HOLY CRAP! I have a program written in Rust, ppbert[1], and I just tried wrapping my StdoutLock object in a BufWriter, and I improved the performance of my pretty-printing by a factor of 2x! I knew to use BufReader for files, I didn't know it was helpful for stdin and stdout! Thank you _so much_ for sharing your experience, I've certainly benefited! Benchmark #1: ppbert -2 *.bert2 Time (mean ± σ): 3.816 s ± 0.115 s [User: 2.494 s, System: 1.321 s] Range (min … max): 3.688 s … 4.028 s Benchmark #2: ppbert-dev -2 *.bert2 Time (mean ± σ): 1.728 s ± 0.045 s [User: 1.493 s, System: 0.234 s] Range (min … max): 1.678 s … 1.843 s Summary 'ppbert-dev -2 *.bert2' ran 2.21x faster than 'ppbert -2 *.bert2' [1] https://github.com/gnuvince/ppbert https://github.com/gnuvince/ppbert
portmanteaufu 8y ago
Can you share the code? Are you using the `--release` flag to build/run it?
- usefulcat 8y ago
  Yes I'm using --release.

steveklabnik 8y ago

You can use your usual tools, like perf, on Rust. What’s the situation with the IO? Are you buffering? Are you holding the lock for a long time or rapidly locking and unlocking? If you can share the code I can take a look.

usefulcat 8y ago
I realized that stdout was acquiring and releasing a mutex on every write operation; fixing that improved things. The good news is that when writing to /dev/null, it's now faster than the C++ version, but writing to stdout is still ~25% slower. I suppose there's something else suboptimal about the way I'm using stdout, but I haven't figured it out yet. Code is here: https://gist.github.com/usefulcat/56f334bc58c97edb073b457b683a9f93 https://gist.github.com/usefulcat/56f334bc58c97edb073b457b68... There is actually one other file but it only contains a couple of struct definitions for the libpcap file and packet headers. Thanks for having a look!
- usefulcat 8y ago
  Update: I wrapped the locked stdout handle in a BufWriter and now the rust version is >25% faster than the C++ version in all cases. I didn't think there was room for that much improvement; I'm really impressed.
  - steveklabnik 8y ago
    Awesome! Glad you got it sorted. Those two things are always footguns...
  - gnuvince 8y ago
    If you have a blog, that would make for an amazing entry, going from 50% slower to 25% faster. Edit: HOLY CRAP! I have a program written in Rust, ppbert[1], and I just tried wrapping my StdoutLock object in a BufWriter, and I improved the performance of my pretty-printing by a factor of 2x! I knew to use BufReader for files, I didn't know it was helpful for stdin and stdout! Thank you _so much_ for sharing your experience, I've certainly benefited! Benchmark #1: ppbert -2 *.bert2 Time (mean ± σ): 3.816 s ± 0.115 s [User: 2.494 s, System: 1.321 s] Range (min … max): 3.688 s … 4.028 s Benchmark #2: ppbert-dev -2 *.bert2 Time (mean ± σ): 1.728 s ± 0.045 s [User: 1.493 s, System: 0.234 s] Range (min … max): 1.678 s … 1.843 s Summary 'ppbert-dev -2 *.bert2' ran 2.21x faster than 'ppbert -2 *.bert2' [1] https://github.com/gnuvince/ppbert https://github.com/gnuvince/ppbert

portmanteaufu 8y ago

Can you share the code? Are you using the `--release` flag to build/run it?

usefulcat 8y ago
Yes I'm using --release.