5 ms·

Interesting how the Rust implementation of glob currently seems to be the slowest out of the linear time implementations. I guess maybe not too much optimisatio

Interesting how the Rust implementation of glob currently seems to be the slowest out of the linear time implementations. I guess maybe not too much optimisation effort was put into it?

kibwen 9y ago
There's no implementation of globbing in Rust's standard library, so we'd need more information regarding which implementation was used for the blog post. I'm guessing one of two possibilities: 1. The glob crate, which is stdlib ejecta from pre-1.0, and appears to be in maintenance mode (it hasn't seen a "real" commit since Jan 2016). 2. The regex crate, which is very actively developed (and in fact inspired by rsc's writings and Go's regex implementation). If it's the former, then indeed it's probably as disinteresting as "nobody has bothered to ever benchmark this crate". But if it's the latter then I bet burntsushi would be very interested!
- taeric 9y ago
  I'd question if it should be considered maintenance mode if it is complete. I understand why monolithic utility and frameworks need constant updating. It seems that in the "micro library" world, though, we should see and encourage "finished" libraries.
  - tomjakubowski 9y ago
    Maintenance mode just suggests that no one is currently trying to make it any faster.
    - kibwen 9y ago
      Indeed, I draw a stark line between "maintenance mode" and "abandoned". The glob crate isn't abandoned, it still accepts patches, but it doesn't appear to be anyone's priority. EDIT: It's also worth mentioning that there's an important mental distinction between a "1.0-level" maintenance mode crate and a pre-1.0 maintenance mode crate, as the former at least implies that somebody considers the crate to be marginally usable and feature-complete. glob is a pre-1.0 crate, so even though patches are accepted, I wouldn't necessarily consider it to be well-vetted without doing more research.
      - taeric 9y ago
        This makes sense, but seems unlikely to be what most people hear when you say something is in maintenance mode. In particular, "keep the lights on" is a death sentence for most projects.
- zokier 9y ago
  On a glance, I don't see any support for globbing in regex crate.
  - kibwen 9y ago
    I was originally sort of hoping that the OP had written some code to implement globbing based on the regex crate, because that sounds like it would be interesting. :) (And because I forgot that the glob crate existed, frankly.)
  - adito 9y ago
    I guess it's the one that's part of ripgrep, https://github.com/BurntSushi/ripgrep/tree/master/globset https://github.com/BurntSushi/ripgrep/tree/master/globset > This crate implements globs by converting them to regular expressions, and executing them with the regex crate.
- rsc 9y ago
  I googled for Rust glob match and ended up at the glob crate. This is my Rust program: extern crate glob; extern crate time; use glob::glob; use time::PreciseTime; fn main() { std::env::set_current_dir("/tmp/glob").expect("cd"); for i in 0..100 { let pattern = "a*".repeat(i)+"b"; let t = PreciseTime::now(); let mul = 100; for j in 0..mul { for entry in glob(&pattern).expect("Failed to read glob pattern") { match entry { Ok(path) => println!("{:?}", path.display()), Err(e) => println!("{:?}", e), } } } let t1 = PreciseTime::now(); println!("{:?} {:?}", i, (t.to(t1).num_nanoseconds().expect("ok") as f64)/1e9/(mul as f64)) } } Honestly I'm not too worried about Rust's performance here: it's on the right side of the fence.
  - kibwen 9y ago
    Thanks for the info! In the course of this conversation I've also been made aware of the globset crate, by the same author as the regex crate: https://crates.io/crates/globset https://crates.io/crates/globset . I'm going to start a conversation about seeing whether anybody might want to start deprecating glob in favor of globset, in the interest of having just one "blessed" library for this.
    - burntsushi 9y ago
      FWIW, the globset crate works by converting globs to regexes (so it is therefore also linear time). The downside is that you need to pay the overhead of regex compilation. In general, it should be pretty small since globs probably don't result in huge regexes, but I bet compilation is slower than what's in the `glob` crate currently. (I've never benchmarked it though, so I'm just guessing.)
      - stouset 9y ago
        Doesn't your regex crate avoid compilation of various types of "simple" regexes, and instead just translates them to simple string library calls? If so, wouldn't most globs be converted to such?

Interesting how the Rust implementation of glob currently seems to be the slowest out of the linear time implementations. I guess maybe not too much optimisation effort was put into it?

kibwen 9y ago

There's no implementation of globbing in Rust's standard library, so we'd need more information regarding which implementation was used for the blog post. I'm guessing one of two possibilities: 1. The glob crate, which is stdlib ejecta from pre-1.0, and appears to be in maintenance mode (it hasn't seen a "real" commit since Jan 2016). 2. The regex crate, which is very actively developed (and in fact inspired by rsc's writings and Go's regex implementation). If it's the former, then indeed it's probably as disinteresting as "nobody has bothered to ever benchmark this crate". But if it's the latter then I bet burntsushi would be very interested!

taeric 9y ago
I'd question if it should be considered maintenance mode if it is complete. I understand why monolithic utility and frameworks need constant updating. It seems that in the "micro library" world, though, we should see and encourage "finished" libraries.
- tomjakubowski 9y ago
  Maintenance mode just suggests that no one is currently trying to make it any faster.
  - kibwen 9y ago
    Indeed, I draw a stark line between "maintenance mode" and "abandoned". The glob crate isn't abandoned, it still accepts patches, but it doesn't appear to be anyone's priority. EDIT: It's also worth mentioning that there's an important mental distinction between a "1.0-level" maintenance mode crate and a pre-1.0 maintenance mode crate, as the former at least implies that somebody considers the crate to be marginally usable and feature-complete. glob is a pre-1.0 crate, so even though patches are accepted, I wouldn't necessarily consider it to be well-vetted without doing more research.
    - taeric 9y ago
      This makes sense, but seems unlikely to be what most people hear when you say something is in maintenance mode. In particular, "keep the lights on" is a death sentence for most projects.
zokier 9y ago
On a glance, I don't see any support for globbing in regex crate.
- kibwen 9y ago
  I was originally sort of hoping that the OP had written some code to implement globbing based on the regex crate, because that sounds like it would be interesting. :) (And because I forgot that the glob crate existed, frankly.)
- adito 9y ago
  I guess it's the one that's part of ripgrep, https://github.com/BurntSushi/ripgrep/tree/master/globset https://github.com/BurntSushi/ripgrep/tree/master/globset > This crate implements globs by converting them to regular expressions, and executing them with the regex crate.
rsc 9y ago
I googled for Rust glob match and ended up at the glob crate. This is my Rust program: extern crate glob; extern crate time; use glob::glob; use time::PreciseTime; fn main() { std::env::set_current_dir("/tmp/glob").expect("cd"); for i in 0..100 { let pattern = "a*".repeat(i)+"b"; let t = PreciseTime::now(); let mul = 100; for j in 0..mul { for entry in glob(&pattern).expect("Failed to read glob pattern") { match entry { Ok(path) => println!("{:?}", path.display()), Err(e) => println!("{:?}", e), } } } let t1 = PreciseTime::now(); println!("{:?} {:?}", i, (t.to(t1).num_nanoseconds().expect("ok") as f64)/1e9/(mul as f64)) } } Honestly I'm not too worried about Rust's performance here: it's on the right side of the fence.
- kibwen 9y ago
  Thanks for the info! In the course of this conversation I've also been made aware of the globset crate, by the same author as the regex crate: https://crates.io/crates/globset https://crates.io/crates/globset . I'm going to start a conversation about seeing whether anybody might want to start deprecating glob in favor of globset, in the interest of having just one "blessed" library for this.
  - burntsushi 9y ago
    FWIW, the globset crate works by converting globs to regexes (so it is therefore also linear time). The downside is that you need to pay the overhead of regex compilation. In general, it should be pretty small since globs probably don't result in huge regexes, but I bet compilation is slower than what's in the `glob` crate currently. (I've never benchmarked it though, so I'm just guessing.)
    - stouset 9y ago
      Doesn't your regex crate avoid compilation of various types of "simple" regexes, and instead just translates them to simple string library calls? If so, wouldn't most globs be converted to such?