4 ms·

Interesting. From what I understand, this was what BurntSushi's regex macro was intended to accomplish: an efficiently compiled regex that optimized away all un

Interesting. From what I understand, this was what BurntSushi's regex macro was intended to accomplish: an efficiently compiled regex that optimized away all unneeded functionality for regex literals. With CTFE, would this get rid of the need for a macro entirely?

kibwen 12y ago
Sadly I'm not enough of an expert to say for sure. It would certainly depend on the specific form that it took, and there are a multitude of options in this space. Of the current contributors who are willing and able to push the boundaries here, I believe that quasiquotation is the preferred approach (motivated by use cases for compile-time codegen in Servo).
- burntsushi 12y ago
  > I believe that quasiquotation is the preferred approach Oh yes yes yes. Quasiquotation makes writing syntax extensions almost easy. It can't cover everything though. For those cases, dropping down into AstBuilder (or the raw AST) isn't completely terrible, but it's a big step down from quasiquoting.
burntsushi 12y ago
The regex macro has two primary benefits: 1. You cannot compile your program with an invalid regex. 2. The regex runs in a specialized VM corresponding to the specific bytecodes for a regex. This results in an across-the-board performance improvement, mostly because there are some places that can now use stack allocation instead of heap allocation. (But it's still a full NFA simulation, so it isn't going to, say, beat RE2/C++ quite yet.) You can take a look at the syntax extension here: https://github.com/rust-lang/rust/blob/master/src/libregex_macros/lib.rs https://github.com/rust-lang/rust/blob/master/src/libregex_m... Basically, if CTFE allows you to write functions that produce AST transforms, then the `regex!` macro should be convertible to that form. Most of the syntax extension is a quasiquoted copy of the general VM: https://github.com/rust-lang/rust/blob/master/src/libregex/vm.rs https://github.com/rust-lang/rust/blob/master/src/libregex/v... --- The actual syntax extension bits are pretty minimal. (There are quite a few tricks employed to get native and dynamic regexes to have the same API. It's the reason why the main `Regex` type is actually an `enum`! https://github.com/rust-lang/rust/blob/master/src/libregex/re.rs#L105 https://github.com/rust-lang/rust/blob/master/src/libregex/r...) Full disclaimer: the `regex!` macro does have a couple downsides. #1, it's a syntax extension, so it has a dynamic dependency on `libsyntax`. #2, it ends up producing a lot of code and bloating your binary if you use it a lot (you might start noticing at after the first few dozen, and you might start cursing it after the first few hundred).

kibwen 12y ago

Sadly I'm not enough of an expert to say for sure. It would certainly depend on the specific form that it took, and there are a multitude of options in this space. Of the current contributors who are willing and able to push the boundaries here, I believe that quasiquotation is the preferred approach (motivated by use cases for compile-time codegen in Servo).

burntsushi 12y ago
> I believe that quasiquotation is the preferred approach Oh yes yes yes. Quasiquotation makes writing syntax extensions almost easy. It can't cover everything though. For those cases, dropping down into AstBuilder (or the raw AST) isn't completely terrible, but it's a big step down from quasiquoting.

burntsushi 12y ago

The regex macro has two primary benefits: 1. You cannot compile your program with an invalid regex. 2. The regex runs in a specialized VM corresponding to the specific bytecodes for a regex. This results in an across-the-board performance improvement, mostly because there are some places that can now use stack allocation instead of heap allocation. (But it's still a full NFA simulation, so it isn't going to, say, beat RE2/C++ quite yet.) You can take a look at the syntax extension here: https://github.com/rust-lang/rust/blob/master/src/libregex_macros/lib.rs https://github.com/rust-lang/rust/blob/master/src/libregex_m... Basically, if CTFE allows you to write functions that produce AST transforms, then the `regex!` macro should be convertible to that form. Most of the syntax extension is a quasiquoted copy of the general VM: https://github.com/rust-lang/rust/blob/master/src/libregex/vm.rs https://github.com/rust-lang/rust/blob/master/src/libregex/v... --- The actual syntax extension bits are pretty minimal. (There are quite a few tricks employed to get native and dynamic regexes to have the same API. It's the reason why the main `Regex` type is actually an `enum`! https://github.com/rust-lang/rust/blob/master/src/libregex/re.rs#L105 https://github.com/rust-lang/rust/blob/master/src/libregex/r...) Full disclaimer: the `regex!` macro does have a couple downsides. #1, it's a syntax extension, so it has a dynamic dependency on `libsyntax`. #2, it ends up producing a lot of code and bloating your binary if you use it a lot (you might start noticing at after the first few dozen, and you might start cursing it after the first few hundred).