3 ms·
> and supports a wider range of character encodings Since this is an uncommon feature, I'd like to emphasize this. :-) In particular, ripgrep will automaticall
by burntsushi 9y ago
> and supports a wider range of character encodings
Since this is an uncommon feature, I'd like to emphasize this. :-) In particular, ripgrep will automatically search UTF-16 encoded files via BOM sniffing. That means you can run ripgrep over a directory on Windows and be confident that it will correctly search both UTF-8 and UTF-16 encoded files automatically without having to think about it.
More generally, it supports all encodings found in the Encoding Standard[1]. However, only UTF-16 is automatically detected (where UTF-8 is the presumed default), so you'll need to explicitly specify `-E sjis` (for example) if you want to search Shift_JIS encoded files.
Also, I didn't really need to do much of anything to get this working. This is all thanks to @hsivonen's encoding_rs[2] crate, which is now (I think) in Firefox.
[1] - https://encoding.spec.whatwg.org/#concept-encoding-get https://encoding.spec.whatwg.org/#concept-encoding-get
[2] - https://github.com/hsivonen/encoding_rs https://github.com/hsivonen/encoding_rs