4 ms·

Interestingly, my numbers are completely different relative to each other when compared with yours... I wonder how the number of runs is determined and if this

Interestingly, my numbers are completely different relative to each other when compared with yours... I wonder how the number of runs is determined and if this affects the results. Benchmarking, the higher ops/sec the better. Firefox 63.0 on Fedora 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,707 ops/sec ±1.93% (60 runs sampled) - Regex-based x 4,078 ops/sec ±0.88% (25 runs sampled) - Trie-based (1st-gen) x 10,038 ops/sec ±1.41% (64 runs sampled) - Trie-based JS (2nd-gen) x 7,258 ops/sec ±1.03% (40 runs sampled) - Trie-based WASM (2nd-gen) x 8,033 ops/sec ±1.08% (44 runs sampled) Done.

DiabloD3 8y ago
Benchmarking, the higher ops/sec the better. Firefox 65.0 on Windows 10 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,647 ops/sec ±2.23% (13 runs sampled) - Regex-based x 1,178 ops/sec ±4.95% (48 runs sampled) - Set-based x 1,854 ops/sec ±1.94% (59 runs sampled) - Trie-based (1st-gen) x 275 ops/sec ±4.61% (56 runs sampled) - Regex-based x 2,947 ops/sec ±2.32% (61 runs sampled) - Trie-based (2nd-gen) x 671 ops/sec ±2.60% (47 runs sampled) Done. - Trie-based (1st-gen) x 9,311 ops/sec ±1.61% (51 runs sampled) - Trie-based JS (2nd-gen) x 7,121 ops/sec ±1.28% (40 runs sampled) - Trie-based WASM (2nd-gen) x 7,983 ops/sec ±2.44% (63 runs sampled) Done. And mine seems to look different than yours, but similarly lopsided.
Nullabillity 8y ago
That's what I got too: Benchmarking, the higher ops/sec the better. Firefox 63.0 on Linux 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,992 ops/sec ±0.71% (15 runs sampled) - Regex-based x 5,148 ops/sec ±0.18% (30 runs sampled) - Trie-based (1st-gen) x 11,797 ops/sec ±0.49% (63 runs sampled) - Trie-based JS (2nd-gen) x 8,471 ops/sec ±0.50% (47 runs sampled) - Trie-based WASM (2nd-gen) x 9,543 ops/sec ±0.48% (52 runs sampled) Done. I don't see why trie matching performance would depend on the host OS... Unless WASM runs out of process and requires some form of IPC, I guess?
- wcdolphin 8y ago
  Interesting that an iPhone XS is the fastest of all the benchmarks above. Benchmarking, the higher ops/sec the better. Safari 12.0 on Apple iPhone (iOS 12.1). Test 100 needles against 16 dictionaries of hostnames - Set-based x 3,763 ops/sec ±0.19% (68 runs sampled) - Regex-based x 392 ops/sec ±0.16% (66 runs sampled) - Trie-based (1st-gen) x 20,669 ops/sec ±0.15% (69 runs sampled) - Trie-based JS (2nd-gen) x 16,926 ops/sec ±0.71% (69 runs sampled) Done.
  - X-Istence 8y ago
    Safari is also a lot faster than Firefox/Chrome on the Trie-based versions (non-wasm)
jannes 8y ago
I got these results on an i7 8700K (not overclocked): Benchmarking, the higher ops/sec the better. Firefox 63.0 on Windows 10 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,917 ops/sec ±1.21% (14 runs sampled) - Trie-based (2nd-gen) x 829 ops/sec ±0.64% (66 runs sampled) Done. - Regex-based x 4,475 ops/sec ±0.37% (27 runs sampled) - Trie-based (1st-gen) x 8,624 ops/sec ±0.29% (48 runs sampled) - Trie-based JS (2nd-gen) x 8,848 ops/sec ±0.43% (49 runs sampled) - Trie-based WASM (2nd-gen) x 10,441 ops/sec ±0.30% (57 runs sampled) Done. Interestingly, my CPU only boosts to 4.3 GHz during this benchmark and CPU utilisation of the Firefox process only goes up to 8%.
gorhill 8y ago
For some reasons, I found that Firefox's results for WASM are always better after a first run -- i.e. clicking "Lookup" again without reloading the page. I don't know what is the explanation of this.
- vanderZwan 8y ago
  Firefox can cache the compiled WASM. So perhaps your benchmarks include compilation warm-up on the first run.

DiabloD3 8y ago

Benchmarking, the higher ops/sec the better. Firefox 65.0 on Windows 10 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,647 ops/sec ±2.23% (13 runs sampled) - Regex-based x 1,178 ops/sec ±4.95% (48 runs sampled) - Set-based x 1,854 ops/sec ±1.94% (59 runs sampled) - Trie-based (1st-gen) x 275 ops/sec ±4.61% (56 runs sampled) - Regex-based x 2,947 ops/sec ±2.32% (61 runs sampled) - Trie-based (2nd-gen) x 671 ops/sec ±2.60% (47 runs sampled) Done. - Trie-based (1st-gen) x 9,311 ops/sec ±1.61% (51 runs sampled) - Trie-based JS (2nd-gen) x 7,121 ops/sec ±1.28% (40 runs sampled) - Trie-based WASM (2nd-gen) x 7,983 ops/sec ±2.44% (63 runs sampled) Done. And mine seems to look different than yours, but similarly lopsided.

Nullabillity 8y ago

That's what I got too: Benchmarking, the higher ops/sec the better. Firefox 63.0 on Linux 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,992 ops/sec ±0.71% (15 runs sampled) - Regex-based x 5,148 ops/sec ±0.18% (30 runs sampled) - Trie-based (1st-gen) x 11,797 ops/sec ±0.49% (63 runs sampled) - Trie-based JS (2nd-gen) x 8,471 ops/sec ±0.50% (47 runs sampled) - Trie-based WASM (2nd-gen) x 9,543 ops/sec ±0.48% (52 runs sampled) Done. I don't see why trie matching performance would depend on the host OS... Unless WASM runs out of process and requires some form of IPC, I guess?

wcdolphin 8y ago
Interesting that an iPhone XS is the fastest of all the benchmarks above. Benchmarking, the higher ops/sec the better. Safari 12.0 on Apple iPhone (iOS 12.1). Test 100 needles against 16 dictionaries of hostnames - Set-based x 3,763 ops/sec ±0.19% (68 runs sampled) - Regex-based x 392 ops/sec ±0.16% (66 runs sampled) - Trie-based (1st-gen) x 20,669 ops/sec ±0.15% (69 runs sampled) - Trie-based JS (2nd-gen) x 16,926 ops/sec ±0.71% (69 runs sampled) Done.
- X-Istence 8y ago
  Safari is also a lot faster than Firefox/Chrome on the Trie-based versions (non-wasm)

jannes 8y ago

I got these results on an i7 8700K (not overclocked): Benchmarking, the higher ops/sec the better. Firefox 63.0 on Windows 10 64-bit. Test 100 needles against 16 dictionaries of hostnames - Set-based x 1,917 ops/sec ±1.21% (14 runs sampled) - Trie-based (2nd-gen) x 829 ops/sec ±0.64% (66 runs sampled) Done. - Regex-based x 4,475 ops/sec ±0.37% (27 runs sampled) - Trie-based (1st-gen) x 8,624 ops/sec ±0.29% (48 runs sampled) - Trie-based JS (2nd-gen) x 8,848 ops/sec ±0.43% (49 runs sampled) - Trie-based WASM (2nd-gen) x 10,441 ops/sec ±0.30% (57 runs sampled) Done. Interestingly, my CPU only boosts to 4.3 GHz during this benchmark and CPU utilisation of the Firefox process only goes up to 8%.

gorhill 8y ago

For some reasons, I found that Firefox's results for WASM are always better after a first run -- i.e. clicking "Lookup" again without reloading the page. I don't know what is the explanation of this.

vanderZwan 8y ago
Firefox can cache the compiled WASM. So perhaps your benchmarks include compilation warm-up on the first run.