4 ms·

The problem with Shiki is that it uses TextMate grammars which in turn require the Oniguruma regex engine. Shiki compiles that to WASM I believe, but it makes S

The problem with Shiki is that it uses TextMate grammars which in turn require the Oniguruma regex engine. Shiki compiles that to WASM I believe, but it makes Shiki larger and slower than a system that uses plain JavaScript regexes or a real parser. I'd only use it if it's absolutely necessary that your syntax highlighting exactly matches VS Code. (It's also a shame that VS Code requires Oniguruma)

benatkin 4y ago
Not all WASM files are in the same ballpark. This one is 500K while esbuild is almost 10MB. https://cdn.jsdelivr.net/npm/shiki@0.14.1/dist/ https://cdn.jsdelivr.net/npm/shiki@0.14.1/dist/ https://cdn.jsdelivr.net/npm/esbuild-wasm/ https://cdn.jsdelivr.net/npm/esbuild-wasm/ That's no knock on esbuild. esbuild benefits from having been built with golang and wasn't designed to be run in the browser. (tinygo can run go in much smaller but you give up some of golang's power) WASM is fast becoming a standard thing.
- spankalee 4y ago
  500k to mostly duplicate an existing platform feature? No thanks.
  - tiagod 4y ago
    What feature are you referring to?
    - benatkin 4y ago
      Regular expressions it seems. I disagree that it's a duplicate or a waste. It's a different regex engine with different features.
supz_k 4y ago
Yes, I have also noticed that Shiki is slower. We use it in our blogging platform for code block highlighting and posts with code blocks are noticeably slower to save than posts without them. Are there any alternatives that supports VS Code syntax but faster?
milkbikis 4y ago
Coincidentally, I recently looked into the slow startup time for shiki and it was mostly from parsing JSONC, not WASM: https://github.com/shikijs/shiki/issues/439 https://github.com/shikijs/shiki/issues/439.
- benatkin 4y ago
  Hmm, I don't see why it's parsing it at all. It could just be inside the rust source code. And it might be faster to run, not just faster to load.
shubhamjain 4y ago
> I'd only use it if it's absolutely necessary that your syntax highlighting exactly matches VS Code. (It's also a shame that VS Code requires Oniguruma) This more important than you state. I have many JS syntax highlighters and most of them are garbage. Like, they don't even recognize some obvious keywords like `const`, `async` and `await` for Javascript. I'd rather chose something with better inbuilt grammar. I am not sure what the slowdown is like, but I don't imagine it'd be much. And with SSR, you can even offload it to a server and cache the generated HTML.
mkl 4y ago
In the examples it doesn't seem to match VS Code at all. It's way less colourful and missing useful features like different colours for nested brackets.

benatkin 4y ago

Not all WASM files are in the same ballpark. This one is 500K while esbuild is almost 10MB. https://cdn.jsdelivr.net/npm/shiki@0.14.1/dist/ https://cdn.jsdelivr.net/npm/shiki@0.14.1/dist/ https://cdn.jsdelivr.net/npm/esbuild-wasm/ https://cdn.jsdelivr.net/npm/esbuild-wasm/ That's no knock on esbuild. esbuild benefits from having been built with golang and wasn't designed to be run in the browser. (tinygo can run go in much smaller but you give up some of golang's power) WASM is fast becoming a standard thing.

spankalee 4y ago
500k to mostly duplicate an existing platform feature? No thanks.
- tiagod 4y ago
  What feature are you referring to?
  - benatkin 4y ago
    Regular expressions it seems. I disagree that it's a duplicate or a waste. It's a different regex engine with different features.

supz_k 4y ago

Yes, I have also noticed that Shiki is slower. We use it in our blogging platform for code block highlighting and posts with code blocks are noticeably slower to save than posts without them. Are there any alternatives that supports VS Code syntax but faster?

milkbikis 4y ago

Coincidentally, I recently looked into the slow startup time for shiki and it was mostly from parsing JSONC, not WASM: https://github.com/shikijs/shiki/issues/439 https://github.com/shikijs/shiki/issues/439.

benatkin 4y ago
Hmm, I don't see why it's parsing it at all. It could just be inside the rust source code. And it might be faster to run, not just faster to load.

shubhamjain 4y ago

> I'd only use it if it's absolutely necessary that your syntax highlighting exactly matches VS Code. (It's also a shame that VS Code requires Oniguruma) This more important than you state. I have many JS syntax highlighters and most of them are garbage. Like, they don't even recognize some obvious keywords like `const`, `async` and `await` for Javascript. I'd rather chose something with better inbuilt grammar. I am not sure what the slowdown is like, but I don't imagine it'd be much. And with SSR, you can even offload it to a server and cache the generated HTML.

mkl 4y ago

In the examples it doesn't seem to match VS Code at all. It's way less colourful and missing useful features like different colours for nested brackets.