4 ms·

I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it

by wbond 10y ago

I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead. It's great that Rust will decode and encode the JSON quickly (I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project), but the project I work on cares about optimizations on the level of in-memory data layout and memory allocations. Moving syntax highlighting into a pluggable system would be a huge performance regression.

steveklabnik 10y ago
There was a talk at RustConf by Raph Levien about his editor, Xi. It almost entirely uses JSON-RPC to communicate through a distributed architecture, and next-level speed is one of his design goals: https://confreaks.tv/videos/rustconf2016-a-modern-editor-built-in-rust https://confreaks.tv/videos/rustconf2016-a-modern-editor-bui... > I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project The latest builds of MSVC claim to have native Rust syntax highlighting and autocomplete, I have not yet tried it myself: https://www.visualstudio.com/news/releasenotes/vs15-relnotes https://www.visualstudio.com/news/releasenotes/vs15-relnotes
- wbond 10y ago
  I'm referring to the MSVC "backend" for Rust, not Visual Studio.
  - steveklabnik 10y ago
    Ah! What things are you looking for in its maturity? In some ways, it's actually the most mature backend; stack probes work with it, for example. Windows isn't my personal area of expertise though.
    - wbond 10y ago
      I've been following https://github.com/rust-lang/rfcs/issues/1061 https://github.com/rust-lang/rfcs/issues/1061 for a while. It seems that high-level pieces are there, but that certain elements useful in day-to-day software engineering still need some more time.
      - retep998 10y ago
        Yay, someone actually follows that issue. If you have the time, you should really consider helping to improve the Rust experience on Windows. There's a huge shortage of Windows developers in the Rust community.
        steveklabnik 10y ago
        Seconded, and you have done so much work on this front. It'd be wonderful if we had more people to advocate for and help do this work.
        deleted 10y ago
        [deleted]
        wbond 10y ago
        Unfortunately I am already overcommitted with open source projects. For now I am content to wait and see how it all plays out. It will be great to have another language where Windows support is really first-rate.
    - Rusky 10y ago
      Probably the biggest drawback of the MSVC backend is that its debugging symbols only have line numbers, no variable/type information. The rest is mostly Windows-specific issues like manifests and subsystems.
      - retep998 10y ago
        Actually, the PDB debug info has variable and type information and a lot of it actually works. There's just a few minor issues here and there with stuff like fat pointers.
  - IsmaOlvey 10y ago
    MSVC is commonly used to refer to Visual Studio, more specifically the C++ compiler that's part of it (originally "Microsoft Visual C++"), which would explain the parent confusing the two.
deleted 10y ago
[deleted]
mattkrause 10y ago
Perhaps I'm not understanding you, but how does the speed of syntax highlighting (or, more generally, the editor) affect your project's performance? I'm trying and failing to imagine some sort of realtime system controlled by someone coding away. (Also, you wouldn't necessarily need to transmit the whole editor state--just changes).
- wbond 10y ago
  I work on Sublime Text.
  - StavrosK 10y ago
    Haha, I found this exchange pretty funny. "My project depends on my editor being super fast." "What are you writing that has that requirement?" "I'm writing my editor."
comex 10y ago
> I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead. The latency of both Unix IPC calls and serialization/deserialization of small amounts of JSON should be measured in single- to double-digit microseconds. A frame at 60fps is 16,000 microseconds, so if this is causing even a single dropped frame, something is seriously wrong. The lexing itself might or might not be a performance worry, depending on the architecture of the whole system, but that's no different from the lexing built into the editor (except perhaps in that people writing editors tend to care more about latency)... (Well, this seems academic, since according to other comments RLS doesn't aim to put syntax highlighting out of process.)

by wbond 10y ago

steveklabnik 10y ago

There was a talk at RustConf by Raph Levien about his editor, Xi. It almost entirely uses JSON-RPC to communicate through a distributed architecture, and next-level speed is one of his design goals: https://confreaks.tv/videos/rustconf2016-a-modern-editor-built-in-rust https://confreaks.tv/videos/rustconf2016-a-modern-editor-bui... > I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project The latest builds of MSVC claim to have native Rust syntax highlighting and autocomplete, I have not yet tried it myself: https://www.visualstudio.com/news/releasenotes/vs15-relnotes https://www.visualstudio.com/news/releasenotes/vs15-relnotes

wbond 10y ago
I'm referring to the MSVC "backend" for Rust, not Visual Studio.
- steveklabnik 10y ago
  Ah! What things are you looking for in its maturity? In some ways, it's actually the most mature backend; stack probes work with it, for example. Windows isn't my personal area of expertise though.
  - wbond 10y ago
    I've been following https://github.com/rust-lang/rfcs/issues/1061 https://github.com/rust-lang/rfcs/issues/1061 for a while. It seems that high-level pieces are there, but that certain elements useful in day-to-day software engineering still need some more time.
    - retep998 10y ago
      Yay, someone actually follows that issue. If you have the time, you should really consider helping to improve the Rust experience on Windows. There's a huge shortage of Windows developers in the Rust community.
      - steveklabnik 10y ago
        Seconded, and you have done so much work on this front. It'd be wonderful if we had more people to advocate for and help do this work.
      - deleted 10y ago
        [deleted]
      - wbond 10y ago
        Unfortunately I am already overcommitted with open source projects. For now I am content to wait and see how it all plays out. It will be great to have another language where Windows support is really first-rate.
  - Rusky 10y ago
    Probably the biggest drawback of the MSVC backend is that its debugging symbols only have line numbers, no variable/type information. The rest is mostly Windows-specific issues like manifests and subsystems.
    - retep998 10y ago
      Actually, the PDB debug info has variable and type information and a lot of it actually works. There's just a few minor issues here and there with stuff like fat pointers.
- IsmaOlvey 10y ago
  MSVC is commonly used to refer to Visual Studio, more specifically the C++ compiler that's part of it (originally "Microsoft Visual C++"), which would explain the parent confusing the two.

deleted 10y ago

[deleted]

mattkrause 10y ago

Perhaps I'm not understanding you, but how does the speed of syntax highlighting (or, more generally, the editor) affect your project's performance? I'm trying and failing to imagine some sort of realtime system controlled by someone coding away. (Also, you wouldn't necessarily need to transmit the whole editor state--just changes).

wbond 10y ago
I work on Sublime Text.
- StavrosK 10y ago
  Haha, I found this exchange pretty funny. "My project depends on my editor being super fast." "What are you writing that has that requirement?" "I'm writing my editor."

comex 10y ago

> I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead. The latency of both Unix IPC calls and serialization/deserialization of small amounts of JSON should be measured in single- to double-digit microseconds. A frame at 60fps is 16,000 microseconds, so if this is causing even a single dropped frame, something is seriously wrong. The lexing itself might or might not be a performance worry, depending on the architecture of the whole system, but that's no different from the lexing built into the editor (except perhaps in that people writing editors tend to care more about latency)... (Well, this seems academic, since according to other comments RLS doesn't aim to put syntax highlighting out of process.)