7 ms·

This is a project I'm really excited for. It seems like it has a chance of potentially replacing Sublime Text for me some day, which is something I've been tryi

by bogwog 4y ago

This is a project I'm really excited for. It seems like it has a chance of potentially replacing Sublime Text for me some day, which is something I've been trying to do for a while with zero success. I worry about the plugin system though. It is using WASI for plugins, so people can use any programming language they want. That seems like it could lead to a horribly fragmented ecosystem. It also means plugins are limited to accessing only the system resources exposed by the WASI runtime, which may make sense for security in, say, a web browser, but for an extensible editor it just feels like an unnecessary obstacle. IMO, any application for editing/powerusers with a plugin system should be using Python exclusively. Not only is Python available everywhere (and often preinstalled), but it has the richest ecosystem out there that supports the most varied domains. And in the rare cases where a plugin does need to do some number crunching that's too slow in Python, it can be implemented with CFFI using C or any language that can expose a compatible C interface. Also, the typical systems programming stuff you'd likely need to do in a code editor/IDE plugin are extremely easy to do in Python using nothing more than the standard library. Imagine trying to maintain a set of plugins all using different programming languages and build systems and package managers. That's a maintenance nightmare.

funklute 4y ago
> IMO, any application for editing/powerusers with a plugin system should be using Python exclusively. While I agree that python is often a good (or even the best) choice, I think the neovim team made a solid argument for lua (easier than C, faster than python). Would you have chosen python for neovim, and if so, how would you have avoided excessive startup times? (keeping in mind that a terminal-based editor gets started many times throughout the day, as opposed to a GUI editor which you might only start once at the beginning of the day) EDIT: btw, I wholeheartedly agree about the plugin system. Even with neovim, it's pretty confusing that many plugins are now available in original vimscript versions, or new lua versions. I tend to go with the lua versions, simply because I don't want the mental overhead of understanding two different plugin languages, in cases where I decide to fork and alter the plugin code.
- frou_dh 4y ago
  Python has a huge standard library full of useful general-purpose stuff. The motto "Batteries Included" might sound a bit boring at this point, but it's as true as ever.
  - funklute 4y ago
    I wasn't disputing that though. However, because of the batteries included, it has significantly more overhead than lua.
    - frou_dh 4y ago
      I don't think the breadth of the standard library necessarily affects startup time. You only pay the cost for a module when someone imports it, surely.
      - funklute 4y ago
        Yes, true - I didn't word that very well. I believe lua is faster than python mostly due to the simplicity of the language implementation itself, but as another comment pointed out, it might not actually be that big a difference (and I don't have any hard numbers at hand).
- bogwog 4y ago
  I actually have a mild distaste for Lua after spending about a month and a half using LiteXL (https://lite-xl.com/ https://lite-xl.com/), which is an editor written entirely in Lua. The thing is a spaghetti mess, with a lot of plugins relying in internal implementation details of core features, and some of them even making changes to the core and other plugins. (but to be fair, that's mostly an issue with the design of LiteXL and not Lua) The thing I don't like about Lua is that it doesn't really offer much. It's a programming language with practically zero features. Projects end up doing things, like inventing their own class system just to do OOP, and that becomes a problem for interop between projects. Performance is good, but there are other options with good performance. Although I will admit that coroutines are a killer feature of Lua that I wish more languages had. For the use-case of neovim, idk. I stopped using vim as a main editor many years ago, but I do know what you mean about the concern with startup times. However, I don't think anyone can say Python's startup times are too slow without actual data to back that up. On my machine (with Python 3.10), a simple hello world runs instantly. I'm too lazy to do a proper measurement, but it's nowhere near what I would consider slow. And idk what neovims plugin ecosystem looks like, but if Python's startup times were a real issue, I don't think I would mind having a persistent daemon to reduce the problem, especially if it means better plugins.
haolez 4y ago
There might be another advantage here for WASI: you could write a compiler for, say, VimScript and it would translate to this editor's APIs. Sounds like an interesting project to me.
rstat1 4y ago
Nice thing about using WASM for plugins is that you wanted to (for some unknown reason) write a plugin for this editor in Python you could because it can be compiled to WASM. >>Imagine trying to maintain a set of plugins all using different programming languages and build systems and package managers. That's a maintenance nightmare. I would imagine once you got all this setup the first time, it wouldn't be that much of a "maintenance nightmare". Literally just running scripts.
- bogwog 4y ago
  > I would imagine once you got all this setup the first time, it wouldn't be that much of a "maintenance nightmare". Literally just running scripts. Famous last words
bern4444 4y ago
Python is and has been a go to language for writing vim plugins. We've seen its limitations in the vim community. The same is true for Javascript and Typescript (which I personally love). NeoVim picked Lua and every Lua based plugin I've seen has been extremely impressive in its speed. I don't love Lua - I don't know it so well and find it a little difficult to read and write - but I can't argue with the results. Python is not the right decision nor is JS. Only other language I could see is rust given its speed, safety, and interoperability with C but Rust also has a lot of ceremony which is great but not as conducive to the plugin author community (a lot of additional overhead).
- bogwog 4y ago
  > Python is and has been a go to language for writing vim plugins. We've seen its limitations in the vim community. The same is true for Javascript and Typescript (which I personally love). Can you link to any example plugins that are written in either python or js/ts and are considered slow? I'm not doubting you, I'm just not a big vim user, so I'm not familiar with many plugins. > NeoVim picked Lua and every Lua based plugin I've seen has been extremely impressive in its speed. I'm sure you're impressed by the perceived speed, but I don't believe you have any real (i.e. data-backed) basis for comparison there. If you do, I'd love to see it. But to counter your anecdotal evidence with another, Sublime Text only supports Python for its plugin system, and the performance of every plugin I have ever used has been extremely fast. Besides network stuff, I've never even noticed plugin operations, since they all seem to finish instantly.
  - bern4444 4y ago
    I believe you complete me was a python plugin. CoC for vim is a LSP client written in typescript which was slow to startup and felt a lot slower than neovim’s native LSP
    - bogwog 4y ago
      Slow startups are potentially a language issue (which can be mitigated/eliminated in various ways, especially for a text editor), but I hope you realize that comparing the performance of two completely different implementations doesn't say anything about the language choice. If the neovim native LSP were a direct translation of the typescript implementation, then that would be useful to compare. However, I doubt that's what happened here since doing that would be a waste of effort. If anything, it's more likely neovim's LSP was designed with knowledge of some of CoC's performance issues in mind (since it pre-dates it). As for youcompleteme, I don't have too much experience with it. I used it years ago and don't remember any performance problems, although that's anecdotal. Regardless, a completion plugin like that typically relies on a cache built from e.g. clang's compilation database feature, meaning that (after the cache is built) the plugin just needs to access that data structure in response to queries. The design of the cache and implementation of the interface between the client and the server are what will likely determine the bottlenecks, not the raw ops/sec of the Python interpreter.
Yakei 4y ago
> I worry about the plugin system though. It is using WASI for plugins, so people can use any programming language they want. That seems like it could lead to a horribly fragmented ecosystem. It also means plugins are limited to accessing only the system resources exposed by the WASI runtime, which may make sense for security in, say, a web browser, but for an extensible editor it just feels like an unnecessary obstacle. Contributor here, mainly on the plugin system. The main reason for using wasi is to have a single runtime to run any plugins on any os. As you mentioned, plugin makers can write their plugins on their favourite languages ( although as of right now no library or wrapper has been done for other languages ), and the end user doesn't have to care. As for the sandboxing part, the only real sandboxing done as of right now is allowing the plugin to only access its own folder. One of the main features of this release is the new plugin system, which we call PSP. It's designed as a superset of LSP, and extended to the scope of an editor. As PSP is the mandatory way to interact with Lapce for plugins, you could run any kind of binary instead of WASI and it would still work (in fact, there have been talks about allowing user to run binaries as an alternative for WASI based plugins)

by bogwog 4y ago

funklute 4y ago

> IMO, any application for editing/powerusers with a plugin system should be using Python exclusively. While I agree that python is often a good (or even the best) choice, I think the neovim team made a solid argument for lua (easier than C, faster than python). Would you have chosen python for neovim, and if so, how would you have avoided excessive startup times? (keeping in mind that a terminal-based editor gets started many times throughout the day, as opposed to a GUI editor which you might only start once at the beginning of the day) EDIT: btw, I wholeheartedly agree about the plugin system. Even with neovim, it's pretty confusing that many plugins are now available in original vimscript versions, or new lua versions. I tend to go with the lua versions, simply because I don't want the mental overhead of understanding two different plugin languages, in cases where I decide to fork and alter the plugin code.

frou_dh 4y ago
Python has a huge standard library full of useful general-purpose stuff. The motto "Batteries Included" might sound a bit boring at this point, but it's as true as ever.
- funklute 4y ago
  I wasn't disputing that though. However, because of the batteries included, it has significantly more overhead than lua.
  - frou_dh 4y ago
    I don't think the breadth of the standard library necessarily affects startup time. You only pay the cost for a module when someone imports it, surely.
    - funklute 4y ago
      Yes, true - I didn't word that very well. I believe lua is faster than python mostly due to the simplicity of the language implementation itself, but as another comment pointed out, it might not actually be that big a difference (and I don't have any hard numbers at hand).
bogwog 4y ago
I actually have a mild distaste for Lua after spending about a month and a half using LiteXL (https://lite-xl.com/ https://lite-xl.com/), which is an editor written entirely in Lua. The thing is a spaghetti mess, with a lot of plugins relying in internal implementation details of core features, and some of them even making changes to the core and other plugins. (but to be fair, that's mostly an issue with the design of LiteXL and not Lua) The thing I don't like about Lua is that it doesn't really offer much. It's a programming language with practically zero features. Projects end up doing things, like inventing their own class system just to do OOP, and that becomes a problem for interop between projects. Performance is good, but there are other options with good performance. Although I will admit that coroutines are a killer feature of Lua that I wish more languages had. For the use-case of neovim, idk. I stopped using vim as a main editor many years ago, but I do know what you mean about the concern with startup times. However, I don't think anyone can say Python's startup times are too slow without actual data to back that up. On my machine (with Python 3.10), a simple hello world runs instantly. I'm too lazy to do a proper measurement, but it's nowhere near what I would consider slow. And idk what neovims plugin ecosystem looks like, but if Python's startup times were a real issue, I don't think I would mind having a persistent daemon to reduce the problem, especially if it means better plugins.

haolez 4y ago

There might be another advantage here for WASI: you could write a compiler for, say, VimScript and it would translate to this editor's APIs. Sounds like an interesting project to me.

rstat1 4y ago

Nice thing about using WASM for plugins is that you wanted to (for some unknown reason) write a plugin for this editor in Python you could because it can be compiled to WASM. >>Imagine trying to maintain a set of plugins all using different programming languages and build systems and package managers. That's a maintenance nightmare. I would imagine once you got all this setup the first time, it wouldn't be that much of a "maintenance nightmare". Literally just running scripts.

bogwog 4y ago
> I would imagine once you got all this setup the first time, it wouldn't be that much of a "maintenance nightmare". Literally just running scripts. Famous last words

bern4444 4y ago

Python is and has been a go to language for writing vim plugins. We've seen its limitations in the vim community. The same is true for Javascript and Typescript (which I personally love). NeoVim picked Lua and every Lua based plugin I've seen has been extremely impressive in its speed. I don't love Lua - I don't know it so well and find it a little difficult to read and write - but I can't argue with the results. Python is not the right decision nor is JS. Only other language I could see is rust given its speed, safety, and interoperability with C but Rust also has a lot of ceremony which is great but not as conducive to the plugin author community (a lot of additional overhead).

bogwog 4y ago
> Python is and has been a go to language for writing vim plugins. We've seen its limitations in the vim community. The same is true for Javascript and Typescript (which I personally love). Can you link to any example plugins that are written in either python or js/ts and are considered slow? I'm not doubting you, I'm just not a big vim user, so I'm not familiar with many plugins. > NeoVim picked Lua and every Lua based plugin I've seen has been extremely impressive in its speed. I'm sure you're impressed by the perceived speed, but I don't believe you have any real (i.e. data-backed) basis for comparison there. If you do, I'd love to see it. But to counter your anecdotal evidence with another, Sublime Text only supports Python for its plugin system, and the performance of every plugin I have ever used has been extremely fast. Besides network stuff, I've never even noticed plugin operations, since they all seem to finish instantly.
- bern4444 4y ago
  I believe you complete me was a python plugin. CoC for vim is a LSP client written in typescript which was slow to startup and felt a lot slower than neovim’s native LSP
  - bogwog 4y ago
    Slow startups are potentially a language issue (which can be mitigated/eliminated in various ways, especially for a text editor), but I hope you realize that comparing the performance of two completely different implementations doesn't say anything about the language choice. If the neovim native LSP were a direct translation of the typescript implementation, then that would be useful to compare. However, I doubt that's what happened here since doing that would be a waste of effort. If anything, it's more likely neovim's LSP was designed with knowledge of some of CoC's performance issues in mind (since it pre-dates it). As for youcompleteme, I don't have too much experience with it. I used it years ago and don't remember any performance problems, although that's anecdotal. Regardless, a completion plugin like that typically relies on a cache built from e.g. clang's compilation database feature, meaning that (after the cache is built) the plugin just needs to access that data structure in response to queries. The design of the cache and implementation of the interface between the client and the server are what will likely determine the bottlenecks, not the raw ops/sec of the Python interpreter.

Yakei 4y ago

> I worry about the plugin system though. It is using WASI for plugins, so people can use any programming language they want. That seems like it could lead to a horribly fragmented ecosystem. It also means plugins are limited to accessing only the system resources exposed by the WASI runtime, which may make sense for security in, say, a web browser, but for an extensible editor it just feels like an unnecessary obstacle. Contributor here, mainly on the plugin system. The main reason for using wasi is to have a single runtime to run any plugins on any os. As you mentioned, plugin makers can write their plugins on their favourite languages ( although as of right now no library or wrapper has been done for other languages ), and the end user doesn't have to care. As for the sandboxing part, the only real sandboxing done as of right now is allowing the plugin to only access its own folder. One of the main features of this release is the new plugin system, which we call PSP. It's designed as a superset of LSP, and extended to the scope of an editor. As PSP is the mandatory way to interact with Lapce for plugins, you could run any kind of binary instead of WASI and it would still work (in fact, there have been talks about allowing user to run binaries as an alternative for WASI based plugins)