4 ms·

Reading through the Github proposal for Go plugin integration, I'm pretty confused about Neovim's plugin architecture. I had thought that Neovim's plugins would

by yepguy 12y ago

Reading through the Github proposal for Go plugin integration, I'm pretty confused about Neovim's plugin architecture. I had thought that Neovim's plugins would be language agnostic, with external processes communicating with Neovim using msgpack-rpc (presumably paired with some method of telling Neovim to start those processes). However, it looks like the proposal for Go integration is very much Go-specific, following the example set by Python-specific integration code. This makes me significantly less excited about Neovim's plugin architecture, since it now seems to me like every language may need support to be manually added to Neovim anyway. What's going on here?

banachtarski 12y ago
Is that not how it works? https://github.com/neovim/neovim/wiki/Plugin-UI-architecture https://github.com/neovim/neovim/wiki/Plugin-UI-architecture
fmoralesc 12y ago
No, the architecture is still pretty much language agnostic. My `providers` branch[1] generalizes the python integration, for example. I discussed the possibility of plugins building on this to provide the equivalent functionality to the current python implementation. In regards to the Go integration, it's been decided it would be better to have a host like in the case of the python plugin, but the main rationale for that is not to create too many co-processes. [1]: https://github.com/fmoralesc/neovim/tree/providers https://github.com/fmoralesc/neovim/tree/providers
- yepguy 12y ago
  Okay, it sounds like my fears are unfounded, but I still don't really understand it. What is a host in this context? I know that the plugin API is still in flux, but I really wish it was documented anyway. Actually, I'm not even sure how anyone was even able to start coding something like this without a design doc existing first.
  - fmoralesc 12y ago
    The current python integration works by having a python interpreter interact with neovim, instead of the python interpreter being in-process. [EDIT: All python code runs in this interpreter.] This is what I call a "host". It provides functions that implement this API: :python // executes python code, eg: :python print 1+1 :pyfile // executes a file, eg: :python pyfile test.py :pydo // applies a python expression on a range, eg: 1,2pydo return int(line)+1 pyeval(...) // returns the value of ..., eg: :echo pyeval("1+1") A ruby-host would have similar functionality, for example, as would a scheme-host, etc. You name it. My proposal in the tracker was to allow such host-plugins to define the equivalent ex commands and viml functions instead of hardcoding them in neovim itself (like it currently does for the python integration commands and functions). This is currently difficult because of some limitations in vim's structure. A go host, however, would possibly need to work differently. The idea for now is to have a process that checks for go files in a folder, and bundles them into a big executable that registers external functions in vim like a dynamic host would. Nothing in the above means a plugin couldn't simply talk to neovim by itself, without using the host architecture. All a plugin needs is to talk to neovim thorough the msgpack-rpc api. > I know that the plugin API is still in flux, but I really wish it was documented anyway. Actually, I'm not even sure how anyone was even able to start coding something like this without a design doc existing first. Agreed. That's the goal of that discussion (I believe), to come up with a good design.
    - advocaat23 12y ago
      Another approach would be to choose a client server architecture but this is probably faster done in a separate project. But the idea would be that we have something like an editor core that exposes a SMALL API (e.g. "change cursor position to x/y", "insert 'foobar'", etc.) in a well-defined protocol. On one hand you can then implement the famous VIM UI on top of that (i.e. in the tty sense, a curses program that communicates with the server, sending stuff like "change word") and on the other hand it is really easy to make it scriptable in any language that supports sockets. Obviously the networking/parsing stuff would be hidden by a neovim <LANG$> module. Obviously this would be a multi-process architecture but the number of processes would be really small and peanuts in a modern system. But anyway, thanks for your work! /update: just read https://github.com/neovim/neovim/wiki/Plugin-UI-architecture https://github.com/neovim/neovim/wiki/Plugin-UI-architecture so it IS the chosen architecture - never mind then!
sciurus 12y ago
Even though the plugins communicate over msgpack-rpc, there needs to be integration to support the best way of finding and executing them. There's a lot of debate in the discussion between a process-per-plugin approach like you assumed (and like the proposed go plugin interface implements) and a plugin-host approach where a single process runs all of a given language's plugins (like the python plugin interface implements).

by yepguy 12y ago

banachtarski 12y ago

Is that not how it works? https://github.com/neovim/neovim/wiki/Plugin-UI-architecture https://github.com/neovim/neovim/wiki/Plugin-UI-architecture

fmoralesc 12y ago

No, the architecture is still pretty much language agnostic. My `providers` branch[1] generalizes the python integration, for example. I discussed the possibility of plugins building on this to provide the equivalent functionality to the current python implementation. In regards to the Go integration, it's been decided it would be better to have a host like in the case of the python plugin, but the main rationale for that is not to create too many co-processes. [1]: https://github.com/fmoralesc/neovim/tree/providers https://github.com/fmoralesc/neovim/tree/providers

yepguy 12y ago
Okay, it sounds like my fears are unfounded, but I still don't really understand it. What is a host in this context? I know that the plugin API is still in flux, but I really wish it was documented anyway. Actually, I'm not even sure how anyone was even able to start coding something like this without a design doc existing first.
- fmoralesc 12y ago
  The current python integration works by having a python interpreter interact with neovim, instead of the python interpreter being in-process. [EDIT: All python code runs in this interpreter.] This is what I call a "host". It provides functions that implement this API: :python // executes python code, eg: :python print 1+1 :pyfile // executes a file, eg: :python pyfile test.py :pydo // applies a python expression on a range, eg: 1,2pydo return int(line)+1 pyeval(...) // returns the value of ..., eg: :echo pyeval("1+1") A ruby-host would have similar functionality, for example, as would a scheme-host, etc. You name it. My proposal in the tracker was to allow such host-plugins to define the equivalent ex commands and viml functions instead of hardcoding them in neovim itself (like it currently does for the python integration commands and functions). This is currently difficult because of some limitations in vim's structure. A go host, however, would possibly need to work differently. The idea for now is to have a process that checks for go files in a folder, and bundles them into a big executable that registers external functions in vim like a dynamic host would. Nothing in the above means a plugin couldn't simply talk to neovim by itself, without using the host architecture. All a plugin needs is to talk to neovim thorough the msgpack-rpc api. > I know that the plugin API is still in flux, but I really wish it was documented anyway. Actually, I'm not even sure how anyone was even able to start coding something like this without a design doc existing first. Agreed. That's the goal of that discussion (I believe), to come up with a good design.
  - advocaat23 12y ago
    Another approach would be to choose a client server architecture but this is probably faster done in a separate project. But the idea would be that we have something like an editor core that exposes a SMALL API (e.g. "change cursor position to x/y", "insert 'foobar'", etc.) in a well-defined protocol. On one hand you can then implement the famous VIM UI on top of that (i.e. in the tty sense, a curses program that communicates with the server, sending stuff like "change word") and on the other hand it is really easy to make it scriptable in any language that supports sockets. Obviously the networking/parsing stuff would be hidden by a neovim <LANG$> module. Obviously this would be a multi-process architecture but the number of processes would be really small and peanuts in a modern system. But anyway, thanks for your work! /update: just read https://github.com/neovim/neovim/wiki/Plugin-UI-architecture https://github.com/neovim/neovim/wiki/Plugin-UI-architecture so it IS the chosen architecture - never mind then!

sciurus 12y ago

Even though the plugins communicate over msgpack-rpc, there needs to be integration to support the best way of finding and executing them. There's a lot of debate in the discussion between a process-per-plugin approach like you assumed (and like the proposed go plugin interface implements) and a plugin-host approach where a single process runs all of a given language's plugins (like the python plugin interface implements).