10 ms·

An experimental Vim fork with multi-threading capabilities

75 points by tarruda 13y ago

epmatsw 13y ago
Hmmm. So the benefit of this is offloading plugin processing onto a separate thread(s)? I guess I see the theoretical benefit of that, but I'm unsure of how useful this actually is...
- jeremysmyth 13y ago
  An indirect attempt to troll for side-effects and hilarious race conditions perhaps?
- taeric 13y ago
  Ever had to sit and wait for vim to parse a gigantic file to get syntax highlighting correct? Just waiting for it to know the line numbers can take a while.
- cma 13y ago
  One thing I run into a lot is that sometimes tab completion scanning ctags locks up the whole process for several seconds.
- seren 13y ago
  Every time you launch a process from inside vim (like grepping multiple files, generating tags for a large project, etc) it totally freezes the user interface. You can not execute any long task in the background, which is really annoying. So having a multi threaded Vim is very much overdue. There is an ongoing discussion on the vim-dev mailing list regarding a proposed patch by Thiago Arruda [1]. I have no opinion on the technical merit of the patch, but as a user, I would warmly welcome a move in that direction. [1] https://groups.google.com/forum/#!topic/vim_dev/65jjGqS1_VQ https://groups.google.com/forum/#!topic/vim_dev/65jjGqS1_VQ
  - njharman 13y ago
    GUI apps should be multi-threaded. But, process handling is OS task. Unix already provides facility for backgrounding of processes. :!long-process&
    - seren 13y ago
      I think there are some use cases, like getting the syntax highlighting being processed by vim in another thread, that would help the responsiveness (or feeling of) if this was supported natively.
    - tarruda 13y ago
      I agree, but how do you get notified that 'long-process' finished running? Even if you poll using CursorHold-like hacks the user still needs to type something.
      - dllthomas 13y ago
        :!{ long-process ; notify-send "done" ; } &
        tarruda 13y ago
        All that does is display a popup on the desktop, it does nothing to notify/callback to the vimscript code that started the process
        dllthomas 13y ago
        Yes. For many things that's what you want. I wasn't trying to say "this will meet all needs and make this entire discussion irrelevant"; I was posting lest anyone find it interesting/useful.
  - recuter 13y ago
    Wouldn't a simple fix be writing a little stub that acts like the Tee command line utility? It would immediately hand back to Vim and pipe onwards the command you sent it. Could be a temporary kludge at least until OP gets going.
- chongli 13y ago
  You haven't tried some of the heavier auto-complete plugins. As it stands right now, they have an annoying tendency to block the UI while you're typing! It's bothered me so much that I've even considered writing my own vi clone based entirely on an asynchronous message queue such as this!
  - rjzzleep 13y ago
    ever tried youcompleteme? the redraw bothers me, but that's a different issue http://valloric.github.io/YouCompleteMe/ http://valloric.github.io/YouCompleteMe/
    - tarruda 13y ago
      The redraw issue is also related to vim not being thread-safe. With this fork youcompleteme can probably do something like 'vim.defer("YcmTriggerSyntastic")' from a background thread to invoke syntastic and redraw the screen.
mattengi 13y ago
- syntax highlight - decent automatic indenting - semantic auto completion popup - on the fly compile - on the fly syntastic check(lint tool) In vim, those settings are very sluggish in my old AMD 2 core 1Ghz cpu laptop. Sublime Text and Emacs both are quite reasonable. I think vim could be more responsive. This slow response problem is only reason for trying other editor. Please make this real.
- grondilu 13y ago
  I don't use the last four of the features you're mentioning, but I can confirm indeed that syntax highlighting can make things terribly slow with large files and/or complex languages (I'm a Perl 6 guy), so yeah if multi-threading can help, I hope it gets implemented soon.
- _delirium 13y ago
  Wanting the "engine" of emacs but the interface conventions of vim, I've been using emacs with Evil (https://gitorious.org/evil/pages/Home https://gitorious.org/evil/pages/Home) a bit lately, and quite liking the combination. It implements a fairly good portion of vim, enough to cover just about everything I regularly use, which was a pleasant surprise (I'm used to "vim compatibility" layers usually being pretty shallow).
CJefferson 13y ago
The patch for this on vim-dev has been completely ignored for other a month by all vim developers, so don't get any hope this will be accepted.
- tarruda 13y ago
  Perhaps with enough pressure from the community there's a chance this will happen? Meanwhile I will keep maintaining the fork up-to-date with upstream, so plugins can make use of the patch through a simple 'if has("messagequeue")' conditional
- dudus 13y ago
  1 month is not a long of time. Specially when you are waiting for core developers of a busy project. Besides that this month had several holidays. Maybe you just need some faith.
  - btipling 13y ago
    Depending on faith will probably lead to bad decisions, a rational appraisal of available evidence and some critical thinking will probably be more useful. And let me present some evidence that may destroy any remnant of that faith you have: https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%5B1-25-false%5D https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%... That's our async/timers patch, we pretty much gave up for now because debugging vim was a nightmare. Right now something is stomping over regexp objects in memory. We have no idea where. We've found unsafe pointer use, etc. Vim code is pretty much all globals, 25k lines and 400+ ifdefs for at least one file, some crazy file encoding, pre C89 C that still compiles on an amiga, mixed tabs and spaces. The vim code base is pretty much the worst code I've ever seen and Bram doesn't seem to really grok our patch. He's also been demanding with his requests that we feel are over the top. That's fine, that's his choice. We're focusing on other plugins right now, we put in many months of work into vim. It kinda works as long as you don't use leader keys.
    - jessaustin 13y ago
      Thanks for the link to an interesting discussion. While I can't speak to the quality (or otherwise) of the vim code, I wouldn't characterize that conversation as "demanding" or "over the top". Even the most recent patch is reported to cause hangs when doing something as simple as pressing the leader key. You can't rationally expect a maintainer of mature software to sign up for that. I doubt Debian or other downstreams would be happy to hear that the vim maintainer had signed up for that. If the fork is maintained for a year or so, fixing problems with the timer code and feeding patches for actual vim bugs back upstream, and adding tests for all of that, at the end of that time this patch will be quite different. It might even be in a form the vim maintainer can merge in good conscience. As an added benefit, by that time some compelling plugins that take advantage of these timers might be written. As it stands now, the use-case for this patch is rather hypothetical. Indeed, if after a year there aren't a few plugins around that make good use of this functionality, it calls this entire effort into question.
shocks 13y ago
Is this a bi like vimproc[1]? 1: https://github.com/Shougo/vimproc.vim https://github.com/Shougo/vimproc.vim
- amadeus 13y ago
  My understanding is that Vimproc can only kick off async processes, it can't actually call back into the main Vim thread once a process has completed.

75 points by tarruda 13y ago

Hmmm. So the benefit of this is offloading plugin processing onto a separate thread(s)? I guess I see the theoretical benefit of that, but I'm unsure of how useful this actually is...

jeremysmyth 13y ago
An indirect attempt to troll for side-effects and hilarious race conditions perhaps?
taeric 13y ago
Ever had to sit and wait for vim to parse a gigantic file to get syntax highlighting correct? Just waiting for it to know the line numbers can take a while.
cma 13y ago
One thing I run into a lot is that sometimes tab completion scanning ctags locks up the whole process for several seconds.
seren 13y ago
Every time you launch a process from inside vim (like grepping multiple files, generating tags for a large project, etc) it totally freezes the user interface. You can not execute any long task in the background, which is really annoying. So having a multi threaded Vim is very much overdue. There is an ongoing discussion on the vim-dev mailing list regarding a proposed patch by Thiago Arruda [1]. I have no opinion on the technical merit of the patch, but as a user, I would warmly welcome a move in that direction. [1] https://groups.google.com/forum/#!topic/vim_dev/65jjGqS1_VQ https://groups.google.com/forum/#!topic/vim_dev/65jjGqS1_VQ
- njharman 13y ago
  GUI apps should be multi-threaded. But, process handling is OS task. Unix already provides facility for backgrounding of processes. :!long-process&
  - seren 13y ago
    I think there are some use cases, like getting the syntax highlighting being processed by vim in another thread, that would help the responsiveness (or feeling of) if this was supported natively.
  - tarruda 13y ago
    I agree, but how do you get notified that 'long-process' finished running? Even if you poll using CursorHold-like hacks the user still needs to type something.
    - dllthomas 13y ago
      :!{ long-process ; notify-send "done" ; } &
      - tarruda 13y ago
        All that does is display a popup on the desktop, it does nothing to notify/callback to the vimscript code that started the process
        dllthomas 13y ago
        Yes. For many things that's what you want. I wasn't trying to say "this will meet all needs and make this entire discussion irrelevant"; I was posting lest anyone find it interesting/useful.
- recuter 13y ago
  Wouldn't a simple fix be writing a little stub that acts like the Tee command line utility? It would immediately hand back to Vim and pipe onwards the command you sent it. Could be a temporary kludge at least until OP gets going.
chongli 13y ago
You haven't tried some of the heavier auto-complete plugins. As it stands right now, they have an annoying tendency to block the UI while you're typing! It's bothered me so much that I've even considered writing my own vi clone based entirely on an asynchronous message queue such as this!
- rjzzleep 13y ago
  ever tried youcompleteme? the redraw bothers me, but that's a different issue http://valloric.github.io/YouCompleteMe/ http://valloric.github.io/YouCompleteMe/
  - tarruda 13y ago
    The redraw issue is also related to vim not being thread-safe. With this fork youcompleteme can probably do something like 'vim.defer("YcmTriggerSyntastic")' from a background thread to invoke syntastic and redraw the screen.

mattengi 13y ago

- syntax highlight - decent automatic indenting - semantic auto completion popup - on the fly compile - on the fly syntastic check(lint tool) In vim, those settings are very sluggish in my old AMD 2 core 1Ghz cpu laptop. Sublime Text and Emacs both are quite reasonable. I think vim could be more responsive. This slow response problem is only reason for trying other editor. Please make this real.

grondilu 13y ago
I don't use the last four of the features you're mentioning, but I can confirm indeed that syntax highlighting can make things terribly slow with large files and/or complex languages (I'm a Perl 6 guy), so yeah if multi-threading can help, I hope it gets implemented soon.
_delirium 13y ago
Wanting the "engine" of emacs but the interface conventions of vim, I've been using emacs with Evil (https://gitorious.org/evil/pages/Home https://gitorious.org/evil/pages/Home) a bit lately, and quite liking the combination. It implements a fairly good portion of vim, enough to cover just about everything I regularly use, which was a pleasant surprise (I'm used to "vim compatibility" layers usually being pretty shallow).

CJefferson 13y ago

The patch for this on vim-dev has been completely ignored for other a month by all vim developers, so don't get any hope this will be accepted.

tarruda 13y ago
Perhaps with enough pressure from the community there's a chance this will happen? Meanwhile I will keep maintaining the fork up-to-date with upstream, so plugins can make use of the patch through a simple 'if has("messagequeue")' conditional
dudus 13y ago
1 month is not a long of time. Specially when you are waiting for core developers of a busy project. Besides that this month had several holidays. Maybe you just need some faith.
- btipling 13y ago
  Depending on faith will probably lead to bad decisions, a rational appraisal of available evidence and some critical thinking will probably be more useful. And let me present some evidence that may destroy any remnant of that faith you have: https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%5B1-25-false%5D https://groups.google.com/forum/#!topic/vim_dev/-4pqDJfHCsM%... That's our async/timers patch, we pretty much gave up for now because debugging vim was a nightmare. Right now something is stomping over regexp objects in memory. We have no idea where. We've found unsafe pointer use, etc. Vim code is pretty much all globals, 25k lines and 400+ ifdefs for at least one file, some crazy file encoding, pre C89 C that still compiles on an amiga, mixed tabs and spaces. The vim code base is pretty much the worst code I've ever seen and Bram doesn't seem to really grok our patch. He's also been demanding with his requests that we feel are over the top. That's fine, that's his choice. We're focusing on other plugins right now, we put in many months of work into vim. It kinda works as long as you don't use leader keys.
  - jessaustin 13y ago
    Thanks for the link to an interesting discussion. While I can't speak to the quality (or otherwise) of the vim code, I wouldn't characterize that conversation as "demanding" or "over the top". Even the most recent patch is reported to cause hangs when doing something as simple as pressing the leader key. You can't rationally expect a maintainer of mature software to sign up for that. I doubt Debian or other downstreams would be happy to hear that the vim maintainer had signed up for that. If the fork is maintained for a year or so, fixing problems with the timer code and feeding patches for actual vim bugs back upstream, and adding tests for all of that, at the end of that time this patch will be quite different. It might even be in a form the vim maintainer can merge in good conscience. As an added benefit, by that time some compelling plugins that take advantage of these timers might be written. As it stands now, the use-case for this patch is rather hypothetical. Indeed, if after a year there aren't a few plugins around that make good use of this functionality, it calls this entire effort into question.

shocks 13y ago

Is this a bi like vimproc[1]? 1: https://github.com/Shougo/vimproc.vim https://github.com/Shougo/vimproc.vim

amadeus 13y ago
My understanding is that Vimproc can only kick off async processes, it can't actually call back into the main Vim thread once a process has completed.