16 ms·

Xi editor: A modern editor with a backend written in Rust

274 points by ivank 10y ago

educar 10y ago
Interesting this is a google project. https://github.com/google/xi-editor/blob/master/rust/rope/src/rope.rs https://github.com/google/xi-editor/blob/master/rust/rope/sr... was quite interesting to read (for someone who has never looked into rust)
- retbull 10y ago
  doesn't s/he say at the bottom that this isn't a google project?
  - raphlinus 10y ago
    It's a project by a Googler (me), but it's a personal (20%) project, not an official Google one.
    - namelezz 10y ago
      What's your thought of Rust? What dev env did you use?
      - raphlinus 10y ago
        Rust is working out very well so far. If you look at the rope implementation, you'll see that it's high level and mathematical almost like a functional language, but performance is first-class. I'm also very much looking forward to retrofitting parallelism using libraries such as Rayon and crossbeam. I'll probably write a retrospective when I'm farther into the project. My current development environment is just Sublime Text for the Rust parts, and Xcode for the parts written in Swift.
        wbond 10y ago
        If you are using the recently-added default Rust syntax for Sublime Text and run into any issues, please let me know by filing an issue at https://github.com/sublimehq/Packages/issues https://github.com/sublimehq/Packages/issues.
        rudedogg 10y ago
        What made you decide to not write everything in Swift? Is it the lack of Windows support?
- geofft 10y ago
  It seems Google has been releasing employees' personal projects at github.com/google if Google owns the copyright. (I'm not a Googler, so I don't know if this just applies to stuff you hack on while at work, or anything plausibly in Google's business interests that you do at home while you're an employee.)
- geofft 10y ago
  It seems Google has been releasing employees' personal projects at github.com/google if Google owns the copyright. (I'm not a Googler, so I don't know if this just applies to stuff you hack on while at work, or anything plausibly in Google's business interests that you do at home while employed by Google.)
  - skybrian 10y ago
    If you don't care whether you own the copyright (just that it's open source), this is the easiest way to get permission.
    - cyphar 10y ago
      The issue with ditching the copyright is that you have no guarantee that they won't make a proprietary fork (if you use a copy left license). But if you don't care about the copyright you almost certainly don't care about ensuring the perpetual freedom of users.
      - burntsushi 10y ago
        > But if you don't care about the copyright you almost certainly don't care about ensuring the perpetual freedom of users. That's ridiculous. If I release something into the public domain, then it is forever in the public domain and is perpetually free to use by anyone.
        cyphar 10y ago
        Ignoring the issues with public domain (that it doesn't exist in many places, so the default you-cant-do-anything copyright rules apply), you misunderstand what I mean by freedom in perpetuity. If someone can take your code and make it proprietary (as well as adding some enticing features), your code is now used to take away users freedom -- even though your original code was free software. Copyleft is the only defence against that form of attack on user freedom -- attack by distribution.
fritzy 10y ago
According to the feature-set, this is basically NeoVIM. * Separation into front-end and back-end: NeoVIM does this. The UI uses the same interface a plugin does. * Asynchronous operations: NeoVIM uses libuv to do this (same asynchronous C++ lib as NodeJS). * Plug-ins over scripting: Yup, NeoVIM does this, as well as dropping vimscript (although it will compile vimscript into Lua for you). * JSON: NeoVIM uses msgpack, which is significantly less overhead than JSON, but structure compatible. The two other features listed are Rust and rope-structures, which are implimentation specifics that don't affect users or plugin writers at all. Edit: The more the merrier! And it's good to see more projects using Rust, but I don't see this as a strong need. See https://github.com/rogual/neovim-dot-app https://github.com/rogual/neovim-dot-app for a nice native Mac UI plugin/client for neovim.
- grblovrflowerrr 10y ago
  I think the overlap in features demonstrates popular appeal for such features in our tools. I think the details of how they're implemented are pretty important to users too so there's space for innovation even if the feature set is the same on the surface. Another feature mentioned in the introduction is this: * Developer friendliness. It should be easy to customize xi editor, whether by adding plug-ins or hacking on the core. I'm vaguely familiar with NeoVIM, is the core "hackable"? This is really subjective territory but I'm guessing the author is implying that the xi core code is also optimized for human understandability and extensibility and that new code is judged by those criteria as well as the other stuff.
- raphlinus 10y ago
  NeoVIM is great, and obviously it has been inspiration. I think it remains to be seen whether NeoVIM can deliver an experience that feels like a modern editor. My experience is that projects which attempt to layer a GUI on top of an existing, mature console UI end up leaking bits of the underlying UI. I'm thinking mostly of the various TeX shells here. A large part of my motivation for starting this project is to explore how suitable Rust is for this kind of work. End users shouldn't have to care, they should judge it strictly by the quality of what it delivers. But I'm hoping this project will be interesting to Rust programmers for a number of reasons.
sitkack 10y ago
If folks like Rust and want questions answered, check out the vibrant http://reddit.com/r/rust http://reddit.com/r/rust
hayd 10y ago
Not to be confused with Yi: https://github.com/yi-editor/yi https://github.com/yi-editor/yi
- cm3 10y ago
  Yep. I use Yi occasionally and it works well, but it needs more users to test and report the rough edges. Though, it's excellent for writing support for new languages, given that it's all Haskell.
_RPM 10y ago
I don't believe that it being written in Rust is a feature. The language it's written in is just an implementation detail.
- mblode 10y ago
  I think it definitely would be considered a feature. For example Atom, Brackets, and Visual Studio Code were all written in JS with CSS and HTML and they all for the most part have fairly poor performance. The fact that rust is a highly performant language and is also relatively higher level than C for example, it offers a benefit for the user.
- groovy2shoes 10y ago
  This is really common among open source projects. I frequently come across feature lists containing "Written in X" for many different values of X. In fact, was just complaining about it to a friend of mine just last night. He pointed out that for open source projects, it really can be a feature: when attracting contributors, the source language is important. It's also important for people who just want to study the source for some reason or another. It can also be an indicator for how readily portable it is: a program written in C is going to be much more easily ported than one written in Visual Basic, for example.
  - sbuttgereit 10y ago
    Yes. I'm not so interested in a new editor unless it's very stand out (and I'm sure this project is good)... but I do have an interest in Rust; in particular in how it's being used and the different kinds of software it's showing up in. It may be a good example of programming that's worth reviewing as I learn or it may just help provide a sense in the kind of community that's growing (or not) around something that's relatively new (in some stable status at least) like Rust. Had it not said "Rust" in the title... I may well have not bothered to look at this at all.
- Joof 10y ago
  As another poster said, it's open source and written for fellow hackers, so language is a feature. Imagine if emacs was written purely in C++.
Scarbutt 10y ago
What default key bindings will it follow? CUA?
- raphlinus 10y ago
  I'm basically planning on following Sublime Text's keybindings except when there's good reason not to.
  - evanrelf 10y ago
    I know you said you haven't worked on plugins yet, but do you think a Vim mode would be possible through plugins?
    - raphlinus 10y ago
      I think a Vim mode will be possible one way or other. I'm not sure having all interaction mediated by plugins is the best way to accomplish it.
fernly 10y ago
TIL: about the rope data structure[1]. Never heard of it in decades in the industry. Disagree with the comment "Cross-platform UI toolkits never look and feel quite right." Qt, for one, uses the native toolkit on Mac and Windows and I can't see a difference in look'n'feel. [1] https://en.wikipedia.org/wiki/Rope_%28data_structure%29 https://en.wikipedia.org/wiki/Rope_%28data_structure%29
- jdub 10y ago
  You can't see it, but boy can you feel it.
  - mwcampbell 10y ago
    And you can literally feel it, or hear it, if you're blind. Non-native GUIs are very often less than fully usable, or even downright inaccessible for less common toolkits, with a screen reader. The one exception I can think of is XUL-based apps (e.g. Firefox and Thunderbird) on Windows.
    - lmm 10y ago
      Qt specifically has very good accessibility support, no?
      - mwcampbell 10y ago
        Qt's accessibility support is only moderately useful in practice. As one anecdote, TeamTalk (http://bearware.dk/ http://bearware.dk/) has a "classic" MFC-based front-end just for blind Windows users.
- pcwalton 10y ago
  > TIL: about the rope data structure[1]. Never heard of it in decades in the industry. And if you've used JavaScript, you may have been using them without realizing it—JS engines will choose rope representations for strings if they observe that the usage patterns would benefit from them.
- 10y ago
anderspitman 10y ago
Awesome. I've thought it would be fun to work on a simple editor based on Rust. Something similar to Vim but without ViML. I really like the idea of using pipes rather than committing to a single plugin language. I'll be very interested to see how that works out in practice.
mwcampbell 10y ago
Interesting architecture, having the core communicate with the GUI front-end over pipes via JSON. That certainly eliminates a whole class of pain around inter-language glue. Any idea what language and APIs the Windows version will use? All of the Microsoft-sanctioned choices for desktop apps (as opposed to the wanna-be mobile/tablet platform introduced with Windows 8) are kind of sucky.
- raphlinus 10y ago
  I've been playing a bit with the various choices (there are so many!) and am leaning in the direction of C# + WPF + Windows.System.Media for the text drawing. Using something based on DirectWrite rather than GDI+ is important to me because of text rendering quality. I have basically ruled out Windows Universal because it feels a lot clunkier. But I'm very much open to ideas from more experienced Windows devs who understand the tradeoffs better.
  - mwcampbell 10y ago
    WPF left a bad taste in my mouth because, as far as I can tell, it hastened the obsolescence of my brother's laptop around 2011. [1] But then, maybe WPF 4 under Windows 7 would have been better on the same hardware. Impossible to say now. The best solution would probably be one that uses both DirectWrite and Direct2D. WPF 4 uses DirectWrite, but AFAIK, it's still on top of the older Direct3D-based milcore graphics system. But to use both DWrite and D2D, I guess you'd have to get down and dirty with C++, or at least use a lot of P/Invoke and COM interop in your C# code. To be honest, I don't understand why GDI isn't good enough. Can you explain what you find lacking in GDI's text rendering, or point me at an existing explanation? [1]: https://news.ycombinator.com/item?id=8444494 https://news.ycombinator.com/item?id=8444494
    - raphlinus 10y ago
      The main thing I'd add to http://www.basschouten.com/blog1.php/font-rendering-gdi-versus-directwrite http://www.basschouten.com/blog1.php/font-rendering-gdi-vers... is that DirectWrite looks very smooth on high-dpi displays while GDI+ is quite grainy.
ouchunyu 10y ago
afd
dj-wonk 10y ago
What's the plan for plugins? Will they need to be compiled?
- raphlinus 10y ago
  I haven't started implementing plugins yet, so details are still vague. General thoughts, they communicate with the core through a simple JSON-based protocol, so can be written in any language you like. I imagine Python will be one of the popular choices. Among other things, they'll be able to annotate the text with rich-text spans, for things like syntax coloring. Also, I'm planning for them to be async, so they won't slow typing even when they're busy computing.
  - kristiandupont 10y ago
    My personal dream would be to have an editor that acts mostly like VIM, but with folding capability like Code Browser (http://tibleiz.net/code-browser/ http://tibleiz.net/code-browser/) -- would that be possible to achieve with this?
  - martanne 10y ago
    How do you plan to encode binary data in your JSON-based protocol? Base64? This will only increase the overhead. While the human readability of JSON is nice, it has some serious flaws when the goal is to handle arbitrary data. I have yet to decide what kind of protocol to use in my own vis editor[1], hence why I'm asking. [1] https://github.com/martanne/vis https://github.com/martanne/vis
lesbaker 10y ago
I like the explicitly stated emphasis on performance and large file support in the goals. I look forward to seeing this develop and will be following the project (and test-driving it when I have time).
unimpressive 10y ago
So what exactly does 'native' mean for Linux distros?
- raphlinus 10y ago
  I'm leaning towards Gtk+ because I'm more familiar with it (I used to be very involved in that community, in fact) and because I know its Unicode support is good.
  - dman 10y ago
    Please consider Qt as well, the implementation is solid and its easy to make apps that are performant and polished.
    - raphlinus 10y ago
      Qt 4 or 5? I had difficulty getting even "hello world" running in 5, and the size of deploy binaries (~16M) was scary. Maybe that doesn't matter so much if it'll primarily be using system libraries.
      - oblio 10y ago
        At this point? 5, definitely :)
      - dman 10y ago
        I use Qt4 and QWidgets - its by far the most pleasant desktop UI development toolkit Ive used. (Ive used qt, gtk, wx, win32 with gdi/gdi+ and a bit of swing).
      - lmm 10y ago
        What went wrong with 5? And yeah use dynamic linking, that's what it's for.
        IshKebab 10y ago
thedz 10y ago
Interesting note: The author (Raph Levien) is also the creator of my defacto programming font Inconsolata. Super cool.
eggy 10y ago
Would this easily port to Windows or Linux? I put my iMac 2011 in storage, but I would like to try this out to learn Rust and building a text editor in general. I see 'other targets are planned'. Any time soon?
- raphlinus 10y ago
  I'm hoping soon but make no promises.
  - eggy 10y ago
    No rush here. I would like to be able to play with the code and Rust. Thanks!
ArmandGrillet 10y ago
Eager to see in which direction will head the project! Raph Levien seems an extremely talented engineer but, as it is a "20% time" project, I am not sure what is the expecting outcome of this editor. When the development of Atom started it was clear that it would replace Sublime Text in the long term, here I don't know if it will replace TextEdit or be the next big thing.
vintermann 10y ago
First thought, yet another editor attempt, not even very far come yet, and it's OSX only. Rather nice font in the screenshots though. Then > The main author is Raph Levien. Oh. Now I see why this is on hacker news. Now I'm interested too. If nothing else, this will be gorgeous.
- Spiritus 10y ago
  I've never heard of him, care to enlighten me?
  - Someone 10y ago
    https://en.m.wikipedia.org/wiki/Raph_Levien https://en.m.wikipedia.org/wiki/Raph_Levien
    - loup-vaillant 10y ago
      https://en.wikipedia.org/wiki/Raph_Levien https://en.wikipedia.org/wiki/Raph_Levien ?
      - sp332 10y ago
        That's the same article in a "desktop" view. Is that what you're asking? Personally I prefer the mobile one.
        loup-vaillant 10y ago
        I'm asking "why anything but the default URL?" I'm asking "why isn't this handled by freaking CSS? We have a goddam tag for this!"
- wodenokoto 10y ago
  So what? Is he supposed to be famous?
- jfindley 10y ago
Myrmornis 10y ago
In particular, it should use little more memory than the buffers being edited. I'm pretty sure memory footprint of a text editor is not a relevant concern on modern laptops. Or is the idea that it will work on completely different low-memory environments from what I'm thinking?
- kennywinker 10y ago
  > I'm pretty sure memory footprint of a text editor is not a relevant concern on modern laptops. https://github.com/atom/atom/issues/1827 https://github.com/atom/atom/issues/1827
- vidarh 10y ago
  A single editor is not the only thing I run. The only reason I have 8GB on my current laptop is because too many people think that it is no big deal if just their app spends more.
- gnuvince 10y ago
  I can't remember where I heard this, but someone said once that better performance opens up more, and sometimes new, ways to work with a program. I use GNU Emacs myself, and I basically start it when I boot my laptop and never shut it down, which means that sometimes it has an uptime of months. I will also often not close buffers that I'm working on and can end up with hundreds of buffers in Emacs. Yet, the performance is entirely unaffected (or at least, the impact is below what this human can notice). If Xi is serious about keeping memory usage low, these kinds of patterns of usage that I and many other Emacs users have adopted are going to be viable; on the other hand, in an editor like Atom, one should probably keep as few buffers opened as possible.
- panic 10y ago
  For software with variable-sized inputs, performance never stops being relevant. Higher performance increases the maximum size of inputs you can handle. For a text editor, the lower the footprint, the larger the files it can edit and the more of them can be edited at once.
kingosticks 10y ago
I've no exposure to Rust but is it accurate to compare it to C++ as offering a much more reliable, and in many ways, higher level programming platform. Are there some quick and simple examples of this?
- Cyph0n 10y ago
  The main advantage is Rust's ownership model. It allows you to catch most memory problems at compile time. https://doc.rust-lang.org/book/ownership.html https://doc.rust-lang.org/book/ownership.html
  - akadien 10y ago
    And it's one of the languages features that has a very steep learning curve. Rust forces your design into its model. Interestingly, rustc also seems to generate programs that leak memory. I recall an issue where the simple and venerable "hello world" program in Rust (i.e., println!) had memory leaks.
    - steveklabnik 10y ago
      That shouldn't be true. Please file bugs!
      - akadien 10y ago
        It was fixed but isn't fixed. https://github.com/rust-lang/rust/issues/19776 https://github.com/rust-lang/rust/issues/19776
        steveklabnik 10y ago
        Great, thanks.
- kimundi 10y ago
  I don't have concrete examples, but in general: For one, the build system/dependency system is much more easier to handle. Rust uses the cargo package manager, and depending on an external library is as simple as adding a single line to the Cargo.toml config file. Likewise, building, running, documenting and testing is all a build-in matter of calling `cargo [build|run|doc|test]`, which makes prototyping and getting started way easier than my experiences with C++ so far. As far as the actual language goes it basically behaves like C++ and has a similar feature set, so you get the same sense of how given source code will end up in the compiled binary, and can write fast efficient code with, for example, features like pointers. The big distinction there is that Rust is memory safe per default, (including pointers in form of safe references) and upholds this invariant with compiletime checks and explicit language barriers (needing to encapsulate "unsafe" operations in unsafe {} blocks), so per default you are protected from whole classes of bugs that are possible in C++, like dangling pointers, memory races, iterator invalidation, etc.
stephenr 10y ago
I haven't used a "text" editor (as compared to a full IDE) for quite a while. Do any of the decent OS X -compatible editors have things like - find usage of a method/class/constant in a project - change method signature and have calls update - rename a class/method/constant/variable and have usages updated This isn't a troll, I'm honestly asking. I'm using IDEA Ultimate at the moment and it works fine, it just has the usual downsides for a Java app.
- adrusi 10y ago
  Those would typically be considered features for an external tool, maybe with the editor configured to provide an interface to those tools. If you write java and value those features, though IDEA will be a better experience.
  - stephenr 10y ago
    Really? An external tool that isn't a text based editor but does have in depth knowledge of the code? I've never heard of such a thing. Also for reference I don't really write Java. I write php, sql, shell scripts, ldif files, html, occasionally JavaScript and/or css, lua...
    - pdpi 10y ago
      > An external tool that isn't a text based editor but does have in depth knowledge of the code? It's called a "compiler" ;) Jokes aside, here's a few tools that might give you an idea what's out there: - Racer for rust - Go fmt - Youcompleteme - ghc-mod
      - stephenr 10y ago
        Right, so specific tools for Rust, Go and Haskell (none of which I use) that maybe have some refactoring support? (The gofmt example I kept finding was renaming a field in a struct) Oh and a plugin specifically for Vim, so not an external tool at all.. So, basically the answer is the same as before: an IDE is still where it's at.
Keats 10y ago
Looks very neat! Found my Rust reading for the week.
cm3 10y ago
Nice project, but contributing to the project requires assiging copyright of any patch to Google, which is the same flaw that exists with FSF projects and a major blocking issue for many potential contributors. Google should make an exception for personal-but-not-Google-company projects and get rid of it or at least remove the copyright assignment clause. Nobody wants to contribute to a project under license X and then have their contributions be relicensed under Y.
ywecur 10y ago
How do you pronounce the name? X-I or Chi?
- hnbro 10y ago
  or shii or zee or
ninjabeans 10y ago
I like the design decisions. Just a heads up, you misspelled "sparse" and accidentally a word after "completely".
username223 10y ago
The "documentation" is just weird. It has Rust, ropes, and JSON! Okay, but is it modal? Does it have syntax highlighting and debugger support? What kind of editor is it trying to be? The docs say nothing relevant to whether or not I would want to use it.
todd8 10y ago
I've never constructed an editor -- but I have used one before ;) so I thought I'd make a few suggestions about the architectural considerations off the top of my head. I'm an Emacs user, but I'm very excited about NeoVim. It looks like your project is another really interesting one. I hope it takes off. I notice that you've choosen Rust as the programming language for the editor, a fine choice. Performance appears to be a key factor in your choice of language. However, realize that performance isn't the reason one should use Rust or C++ for the backend of an editor. Let's go though some simple back of the envelope calculations. A realistic goal for editors is to keep ordinary command and keystroke latency under 100ms, this should appear to users as if operations are performed instantaniously. You've picked a more aggressive goal of a maximum latency of 16ms, so let's go with that. Today, a slow processor runs perhaps 5,000,000,000 instructions per second (speed of Raspberry Pi 2). By the time the editor is ready for general use, most programmer's development systems will probably run something like 20 times faster or 100 Billion instructions per second. For this rough calculation I'll use the speed of a Raspberry Pi 2. In 16ms that's 8 million instructions. What this means is that in the backend of the editor we have plenty of time to perform the editing operations no matter what language is used as long as it is a straight code path through the editor. The pedestrian operations of the editor, like reading a keystroke or inserting a letter into the text don't require Rust or C++ or C. They can be done with practically any programming language, even a language that runs a hundred times slower than C++ like Python. It is the iterated operations where performance will be a concern: searching big files, complex regular expression searches, syntax highlighting after each keystroke, autocomplete, refactoring, the GUI and interfacing to the GUI. I couldn't agree more with your requirement that the editor shouldn't hang or crash or lose work. Further, a modern editor is a big project and won't gain traction without additional contributors. From a performance perspective my gut feeling is that any language that performs no worse than 10 times slower than C is a fine choice. For safety, either managed memory (with GC etc.) or an approach like Rust is essential to allow code to be contributed. The JVM languages, while being attractive in many ways carry the burden of a big infrastructure; Jetbrains has figured it out but even after all these years Eclipse feels so klunky--this make me rule out Java, Clojure and Scala. I think this leaves at least Rust, Go, OCaml, Common Lisp, Dart, and Swift. (I'd probably pick Go hoping to get more contributors.) What about ropes, a clever and much more modern data structure for holding strings than ancient Emac's gap buffer. Ropes do indead have better asymptotic performace for many of the basic string operations: delete an arbitrary character, insert a substring, etc. What about all of the other operations that you'd like your editor to do? Fast searching, backwards or forwards? Handling UTF-8? Doing regular expression matching. Ropes turn the text being edited into a tree of text segments so high speed searching is much more difficult. An abstraction layer over the data structure to make it appear to be indexed like an ordinary array will make some things run much slower, for example, Boyer-Moore string searches, regular expression searching (done for code highlighting), etc. Ropes or other persistent data structures for sequences will work because they can present an abstract indexed access interface to the rest of the program, but you will always be dealing with it though this abstraction layer just when you'd rather not. Processors can move several bytes per instruction cycle so the gap buffer can close it's gap while editing a 10MB file in less than a millisecond. This leaves the data in one contiguous block. So for simple things the gap buffer is much faster than rope structures because for most simple activity all of the additions and deletions are at the cursor, where the gap is located, and for complex things there is a 1ms overhead before the operation starts (to close the gap) but then things run basically full processor speed without the requirement to go though the abstraction layers. The places where I see this as likely to be important in modern editors is in the design of incremental backups, persistent text attributes, memory mapped access to underlying files, and coordination with asynchronous plugins working on the buffer. I'll be curious to see how this works out for Xi. Keystrokes and plugins. Emacs can end up running lisp code, through a pretty slow lisp interpreter, on every keystroke. It's completely imperceptable. So performance isn't a reason to treat keybindings differently than other plugins. The overhead of IPC to a plugin is measured in micro seconds. While supporting Sublime Text keybindings is fine. The programmers most likely to be sensitive to keybindings will probably want either Vim or Emacs keybindings. A good way to support multiple keybindings will make the editor appeal to a much broader set of users. Plugins seem ideal for this. A key comes in, is processed by a plugin and a few elemental backend operations are invoked by the plugin in response to the key. This seems completely reasonable. The real issue with plugins is direct access to the edit buffer if the plugins run in a separate process and address space. This is complex especially if the buffer is being accessed asynchronously. How is syntax highlighting done in a plugin for example. Where does the syntax highlighting actually get stored? Does the plugin have to be informed each time the user scrolls the screen or types a key (probably yes). What asynchronous commands does the plugin send back to the editor to update the display. Is there a separate plugin working on autocomplete? Does it also have to parse the source code just like the syntax highlighting plugin? Well those are things to figure out, but the keystroke bindings seem much easier to address in a plugin. Plugins provide a convienient place to address the semantic difference between a modal editing model (used extensively by Vim and to a degree by Emacs editors) and the underlying editor backends capabilities.

274 points by ivank 10y ago

Interesting this is a google project. https://github.com/google/xi-editor/blob/master/rust/rope/src/rope.rs https://github.com/google/xi-editor/blob/master/rust/rope/sr... was quite interesting to read (for someone who has never looked into rust)

retbull 10y ago
doesn't s/he say at the bottom that this isn't a google project?
- raphlinus 10y ago
  It's a project by a Googler (me), but it's a personal (20%) project, not an official Google one.
  - namelezz 10y ago
    What's your thought of Rust? What dev env did you use?
    - raphlinus 10y ago
      Rust is working out very well so far. If you look at the rope implementation, you'll see that it's high level and mathematical almost like a functional language, but performance is first-class. I'm also very much looking forward to retrofitting parallelism using libraries such as Rayon and crossbeam. I'll probably write a retrospective when I'm farther into the project. My current development environment is just Sublime Text for the Rust parts, and Xcode for the parts written in Swift.
      - wbond 10y ago
        If you are using the recently-added default Rust syntax for Sublime Text and run into any issues, please let me know by filing an issue at https://github.com/sublimehq/Packages/issues https://github.com/sublimehq/Packages/issues.
      - rudedogg 10y ago
        What made you decide to not write everything in Swift? Is it the lack of Windows support?
geofft 10y ago
It seems Google has been releasing employees' personal projects at github.com/google if Google owns the copyright. (I'm not a Googler, so I don't know if this just applies to stuff you hack on while at work, or anything plausibly in Google's business interests that you do at home while you're an employee.)
geofft 10y ago
It seems Google has been releasing employees' personal projects at github.com/google if Google owns the copyright. (I'm not a Googler, so I don't know if this just applies to stuff you hack on while at work, or anything plausibly in Google's business interests that you do at home while employed by Google.)
- skybrian 10y ago
  If you don't care whether you own the copyright (just that it's open source), this is the easiest way to get permission.
  - cyphar 10y ago
    The issue with ditching the copyright is that you have no guarantee that they won't make a proprietary fork (if you use a copy left license). But if you don't care about the copyright you almost certainly don't care about ensuring the perpetual freedom of users.
    - burntsushi 10y ago
      > But if you don't care about the copyright you almost certainly don't care about ensuring the perpetual freedom of users. That's ridiculous. If I release something into the public domain, then it is forever in the public domain and is perpetually free to use by anyone.
      - cyphar 10y ago
        Ignoring the issues with public domain (that it doesn't exist in many places, so the default you-cant-do-anything copyright rules apply), you misunderstand what I mean by freedom in perpetuity. If someone can take your code and make it proprietary (as well as adding some enticing features), your code is now used to take away users freedom -- even though your original code was free software. Copyleft is the only defence against that form of attack on user freedom -- attack by distribution.

fritzy 10y ago

According to the feature-set, this is basically NeoVIM. * Separation into front-end and back-end: NeoVIM does this. The UI uses the same interface a plugin does. * Asynchronous operations: NeoVIM uses libuv to do this (same asynchronous C++ lib as NodeJS). * Plug-ins over scripting: Yup, NeoVIM does this, as well as dropping vimscript (although it will compile vimscript into Lua for you). * JSON: NeoVIM uses msgpack, which is significantly less overhead than JSON, but structure compatible. The two other features listed are Rust and rope-structures, which are implimentation specifics that don't affect users or plugin writers at all. Edit: The more the merrier! And it's good to see more projects using Rust, but I don't see this as a strong need. See https://github.com/rogual/neovim-dot-app https://github.com/rogual/neovim-dot-app for a nice native Mac UI plugin/client for neovim.

grblovrflowerrr 10y ago
I think the overlap in features demonstrates popular appeal for such features in our tools. I think the details of how they're implemented are pretty important to users too so there's space for innovation even if the feature set is the same on the surface. Another feature mentioned in the introduction is this: * Developer friendliness. It should be easy to customize xi editor, whether by adding plug-ins or hacking on the core. I'm vaguely familiar with NeoVIM, is the core "hackable"? This is really subjective territory but I'm guessing the author is implying that the xi core code is also optimized for human understandability and extensibility and that new code is judged by those criteria as well as the other stuff.
raphlinus 10y ago
NeoVIM is great, and obviously it has been inspiration. I think it remains to be seen whether NeoVIM can deliver an experience that feels like a modern editor. My experience is that projects which attempt to layer a GUI on top of an existing, mature console UI end up leaking bits of the underlying UI. I'm thinking mostly of the various TeX shells here. A large part of my motivation for starting this project is to explore how suitable Rust is for this kind of work. End users shouldn't have to care, they should judge it strictly by the quality of what it delivers. But I'm hoping this project will be interesting to Rust programmers for a number of reasons.

sitkack 10y ago

If folks like Rust and want questions answered, check out the vibrant http://reddit.com/r/rust http://reddit.com/r/rust

hayd 10y ago

Not to be confused with Yi: https://github.com/yi-editor/yi https://github.com/yi-editor/yi

cm3 10y ago
Yep. I use Yi occasionally and it works well, but it needs more users to test and report the rough edges. Though, it's excellent for writing support for new languages, given that it's all Haskell.

_RPM 10y ago

I don't believe that it being written in Rust is a feature. The language it's written in is just an implementation detail.

mblode 10y ago
I think it definitely would be considered a feature. For example Atom, Brackets, and Visual Studio Code were all written in JS with CSS and HTML and they all for the most part have fairly poor performance. The fact that rust is a highly performant language and is also relatively higher level than C for example, it offers a benefit for the user.
groovy2shoes 10y ago
This is really common among open source projects. I frequently come across feature lists containing "Written in X" for many different values of X. In fact, was just complaining about it to a friend of mine just last night. He pointed out that for open source projects, it really can be a feature: when attracting contributors, the source language is important. It's also important for people who just want to study the source for some reason or another. It can also be an indicator for how readily portable it is: a program written in C is going to be much more easily ported than one written in Visual Basic, for example.
- sbuttgereit 10y ago
  Yes. I'm not so interested in a new editor unless it's very stand out (and I'm sure this project is good)... but I do have an interest in Rust; in particular in how it's being used and the different kinds of software it's showing up in. It may be a good example of programming that's worth reviewing as I learn or it may just help provide a sense in the kind of community that's growing (or not) around something that's relatively new (in some stable status at least) like Rust. Had it not said "Rust" in the title... I may well have not bothered to look at this at all.
Joof 10y ago
As another poster said, it's open source and written for fellow hackers, so language is a feature. Imagine if emacs was written purely in C++.

Scarbutt 10y ago

What default key bindings will it follow? CUA?

raphlinus 10y ago
I'm basically planning on following Sublime Text's keybindings except when there's good reason not to.
- evanrelf 10y ago
  I know you said you haven't worked on plugins yet, but do you think a Vim mode would be possible through plugins?
  - raphlinus 10y ago
    I think a Vim mode will be possible one way or other. I'm not sure having all interaction mediated by plugins is the best way to accomplish it.

fernly 10y ago

TIL: about the rope data structure[1]. Never heard of it in decades in the industry. Disagree with the comment "Cross-platform UI toolkits never look and feel quite right." Qt, for one, uses the native toolkit on Mac and Windows and I can't see a difference in look'n'feel. [1] https://en.wikipedia.org/wiki/Rope_%28data_structure%29 https://en.wikipedia.org/wiki/Rope_%28data_structure%29

jdub 10y ago
You can't see it, but boy can you feel it.
- mwcampbell 10y ago
  And you can literally feel it, or hear it, if you're blind. Non-native GUIs are very often less than fully usable, or even downright inaccessible for less common toolkits, with a screen reader. The one exception I can think of is XUL-based apps (e.g. Firefox and Thunderbird) on Windows.
  - lmm 10y ago
    Qt specifically has very good accessibility support, no?
    - mwcampbell 10y ago
      Qt's accessibility support is only moderately useful in practice. As one anecdote, TeamTalk (http://bearware.dk/ http://bearware.dk/) has a "classic" MFC-based front-end just for blind Windows users.
pcwalton 10y ago
> TIL: about the rope data structure[1]. Never heard of it in decades in the industry. And if you've used JavaScript, you may have been using them without realizing it—JS engines will choose rope representations for strings if they observe that the usage patterns would benefit from them.
10y ago

anderspitman 10y ago

Awesome. I've thought it would be fun to work on a simple editor based on Rust. Something similar to Vim but without ViML. I really like the idea of using pipes rather than committing to a single plugin language. I'll be very interested to see how that works out in practice.

mwcampbell 10y ago

Interesting architecture, having the core communicate with the GUI front-end over pipes via JSON. That certainly eliminates a whole class of pain around inter-language glue. Any idea what language and APIs the Windows version will use? All of the Microsoft-sanctioned choices for desktop apps (as opposed to the wanna-be mobile/tablet platform introduced with Windows 8) are kind of sucky.

raphlinus 10y ago
I've been playing a bit with the various choices (there are so many!) and am leaning in the direction of C# + WPF + Windows.System.Media for the text drawing. Using something based on DirectWrite rather than GDI+ is important to me because of text rendering quality. I have basically ruled out Windows Universal because it feels a lot clunkier. But I'm very much open to ideas from more experienced Windows devs who understand the tradeoffs better.
- mwcampbell 10y ago
  WPF left a bad taste in my mouth because, as far as I can tell, it hastened the obsolescence of my brother's laptop around 2011. [1] But then, maybe WPF 4 under Windows 7 would have been better on the same hardware. Impossible to say now. The best solution would probably be one that uses both DirectWrite and Direct2D. WPF 4 uses DirectWrite, but AFAIK, it's still on top of the older Direct3D-based milcore graphics system. But to use both DWrite and D2D, I guess you'd have to get down and dirty with C++, or at least use a lot of P/Invoke and COM interop in your C# code. To be honest, I don't understand why GDI isn't good enough. Can you explain what you find lacking in GDI's text rendering, or point me at an existing explanation? [1]: https://news.ycombinator.com/item?id=8444494 https://news.ycombinator.com/item?id=8444494
  - raphlinus 10y ago
    The main thing I'd add to http://www.basschouten.com/blog1.php/font-rendering-gdi-versus-directwrite http://www.basschouten.com/blog1.php/font-rendering-gdi-vers... is that DirectWrite looks very smooth on high-dpi displays while GDI+ is quite grainy.

ouchunyu 10y ago

afd

dj-wonk 10y ago

What's the plan for plugins? Will they need to be compiled?

raphlinus 10y ago
I haven't started implementing plugins yet, so details are still vague. General thoughts, they communicate with the core through a simple JSON-based protocol, so can be written in any language you like. I imagine Python will be one of the popular choices. Among other things, they'll be able to annotate the text with rich-text spans, for things like syntax coloring. Also, I'm planning for them to be async, so they won't slow typing even when they're busy computing.
- kristiandupont 10y ago
  My personal dream would be to have an editor that acts mostly like VIM, but with folding capability like Code Browser (http://tibleiz.net/code-browser/ http://tibleiz.net/code-browser/) -- would that be possible to achieve with this?
- martanne 10y ago
  How do you plan to encode binary data in your JSON-based protocol? Base64? This will only increase the overhead. While the human readability of JSON is nice, it has some serious flaws when the goal is to handle arbitrary data. I have yet to decide what kind of protocol to use in my own vis editor[1], hence why I'm asking. [1] https://github.com/martanne/vis https://github.com/martanne/vis

lesbaker 10y ago

I like the explicitly stated emphasis on performance and large file support in the goals. I look forward to seeing this develop and will be following the project (and test-driving it when I have time).

unimpressive 10y ago

So what exactly does 'native' mean for Linux distros?

raphlinus 10y ago
I'm leaning towards Gtk+ because I'm more familiar with it (I used to be very involved in that community, in fact) and because I know its Unicode support is good.
- dman 10y ago
  Please consider Qt as well, the implementation is solid and its easy to make apps that are performant and polished.
  - raphlinus 10y ago
    Qt 4 or 5? I had difficulty getting even "hello world" running in 5, and the size of deploy binaries (~16M) was scary. Maybe that doesn't matter so much if it'll primarily be using system libraries.
    - oblio 10y ago
      At this point? 5, definitely :)
    - dman 10y ago
      I use Qt4 and QWidgets - its by far the most pleasant desktop UI development toolkit Ive used. (Ive used qt, gtk, wx, win32 with gdi/gdi+ and a bit of swing).
    - lmm 10y ago
      What went wrong with 5? And yeah use dynamic linking, that's what it's for.
      - IshKebab 10y ago

thedz 10y ago

Interesting note: The author (Raph Levien) is also the creator of my defacto programming font Inconsolata. Super cool.

eggy 10y ago

Would this easily port to Windows or Linux? I put my iMac 2011 in storage, but I would like to try this out to learn Rust and building a text editor in general. I see 'other targets are planned'. Any time soon?

raphlinus 10y ago
I'm hoping soon but make no promises.
- eggy 10y ago
  No rush here. I would like to be able to play with the code and Rust. Thanks!

ArmandGrillet 10y ago

Eager to see in which direction will head the project! Raph Levien seems an extremely talented engineer but, as it is a "20% time" project, I am not sure what is the expecting outcome of this editor. When the development of Atom started it was clear that it would replace Sublime Text in the long term, here I don't know if it will replace TextEdit or be the next big thing.

vintermann 10y ago

First thought, yet another editor attempt, not even very far come yet, and it's OSX only. Rather nice font in the screenshots though. Then > The main author is Raph Levien. Oh. Now I see why this is on hacker news. Now I'm interested too. If nothing else, this will be gorgeous.

Spiritus 10y ago
I've never heard of him, care to enlighten me?
- Someone 10y ago
  https://en.m.wikipedia.org/wiki/Raph_Levien https://en.m.wikipedia.org/wiki/Raph_Levien
  - loup-vaillant 10y ago
    https://en.wikipedia.org/wiki/Raph_Levien https://en.wikipedia.org/wiki/Raph_Levien ?
    - sp332 10y ago
      That's the same article in a "desktop" view. Is that what you're asking? Personally I prefer the mobile one.
      - loup-vaillant 10y ago
        I'm asking "why anything but the default URL?" I'm asking "why isn't this handled by freaking CSS? We have a goddam tag for this!"
wodenokoto 10y ago
So what? Is he supposed to be famous?
jfindley 10y ago

Myrmornis 10y ago

In particular, it should use little more memory than the buffers being edited. I'm pretty sure memory footprint of a text editor is not a relevant concern on modern laptops. Or is the idea that it will work on completely different low-memory environments from what I'm thinking?

kennywinker 10y ago
> I'm pretty sure memory footprint of a text editor is not a relevant concern on modern laptops. https://github.com/atom/atom/issues/1827 https://github.com/atom/atom/issues/1827
vidarh 10y ago
A single editor is not the only thing I run. The only reason I have 8GB on my current laptop is because too many people think that it is no big deal if just their app spends more.
gnuvince 10y ago
I can't remember where I heard this, but someone said once that better performance opens up more, and sometimes new, ways to work with a program. I use GNU Emacs myself, and I basically start it when I boot my laptop and never shut it down, which means that sometimes it has an uptime of months. I will also often not close buffers that I'm working on and can end up with hundreds of buffers in Emacs. Yet, the performance is entirely unaffected (or at least, the impact is below what this human can notice). If Xi is serious about keeping memory usage low, these kinds of patterns of usage that I and many other Emacs users have adopted are going to be viable; on the other hand, in an editor like Atom, one should probably keep as few buffers opened as possible.
panic 10y ago
For software with variable-sized inputs, performance never stops being relevant. Higher performance increases the maximum size of inputs you can handle. For a text editor, the lower the footprint, the larger the files it can edit and the more of them can be edited at once.

kingosticks 10y ago

I've no exposure to Rust but is it accurate to compare it to C++ as offering a much more reliable, and in many ways, higher level programming platform. Are there some quick and simple examples of this?

Cyph0n 10y ago
The main advantage is Rust's ownership model. It allows you to catch most memory problems at compile time. https://doc.rust-lang.org/book/ownership.html https://doc.rust-lang.org/book/ownership.html
- akadien 10y ago
  And it's one of the languages features that has a very steep learning curve. Rust forces your design into its model. Interestingly, rustc also seems to generate programs that leak memory. I recall an issue where the simple and venerable "hello world" program in Rust (i.e., println!) had memory leaks.
  - steveklabnik 10y ago
    That shouldn't be true. Please file bugs!
    - akadien 10y ago
      It was fixed but isn't fixed. https://github.com/rust-lang/rust/issues/19776 https://github.com/rust-lang/rust/issues/19776
      - steveklabnik 10y ago
        Great, thanks.
kimundi 10y ago
I don't have concrete examples, but in general: For one, the build system/dependency system is much more easier to handle. Rust uses the cargo package manager, and depending on an external library is as simple as adding a single line to the Cargo.toml config file. Likewise, building, running, documenting and testing is all a build-in matter of calling `cargo [build|run|doc|test]`, which makes prototyping and getting started way easier than my experiences with C++ so far. As far as the actual language goes it basically behaves like C++ and has a similar feature set, so you get the same sense of how given source code will end up in the compiled binary, and can write fast efficient code with, for example, features like pointers. The big distinction there is that Rust is memory safe per default, (including pointers in form of safe references) and upholds this invariant with compiletime checks and explicit language barriers (needing to encapsulate "unsafe" operations in unsafe {} blocks), so per default you are protected from whole classes of bugs that are possible in C++, like dangling pointers, memory races, iterator invalidation, etc.

stephenr 10y ago

I haven't used a "text" editor (as compared to a full IDE) for quite a while. Do any of the decent OS X -compatible editors have things like - find usage of a method/class/constant in a project - change method signature and have calls update - rename a class/method/constant/variable and have usages updated This isn't a troll, I'm honestly asking. I'm using IDEA Ultimate at the moment and it works fine, it just has the usual downsides for a Java app.

adrusi 10y ago
Those would typically be considered features for an external tool, maybe with the editor configured to provide an interface to those tools. If you write java and value those features, though IDEA will be a better experience.
- stephenr 10y ago
  Really? An external tool that isn't a text based editor but does have in depth knowledge of the code? I've never heard of such a thing. Also for reference I don't really write Java. I write php, sql, shell scripts, ldif files, html, occasionally JavaScript and/or css, lua...
  - pdpi 10y ago
    > An external tool that isn't a text based editor but does have in depth knowledge of the code? It's called a "compiler" ;) Jokes aside, here's a few tools that might give you an idea what's out there: - Racer for rust - Go fmt - Youcompleteme - ghc-mod
    - stephenr 10y ago
      Right, so specific tools for Rust, Go and Haskell (none of which I use) that maybe have some refactoring support? (The gofmt example I kept finding was renaming a field in a struct) Oh and a plugin specifically for Vim, so not an external tool at all.. So, basically the answer is the same as before: an IDE is still where it's at.

Keats 10y ago

Looks very neat! Found my Rust reading for the week.

cm3 10y ago

Nice project, but contributing to the project requires assiging copyright of any patch to Google, which is the same flaw that exists with FSF projects and a major blocking issue for many potential contributors. Google should make an exception for personal-but-not-Google-company projects and get rid of it or at least remove the copyright assignment clause. Nobody wants to contribute to a project under license X and then have their contributions be relicensed under Y.

ywecur 10y ago

How do you pronounce the name? X-I or Chi?

hnbro 10y ago
or shii or zee or

ninjabeans 10y ago

I like the design decisions. Just a heads up, you misspelled "sparse" and accidentally a word after "completely".

username223 10y ago

The "documentation" is just weird. It has Rust, ropes, and JSON! Okay, but is it modal? Does it have syntax highlighting and debugger support? What kind of editor is it trying to be? The docs say nothing relevant to whether or not I would want to use it.

todd8 10y ago

I've never constructed an editor -- but I have used one before ;) so I thought I'd make a few suggestions about the architectural considerations off the top of my head. I'm an Emacs user, but I'm very excited about NeoVim. It looks like your project is another really interesting one. I hope it takes off. I notice that you've choosen Rust as the programming language for the editor, a fine choice. Performance appears to be a key factor in your choice of language. However, realize that performance isn't the reason one should use Rust or C++ for the backend of an editor. Let's go though some simple back of the envelope calculations. A realistic goal for editors is to keep ordinary command and keystroke latency under 100ms, this should appear to users as if operations are performed instantaniously. You've picked a more aggressive goal of a maximum latency of 16ms, so let's go with that. Today, a slow processor runs perhaps 5,000,000,000 instructions per second (speed of Raspberry Pi 2). By the time the editor is ready for general use, most programmer's development systems will probably run something like 20 times faster or 100 Billion instructions per second. For this rough calculation I'll use the speed of a Raspberry Pi 2. In 16ms that's 8 million instructions. What this means is that in the backend of the editor we have plenty of time to perform the editing operations no matter what language is used as long as it is a straight code path through the editor. The pedestrian operations of the editor, like reading a keystroke or inserting a letter into the text don't require Rust or C++ or C. They can be done with practically any programming language, even a language that runs a hundred times slower than C++ like Python. It is the iterated operations where performance will be a concern: searching big files, complex regular expression searches, syntax highlighting after each keystroke, autocomplete, refactoring, the GUI and interfacing to the GUI. I couldn't agree more with your requirement that the editor shouldn't hang or crash or lose work. Further, a modern editor is a big project and won't gain traction without additional contributors. From a performance perspective my gut feeling is that any language that performs no worse than 10 times slower than C is a fine choice. For safety, either managed memory (with GC etc.) or an approach like Rust is essential to allow code to be contributed. The JVM languages, while being attractive in many ways carry the burden of a big infrastructure; Jetbrains has figured it out but even after all these years Eclipse feels so klunky--this make me rule out Java, Clojure and Scala. I think this leaves at least Rust, Go, OCaml, Common Lisp, Dart, and Swift. (I'd probably pick Go hoping to get more contributors.) What about ropes, a clever and much more modern data structure for holding strings than ancient Emac's gap buffer. Ropes do indead have better asymptotic performace for many of the basic string operations: delete an arbitrary character, insert a substring, etc. What about all of the other operations that you'd like your editor to do? Fast searching, backwards or forwards? Handling UTF-8? Doing regular expression matching. Ropes turn the text being edited into a tree of text segments so high speed searching is much more difficult. An abstraction layer over the data structure to make it appear to be indexed like an ordinary array will make some things run much slower, for example, Boyer-Moore string searches, regular expression searching (done for code highlighting), etc. Ropes or other persistent data structures for sequences will work because they can present an abstract indexed access interface to the rest of the program, but you will always be dealing with it though this abstraction layer just when you'd rather not. Processors can move several bytes per instruction cycle so the gap buffer can close it's gap while editing a 10MB file in less than a millisecond. This leaves the data in one contiguous block. So for simple things the gap buffer is much faster than rope structures because for most simple activity all of the additions and deletions are at the cursor, where the gap is located, and for complex things there is a 1ms overhead before the operation starts (to close the gap) but then things run basically full processor speed without the requirement to go though the abstraction layers. The places where I see this as likely to be important in modern editors is in the design of incremental backups, persistent text attributes, memory mapped access to underlying files, and coordination with asynchronous plugins working on the buffer. I'll be curious to see how this works out for Xi. Keystrokes and plugins. Emacs can end up running lisp code, through a pretty slow lisp interpreter, on every keystroke. It's completely imperceptable. So performance isn't a reason to treat keybindings differently than other plugins. The overhead of IPC to a plugin is measured in micro seconds. While supporting Sublime Text keybindings is fine. The programmers most likely to be sensitive to keybindings will probably want either Vim or Emacs keybindings. A good way to support multiple keybindings will make the editor appeal to a much broader set of users. Plugins seem ideal for this. A key comes in, is processed by a plugin and a few elemental backend operations are invoked by the plugin in response to the key. This seems completely reasonable. The real issue with plugins is direct access to the edit buffer if the plugins run in a separate process and address space. This is complex especially if the buffer is being accessed asynchronously. How is syntax highlighting done in a plugin for example. Where does the syntax highlighting actually get stored? Does the plugin have to be informed each time the user scrolls the screen or types a key (probably yes). What asynchronous commands does the plugin send back to the editor to update the display. Is there a separate plugin working on autocomplete? Does it also have to parse the source code just like the syntax highlighting plugin? Well those are things to figure out, but the keystroke bindings seem much easier to address in a plugin. Plugins provide a convienient place to address the semantic difference between a modal editing model (used extensively by Vim and to a degree by Emacs editors) and the underlying editor backends capabilities.