15 ms·
I Make Dreams Come True
- collint 15y agoI agree with the sentiments on OT. It really ought to be baked into everything. It has features you want, such as undo/redo, collaborative editing and optimistic locking. I've been working on an OT implementation for the past few weeks. It really is some brain wrinkling stuff. The adoption of data-bining we're seeing in the newer JavaScript client frameworks/libraries could be paving the way for something like OT to go mainstream. OT is a bit like pulling your brain through a wormhole#. I think it's going to take a really easy to plug-in library to have something like OT get any traction. # That is, OT is fairly straightforward if you're editing a document. It gets weird when you have to map a graph of objects onto/from a stream of operations.
- raganwald 15y agoOT is fairly straightforward if you're editing a document Even with a “plain text” document, there is much nerd sniping to be done when you consider that OT works best at a semantic level, not a character level. A trivial example is that conflicting edits to words and numbers should often be treated differently. This may not generalize very well, a domain-specific editor for one application may want to use slightly different transformation semantics than another.
- collint 15y agoI try not spend too much time on semantics at the transformation level. There are already hundreds of cases that must be understood transformed just to handle concurrent operations on an XML document. If you want the transformer to handle semantics I fear you're going to balloon in complexity to the point of deep sadness. ( What do we do if I'm typing a number inside your word? Word inside a number? Delete a number inside a word? What if you word is the name of city? etc. forever ) I think there is a better ROI on making the intent of your collaborators clear, such as with cursor/mouse indicators. If I can see you're editing a number, I can know to steer clear, or at least be ready for the consequences.
- josephg 15y agoI've written ShareJS, which performs OT over generic JSON objects. Getting OT-for-JSON working involved a lot of cases, but it was certainly tractable. The tricky algorithmic code is 440 lines long. XML would be much more complicated to get right, but certainly possible if you're mad keen on XML.
- beza1e1 15y agoDid you really need it on the JSON level? It goes over the wire, hence it is essentially just a string.
- josephg 15y agoYes, you do. Say you have an object: {x:1} Two people try and add a y:2 to the object. If you treat the object as a string, you can end up with: {y:2,x:1,y:2} ... which is invalid JSON. If you do string-level operations, there's heaps of ways the JSON can end up invalid or just plain wrong. In comparison, using OT if two people apply a "Insert y:2 into the object" operation, the object will be edited correctly at all sites.
- raganwald 15y agoJoesph's reply is absolutely correct. I just wanted to add that JSON is a special case of what I noted above: OT is more of a protocol than an algorithm, it takes in operations and performs a transformation on them. It's up to you to decide what those operations are and how to transform them. The canonical example of a text document with operations consisting of inserting or deleting strings of characters is a little error-prone, but in the context of humans editing text in real time, there's an assumption that they will correct any visible artefacts. However, when you have structured data created and consumed by algorithms, your tolerance for artefacts goes down to nothing, and you need to choose a different set of operations and carefully design the transformation function. In between structured data and unstructured text there is also text with semantics. As noted above, you can ignore that and leave it up to humans to fix it, or you can bake some smarts about the text in. There are existing products that treat numbers as a special case, for example. It may not be worth the effort, that's a design choice. Anyhow, this is a lot of blowhardiness just to say that OT isn't an error-free way to synchonize strings, nor is it just one algorithm, it's a protocol and part of the design process is deciding when and how to go beyond plain text and the obvious transformations on the operations.
- ilaksh 15y agoI always thought that Google Wave was about baking OT into everything.
- ilaksh 15y ago"The adoption of data-bining we're seeing in the newer JavaScript client frameworks/libraries could be paving the way for something like OT to go mainstream." Socket.IO plus some kind of generic data framework with a collection type.. you can get pretty far with most CRUD type applications if you just have a way (something OT algorithms handle) to resolve issues related to positioning in a list I think, and that could be generalized to take care of everything from sequences of characters in text to items in a spreadsheet. When you are talking about graphs.. what sort of issues are you having?
- josephg 15y agoI'm working on the same thing for node.js. Come help me write ShareJS: http://sharejs.org/ http://sharejs.org/ Also, plaintext and JSON data types are generic and pluggable into other systems. I'm not sure what you're trying to do, but they might be useful.
- jmcqk6 15y agoThanks for this. I came across your project a few months ago and bookmarked for when I have more time. Now I'm getting to that point.
- gregw100 15y agoOne thing I take from this article is that the things that give us that "Dreamy" feeling are (especially in tech) things that turn ordinarily mundane or complicated things into simple and easy things. For example, the GUI basically turned accessing files (along with other commands) from mundane tasks into easy and attractive tasks, effectively making it far more useful by lowering the learning curve and lessening the complexity. Movies are another example of turning something (plays) into a more exciting, entertaining, and accessible form. It seems that what we consider "Dreamy" is simplicity. Perhaps that's why consumers gawk over Apple device's and describe them as dreamy. It makes one wonder, why is it that we consider simplicity so attractive? One reason could be that it is purely natural human instinct. We do not like not understanding things and we constantly try to understand that which we don't. When people create things that increase other's ability to understand whatever product or service they are offering, they are creating desires, also known as dreams. In a sense, that's the only reason that businesses exist, or at least it's the only reason businesses should exist.
- Pheter 15y agoI think the reason why we are drawn to simple things is down to, among other reasons, something that I've observed with how the human mind fills in blanks. Ever noticed how someone looks attractive from behind but then they turn round and are not so much? It seems to me that when our brain fills in the gaps it assumes that the subject is aesthetically pleasing. (Well, either our brain defaults to expecting something to look nice, or it fills in the blanks with an average and we find the average to be attractive.) So coming back to simplistic design and why it evokes a positive emotional response from us: Perhaps it's because with a simple design our brain has more 'blanks' to fill in. If anyone knows of a name for this phenomenon and further information exploring it I would love to learn about it. Same goes for if what I'm saying is nonsense and there is an alternate theory disproving mine!
- sliverstorm 15y agoI rather suspect it's simpler than you think. Attractive-from-behind people are attractive by virtue of their figure. Before they turn around, everything we have seen we find attractive. Unless you find every person you see from behind first, initially attractive. But I know I certainly don't.
- rmason 15y agoThis post reminded me of some discussions I've had with a friend about open source. He always says that open source software is written by geeks for geeks. His point was that the software might be free but there was too high a cost struggling with difficult installs and ugly user interfaces. You can stand out if you bridge that gap. Did you catch what Matt Mullenweeg said yesterday at Startup School why WordPress succeeded in the early days? He said it was because WP had the quickest and easiest install of any blog software. Bringing tech we use every day to a wider audience is a viable business model.
- ilaksh 15y agoI have been thinking along similar lines, in the context of my latest open source interest, Node.js and the multitude of packages available through NPM. My main goal right now, aside from building a Node.js hosting service, is to create a platform for content and data management built on Node.JS, CoffeeScript, and NPM, with a plugin and component architecture. I dream about killing WordPress.
- marcocampos 15y agoIf your dream is to kill the competition instead of building a better product, well... you're doing it wrong. Buzzwords and fancy tech don't mean crap to the end user.
- Periodic 15y agoMost of the technologies I dream about killing are the ones that I feel are suboptimal and downright frustrating but I am forced to use for lack of a better solution or due to economic forces. I want to kill them because I want them to disappear by being replaced by something better. For example, I hate most of the Microsoft Office suite, but there really isn't anything that's better for the average user to author documents. I dream of making something that's so good that people will switch and I won't have to deal with Office ever again.
- ilaksh 15y agoFirst of all, I do not use any of these terms in a non-specific non-technical 'buzz' fashion. I am actively working on implementing this system and the technology choices I made are not simply following trends, there are real advantages to working with things like Node.JS, NPM and CoffeeScript. And just because I didn't get into detail in regards to the advantages doesn't mean that wasn't the an important part of the motivation. For a platform like this, there are two sets of users: the non-technical users and the software developers building plugins and components for the system (not buzzwords). Let me explain exactly what each of these words means technically in this context so that they cannot be mistaken for buzzwords. I will admit that 'plugins' and 'components' can often mean different things to different people, but I don't believe there are better words to describe what I am building. Plugins are extensions to the system which are easily added, integrated and configured. This is a feature which is missing from a number of sophisticated web frameworks, but not missing from WordPress. The biggest problem with WordPress is the language it is built on (PHP) and the architecture of the system. I think that many web developers would welcome a system with something like the types of plugins offered by WordPress, but built on a much cleaner language like CoffeeScript, Node.JS, BDD (not a buzzword, I mean specifically developing starting with Vows and Jasmine, those are actual specific libraries for BDD as per examples such as here http://vowsjs.org/ http://vowsjs.org/) and a component architecture (I will explain exactly what I mean by that). Components are reusable, interact on the basis of known interfaces, cohesive but decoupled, and substitutable pieces of software, which often, as in this case, present graphical user interfaces. I intended to use them primarily (but not exclusively) to compose the user interface screens and to provide easy configuration and integration for different aspects of the system. Building the screens from drag-and-drop, composed components with on-screen configuration rather than using mixed PHP, HTML and CSS code provides a major advantage (obviously) for end users and also (perhaps less obviously for many, unfortunately) developers. Having reusable, easily configurable components available with built-in UIs could make plugin and application development much more efficient for developers than it is with just (for example) libraries of modules.
- dhotson 15y agoI had a similar reaction after seeing Etherpad’s collaborative text editing. I remember my jaw hitting the floor when watching a replay of Paul Graham writing and editing an essay keystroke by keystroke. Now that I’ve seen it—I want everything to be real time and collaborative. I think it’s one of those things where once people have seen it, there’s no going back.
- quizbiz 15y agoI really wish Google docs integrated their versioning in a visual way like Etherpad. I wonder what the team is up to these days. Are they still at Google?
- jakevoytko 15y ago"I really wish Google docs integrated their versioning in a visual way like Etherpad" What did you like about it?
- quizbiz 15y agoJake, thanks a ton for your comment. I absolutely love that Hacker News is a place where users and engineers can talk openly and comfortably. It's been a while but if I remember correctly, there was a slider on the top right with a play button. It allowed me to scroll through the history of the doc akin to Time Machine. That visual playful approach is much more engaging than a drop down list of versions. Titles and dates really tell you nothing about the nature of the document. Watching an essay evolve with a click of a button was awesome and uniquely informative. A killer feature that should not be sacrificed. It was certainly a feature that could help Google Docs position itself above native applications.
- jmcqk6 15y agoI'm an instructor for a university course that is taught in a computer lab, so that every student has a computer in front of them. Each week, I setup two "community notes" pages using some etherpad clone and let them write their notes for the day in that fashion. I have to setup two, because apparently there is a hard limit of 12 users per etherpad, but it's completely changed my students' view of notes. There's no more clamoring each week for the powerpoint that I refuse to make. They're now using each other to build notes that are better than anything they could build individually. When we get this technology into everything, the world will be a much better place.
- acgourley 15y agoMaking dreams come true is the same as "adding business value" as long as your business is consumer facing.
- raganwald 15y agoTrue, personal value is often aligned with realizing dreams. But you can make an amazing amount of money making people’s dreams come true in the workplace was well. How do you feel about Visicalc, fax machines, FedEx, and others examples of things sold to businesses that provoked the same “Why didn’t anyone think of this before?" reaction? Sure, people justified these purchases with the money they’d make or save using them, but often people looked at them, stared, then a lightbulb came on and they wanted them. Businesses are staffed by people. Yes, you can sell them Enterprise Software and Services they hate, and laugh at me from your massive mansion. But you can also sell them software and services that make their dreams come true.
- sgentle 15y agoIf anyone is keen to hack on OT or OT-related products, a friend of mine has a great Javascript/Coffeescript library called ShareJS that makes building OT-powered web apps pretty easy. Demo: http://sharejs.org/hello-ace.html http://sharejs.org/hello-ace.html Github: https://github.com/josephg/ShareJS https://github.com/josephg/ShareJS
- ilaksh 15y agoI was just googling for socket.io and operational transform a minute ago and that popped up. The slideshow I saw though said it just uses Ace to handle text. Can it do OT on types other than text now? EDIT:.. actually I see the JSON example. The hex thing game, I notice it has a fixed length in that array.. does this sharejs OT library handle cases where a variable size array has concurrent updates, deletes and inserts?
- josephg 15y agoHi! Author of ShareJS here. ShareJS works fine with variable length arrays. Also, ShareJS isn't limited to ace. Textareas work as well. Here's the same document that was linked, editable from a textarea: http://sharejs.org/textarea.html http://sharejs.org/textarea.html
- thornad 15y agoIf you like dreams and technology, you may also like this: www.technosophics.com Try it, it is all free. Eventually it will also be open-source.
- toblender 15y agoI've come to realize that I'm an average programmer in the world of programming. I'm not a rock star that knows how to optimize every slice of code. However, when I go to other industries and bring my programming skills to improve their processes, I am basically a wizard in their eyes.