8 ms·
Announcing Firepad — Our Open Source Collaborative Text Editor
- karl_gluck 14y agoVery cool technology! I really appreciate your decision to allow us to just give it a try without having to set up an account or log in.
- gnuvince 14y agoAnd it's web based, and thus I will never use it.
- macspoofing 14y agoThat's a strange restriction you put on yourself? Today you have to have a good reason for your product to NOT be web-based.
- yuchi 14y agoI'm interested too. Is the browser environment which bugs you or the web "platform"? (you know, HTML and brothers)
- lutusp 14y ago> Today you have to have a good reason for your product to NOT be web-based. For a document that the visitor cannot afford to lose, or for anything in which privacy, security or personal control are issues, a Web-based document is obviously a mistake. Why do you think the big players in cloud-based and Web-based storage and applications are having such a hard time getting people to adopt them? The answer is obvious -- the drawbacks greatly outweigh the advantages. The risk of losing sensitive content or having it be compromised or stolen is too great. Even games, a less serious endeavor, suffer when they adopt a Web-based approach. Look at the brouhaha that followed from making the most recent SimCity version work online-only: http://www.gamesindustry.biz/articles/2013-03-15-ea-defends-always-online-simcity http://www.gamesindustry.biz/articles/2013-03-15-ea-defends-... A quote: ------------------------------------ "So, could we have built a subset offline mode? Yes," Bradshaw admitted. "But we rejected that idea because it didn't fit with our vision...The SimCity we delivered captures the magic of its heritage but catches up with ever-improving technology." A number of upset fans in the comments section to Bradshaw's update were not assuaged by the explanation. As one user going by the handle klymen wrote, "With all due respect, to write an article about why the new SimCity has to be always online and not to mention DRM or the anti-piracy measure even once, is dishonest and downright disrespectful. It seems like that you don't think very highly of your audience. Yes, the DRM issue is a sensitive topic, but to avoid it, not to mention it as one of the reason why SimCity is always online, shows absolute disconnect and contempt for your fans. We're not stupid." ------------------------------------ The above is just about a game, not a business transaction or potentially sensitive communication. As to data loss, consider incidents like this: Title: Amazon's Cloud Crash Disaster Permanently Destroyed Many Customers' Data Link: http://articles.businessinsider.com/2011-04-28/tech/29958976_1_amazon-customer-customers-data-data-loss http://articles.businessinsider.com/2011-04-28/tech/29958976... Quote: "In addition to taking down the sites of dozens of high-profile companies for hours (and, in some cases, days), Amazon's huge EC2 cloud services crash permanently destroyed some data. The data loss was apparently small relative to the total data stored, but anyone who runs a web site can immediately understand how terrifying a prospect any data loss is." In conclusion, and to second the SimCity gamer's quote above, we're not stupid.
- daleharvey 14y agoSim City wasnt written in web technology and needed to be online, its possible to write applications using web technology that dont need to be online. Those arguments have nothing to do with each other, 'the web' doesnt equal 'store everything in cloud based servers'
- lutusp 14y agoMy point was that forcing SimCity's fans to adopt a cloud-based version of the game was a big mistake, and everyone involved accepts this now. > Sim City wasnt written in web technology and needed to be online The present version of SimCity requires players to log on. The game will not work without a Web connection. This has created a huge outcry from fans of the game. > its possible to write applications using web technology that dont need to be online. If we define "online" as "connected to a network", then yes, Web-based applications require one to be online. And most Web-based applications require an Internet connection. > Those arguments have nothing to do with each other ... Cloud-based storage and Web-based applications have nothing to do with each other? Most Web-based applications store their results in the cloud. If you adopt Web-based applications, you're also adopting cloud-based storage at least temporarily.
- macspoofing 14y ago>My point was that forcing SimCity's fans to adopt a cloud-based version of the game was a big mistake, and everyone involved accepts this now. The BIG problem with SimCity wasn't that it had online features (those are great), but that it did not let you play offline for any good reason, which annoyed the heck out of people. And how does that example prove anything? A mediocre game (single or multiplayer, online or offline) gets released every other week. >Cloud-based storage and Web-based applications have nothing to do with each other? They don't. You can have a web-based app, that is hosted locally, or you can have an installed app, that cannot function without a cloud-service (e.g. almost every mobile app). >Most Web-based applications store their results in the cloud. I mentioned this in another post, but almost every business, big or small runs some of kind of a intranet, where they control the entire stack. The reality is that we're moving away from that setup as well. So no, web-based and cloud-based aren't the same thing.
- toddmorey 14y agoOne interesting and important bit about how Firebase works is that all write operations to the data happen locally, and then they are synced back to Firebase (and to other connected clients) on a best-effort basis. That approach is designed to prevent network lag / connectivity issues from impacting the responsiveness of the application. Advancements like that are starting to make newer web apps feel more and more like their desktop counterparts.
- yesimahuman 14y agoWhile your preferences are perfectly valid, I think it's safe to say web based editors and tools are very popular to a large amount of people. For example, everyone I know loves Google Docs and few of them use MS Office. That's real and shouldn't be discounted so quickly.
- yuchi 14y agoI'm happy to see [Substance](http://substance.io http://substance.io) work (Tim's OT) to spread to other related project!
- mikelehen 14y agoYes! His work is great. Made for a great starting point for building Firepad. :-)
- macspoofing 14y agoIs it based on etherpad?
- yuchi 14y agoNo it's based on [Substance](http://substance.io http://substance.io) OT library, but uses CodeMirror instead of the [Surface](http://interior.substance.io/modules/surface.html http://interior.substance.io/modules/surface.html) (Substance Team member here)
- unwind 14y agoBy the way, unfortunately HN doesn't do Markdown. Links should just be links, no separate display text.
- yuchi 14y agoSorry, just used to find md pretty everywhere to the point I forget HN does not support it! Well, then, _pardon my markdown_ :)
- ianstormtaylor 14y agoI actually like reading the Markdown, it's intuitive and then I get to have display text that makes the sentence easy to parse.
- yuvipanda 14y agoSweet! This will be fun to integrate into Wikipedia for editing... /me plots
- nahimn 14y agoThe homepage turned into a giant chat room
- mbrock 14y agoDoes this depend on proprietary server tech hosted by Firebase?
- toddmorey 14y agoYes, it does. "Firepad has no server dependencies and instead relies on Firebase [hosted service] for real-time data synchronization." So the front-end editor portion of it is open source, but the data sync between connected clients happens via Firebase. You can find more info at the bottom of this page: http://www.firepad.io/ http://www.firepad.io/
- mbrock 14y agoThanks. Seems cool, I just wasn't sure, and phrasing like "Firepad has no server dependencies" had me picturing some advanced WebRTC-based peer syncing or something, which is pretty far-fetched but you never know these days...
- derefr 14y agoTechnically, the whole point of OT is to allow for "some advanced [...] peer syncing or something"; any OT peer can actually synchronize state against any other peer. The original OT applications were all peer-to-peer apps. It's very similar to git, actually: everyone is on a particular commit[1], where each commit has another commit as its parent; clients can create regular commits, and thus move apart in state-space; and then they can accept others' commits to move back together (creating a merge commit in the process.) Very decentralized. Ever since Etherpad/Google Wave/Google Docs, though, OT has been consistently mangled into a hub-and-spoke design, where there's a server who keeps a canonical state, and the OT tie-breaking algorithm is always decided in the server's favor. Used in this fashion, it's not much different (actually a bit higher-overhead than) just having the server arbitrarily accept and temporally-order input operations in a way it likes, and then send back a single overwriting transformation to all clients to move them from their last server-known commits to the new server-canonical commit. (In other words, what any multiplayer game's logic server does.) [1] One major difference is that OT "commits" are referred to by their vector-clock, instead of by their SHA. Cryptographic hashes are nice, and would give OT even cooler properties if it could use them, but they're a bit too slow for something that gets regenerated with every character typed. CRCs might be fast enough, but in an actually-distributed peer-to-peer system you'll get real collisions fast.
- prg318 14y agoThis is a really neat idea! It would be nice to see the editor widget be re-sizable (like an HTML text area) so that you don't have to deal with scroll bars for larger portions of text. Pretty impressive though!
- mikelehen 14y agoThis should be a pretty easy thing to add, and it's all open source... :-)
- joeblau 14y agoYou guys are awesome! I love your hot sauce!
- Johnyma22 14y agoCheck out the Etherpad plugins, stuff like showing cursor position is available http://beta.etherpad.org http://beta.etherpad.org has it enabled for example.. Etherpad is extendable and does support code editing with syntax highlighting, you just have to jump into /admin/plugins and click "install" on whatever plugin you want :)
- glavata 14y agoLoved etherpad before, and still do :)
- civilian 14y agoI think there's a bug with people's text cursors writing over eachother. http://imgur.com/1IvMccd http://imgur.com/1IvMccd
- mikelehen 14y agoCan you provide more detail? Here or firepad@firebase.com or as a github issue? From the screenshot it's not immediately obvious to me what went wrong. People are certainly allowed to write over each other (e.g click in the middle of your sentence and write something). If that's not what happened, let me know
- saurik 14y agoSo, as with most things built with Firebase, I have to ask how the security works. Last I talked to the Firebase team, they were building expression-only rules for managing server-side validation. This allows you to express some reasonable subset of permissions, but not all possible ones. In this case, the OT history for the document (required to synchronize clients) is stored in Firebase (with each op being a separate object with a massive ID, I imagine this will become awkward with large numbers of old documents, but I digress). Additionally, snapshots are occasionally stored. Rather better than previous offerings I've seen using Firebase, this demonstration has been put together to solve the first few obvious problems: I am not allowed to dump the set of documents[1], nor am I allowed to arbitrarily corrupt the database by deleting random objects[2]. So far, so good. [1]: https://news.ycombinator.com/item?id=4780495 https://news.ycombinator.com/item?id=4780495 [2]: https://news.ycombinator.com/item?id=3824775 https://news.ycombinator.com/item?id=3824775 A set of example security rules for Firepad is actually provided as part of the GitHub project, so we can do some analysis of the kinds of checks we will need to bypass in order to break this particular demo ;P. (Of course, this just makes us faster, it isn't what makes this possible.) https://github.com/firebase/firepad/blob/master/examples/security/secret-url.json https://github.com/firebase/firepad/blob/master/examples/sec... https://github.com/firebase/firepad/blob/master/examples/security/validate-auth.json https://github.com/firebase/firepad/blob/master/examples/sec... Reading these, it turns out that the only verification that is being done on the snapshots is that 1) they look reasonably valid (have the correct set of fields) and 2) they have the correct author field associated with them (as in, the same one that is used on the history revision item). However, it doesn't do any consistency checks on the data itself. It doesn't even verify that the snapshot we are uploading is different than the one currently on the server, so the problem of corrupting the state is really easy: we just need to pull the current snapshot and modify its data. The only pain we could run into is that it could also verify that the author of the snapshot is the current user; but that doesn't help: all we need to do is to make a quick edit to the document and then use our new revision (which we legitimately own) to build our new corrupted snapshot. That said, while that check is present in the example rules on GitHub, that check isn't actually used in the deployed copy of Firepad on this server as this server is entirely anonymous and thereby none of the users have any auth information at all... we can just pretend to be other users. For users who wish to follow along at home, you just need to have node.js installed, and then do "npm install firebase". You can then use the following script to destroy any document you want: you just need to set the "room" variable to the ID # of the document you want to modify. #!/usr/bin/nodejs var Firebase = require('firebase'); // parseInt(window.location.hash.substring(1)) var room = 44; var shard = room % 15; var db = new Firebase('https://firebase-firepad' + shard + '.firebaseio.com/' + room); var check = db.child('checkpoint'); check.once('value', function (value) { value = value.val(); check.set({ a: value.a, id: value.id, o: [''], // random data would be better // but I'm both lazy and busy today ;P }); }); When the client then restores this snapshot and attempts to play back the resulting history to "catch up" it will instead end up outputting tons of errors to the console, as the operations stored in the history will be referring to document positions that no longer exist or are different. Firepad: Invalid operation. https://firebase-firepad14.firebaseio.com/44 C2GK The client then has two options: it can either skip the history entry or it can decide the entire document is corrupt. In this case, it seems that Firebase believed the better of the two options was to simply skip these operations: new clients then manage to resync their state. But, existing clients now have desynchronized state, so all operations that are being synchronized live between the various clients on either side of this split (ones that started from this snapshot, and ones that pre-existed it) are going to result in this error; that didn't really help. To be very explicit for a second: this is a different scenario than just "well, its an anonymous system, so anyone can delete the data": we didn't just go in and delete the data in the document, we actually corrupted the state of the document, rendering further attempts to edit it useless. It is currently my belief that Firebase's security rules system is simply not powerful enough to secure an OT-based text editor, whether or not it uses snapshots (at least assuming it supports offline; there might be tricks you can play if all users are required to be online at all times). (edit: I am finding it interesting that my previous security analyses of Firebase projects, combined with example code, had been voted up quite high, and this one has now been downvoted to 0. I wonder if people just don't care as much about security anymore? Is it because it is open source? Is the Firebase team themselves going around voting down? ;P)