6 ms·
hi I am trying very hard not to be dumb, but this is beating me seemingly Kanso? and Hoodie both have JS stubs at the client and server end, these synch with
by lifeisstillgood 14y ago
hi
I am trying very hard not to be dumb, but this is beating me
seemingly Kanso? and Hoodie both have JS stubs at the client and server end, these synch with each other, then synch with whatever you are using at client (say backbone) and server (node? CouchDB? My Postgres server behind my python app?)
If thats roughly right, why do I do this? I control my server apps no? Why wrap them in another layer? Json->backbone is hardly a difficult synch operation - I mean its called synch().
I really am interested in understanding the philosophy and driving ideas here - everyone has a moment of clarity where they can see how to build an app or a framework, and how it will come together - I am not attacking you (yet !) - please describe your moment of clarity.
- janl 14y agop2p sync is actually a hard problem, you might want to look into the theory :) The value of Hoodie is putting everything together into a way that a jQuery / backbone dev can be productive with. It sounds like you are way more advanced, as such, you will have a way easier time managing all the pieces. > Why? Because we like building apps that way :)
- lifeisstillgood 14y agopeer-to-peer synch? So this is not client server? But browser to browser? WebRTC-enabled? Much cooler. And not even eventually consistent. Please point me at theories :-) jQuery / backbone dev can be productive with I am not clear on that - is that "someone who does not / cannot develop on a tradiational server backend (LAMP etc). How does Hoodie help them? How do you handle network downtime or two clients changing one backend resource with downtime? Why? - that was more what are the benefits to me as a LAMP / backbone dev to using Hoodie, as opposed to are you enjoying yourself, which seems evident :-)
- janl 14y agothis is many-client-server. p2p sync enables you opening your Hoodie app on your browser, smartphone and tablet at the same time and make it all work, even when some or all of the devices are offline for short or long periods of time :) > I am not clear on that - is that "someone who does not / cannot develop on a tradiational server backend (LAMP etc). How does Hoodie help them $ hoodie new app $ cd app $ jitsu create $ jitsu push (simplified) > How do you handle network downtime? Waiting. > or two clients changing one backend resource with downtime? p2p sync with conflict detection as provided by CouchDB. > that was more what are the benefits to me as a LAMP / backbone dev to using Hoodie, as opposed to are you enjoying yourself, which seems evident :-) While you are capable of doing all the heavy lifting, why not focus on building great apps? Or if your thing is running backends for people who build the apps, why not give them Hoodie and take the day off? :)
- tarabukka 14y ago> p2p sync enables you... That isn't what "peer to peer" means, though. Peer to peer means that peers (the aforementioned browser, smartphone and tablet) communicate directly with each other. Also, how does CouchDB keep things secure if clients can sync apparently any data? I'm assuming there's something there, but nobody else has explained this.
- Lazare 14y agoThe trick is twofold: First, CouchDB supports security and validation. Second, for CouchDB replication is just another client connecting to a DB; it's handled the same way anything else is. As a result, we have the notion of filtered replication. The server-side CouchDB won't tell anyone secrets they don't need to know - be they clients or client-side DBs. At at the other end, the client-side CouchDB has some validation to stop "bad" data going into the database by accident, and then the server-side CouchDB has the same validation again. People can compromise their client DB all they want, but eventually it all ends up on the wire as plain old HTTP requests interacting with a plain old REST API. In many ways - as far as security goes - the client DB is a red herring. The server ONLY speaks a well-defined REST API, and has validation and security to deal with malformed or malicious API requests. The fact that those requests are generated by a client-side DB based on data entered into it via JS commands is neither here nor there; a Backbone app would generate the exact same requests based on more-or-less the same JS commands. If you can secure any REST API, you can secure CouchDB.
- janl 14y agoYet again, spot on, thanks @Lazare!
- nborwankar 14y agoHi Jan, another way to think about this (from the old days) is that disconnected mode data is a given because the rate of growth of data is far greater than the rate of growth of bandwidth. So everyone can't always be accessing all their data in the cloud through an always on infinite bandwidth connection. So you're going to need to carry a subset of your data with you - the subset that's "hot". Now having said that keeping hot data in sync with the larger data store is a hard problem - p2p or not. And what these architectures (CouchDB, PouchDB, ...) do is take the sync problem and make it an infrastructure issue so it only has to be solved once by the infrastructure creator (CouchDB team) rather than again and again and again by each app developer who wakes up in the middle of the night and realizes they have to solve "sync" in their app as an application problem. Then they have a nightmare and when they wake up they are babbling. The additional anti-pattern-badness with sync-in-the-app is that that kind of sync is usually incompatible with another app's bespoke sync. But when sync is in the infrastructure then a much larger group of people who use that infrastructure can share data across apps if they want to without having necessarily to share a schema. This is the real power of making sync part of the underlying computing fabric. Finally when this infrastructure is open source the real -danger of lock-in such as when you use Dropbox is mitigated, should you want to have that freedom. Cheers.
- Lazare 14y agoOkay, first off, let's discuss a standard three tier webapp architecture: Browser <---> Server <---> DB You get some HTML/CSS/JS from the server, you display stuff to the user. As the user interacts with your app, HTTP requests hit your server which processes them. As needed, the server will make requests to the DB, which will reply with data, which then gets passed back to the browser. Everyone knows how this stack works, I hope. :) It's simple and it justs works. Almost every website you use works this way. But for webapps, it's not always ideal. Let's discuss some of the issues: First off, this architecture is "always online". If the browser loses connection to the server, then when your user clicks on a button nothing will happen. Not too bad in a desktop environment, but with laptops or - especially - mobile clients this isn't ideal. Worse, what if you're trying to keep the state in sync with multiple clients (like Trello does)? You're probably using websockets or long polling or similar techniques so that the server can tell clients about the actions of other clients. A network hiccup might cause you to miss one of these updates; detecting and recovering from this sort of sync error is problematic. Generally the solution is to throw away the last few bits of work the user did, and force a full reload and re-fetch the current state. Which is fine, in most cases. Unless the work the user is difficult to replicate, or if the application state is quite large. Trello takes a second or two to grab the current cards; if your app takes 30s to grab its data, you may want to avoid this though. :) In fact, the more you want to replicate the feel and functionality of a classic desktop app, while allowing multiple users and clients, the more you start to struggle against this architecture. What other architectures are there? Well, there's one obvious one: Browser <---> CouchDB <---> Workers CouchDB is kind of unique among databases because it speaks HTTP natively. It can serve your app to the client, and then turn around and have the clients connect directly to it. It can't, eg, send emails, but what you can do is use CouchDB as a queue. Add a document that says "send an email to X", then you spin up some worker processes that monitor it and process any documents like that as they appear. CouchDB also has a _changes feed that makes a lot of sync tasks very easy. It also does master/master replication, and is generally a really nice database to work with from a devops point of view. Still, we're missing a crucial bit of magic to really improve over a classic three tier webapp. And that is... ...local DBs, in the form of PouchDB for browsers or TouchDB (apparently called Couchbase Mobile these days, although confusingly it's unrelated to Couchbase, which in turn is unrelated to CouchDB. Don't ask.) for Android/iOS. CouchDB is really really good at syncing and replicating, so now we have a new architecture: Browser/Mobile App <--> PouchDB/TouchDB <--> CouchDB <--> Workers This isn't appropriate for every task. For one thing, it's quite a lot more work to implement (partly due to inferior tooling which projects like Hoodie are, thankfully, addressing). But if you really want the best possible user experience, you want it to work even if your on an airplane or in a tunnel, you want a really snappy UI, you want each client to see the changes made by other clients in as close to real time as practical, you want a consistent experience for native mobile apps and webapps running in a browser...then this is the architecture for you. So to directly answer your questions: First, Kanso (by default) wants to do something like: Browser <--> CouchDB Which is a really slick, really easy way to do single page applications that don't need local persistence or server processes. But it's not hard to add them on, in which case you have the architecture above. You seem to be imagining something like this: Hoodie Client <--> Hoodie Server <--> CouchDB But that's not the idea at all. You're also asking why not do this: Backbone <---> Python <---> Postgres And that's a perfectly fine architecture. It works, it's rock solid, and it's easy to code. For many apps (maybe most) it's the best possible architecture (well, I'd suggest Knockout over Backbone, but that's personal preference). But for the project I'm working on right now, we decided that wasn't quite good enough. :)
- espylaub 14y ago"Moment of clarity" is a great cue. To me, Hoodie is enormously empowering (which is why I'm part of the Hoodie team). As a self-taught, non comp-sci frontend dev with a humanities degree I can get all the backend/DB stuff sorted, definitely, but I'd actually rather not. It's difficult and occasionally arcane. I don't enjoy it. And, let's face it, I can make terrible mistakes. With Hoodie, I can quickly build apps by myself, on a solid, proven base, with super-fast setup, easy deployment, offline capability and syncing. And these last two don't even need to be actively invoked, they're just there and they just work. That's practically magical from where I stand. Hoodie allows me to build things in days that would usually take me weeks and in some cases require additional people. This gives me more time to experiment, refine, user test etc. It also lets me delegate with ease: I don't want or need full control and responsibility over the server and the db and many security aspects, not for what I do. I'll happily pay nodejitsu and iriscouch for that, because they know more about this stuff than I could ever learn in the time it takes me to make the money to pay them. That was my moment of clarity: Hoodie lets me get on with what I want to do and what I'm good at, and it's really got my back on many of the things I have neither the skill nor the patience nor the money for.