6 ms·

Show HN: Crudlet.js – A common, streamable interface for synchronizing data

94 points by knes 12y ago

namuol 12y ago
What problem is this solving?
- crcn 12y ago
  I'm a front-end developer, and there's lots of service features to think about when building a single page app: Should app data be generated from the server initially? Should the app support offline read-only mode? Should the app support realtime data? Should operations be batched and sent to a data source? Should the app be isomorphic? And others. I built crudlet to help encapsulate services so that these sorta features can be added anytime without affecting the rest of the application. That, and also to help decouple applications from a particular service. For instance, it'd be pretty easy to swap firebase with parse, or socket.io for pubnub if you want.
egeozcan 12y ago
This looks great. I think it deserves more examples though. Looking again, I see from the commits that you've already been working on the docs. Again, great! I may give this a try in my toy project. Edit: I'm currently using Baobab[1] to manage state in a React app and I'm thinking about writing an adapter for it. I wonder if that makes sense. I'm leaning towards "yes". [1]: https://github.com/Yomguithereal/baobab https://github.com/Yomguithereal/baobab
roneesh 12y ago
Echoing other comments, this does look pretty cool. Would appreciate a comment from OP on what the use case is with this library? Maybe an example app or ideal use case?
- crcn 12y ago
  Author here - the primary motivation behind this library was to make APIs interoperable with other data stores. For example: https://gist.github.com/crcn/be352008bca201e8ac00 https://gist.github.com/crcn/be352008bca201e8ac00 Currently working on more examples - didn't realize this would appear on HN so quickly!
markthethomas 12y ago
Looks awesome! Agreed w/below that more usage examples would be great!
coderzach 12y ago
It would be awesome if it persisted all the operations to a log. That way, when I attach a new endpoint I could get it "caught up". And then, assuming your handlers are idempotent, running db migrations or adding a new representation of your data would be as easy as attaching a modified handler and running through the log from the beginning.
- jchrisa 12y ago
  Sounds like you might want http://pouchdb.com/ http://pouchdb.com/ which has offline support including conflict management.
- crcn 12y ago
  Shouldn't be too difficult to implement. Here's one possible way to do it: https://gist.github.com/crcn/cac5b5826590abed921d https://gist.github.com/crcn/cac5b5826590abed921d
ollysb 12y ago
There seems to be a lot of overlap with orbit.js[1], how do they compare? [1] https://github.com/orbitjs/orbit.js https://github.com/orbitjs/orbit.js
- crcn 12y ago
  They definitely solve similar problems. Bigger difference is that crudlet acts more like a utility belt library for interacting with service adapters that support the same operations. You can really do just about anything you want. For example, here's crudlet used as an event bus system: https://gist.github.com/crcn/cfe215ab3b604fb0b441 https://gist.github.com/crcn/cfe215ab3b604fb0b441 Crudlet also uses node streams instead of promises.
rvdm 12y ago
I've been working on some offline first real time collaboration apps, will definitely give Crudlet a try. Maybe promote the existing Todo example to the site of the repo so it's easily accessible for first time visitors?
marknadal 12y ago
Great work, just a warning: your http://crudlet.herokuapp.com/#/live-todos http://crudlet.herokuapp.com/#/live-todos example gets stuck in an infinite loop of inserting whatever todo item. Do you have any other demo examples? I work on data synchronization problems as well (I'm the guy who built http://gunDB.io http://gunDB.io) I'd love to talk. I assume you are using OT? OT requires a "master" server, right - or do you handle decentralized conflicts in other ways? I'm glad you already have a query doc working, I'm still adding such a feature. Let's chat, send me an email: mark@gunDB.io . Good work!
- crcn 12y ago
  Thanks for the heads up - the todos example should be fixed. Right now the todos example is pretty simple - it just hooks up with a socket.io (master) server which re-broadcasts all operations to each connected client.
redindian75 12y ago
Is this something similar to what Meteor[1] does? [1] http://www.meteor.com http://www.meteor.com

94 points by knes 12y ago

namuol 12y ago

What problem is this solving?

crcn 12y ago
I'm a front-end developer, and there's lots of service features to think about when building a single page app: Should app data be generated from the server initially? Should the app support offline read-only mode? Should the app support realtime data? Should operations be batched and sent to a data source? Should the app be isomorphic? And others. I built crudlet to help encapsulate services so that these sorta features can be added anytime without affecting the rest of the application. That, and also to help decouple applications from a particular service. For instance, it'd be pretty easy to swap firebase with parse, or socket.io for pubnub if you want.

egeozcan 12y ago

This looks great. I think it deserves more examples though. Looking again, I see from the commits that you've already been working on the docs. Again, great! I may give this a try in my toy project. Edit: I'm currently using Baobab[1] to manage state in a React app and I'm thinking about writing an adapter for it. I wonder if that makes sense. I'm leaning towards "yes". [1]: https://github.com/Yomguithereal/baobab https://github.com/Yomguithereal/baobab

roneesh 12y ago

Echoing other comments, this does look pretty cool. Would appreciate a comment from OP on what the use case is with this library? Maybe an example app or ideal use case?

crcn 12y ago
Author here - the primary motivation behind this library was to make APIs interoperable with other data stores. For example: https://gist.github.com/crcn/be352008bca201e8ac00 https://gist.github.com/crcn/be352008bca201e8ac00 Currently working on more examples - didn't realize this would appear on HN so quickly!

markthethomas 12y ago

Looks awesome! Agreed w/below that more usage examples would be great!

coderzach 12y ago

It would be awesome if it persisted all the operations to a log. That way, when I attach a new endpoint I could get it "caught up". And then, assuming your handlers are idempotent, running db migrations or adding a new representation of your data would be as easy as attaching a modified handler and running through the log from the beginning.

jchrisa 12y ago
Sounds like you might want http://pouchdb.com/ http://pouchdb.com/ which has offline support including conflict management.
crcn 12y ago
Shouldn't be too difficult to implement. Here's one possible way to do it: https://gist.github.com/crcn/cac5b5826590abed921d https://gist.github.com/crcn/cac5b5826590abed921d

ollysb 12y ago

There seems to be a lot of overlap with orbit.js[1], how do they compare? [1] https://github.com/orbitjs/orbit.js https://github.com/orbitjs/orbit.js

crcn 12y ago
They definitely solve similar problems. Bigger difference is that crudlet acts more like a utility belt library for interacting with service adapters that support the same operations. You can really do just about anything you want. For example, here's crudlet used as an event bus system: https://gist.github.com/crcn/cfe215ab3b604fb0b441 https://gist.github.com/crcn/cfe215ab3b604fb0b441 Crudlet also uses node streams instead of promises.

rvdm 12y ago

I've been working on some offline first real time collaboration apps, will definitely give Crudlet a try. Maybe promote the existing Todo example to the site of the repo so it's easily accessible for first time visitors?

marknadal 12y ago

Great work, just a warning: your http://crudlet.herokuapp.com/#/live-todos http://crudlet.herokuapp.com/#/live-todos example gets stuck in an infinite loop of inserting whatever todo item. Do you have any other demo examples? I work on data synchronization problems as well (I'm the guy who built http://gunDB.io http://gunDB.io) I'd love to talk. I assume you are using OT? OT requires a "master" server, right - or do you handle decentralized conflicts in other ways? I'm glad you already have a query doc working, I'm still adding such a feature. Let's chat, send me an email: mark@gunDB.io . Good work!

crcn 12y ago
Thanks for the heads up - the todos example should be fixed. Right now the todos example is pretty simple - it just hooks up with a socket.io (master) server which re-broadcasts all operations to each connected client.

redindian75 12y ago

Is this something similar to what Meteor[1] does? [1] http://www.meteor.com http://www.meteor.com