10 ms·

PouchDB, the JavaScript Database That Syncs

321 points by _qc3o 10y ago

mike-cardwell 10y ago
Anyone use this system in production? Care to share your experiences?
- onion2k 10y ago
  I'm using it in a Chrome extension (which is technically in production but only has one user, me). It's excellent. It took a bit of work to get my head around what _rev means, and I'm not using the sync functionality yet (but I will, hence using it now) but I've not found it to be at all problematic. I'd definitely recommend using it if it fits your project.
  - basic1_0 10y ago
    that isn't production :(
- seeekr 10y ago
  Have been using it for a long time inside a Chrome extension. The use case was to keep user's preferences and data/history stored on both client and server and in sync with each other easily and PouchDB accomplished that task superbly. Was a pleasure to use, no issues that I can remember. Would use it again for similar use cases.
- fredguth 10y ago
  We have been using it in an Ember app and it was very easy to setup. There is an Ember Data adapter that works well. Main advantage is to have an app that works offline and all sync happening under the hood. If you are going to give a try, highly recommend watching Nolan Lawson's videos in YouTube.
- rglullis 10y ago
  The current project I am working on has been developing an offline-first, messaging/calling app that started as a webapp but was decided to also have native apps for Android/iOS. But because the web app came first, the web-centered architecture was kept, and the javascript developers have chosen PouchDB/CouchDB as the single data storage/sync mechanism. I was peripherally involved in the part of the services that dealt with user data - basically, massaging data from the address book and call history, dealing with the backend. This is a collection of things I believe I can say after working on this for almost two years: - PouchDB + CouchDB works well, as long as you are already used to model your application in terms of CouchApps. No SQL, no E-R representation of your data, etc. If you are not comfortable with the couchDB model and don't know your way around views, you are not going to have a good time. - This is not an issue with PouchDB, but it if you are planning on having native apps, I'd strongly advise against. All of our native app developers struggled with the change in mindset and the not so mature state of couchbase for Android/iOS. And because they can use SQLite, forcing them to adopt couchbase was not but pain in our team. - You need to figure out security and authentication: if you take the direct approach of using one couchdb per user and just replicate that, you need to figure out on your own how to secure access. CouchDB only supports OAuth 1 out of the box. We are using oAuth 2 for our services, and I basically had to implement a proxy server that checked oauth tokens before passing requests to our couchdb server. - CouchDB that does not allow cross-database views, so if you take the "one-database-per-user" approach and you need to check anything that spans more than one database, you are on your own to create more databases/replicate/index/aggregate the data. - The solution for sync helps, and continuous replication makes for a good demo, but it is not magic. Imagine if you have already accumulated some good amount of data in your database. The moment you start your application on a different browser, PouchDB will start desperately to sync everything you have. You need to know how filtered replication works. - Very resource intensive, more so if you keep continuous replication. Constant usage of CPU and network I/O can make your app feel sluggish.
  - nolanl 10y ago
    (PouchDB contributor here.) Just a quick note that slow replication is one of the things that is largely fixed in CouchDB 2.0 thanks to the new _bulk_get API (https://issues.apache.org/jira/browse/COUCHDB-2310 https://issues.apache.org/jira/browse/COUCHDB-2310) and is set to get even better in CouchDB 3.0 once it has HTTP 2. As for your app feeling sluggish, my hunch is that you're seeing this in a Chromium or Gecko browser (e.g. Android WebView) in which cases IndexedDB does quite a lot of heavy operations on the UI thread (http://nolanlawson.com/2015/09/29/indexeddb-websql-localstorage-what-blocks-the-dom/ http://nolanlawson.com/2015/09/29/indexeddb-websql-localstor...), which can be mitigated by moving it to a Web Worker (as Pokedex.org does) or a Service Worker (as HospitalRun.io does).
  - skruger 10y ago
    Check out Cloudant Envoy to get around the "one-database-per-user" anti-pattern: https://github.com/cloudant-labs/envoy https://github.com/cloudant-labs/envoy
krrishd 10y ago
My bad if the answer is obvious but (besides backward compatibility/shims + abstraction), what's the benefit of using PouchDB as opposed to vanilla localStorage functionality?
- pokstad 10y ago
  It can sync data between CouchDB servers (e.g. IBM Cloudant) and the browser.
  - erikb 10y ago
    Honestly I didn't know that this is a feature people want (no criticism intended). So, you have a database object in your browser and that takes care of getting the data to the server himself? Otherwise every website syncs data between its browser instances and its database, right? That's how we get state.
    - daleharvey 10y ago
      Using a synced data model provides a few features for users, data access is efficient and far far faster (disk/memory is faster than network) and it allows offline usage (+ failure tolerance, the server goes down the client still works). It can also enable p2p use cases and a bunch of other things, the above 2 are the main drivers though
    - candeira 10y ago
      The sync happens in an unmanaged fashion, without the user or the application programmer having to care about the state of the connection. Your PouchDB application works locally on your device, whether the connection is up and down, and the data is synched with the remote database whenever there is connection. The alternatives to this PouchDB to CouchDB synching mechanism would have to be either: - the user checks whether the connection is up, and manually manages the sync, or - the application programmer saves the user the trouble by adding code that checks whether the connection is up, and automatically manages the sync
- daleharvey 10y ago
awjr 10y ago
For one of the best uses of PouchDB that blew my mind when I read about it, read this http://www.pocketjavascript.com/blog/2015/11/23/introducing-pokedex-org http://www.pocketjavascript.com/blog/2015/11/23/introducing-...
- awjr 10y ago
  The article I linked has been discussed on here before https://news.ycombinator.com/item?id=10619933 https://news.ycombinator.com/item?id=10619933
cyberferret 10y ago
Been experimenting with PouchDB for a while and really liking the simplicity of the project. Looking to implement it in a mobile hybrid app to allow users to take data 'offline' then sync up later. The only downside I've found so far is that the PouchDB Inspector on my Chrome browser tends to go rogue from time to time and suck up > 50% of the CPU time and has to be shutdown manually.
dmihal 10y ago
Cool library! I'm a huge fan of Meteor.js, which provides a similar isomorphic database. Cool to see this implemented in such a lightweight package!
daleharvey 10y ago
Surprised to see this on HN, but I am one of the maintainers of this (I expect Nolan will end up finding this too) so happy to answer any questions about it.
- grizzles 10y ago
  Is the project sponsored by Couchbase? If not, have you thought about publishing a storage API that the replication protocol calls so that it could be adapted to work with any backend db? What about P2P sync - (eg. serverless) has anyone thought of making PouchDB do that?
  - daleharvey 10y ago
    Not sponsored by Couchbase at all (I worked at Couchbase during its formation, this started near the end of my time there). For pluggable storage engines our node.js adapter uses leveldown, so any *down backend can be plugged in. https://pouchdb.com/adapters.html https://pouchdb.com/adapters.html has some more information about this. For p2p yup its been something quite a few people have been using pouchdb for, one of the more notable examples has been http://thaliproject.org/ http://thaliproject.org/, the core storage format is entirely compatible with p2p (inherited from couchdb)
  - davidbanham 10y ago
    The idea of making pouch sync with other databases is covered in their faq: https://pouchdb.com/faq.html#sync_non_couchdb https://pouchdb.com/faq.html#sync_non_couchdb Basically, your other data base probably handles conflicts in a different way to Couch that would make syncing this way nonsensical. If you're happy to handle that yourself, the Couch replication protocol is well documented and there are plenty of libraries written for it.
  - fredguth 10y ago
    AFAIK, Couchbase is a CouchDB as a service tuned for speed. PouchDB is a CouchDB implementation in the browser. I don't think your sync with other db would be easy and I believe there is no such desire in the project roadmap.
PaulMest 10y ago
I've been using PouchDB in a React Native app for about 6 months with SQLite as its backing store so that we can use more than 50MB of storage on the device. It has been working pretty well to persist data into an offline cache and then sync to a CouchDB 2.0 database in Digital Ocean. Getting it to work inside React Native was initially very challenging. Keeping our shim up to date with the recent changes to PouchDB has also been challenging. We are currently using PouchDB 5.4.5 because there was a breaking change in 6.x and I haven't had a chance to dive into it to figure out what is going wrong. The PouchDB community (especially Nolan Lawson) does a great job of showing examples, answering questions, and responding to feedback.
- m_mueller 10y ago
  Did you also consider Couchbase Lite? I'd be interested in your decision making.
  - davidbanham 10y ago
    I've used pouch on a few projects and also attempted to use cb lite. I found pouch dramatically easier to get up and running with. Cb lite was complicated since it needs a native plugin rather than being just js. That wasn't too bad. The thing that put the nail in it for me was all the setup on the server side with sync gateways, multiple databases that then sync with one another, etc etc.
    - m_mueller 10y ago
      CBL has worked OK so far for us with just plain CouchDB serverside, no sync gateways. But I can see where you're coming from, if your stack is centered around JS I think pouchDB makes more sense. We have potentially large DBs (depends on usecase) where browser context isn't enough and/or provides a bad user experience.
      - davidbanham 10y ago
        Oh so you're using CBL with a couchdb backend? I didn't even realise that was possible. I thought it only synced with Couchbase which is where all the complexity came in.
daurnimator 10y ago
I really love the project, but I never used it in production due to the large file size :(
- fiatjaf 10y ago
  Your solution has arrived: https://pouchdb.com/custom.html https://pouchdb.com/custom.html
- mungoid 10y ago
  I just now learned about this DB and was intrigued but havent researched much about the cons yet, so I am curious to hear your thoughts about the file size?
rahilsondhi 10y ago
I'm interested in PouchDB to make my JavaScript app easily sync to the server, but I don't want to switch my server's database from Postgres to CouchDB. Surely I'm not the only one in this situation?
- daleharvey 10y ago
  You aren't, its mentioned in our faq - https://pouchdb.com/faq.html#sync_non_couchdb https://pouchdb.com/faq.html#sync_non_couchdb. A lot of people would like their current data to just be able to sync, but it almost always needs changes in the way data is stored and complementary changes to the application code
- lacampbell 10y ago
  Any experience doing it the hard way? I'm in the same boat. Really don't want to throw away what postgres provides.
cherryblakes 10y ago
Hello guys and how are you doing today, Do you need require the services of an accomplished and professional hacker for any of the following: -Facebook account hacking -Instagram account hacking -Mobile phone hacking -Whatsapp hacking -Kik account hacking -GPS tracking -Website hacking -Bank account hacking -Account recovery -DDOS attack -Email account hacking Then you should contact:danielwellingtonhacks@gmail.com for his professional services. He is one of the best hackers out there.
ioquatix 10y ago
Since when did arguments become functions? db.replicate.to('http://example.com/mydb'); IMHO, should be db.replicate(to: 'http://example.com/mydb');
- Spivak 10y ago
  I'm personally really a fan of this syntax mostly because I like how it reads when you chain them together. db.replicate .from('http://example.com/mydb1') .to('http://example.com/mydb2') .on('complete', function() { }) .on('error', function() { }) .then(function() { });
  - Mouse47 10y ago
    It's not clear which 'arguments' are required when it's done this way. What's stopping you from calling 'from' without 'to'?
    - BinaryIdiot 10y ago
      I think it would just not do anything. I like this type of syntax but only when the required parameters are done up front. Then all chainable are optional / modifiers / events. This is how jQuery works and how I modeled msngr.js. Unless Pouch has anything required missing I think it's fine though maybe a little unintuitive (like can you do multiple froms? Multiple tos? Etc)
  - zepolen 10y ago
    What happens when you do: db.replicate .from('http://example.com/mydb1') or db.replicate .from('http://example.com/mydb1') .from('http://example.com/mydb2') .to('http://example.com/mydb3') or db.replicate .from('http://example.com/mydb1') .to('http://example.com/mydb2') .to('http://example.com/mydb3') ? serious question
    - azr89 10y ago
slashblake 10y ago
Yeah fuck REST!!
ojanik 10y ago
We've been running PouchDB in production for ~15 months now. We chose it because it was a greenfield project and it gave us 2 things: Easy offline support and real-time syncing that makes it easy to create collaboration a-la Google Docs. Because the entire thing is a web app with app cache manifest deploying new versions is very little hassle. In terms of architecture we have about 250 tenants with separate Couch databases per each. We're still running Couch 1.6. We have yet to evaluate Couch 2.0. It's been mostly smooth ride for the most part but this being a very unusual architecture we had to tackle few interesting problems that came along. 1. Load times. Once you get over certain db size the initial load time from clean slate takes ages due to PouchDB being super chatty. I'm talking about 15-30 mins to do initial sync of 20-30mb database. We had to resort to pouch-dump to produce dump files periodically. That helped a lot. I think this issue has been rectified with Couch 2.0 and sync protocol update. 2. Browser limits. Once we hit the inherent capacity of some browsers (namely Safari on iOS, 50mb) we had to get creative. Now we're running 2 CouchDB databases for each tenant where 1 has full data and the other only contains last 7-8 days. Pouch syncs to the latter one. We run filtered replications between the full db and the reduced db and do periodic purging. On the client side if a customer tries to go back more than 7 days we just use the Pouch in online only mode where it acts as a client library to remote couch and doesn't sync locally. 3. Dealing with conflicts. This might matter or it might not depending on the domain but you have to be aware of data conflicts. Because CouchDB/PouchDb is eventually consistent multi-master setup and you will get data conflicts where people update the same entity based on the same source revision. PouchDB has nice hooks to let you deal with this but you have to architect for it. 4. Custom back-end logic. Because Pouch talks directly to Couch you can't exactly execute custom back-end logic when needed. We had to introduce a REST back-channel to make sure our back-end runs extra logic when needed. 5. We had some nasty one-off surprises. Last one was with an object that had 1700 or so revisions in couch and once it synced to PouchDB it would crash the Chrome tab in a matter of seconds. Due to the way PouchDB stores revision tree (lot's of nested arrays) Chrome would choke during JSON.parse() call and eat up memory until crash. We resolved this one by reducing the revision history limit that is kept.
- deleted 10y ago
  [deleted]
- 10y ago
azr89 10y ago
We've been using Pouch in a progressive web app designed to be used on the field in remote locations, and while there was a learning curve in understanding how the replication protocol works, and as highlighted in another comment the way Chrome stores data for a web app - we can't be happier with pouch/couch. Additionally, moving out of Cloudant and into CouchDB with an openresty based reverse proxy has made things even better, and really fun. This is one of those stacks that feels easy and simple at the same time. (Ref:https://www.infoq.com/presentations/Simple-Made-Easy https://www.infoq.com/presentations/Simple-Made-Easy).
- karmelapple 10y ago
  Any guidance on moving from Cloudant to CouchDB? Are you hosting it yourself? If so, has the amount of maintenance been more than you expected, or was it mostly setup time and then forget about it?
  - azr89 10y ago
    Yup, hosting it ourselves. Its a peach. There are few things that it doesnt come with out of the box - clustering, Full text search, geoindexing, chained map reduce, auto compaction, index auto-updation. Once thats done, if anything it was more forget about it than Cloudant, which bills on requests / thoroughput. This can catch you out because continuous replications between databases on the same cloudant account are also counted as requests and billed as such. And continuous replication is very chatty. So if you have a particularly creative multi-master setup, like a per user db -> masterdb kind of thing going, this can eat up your thoroughput / push up your bills with no practical benefit. Its really openresty + couch that does it for me. The idea of writing security / validations / routing etc right into ngnix combines beautifully with the CouchDB way of thinking.
    - kocolosk 10y ago
      Ah, yeah, you weren't the only one bitten by that. We actually went and changed the Cloudant metering model recently so that you're billed on provisioned throughput rather than total request volume. You get dramatically more predictable billing, with the tradeoff that clients need to handle a 429 Too Many Requests response if the Cloudant instance is under-sized. More here: https://www.ibm.com/blogs/bluemix/2016/09/new-cloudant-lite-standard-plans-are-live-in-bluemix-public/ https://www.ibm.com/blogs/bluemix/2016/09/new-cloudant-lite-...
crooked-v 10y ago
PouchDB's replication capability is interesting, but is there a way to make it lazy load to the local DB instead of doing everything up front? I hesitate to use it for a web project with 10+ MB of docs where it would otherwise be ideal.
- WorldMaker 10y ago
  You can provide a server-side filter function to replication and progressively filter partial replications until eventually everything gets replicated. At that point it becomes a question of architecture of your documents: how much is needed to replicate before a user may be productive? You can also explore pouchdb-replication-stream to build bundles that PouchDB can bootstrap from a little bit faster than a chatty replication. That said, I've found initial replications of large databases (one I've worked with this week is a 25+ MB CouchDB database full of photos) is quick enough (and mostly bandwidth constrained) that I haven't had much in the way of concern over it.
imtringued 10y ago
Could I in theory host my own pouchdb instance to keep using applications that are no longer supported by their vendors? Imagine an IoT vendor uses pouchdb shuts down. My devices don't work anymore but I can host my own pouchdb instance. A javascript based database is a bit heavy but it's an acceptable tradeoff to me.
- nolanl 10y ago
  In principle yes any CouchDB/PouchDB database can sync with one another because they all share the same protocol, but OTOH I've seen very few examples of apps that offer users a simple "export to your own CouchDB/PouchDB" feature.
aeharding 10y ago
I've been running PouchDB + CouchDB 2.0 in production for a while now (financier.io). It's worked great, and I really recommend you to check it out. A couple things: 1. CouchDB 2.0 is still rough around the edges, particularly with its new Fauxton interface (ex: completely broken when proxied into subfolder). 2. CouchDB 2.0 brings the _bulk_get API which has improved sync by an order of magnitude(s). 3. I do have custom logic for logging in overriding the _session API in order to do rate limiting. (I proxy with nginx for IP rate limiting and node.js for failed password attempts rate limiting.) I also have custom logic for provisioning a new CouchDB database per user and setting up permissions. 4. I host on Digital Ocean but I use their new block storage solution so that a growing db does not become unwieldy/expensive. 5. My SaaS subscription system is kind of unique: When your subscription expires you'll simply lose write access to the server (CouchDB), but you can still pull down your data to PouchDB.
espeed 10y ago
Datomic-ClojureScript in the browser backed by PouchDB would be killer. Datomic already has a CouchDB-compatible datastore via its support for Couchbase. That would mean Datomic-ClojureScript on PouchDB could run in the browser and sync with Datomic-Clojure backed by CouchDB/Couchbase on the the server -- that would be killer and give Datomic massive reach. Has the Datomic team considered PouchDB as a possible datastore for Datomic-ClojureScript in the browser? NB: Just posted the question to the Datomic discussion group: https://groups.google.com/d/msg/datomic/uqBQE4QlnzI/VuZ14pqOBQAJ https://groups.google.com/d/msg/datomic/uqBQE4QlnzI/VuZ14pqO...

321 points by _qc3o 10y ago

mike-cardwell 10y ago

Anyone use this system in production? Care to share your experiences?

onion2k 10y ago
I'm using it in a Chrome extension (which is technically in production but only has one user, me). It's excellent. It took a bit of work to get my head around what _rev means, and I'm not using the sync functionality yet (but I will, hence using it now) but I've not found it to be at all problematic. I'd definitely recommend using it if it fits your project.
- basic1_0 10y ago
  that isn't production :(
seeekr 10y ago
Have been using it for a long time inside a Chrome extension. The use case was to keep user's preferences and data/history stored on both client and server and in sync with each other easily and PouchDB accomplished that task superbly. Was a pleasure to use, no issues that I can remember. Would use it again for similar use cases.
fredguth 10y ago
We have been using it in an Ember app and it was very easy to setup. There is an Ember Data adapter that works well. Main advantage is to have an app that works offline and all sync happening under the hood. If you are going to give a try, highly recommend watching Nolan Lawson's videos in YouTube.
rglullis 10y ago
The current project I am working on has been developing an offline-first, messaging/calling app that started as a webapp but was decided to also have native apps for Android/iOS. But because the web app came first, the web-centered architecture was kept, and the javascript developers have chosen PouchDB/CouchDB as the single data storage/sync mechanism. I was peripherally involved in the part of the services that dealt with user data - basically, massaging data from the address book and call history, dealing with the backend. This is a collection of things I believe I can say after working on this for almost two years: - PouchDB + CouchDB works well, as long as you are already used to model your application in terms of CouchApps. No SQL, no E-R representation of your data, etc. If you are not comfortable with the couchDB model and don't know your way around views, you are not going to have a good time. - This is not an issue with PouchDB, but it if you are planning on having native apps, I'd strongly advise against. All of our native app developers struggled with the change in mindset and the not so mature state of couchbase for Android/iOS. And because they can use SQLite, forcing them to adopt couchbase was not but pain in our team. - You need to figure out security and authentication: if you take the direct approach of using one couchdb per user and just replicate that, you need to figure out on your own how to secure access. CouchDB only supports OAuth 1 out of the box. We are using oAuth 2 for our services, and I basically had to implement a proxy server that checked oauth tokens before passing requests to our couchdb server. - CouchDB that does not allow cross-database views, so if you take the "one-database-per-user" approach and you need to check anything that spans more than one database, you are on your own to create more databases/replicate/index/aggregate the data. - The solution for sync helps, and continuous replication makes for a good demo, but it is not magic. Imagine if you have already accumulated some good amount of data in your database. The moment you start your application on a different browser, PouchDB will start desperately to sync everything you have. You need to know how filtered replication works. - Very resource intensive, more so if you keep continuous replication. Constant usage of CPU and network I/O can make your app feel sluggish.
- nolanl 10y ago
  (PouchDB contributor here.) Just a quick note that slow replication is one of the things that is largely fixed in CouchDB 2.0 thanks to the new _bulk_get API (https://issues.apache.org/jira/browse/COUCHDB-2310 https://issues.apache.org/jira/browse/COUCHDB-2310) and is set to get even better in CouchDB 3.0 once it has HTTP 2. As for your app feeling sluggish, my hunch is that you're seeing this in a Chromium or Gecko browser (e.g. Android WebView) in which cases IndexedDB does quite a lot of heavy operations on the UI thread (http://nolanlawson.com/2015/09/29/indexeddb-websql-localstorage-what-blocks-the-dom/ http://nolanlawson.com/2015/09/29/indexeddb-websql-localstor...), which can be mitigated by moving it to a Web Worker (as Pokedex.org does) or a Service Worker (as HospitalRun.io does).
- skruger 10y ago
  Check out Cloudant Envoy to get around the "one-database-per-user" anti-pattern: https://github.com/cloudant-labs/envoy https://github.com/cloudant-labs/envoy

krrishd 10y ago

My bad if the answer is obvious but (besides backward compatibility/shims + abstraction), what's the benefit of using PouchDB as opposed to vanilla localStorage functionality?

pokstad 10y ago
It can sync data between CouchDB servers (e.g. IBM Cloudant) and the browser.
- erikb 10y ago
  Honestly I didn't know that this is a feature people want (no criticism intended). So, you have a database object in your browser and that takes care of getting the data to the server himself? Otherwise every website syncs data between its browser instances and its database, right? That's how we get state.
  - daleharvey 10y ago
    Using a synced data model provides a few features for users, data access is efficient and far far faster (disk/memory is faster than network) and it allows offline usage (+ failure tolerance, the server goes down the client still works). It can also enable p2p use cases and a bunch of other things, the above 2 are the main drivers though
  - candeira 10y ago
    The sync happens in an unmanaged fashion, without the user or the application programmer having to care about the state of the connection. Your PouchDB application works locally on your device, whether the connection is up and down, and the data is synched with the remote database whenever there is connection. The alternatives to this PouchDB to CouchDB synching mechanism would have to be either: - the user checks whether the connection is up, and manually manages the sync, or - the application programmer saves the user the trouble by adding code that checks whether the connection is up, and automatically manages the sync
daleharvey 10y ago

awjr 10y ago

For one of the best uses of PouchDB that blew my mind when I read about it, read this http://www.pocketjavascript.com/blog/2015/11/23/introducing-pokedex-org http://www.pocketjavascript.com/blog/2015/11/23/introducing-...

awjr 10y ago
The article I linked has been discussed on here before https://news.ycombinator.com/item?id=10619933 https://news.ycombinator.com/item?id=10619933

cyberferret 10y ago

Been experimenting with PouchDB for a while and really liking the simplicity of the project. Looking to implement it in a mobile hybrid app to allow users to take data 'offline' then sync up later. The only downside I've found so far is that the PouchDB Inspector on my Chrome browser tends to go rogue from time to time and suck up > 50% of the CPU time and has to be shutdown manually.

dmihal 10y ago

Cool library! I'm a huge fan of Meteor.js, which provides a similar isomorphic database. Cool to see this implemented in such a lightweight package!

daleharvey 10y ago

Surprised to see this on HN, but I am one of the maintainers of this (I expect Nolan will end up finding this too) so happy to answer any questions about it.

grizzles 10y ago
Is the project sponsored by Couchbase? If not, have you thought about publishing a storage API that the replication protocol calls so that it could be adapted to work with any backend db? What about P2P sync - (eg. serverless) has anyone thought of making PouchDB do that?
- daleharvey 10y ago
  Not sponsored by Couchbase at all (I worked at Couchbase during its formation, this started near the end of my time there). For pluggable storage engines our node.js adapter uses leveldown, so any *down backend can be plugged in. https://pouchdb.com/adapters.html https://pouchdb.com/adapters.html has some more information about this. For p2p yup its been something quite a few people have been using pouchdb for, one of the more notable examples has been http://thaliproject.org/ http://thaliproject.org/, the core storage format is entirely compatible with p2p (inherited from couchdb)
- davidbanham 10y ago
  The idea of making pouch sync with other databases is covered in their faq: https://pouchdb.com/faq.html#sync_non_couchdb https://pouchdb.com/faq.html#sync_non_couchdb Basically, your other data base probably handles conflicts in a different way to Couch that would make syncing this way nonsensical. If you're happy to handle that yourself, the Couch replication protocol is well documented and there are plenty of libraries written for it.
- fredguth 10y ago
  AFAIK, Couchbase is a CouchDB as a service tuned for speed. PouchDB is a CouchDB implementation in the browser. I don't think your sync with other db would be easy and I believe there is no such desire in the project roadmap.

PaulMest 10y ago

I've been using PouchDB in a React Native app for about 6 months with SQLite as its backing store so that we can use more than 50MB of storage on the device. It has been working pretty well to persist data into an offline cache and then sync to a CouchDB 2.0 database in Digital Ocean. Getting it to work inside React Native was initially very challenging. Keeping our shim up to date with the recent changes to PouchDB has also been challenging. We are currently using PouchDB 5.4.5 because there was a breaking change in 6.x and I haven't had a chance to dive into it to figure out what is going wrong. The PouchDB community (especially Nolan Lawson) does a great job of showing examples, answering questions, and responding to feedback.

m_mueller 10y ago
Did you also consider Couchbase Lite? I'd be interested in your decision making.
- davidbanham 10y ago
  I've used pouch on a few projects and also attempted to use cb lite. I found pouch dramatically easier to get up and running with. Cb lite was complicated since it needs a native plugin rather than being just js. That wasn't too bad. The thing that put the nail in it for me was all the setup on the server side with sync gateways, multiple databases that then sync with one another, etc etc.
  - m_mueller 10y ago
    CBL has worked OK so far for us with just plain CouchDB serverside, no sync gateways. But I can see where you're coming from, if your stack is centered around JS I think pouchDB makes more sense. We have potentially large DBs (depends on usecase) where browser context isn't enough and/or provides a bad user experience.
    - davidbanham 10y ago
      Oh so you're using CBL with a couchdb backend? I didn't even realise that was possible. I thought it only synced with Couchbase which is where all the complexity came in.

daurnimator 10y ago

I really love the project, but I never used it in production due to the large file size :(

fiatjaf 10y ago
Your solution has arrived: https://pouchdb.com/custom.html https://pouchdb.com/custom.html
mungoid 10y ago
I just now learned about this DB and was intrigued but havent researched much about the cons yet, so I am curious to hear your thoughts about the file size?

rahilsondhi 10y ago

I'm interested in PouchDB to make my JavaScript app easily sync to the server, but I don't want to switch my server's database from Postgres to CouchDB. Surely I'm not the only one in this situation?

daleharvey 10y ago
You aren't, its mentioned in our faq - https://pouchdb.com/faq.html#sync_non_couchdb https://pouchdb.com/faq.html#sync_non_couchdb. A lot of people would like their current data to just be able to sync, but it almost always needs changes in the way data is stored and complementary changes to the application code
lacampbell 10y ago
Any experience doing it the hard way? I'm in the same boat. Really don't want to throw away what postgres provides.

cherryblakes 10y ago

Hello guys and how are you doing today, Do you need require the services of an accomplished and professional hacker for any of the following: -Facebook account hacking -Instagram account hacking -Mobile phone hacking -Whatsapp hacking -Kik account hacking -GPS tracking -Website hacking -Bank account hacking -Account recovery -DDOS attack -Email account hacking Then you should contact:danielwellingtonhacks@gmail.com for his professional services. He is one of the best hackers out there.

ioquatix 10y ago

Since when did arguments become functions? db.replicate.to('http://example.com/mydb'); IMHO, should be db.replicate(to: 'http://example.com/mydb');

Spivak 10y ago
I'm personally really a fan of this syntax mostly because I like how it reads when you chain them together. db.replicate .from('http://example.com/mydb1') .to('http://example.com/mydb2') .on('complete', function() { }) .on('error', function() { }) .then(function() { });
- Mouse47 10y ago
  It's not clear which 'arguments' are required when it's done this way. What's stopping you from calling 'from' without 'to'?
  - BinaryIdiot 10y ago
    I think it would just not do anything. I like this type of syntax but only when the required parameters are done up front. Then all chainable are optional / modifiers / events. This is how jQuery works and how I modeled msngr.js. Unless Pouch has anything required missing I think it's fine though maybe a little unintuitive (like can you do multiple froms? Multiple tos? Etc)
- zepolen 10y ago
  What happens when you do: db.replicate .from('http://example.com/mydb1') or db.replicate .from('http://example.com/mydb1') .from('http://example.com/mydb2') .to('http://example.com/mydb3') or db.replicate .from('http://example.com/mydb1') .to('http://example.com/mydb2') .to('http://example.com/mydb3') ? serious question
  - azr89 10y ago

slashblake 10y ago

Yeah fuck REST!!

ojanik 10y ago

We've been running PouchDB in production for ~15 months now. We chose it because it was a greenfield project and it gave us 2 things: Easy offline support and real-time syncing that makes it easy to create collaboration a-la Google Docs. Because the entire thing is a web app with app cache manifest deploying new versions is very little hassle. In terms of architecture we have about 250 tenants with separate Couch databases per each. We're still running Couch 1.6. We have yet to evaluate Couch 2.0. It's been mostly smooth ride for the most part but this being a very unusual architecture we had to tackle few interesting problems that came along. 1. Load times. Once you get over certain db size the initial load time from clean slate takes ages due to PouchDB being super chatty. I'm talking about 15-30 mins to do initial sync of 20-30mb database. We had to resort to pouch-dump to produce dump files periodically. That helped a lot. I think this issue has been rectified with Couch 2.0 and sync protocol update. 2. Browser limits. Once we hit the inherent capacity of some browsers (namely Safari on iOS, 50mb) we had to get creative. Now we're running 2 CouchDB databases for each tenant where 1 has full data and the other only contains last 7-8 days. Pouch syncs to the latter one. We run filtered replications between the full db and the reduced db and do periodic purging. On the client side if a customer tries to go back more than 7 days we just use the Pouch in online only mode where it acts as a client library to remote couch and doesn't sync locally. 3. Dealing with conflicts. This might matter or it might not depending on the domain but you have to be aware of data conflicts. Because CouchDB/PouchDb is eventually consistent multi-master setup and you will get data conflicts where people update the same entity based on the same source revision. PouchDB has nice hooks to let you deal with this but you have to architect for it. 4. Custom back-end logic. Because Pouch talks directly to Couch you can't exactly execute custom back-end logic when needed. We had to introduce a REST back-channel to make sure our back-end runs extra logic when needed. 5. We had some nasty one-off surprises. Last one was with an object that had 1700 or so revisions in couch and once it synced to PouchDB it would crash the Chrome tab in a matter of seconds. Due to the way PouchDB stores revision tree (lot's of nested arrays) Chrome would choke during JSON.parse() call and eat up memory until crash. We resolved this one by reducing the revision history limit that is kept.

deleted 10y ago
[deleted]
10y ago

azr89 10y ago

We've been using Pouch in a progressive web app designed to be used on the field in remote locations, and while there was a learning curve in understanding how the replication protocol works, and as highlighted in another comment the way Chrome stores data for a web app - we can't be happier with pouch/couch. Additionally, moving out of Cloudant and into CouchDB with an openresty based reverse proxy has made things even better, and really fun. This is one of those stacks that feels easy and simple at the same time. (Ref:https://www.infoq.com/presentations/Simple-Made-Easy https://www.infoq.com/presentations/Simple-Made-Easy).

karmelapple 10y ago
Any guidance on moving from Cloudant to CouchDB? Are you hosting it yourself? If so, has the amount of maintenance been more than you expected, or was it mostly setup time and then forget about it?
- azr89 10y ago
  Yup, hosting it ourselves. Its a peach. There are few things that it doesnt come with out of the box - clustering, Full text search, geoindexing, chained map reduce, auto compaction, index auto-updation. Once thats done, if anything it was more forget about it than Cloudant, which bills on requests / thoroughput. This can catch you out because continuous replications between databases on the same cloudant account are also counted as requests and billed as such. And continuous replication is very chatty. So if you have a particularly creative multi-master setup, like a per user db -> masterdb kind of thing going, this can eat up your thoroughput / push up your bills with no practical benefit. Its really openresty + couch that does it for me. The idea of writing security / validations / routing etc right into ngnix combines beautifully with the CouchDB way of thinking.
  - kocolosk 10y ago
    Ah, yeah, you weren't the only one bitten by that. We actually went and changed the Cloudant metering model recently so that you're billed on provisioned throughput rather than total request volume. You get dramatically more predictable billing, with the tradeoff that clients need to handle a 429 Too Many Requests response if the Cloudant instance is under-sized. More here: https://www.ibm.com/blogs/bluemix/2016/09/new-cloudant-lite-standard-plans-are-live-in-bluemix-public/ https://www.ibm.com/blogs/bluemix/2016/09/new-cloudant-lite-...

crooked-v 10y ago

PouchDB's replication capability is interesting, but is there a way to make it lazy load to the local DB instead of doing everything up front? I hesitate to use it for a web project with 10+ MB of docs where it would otherwise be ideal.

WorldMaker 10y ago
You can provide a server-side filter function to replication and progressively filter partial replications until eventually everything gets replicated. At that point it becomes a question of architecture of your documents: how much is needed to replicate before a user may be productive? You can also explore pouchdb-replication-stream to build bundles that PouchDB can bootstrap from a little bit faster than a chatty replication. That said, I've found initial replications of large databases (one I've worked with this week is a 25+ MB CouchDB database full of photos) is quick enough (and mostly bandwidth constrained) that I haven't had much in the way of concern over it.

imtringued 10y ago

Could I in theory host my own pouchdb instance to keep using applications that are no longer supported by their vendors? Imagine an IoT vendor uses pouchdb shuts down. My devices don't work anymore but I can host my own pouchdb instance. A javascript based database is a bit heavy but it's an acceptable tradeoff to me.

nolanl 10y ago
In principle yes any CouchDB/PouchDB database can sync with one another because they all share the same protocol, but OTOH I've seen very few examples of apps that offer users a simple "export to your own CouchDB/PouchDB" feature.

aeharding 10y ago

I've been running PouchDB + CouchDB 2.0 in production for a while now (financier.io). It's worked great, and I really recommend you to check it out. A couple things: 1. CouchDB 2.0 is still rough around the edges, particularly with its new Fauxton interface (ex: completely broken when proxied into subfolder). 2. CouchDB 2.0 brings the _bulk_get API which has improved sync by an order of magnitude(s). 3. I do have custom logic for logging in overriding the _session API in order to do rate limiting. (I proxy with nginx for IP rate limiting and node.js for failed password attempts rate limiting.) I also have custom logic for provisioning a new CouchDB database per user and setting up permissions. 4. I host on Digital Ocean but I use their new block storage solution so that a growing db does not become unwieldy/expensive. 5. My SaaS subscription system is kind of unique: When your subscription expires you'll simply lose write access to the server (CouchDB), but you can still pull down your data to PouchDB.

espeed 10y ago

Datomic-ClojureScript in the browser backed by PouchDB would be killer. Datomic already has a CouchDB-compatible datastore via its support for Couchbase. That would mean Datomic-ClojureScript on PouchDB could run in the browser and sync with Datomic-Clojure backed by CouchDB/Couchbase on the the server -- that would be killer and give Datomic massive reach. Has the Datomic team considered PouchDB as a possible datastore for Datomic-ClojureScript in the browser? NB: Just posted the question to the Datomic discussion group: https://groups.google.com/d/msg/datomic/uqBQE4QlnzI/VuZ14pqOBQAJ https://groups.google.com/d/msg/datomic/uqBQE4QlnzI/VuZ14pqO...