8 ms·

CouchDB. It's rare that a piece of software is bad enough to give its language and runtime a bad name, but it's actually that bad.

lallysingh 8y ago
I've tried to use it. I hadn't realized it was erlang based, but now I respect erlang less
- brightball 8y ago
  I haven’t used it personally, but I looked at it enough to know that its goals are different from most DBs. Isn’t it supposed to be mainly REST based and have a goal of being able to sync multiple nodes with expected connection loss in between? I’ve never needed that on a project but it seems like it would work for some use cases. I hear good things about PouchDB.
  - HumanDrivenDev 8y ago
    I don't quite get pouchdb. Do you need to store multiple revisions of the same document in a browser database? Locks/transactions seem like a better approach here for writing to data client side.
    - brightball 8y ago
      The main point is to sync server side data to a client side database, smoothly, so that the application can continue to function offline and then resync once the connection is re-established. Couch has always been good at this use case where systems on both ends may have changed in the disconnect time.
      - HumanDrivenDev 8y ago
        but you could store Ids and revisions on any key/value store on the client. Why does the client need pouchdbs multiple revisions?
      - jinjin2 8y ago
        Realm looks like a much more modern solution to that problem.
        brightball 8y ago
        Not familiar with Realm
    - daleharvey 8y ago
      Not entirely sure what you misunderstand, PouchDB stores multiple revisions because thats how CouchDB's sync protocol works, the point of PouchDB was to match CouchDB semantics. PouchDB (as CouchDB) provides options to control how much you track (revs_limit, auto_compaction) and there are improvements we could do to handle tracking less information better We use transactions under the hood actually writing data to indexeddb
      - HumanDrivenDev 8y ago
        I mean I both get it, and don't get it. CouchDB is all about master-master, and pouchdb just brings that to the browser. But at the same time I don't understand why you'd want a 'master' at the edge of the network, or the overhead of multiple revisions when a users browser isn't going to have tones of concurrent connections. I fully admit I'm not very well-versed on databases or distributed computing, happy to have things clarified.
        daleharvey 8y ago
        I had to make that distinction fairly early on wether PouchDB would be an edge client or a full node and I decided full node, tradeoffs either way but pros of being a full node: 1. Additional (p2p) use cases, honestly I dont use CouchDB much these days, I use express-pouchdb-server so I can embed pouchdb into my server easier than any other database, there is also pouchdb-server and p2p projects like Thali (http://thaliproject.org/ http://thaliproject.org/) 2. CouchDB existed, sync is super hard, writing an edge client would have meant designing it myself, writing a couchdb clone meant I needed to make a lot less decisions about how it worked while being very confident it would work, The test suite runs the same tests against pouchdb/couchdb to ensure compatibility (and replication via each configuration)
- chriswarbo 8y ago
  I don't follow your reasoning here: why would the badness of an application propagate up to the technology it's based on? Even the best systems in the world can be abused. I can understand it working the other way around, e.g. an application initially seems good, but opinion of it drops when finding out it's written in COBOL. I can understand if the CouchDB authors agreed with criticism of their application, and explained them as being inherent to the language. AFAIK they haven't. EDIT: Ah, looks like the author wrote CouchDB! I can understand if Erlang and its proponents held CouchDB up as a shining example of their philosophy. AFAIK they don't (e.g. it doesn't appear on https://www.erlang.org/community https://www.erlang.org/community ).
joshlemer 8y ago
Any complaints in particular? I had been contemplating CouchDB for a new application at work, maybe you have a link I could read about it? I thought that the incrementally updating mapreduce style views looked really powerful, especially when combined with the changes subscriptions / long polling features.
- facetube 8y ago
  God, where to start. Map/reduce views ("secondary indexes") are currently built by serializing JSON down a pipe to an external process called couchjs that links against a seven year old version of Mozilla Spidermonkey (1.8.5, released on March 31st 2011, and the only version it currently works with). This external process then pipes the map results back, again as JSON. Forget zerocopy; we're serializing and deserializing twice per document. Almost all of this "external view server" (the only kind AFAICT) is actually written in Javascript on that seven year old engine. Nothing is streaming; your map function is sandbox-evaled in that process, the results of a single call are stuffed into a Javascript array via .push, then stringified and printed on standard output as a single line. The entire internal "view server" protocol is based on a pipe of single-line JSON commands and data payloads. Performance (and likely security, due to age of the engine) in this subsystem is extremely poor IME. You can write map/reduce views in Erlang itself as an alternative, but that engine is disabled by default due to security concerns – it's "not sandboxed". Performance in this area also appeared to be proportional to the code size of the map function, which sounded a lot like it was being serialized down a pipe with no caching. If you make a change to a single view in a design document, the database system will rebuild all indexes in that design document unconditionally, even if the other view code hasn't changed. The only way to prevent this is to store a single view per design document, and manage those documents yourself. Once you've endured your multi-day index rebuild on a couple million documents, you'll also have to issue an extra "cleanup" command to delete the old (now essentially useless) indexes. It's a full rebuild, so you need 2x the space at a minimum to pull it off. The data itself is stored in a single file per shard, opened in O_APPEND mode, effectively serializing all writes on the shard. You also will not see kswapd active, ever, because the OS VMM is not being used effectively – no shared memory for IPC, and no use of mmap AFAICT. Server-side changes feed filters appear to be extremely slow when compared to just filtering the results in the client. Authorization: limiting read access will be difficult at best. Transactional behavior: no, aside from a single bulk write feature (make sure to use "all or nothing" mode though). Logging: signal to noise ratio is poor. Giant stack traces for minor events. You may even see strings emitted as UTF-32 arrays of integers in base 10. Hope you brought an ASCII table at least. Documentation: The following passage from section 5.2.5 of the CouchDB v2.1.1 manual just about says it all: "Views with the JavaScript query server are extremely slow to generate when there are a non-trivial number of documents to process. The generation process won’t even saturate a single CPU let alone your I/O. The cause is the latency involved in the CouchDB server and separate couchjs query server, dramatically indicating how important it is to take latency out of your implementation." I can't say anything nice here about that last sentence, so I'm just going to keep my mouth shut. But worst of all: every time it starts up, it tells me "it's time to relax". Nothing about this is relaxing. (These are my own private individually-held views, and do not reflect the views of any employer or organization)
HumanDrivenDev 8y ago
Huh? I have never heard of anyone saying erlang is bad because of couchdb. I've heard criticisms of couchdb itself, sure. But they've been more along the lines of "MVCC multi-master document stores are not a good solution for this problem" more than "CouchDB is a bad MVCC multi-master document store".
- facetube 8y ago
  It doesn't use real node-local/shard-local MVCC AFAICT. It's O_APPEND on flat files.
  - facetube 8y ago
    Sorry, wasn't clear if you were talking about MVCC in a replication context or in a node-local context. It does the former, it definitely does not do the latter.
    - HumanDrivenDev 8y ago
      I don't know that much about databases, could you go into more detail?
      - facetube 8y ago
        From the point of view of a single cluster or node (ignoring replication for a moment), it just doesn't have any strong notion of transactions. There's no write-ahead log, no rollback, and no situation where you'd be able to operate in a mode equivalent to something like `SERIALIZABLE` on a relational database.
        HumanDrivenDev 8y ago
        oh right, I understand what you mean now. I guess most people should know that going in. Are their good master-master systems out there with local transactions?
deleted 8y ago
[deleted]

lallysingh 8y ago

I've tried to use it. I hadn't realized it was erlang based, but now I respect erlang less

brightball 8y ago
I haven’t used it personally, but I looked at it enough to know that its goals are different from most DBs. Isn’t it supposed to be mainly REST based and have a goal of being able to sync multiple nodes with expected connection loss in between? I’ve never needed that on a project but it seems like it would work for some use cases. I hear good things about PouchDB.
- HumanDrivenDev 8y ago
  I don't quite get pouchdb. Do you need to store multiple revisions of the same document in a browser database? Locks/transactions seem like a better approach here for writing to data client side.
  - brightball 8y ago
    The main point is to sync server side data to a client side database, smoothly, so that the application can continue to function offline and then resync once the connection is re-established. Couch has always been good at this use case where systems on both ends may have changed in the disconnect time.
    - HumanDrivenDev 8y ago
      but you could store Ids and revisions on any key/value store on the client. Why does the client need pouchdbs multiple revisions?
    - jinjin2 8y ago
      Realm looks like a much more modern solution to that problem.
      - brightball 8y ago
        Not familiar with Realm
  - daleharvey 8y ago
    Not entirely sure what you misunderstand, PouchDB stores multiple revisions because thats how CouchDB's sync protocol works, the point of PouchDB was to match CouchDB semantics. PouchDB (as CouchDB) provides options to control how much you track (revs_limit, auto_compaction) and there are improvements we could do to handle tracking less information better We use transactions under the hood actually writing data to indexeddb
    - HumanDrivenDev 8y ago
      I mean I both get it, and don't get it. CouchDB is all about master-master, and pouchdb just brings that to the browser. But at the same time I don't understand why you'd want a 'master' at the edge of the network, or the overhead of multiple revisions when a users browser isn't going to have tones of concurrent connections. I fully admit I'm not very well-versed on databases or distributed computing, happy to have things clarified.
      - daleharvey 8y ago
        I had to make that distinction fairly early on wether PouchDB would be an edge client or a full node and I decided full node, tradeoffs either way but pros of being a full node: 1. Additional (p2p) use cases, honestly I dont use CouchDB much these days, I use express-pouchdb-server so I can embed pouchdb into my server easier than any other database, there is also pouchdb-server and p2p projects like Thali (http://thaliproject.org/ http://thaliproject.org/) 2. CouchDB existed, sync is super hard, writing an edge client would have meant designing it myself, writing a couchdb clone meant I needed to make a lot less decisions about how it worked while being very confident it would work, The test suite runs the same tests against pouchdb/couchdb to ensure compatibility (and replication via each configuration)
chriswarbo 8y ago
I don't follow your reasoning here: why would the badness of an application propagate up to the technology it's based on? Even the best systems in the world can be abused. I can understand it working the other way around, e.g. an application initially seems good, but opinion of it drops when finding out it's written in COBOL. I can understand if the CouchDB authors agreed with criticism of their application, and explained them as being inherent to the language. AFAIK they haven't. EDIT: Ah, looks like the author wrote CouchDB! I can understand if Erlang and its proponents held CouchDB up as a shining example of their philosophy. AFAIK they don't (e.g. it doesn't appear on https://www.erlang.org/community https://www.erlang.org/community ).

joshlemer 8y ago

Any complaints in particular? I had been contemplating CouchDB for a new application at work, maybe you have a link I could read about it? I thought that the incrementally updating mapreduce style views looked really powerful, especially when combined with the changes subscriptions / long polling features.

facetube 8y ago
God, where to start. Map/reduce views ("secondary indexes") are currently built by serializing JSON down a pipe to an external process called couchjs that links against a seven year old version of Mozilla Spidermonkey (1.8.5, released on March 31st 2011, and the only version it currently works with). This external process then pipes the map results back, again as JSON. Forget zerocopy; we're serializing and deserializing twice per document. Almost all of this "external view server" (the only kind AFAICT) is actually written in Javascript on that seven year old engine. Nothing is streaming; your map function is sandbox-evaled in that process, the results of a single call are stuffed into a Javascript array via .push, then stringified and printed on standard output as a single line. The entire internal "view server" protocol is based on a pipe of single-line JSON commands and data payloads. Performance (and likely security, due to age of the engine) in this subsystem is extremely poor IME. You can write map/reduce views in Erlang itself as an alternative, but that engine is disabled by default due to security concerns – it's "not sandboxed". Performance in this area also appeared to be proportional to the code size of the map function, which sounded a lot like it was being serialized down a pipe with no caching. If you make a change to a single view in a design document, the database system will rebuild all indexes in that design document unconditionally, even if the other view code hasn't changed. The only way to prevent this is to store a single view per design document, and manage those documents yourself. Once you've endured your multi-day index rebuild on a couple million documents, you'll also have to issue an extra "cleanup" command to delete the old (now essentially useless) indexes. It's a full rebuild, so you need 2x the space at a minimum to pull it off. The data itself is stored in a single file per shard, opened in O_APPEND mode, effectively serializing all writes on the shard. You also will not see kswapd active, ever, because the OS VMM is not being used effectively – no shared memory for IPC, and no use of mmap AFAICT. Server-side changes feed filters appear to be extremely slow when compared to just filtering the results in the client. Authorization: limiting read access will be difficult at best. Transactional behavior: no, aside from a single bulk write feature (make sure to use "all or nothing" mode though). Logging: signal to noise ratio is poor. Giant stack traces for minor events. You may even see strings emitted as UTF-32 arrays of integers in base 10. Hope you brought an ASCII table at least. Documentation: The following passage from section 5.2.5 of the CouchDB v2.1.1 manual just about says it all: "Views with the JavaScript query server are extremely slow to generate when there are a non-trivial number of documents to process. The generation process won’t even saturate a single CPU let alone your I/O. The cause is the latency involved in the CouchDB server and separate couchjs query server, dramatically indicating how important it is to take latency out of your implementation." I can't say anything nice here about that last sentence, so I'm just going to keep my mouth shut. But worst of all: every time it starts up, it tells me "it's time to relax". Nothing about this is relaxing. (These are my own private individually-held views, and do not reflect the views of any employer or organization)

HumanDrivenDev 8y ago

Huh? I have never heard of anyone saying erlang is bad because of couchdb. I've heard criticisms of couchdb itself, sure. But they've been more along the lines of "MVCC multi-master document stores are not a good solution for this problem" more than "CouchDB is a bad MVCC multi-master document store".

facetube 8y ago
It doesn't use real node-local/shard-local MVCC AFAICT. It's O_APPEND on flat files.
- facetube 8y ago
  Sorry, wasn't clear if you were talking about MVCC in a replication context or in a node-local context. It does the former, it definitely does not do the latter.
  - HumanDrivenDev 8y ago
    I don't know that much about databases, could you go into more detail?
    - facetube 8y ago
      From the point of view of a single cluster or node (ignoring replication for a moment), it just doesn't have any strong notion of transactions. There's no write-ahead log, no rollback, and no situation where you'd be able to operate in a mode equivalent to something like `SERIALIZABLE` on a relational database.
      - HumanDrivenDev 8y ago
        oh right, I understand what you mean now. I guess most people should know that going in. Are their good master-master systems out there with local transactions?

deleted 8y ago

[deleted]