8 ms·

There are little technical details. As far as i understand the thesis is: CouchDB provides all the groundwork for distributed state, so there is no need to buil

by beza1e1 16y ago

There are little technical details. As far as i understand the thesis is: CouchDB provides all the groundwork for distributed state, so there is no need to build a lot sync/auth/notify code like Diaspora does. Really?

Tichy 16y ago
Will be interesting to see how well it works with 1 million nodes.
- technoweenie 16y ago
  Isn't that the point with these projects? Managing a central service for 500 million users is hard. But with a decentralized setup, I won't need to connect to a million nodes. I don't know quite that many people. Course, a friend request changes from a simple button click to getting my local couchdb instance to do a filtered sync with your couchdb instance.
  - Tichy 16y ago
    You might have a point there. Although some people on Twitter have 100000 Followers and more.
    - messel 16y ago
      I think if you have 1million active followers some kind of push publishing setup could help. But as far as I know there's nothing in the tech of couchdb that prevents replicating to a million other nodes.
      - maggit 16y ago
        One obstacle is that you cannot have one million open sockets (for long polling) at the same time on one node. I don't know how easy this is to mitigate.
        messel 16y ago
        Hmm. The changes are posted through comet in continuous replication? That would mean another distribution system would be better for heavily published nodes.
        wmf 16y ago
        You can have a million open sockets in Erlang, although it requires serious RAM: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 http://www.metabrew.com/article/a-million-user-comet-applica...
antileet 16y ago
I don't think it's that capable of doing all that. As far as I understand, CouchDB does a "dumb" version-control type synchronization. Server A can copy all the data that Server B has and do it in a nice clean manner over HTTP. I can think of several disadvantages (though I may be wrong and naive): 1. If you're running a node, and I'm running a node, my data schematics might not be compatible with yours. If I write a uniform backward-compatible RPC, then it'll be a little better. 2. I can't selectively allow you to copy some data (photos) and not others. If you are a user on my database, you can access everything you have permissions to. It's hard to set fine-grained permissions. 3. Push notifications will be hard. I'd have to poll the database for changes. For that matter, the same can be achieved by putting a giant XML file on Git, and having each person set up a remote, or using a complicated set of shell scripts and rsync. Couch's replication facility is really neat, especially if you want to scale horizontally or if you're adding redundancy, etc. CouchDB is becoming the new hammer. It'll get tempting to write source control, file backup, etc, using this database because of the data sync, and decent query capabilities. But I feel that building a social network on top of CouchDB's replication might not be the best of ideas.
- daleharvey 16y ago
  couchdb has filtered replication so you can selectively choose what gets replicated and it has continous replication with a comet changes feed, so push notifications are also built in. the data schematics are a bit of a non issue, at some point someone needs to agree with what the data format is. basically couchdb already has a lot of this functionality baked in as first class citizens, I agree that there is a lot of situations where people will be hammering couch into a hole that doesnt fit (source control would be a good one), but I certainly dont think this is one of them.
  - messel 16y ago
    Yeah I wanted to build a stand alone media sharing app (big data files, images, movies, etc), but now I may not need one. Continuous replication of parts of subscribed databases was the idea I was going to go with.
  - 16y ago

by beza1e1 16y ago

Tichy 16y ago

Will be interesting to see how well it works with 1 million nodes.

technoweenie 16y ago
Isn't that the point with these projects? Managing a central service for 500 million users is hard. But with a decentralized setup, I won't need to connect to a million nodes. I don't know quite that many people. Course, a friend request changes from a simple button click to getting my local couchdb instance to do a filtered sync with your couchdb instance.
- Tichy 16y ago
  You might have a point there. Although some people on Twitter have 100000 Followers and more.
  - messel 16y ago
    I think if you have 1million active followers some kind of push publishing setup could help. But as far as I know there's nothing in the tech of couchdb that prevents replicating to a million other nodes.
    - maggit 16y ago
      One obstacle is that you cannot have one million open sockets (for long polling) at the same time on one node. I don't know how easy this is to mitigate.
      - messel 16y ago
        Hmm. The changes are posted through comet in continuous replication? That would mean another distribution system would be better for heavily published nodes.
      - wmf 16y ago
        You can have a million open sockets in Erlang, although it requires serious RAM: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 http://www.metabrew.com/article/a-million-user-comet-applica...

antileet 16y ago

I don't think it's that capable of doing all that. As far as I understand, CouchDB does a "dumb" version-control type synchronization. Server A can copy all the data that Server B has and do it in a nice clean manner over HTTP. I can think of several disadvantages (though I may be wrong and naive): 1. If you're running a node, and I'm running a node, my data schematics might not be compatible with yours. If I write a uniform backward-compatible RPC, then it'll be a little better. 2. I can't selectively allow you to copy some data (photos) and not others. If you are a user on my database, you can access everything you have permissions to. It's hard to set fine-grained permissions. 3. Push notifications will be hard. I'd have to poll the database for changes. For that matter, the same can be achieved by putting a giant XML file on Git, and having each person set up a remote, or using a complicated set of shell scripts and rsync. Couch's replication facility is really neat, especially if you want to scale horizontally or if you're adding redundancy, etc. CouchDB is becoming the new hammer. It'll get tempting to write source control, file backup, etc, using this database because of the data sync, and decent query capabilities. But I feel that building a social network on top of CouchDB's replication might not be the best of ideas.

daleharvey 16y ago
couchdb has filtered replication so you can selectively choose what gets replicated and it has continous replication with a comet changes feed, so push notifications are also built in. the data schematics are a bit of a non issue, at some point someone needs to agree with what the data format is. basically couchdb already has a lot of this functionality baked in as first class citizens, I agree that there is a lot of situations where people will be hammering couch into a hole that doesnt fit (source control would be a good one), but I certainly dont think this is one of them.
- messel 16y ago
  Yeah I wanted to build a stand alone media sharing app (big data files, images, movies, etc), but now I may not need one. Continuous replication of parts of subscribed databases was the idea I was going to go with.
- 16y ago