3 ms·

The data analysis required to improve search relevance, video recommendation, A/B testing, feature improvement - basically anything, relies on crunching data on

by dsacco 10y ago

The data analysis required to improve search relevance, video recommendation, A/B testing, feature improvement - basically anything, relies on crunching data on as many users as possible. For example, you cannot get to Google or Facebook's level of artificial intelligence research and performance achievements without truly vast amounts of data. This criticism of decentralization is in addition to my parallel comment in this thread about how you cannot realistically and significantly improve privacy with feature parity. The performance and reliability would also be difficult to manage without centralized servers. You're not going to be maintaining the same uptime guarantees. Finally, how are you going to motivate development in a decentralized manner? You're removing capitalist incentives to improve the product. This would have to be managed by a consortium of companies, which doesn't sound like a much better situation than we have now, or it would have to be an open protocol. If companies weren't earning a profit on developing features for further user demand, how would the products improve as well as they do now?

AnthonyMouse 10y ago
> For example, you cannot get to Google or Facebook's level of artificial intelligence research and performance achievements without truly vast amounts of data. This argument is self-defeating. If any significant plurality (e.g. a third) of people opt into the "privacy for analysis" setting then you still have "truly vast amounts of data" and there is no trouble. Whereas if so few people want to make that exchange when the choice is made explicit that it can't even work properly then it doesn't matter how well it works because nearly everyone doesn't use it. > The performance and reliability would also be difficult to manage without centralized servers. You're not going to be maintaining the same uptime guarantees. Uptime is just math. You decide how much service uptime you need and based on the average device uptime that determines how much redundancy is necessary to achieve it. Consumer devices have lower uptime so you need somewhat more redundancy. And in practice not even much of that, because you want "close copies" (to improve latency/efficiency) anyway, so if you have enough copies for that then a failure doesn't reduce uptime, it just requires you to use a far copy that once. Moreover, if you have something that really does need specific uptime guarantees or is unusually likely to incur a DDoS, nothing stops you from pinning it to a P2P node hosted on the likes of AWS or CloudFlare. Then to lose uptime you have to lose the entire P2P network and CloudFlare. > Finally, how are you going to motivate development in a decentralized manner? Much the same way as we motivate development of Windows, Linux and Wikipedia.
- smallnamespace 10y ago
  If you depend on P2P to avoid centralized servers, you're just moving the data security issues to devices that are much, much harder to lock down. That can make privacy much worse in a practical sense. Not to mention the difficulties of reindexing and fast retrieval over P2P. If I need to add an additional index on my centralized servers, the development cycle is much shorter than forcing every peer to reindex itself.
  - AnthonyMouse 10y ago
    > If you depend on P2P to avoid centralized servers, you're just moving the data security issues to devices that are much, much harder to lock down. That can make privacy much worse in a practical sense. There is no magic security pixie dust inside a data center. If the server is vulnerable and your data is on the server then you're the same amount of screwed as if the P2P app is vulnerable and your data is on the client. Possibly more screwed because central servers have data for multiple users which give the attackers more incentive to break into them. > If I need to add an additional index on my centralized servers, the development cycle is much shorter than forcing every peer to reindex itself. And why is that?
    - smallnamespace 10y ago
      I'm not saying it's magic, I'm saying that securing a P2P network is more difficult because the threat surface is much larger and because you have much less control over your stack. At the very least, I (should) have physical security over my own servers, and the intra-DC data links. You can't say that at all about a distributed P2P network, so you'll need to put in more elbow grease to address those vectors. For example, if I find a severe vulnerability, I can immediately patch my own servers. If the vulnerability is in the P2P client, it may be impossible to guarantee that every last client gets patched. > And why is that? Because there's obviously an inverse tradeoff between control and development speed. Managing distributed state is hard enough when it's on your machines, it's exponentially harder when it's random devices somewhere on the Internet. Your guarantees are much looser. For example: 1. Your data is less local so there is less effective latency. 2. You're spending computation and bandwidth that other people are paying for If I need to get some big batch job done quickly and the data is on my servers, then I just up a bunch of instances and get it done. If the data is on a bunch of smartphones somewhere, I can't suddenly grab a bunch more of their computational capacity and bandwidth without pissing off a bunch of users, can I? And what if I need to change the protocol? We're back to the problem of patching all the clients. I'm not saying these technical hurdles are insoluble. I'm saying they are real and to date nobody has actually solved them.

by dsacco 10y ago

AnthonyMouse 10y ago

> For example, you cannot get to Google or Facebook's level of artificial intelligence research and performance achievements without truly vast amounts of data. This argument is self-defeating. If any significant plurality (e.g. a third) of people opt into the "privacy for analysis" setting then you still have "truly vast amounts of data" and there is no trouble. Whereas if so few people want to make that exchange when the choice is made explicit that it can't even work properly then it doesn't matter how well it works because nearly everyone doesn't use it. > The performance and reliability would also be difficult to manage without centralized servers. You're not going to be maintaining the same uptime guarantees. Uptime is just math. You decide how much service uptime you need and based on the average device uptime that determines how much redundancy is necessary to achieve it. Consumer devices have lower uptime so you need somewhat more redundancy. And in practice not even much of that, because you want "close copies" (to improve latency/efficiency) anyway, so if you have enough copies for that then a failure doesn't reduce uptime, it just requires you to use a far copy that once. Moreover, if you have something that really does need specific uptime guarantees or is unusually likely to incur a DDoS, nothing stops you from pinning it to a P2P node hosted on the likes of AWS or CloudFlare. Then to lose uptime you have to lose the entire P2P network and CloudFlare. > Finally, how are you going to motivate development in a decentralized manner? Much the same way as we motivate development of Windows, Linux and Wikipedia.

smallnamespace 10y ago
If you depend on P2P to avoid centralized servers, you're just moving the data security issues to devices that are much, much harder to lock down. That can make privacy much worse in a practical sense. Not to mention the difficulties of reindexing and fast retrieval over P2P. If I need to add an additional index on my centralized servers, the development cycle is much shorter than forcing every peer to reindex itself.
- AnthonyMouse 10y ago
  > If you depend on P2P to avoid centralized servers, you're just moving the data security issues to devices that are much, much harder to lock down. That can make privacy much worse in a practical sense. There is no magic security pixie dust inside a data center. If the server is vulnerable and your data is on the server then you're the same amount of screwed as if the P2P app is vulnerable and your data is on the client. Possibly more screwed because central servers have data for multiple users which give the attackers more incentive to break into them. > If I need to add an additional index on my centralized servers, the development cycle is much shorter than forcing every peer to reindex itself. And why is that?
  - smallnamespace 10y ago
    I'm not saying it's magic, I'm saying that securing a P2P network is more difficult because the threat surface is much larger and because you have much less control over your stack. At the very least, I (should) have physical security over my own servers, and the intra-DC data links. You can't say that at all about a distributed P2P network, so you'll need to put in more elbow grease to address those vectors. For example, if I find a severe vulnerability, I can immediately patch my own servers. If the vulnerability is in the P2P client, it may be impossible to guarantee that every last client gets patched. > And why is that? Because there's obviously an inverse tradeoff between control and development speed. Managing distributed state is hard enough when it's on your machines, it's exponentially harder when it's random devices somewhere on the Internet. Your guarantees are much looser. For example: 1. Your data is less local so there is less effective latency. 2. You're spending computation and bandwidth that other people are paying for If I need to get some big batch job done quickly and the data is on my servers, then I just up a bunch of instances and get it done. If the data is on a bunch of smartphones somewhere, I can't suddenly grab a bunch more of their computational capacity and bandwidth without pissing off a bunch of users, can I? And what if I need to change the protocol? We're back to the problem of patching all the clients. I'm not saying these technical hurdles are insoluble. I'm saying they are real and to date nobody has actually solved them.