6 ms·

The interesting thing about all these services is that they're trying to abstract away the same types of problems that all developers come across: Identity, con

by amirmc 12y ago

The interesting thing about all these services is that they're trying to abstract away the same types of problems that all developers come across: Identity, connectivity, sync/backup (and to some extent, deployment). A different (and more disruptive?) approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application. i.e something like the app-store model but it connects to the user's 'personal cloud' (or the desktops of old - if you prefer). Such a system needs to be FOSS at its core but with a way for developers to get paid for providing value to end-users (who themselves get control of their data/networks). Of course, this won't happen overnight, but the alternative is that everyone ends up using proprietary silos, with huge lock-in and innovation suffers as the tech giants get distracted with lawsuits. I'm working with others on the distributed systems infrastructure we need to make this possible [1], so that we can get to a place where everyone can have their own piece of the cloud. In fact, one of the major components of this is Mirage, which has been discussed quite a bit on HN recently [2]. [1] http://nymote.org/blog/2013/introducing-nymote/ http://nymote.org/blog/2013/introducing-nymote/ [2] https://news.ycombinator.com/item?id=8020125 https://news.ycombinator.com/item?id=8020125 and https://news.ycombinator.com/item?id=8005130 https://news.ycombinator.com/item?id=8005130 and https://news.ycombinator.com/item?id=7726748 https://news.ycombinator.com/item?id=7726748

michaelmior 12y ago
Do users really care about this? My guess is that 99% don't and don't want to deal with these details. The project sounds cool and like something the HN crowd might want, but my intuition is that most users won't understand the implications. It also adds an extra step in the data storage process within the app, which increases friction and seems like it would be hard to get developers to adopt. All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems.
- amirmc 12y ago
  Yes, users care about this but like most things you have to segment the market appropriately. I meet more and more people who are wary of putting all their trust/data/etc into only one provider, be that Google, Apple or anyone else. It's not just a privacy thing but a growing awareness of the sheer dependency on that one company. (Aside: I have to say FB is being pretty clever by buying up apps and letting them stay 'independent' - users don't notice that as much and reminds me of FMCG.). We don't need most users in the beginning. Just those that are technically savvy enough to understand the landscape and the way the world is headed. From there we can develop more stuff and eventually get it into a state ready for the masses. For example, I can run my personal website as a unikernel [1], which is not suitable for mom-and-pop but great for anyone currently using any static site generators. As more libraries get released I can build on that an eventually run more of my infrastructure this way (my prototypical examples are mail, contacts and calendars). In terms of the friction for developers, we simply have to make the libraries and tooling fun and easy to use. With well-designed systems it's not necessarily any more friction that what developers have to do now (i.e build their own backend). > "All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems." We're working on it. [1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/ http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li...
  - josh2600 12y ago
    I thought like this once. Futurists have a tendency to imagine a world of changed human behavior and it's compelling to do so. The reality is that the future rarely arrives as sweeping change, but rather as metaphor and specialization. Whereas you can imagine others adopting new patterns of behavior because you understand the underlying reasons why such behavior is reasonable, the metaphor through which you explain this change is not readily understood. Why, as a User do I want this? If the answer is control and privacy, you might be barking up the wrong tree (time and again we've shown that those are not things consumers want or are willing to pay for). If you want to drive dynamic change in the world, you have to change the underlying structure of complicated systems while steadfastly avoiding changes in user behavior. It turns out this is quite hard. I applaud your efforts but encourage you to avoid the rabbit hole of endless specialization and to improve the marketing metaphor/rhetoric.
    - walterbell 12y ago
      Rhetoric from a16z in a thread of 50+ tweets: "Bitcoin, billions of phones, and NSA eavesdropping are among trends that may eventually result in cloud successor: data-local computing" https://twitter.com/balajis/status/487403585445978112 https://twitter.com/balajis/status/487403585445978112
    - quadrangle 12y ago
      > time and again we've shown that those are not things consumers want or are willing to pay for CITATION please. I think we have never seen such things. We've not had studies that control for all other factors and then conclude that users don't want control and privacy. I think your claim is flat out wrong. You're taking the fact that most users won't sacrifice by using a lower-quality, obscure thing for privacy and control, and leaping to the baseless conclusion that nobody cares about these things. I think you're completely wrong.
      - josh2600 12y ago
        There have not been any case studies that have empirically demonstrated this to be true. If that is your basis for evaluating my statements then I cannot support my claim in a manner that you would find satisfactory. I would however, anecdotally highlight the progression of social services from places of privacy to places of publicity. One could argue that sites like "The Well" were the original social networks but they did not grow (whether by choice is of course a matter for debate). Over time, each successive social startup defaulted to a more open stance and scaled an order of magnitude in users with each step. The trend suggests that all things being equal, users will say they want privacy but want the features publishing and syndication provides (voting purely with their registrations and advertising value). I agree that privacy has value, but I would argue that the anecdotal evidence suggests that users want less private services (and yes I'm aware that correlation is not causation, but I think this observation is apt). This is not law or dogma but rather a thought and I'm sorry if it was communicated poorly previously.
- mynegation 12y ago
  I do, but, granted, I am a typical HN reader, not a typical user. I want my data either to be encrypted by the key that only I have, before being sent to the online storage service, and/or being sent to the computer that I physically control (NAS at my home). The reasoning that users do not want to maintain that infrastructure is a very valid one. However, it is like having a car vs. using public transportation. Most car owners do not maintain their car themselves, but at least you can remove your personal items from the trunk before you drive it to the mechanic. Now the question is does this analogy extends to the advantages of using your own car vs public transportation. My guess is that need for privacy could be one big driver, but the company that can make home computing infrastructure an item of conspicuous consumption will be a big winner. Think Apple or Sonos, but in terms of what comes next (Augmented Reality, Voice HCI etc).
- rektide 12y ago
  Users won't care, until someone else builds on the liberated data of your app.
- api 12y ago
  The problem isn't that users don't care, but that there's no revenue model. As a result none of these decentralized networking or platform companies can get funding. Without funding, they don't have the resources to build truly compelling products. You can prototype something in your spare time, but to polish something to be truly competitive outside enthusiast circles you need to dedicate serious time to it. Even if users did care there is no mechanism for them to pay for it, so it doesn't happen.
ebiester 12y ago
The big thing is each individual user would then be responsible for maintaining their own individual cloud. Otherwise, they'll just outsource that out and we're no better than before. How do you get grandma on her own personal cloud?
- rstupek 12y ago
  You probably don't. You also don't get grandma to run her own power plant, telephone switching network, etc. As long as her "stuff" is portable, it really doesn't matter who provides the service
  - mtbcoder 12y ago
    If her "stuff" is that portable and the service provider has at least some kind of read access, how would one prevent the service provider from simply keeping a mirror of Grandma's data on their own servers?
rschmitty 12y ago
> A different approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application. User Dependency Injection!
CaveTech 12y ago
Isn't that what App.net was supposed to be? By most standards that project was a huge failure. It's hard to proposition a user to start managing their own silo.
- api 12y ago
  App.net was centrally hosted and centrally controlled, so it wasn't really all that different from its competitors. It also had execution issues.
johnchristopher 12y ago
Wouldn't Dropbox Apps, even though it isn't open source, be a good start for your requirements ?
tdaltonc 12y ago
I'm really intrigued by this model of internet infrastructure. Would you mind helping me understand the relevant difference between what you, Sandstorm[1], and OpenPDS[2] are doing? [1] https://sandstorm.io/ https://sandstorm.io/ [2] http://openpds.media.mit.edu/ http://openpds.media.mit.edu/
- walterbell 12y ago
  OpenPDS = US gov-defined policy (NSTIC) for access to personal data Sandstorm = packaging & separation of web apps from policy-managed data Nymote & Mirage = UK systems research (OS, storage) on strong enforcement of claims similar but not limited to those being made in the above two projects
- kentonv 12y ago
  (Sandstorm author here.) Based on a brief skim of the Mirage docs, it looks like a really big difference between Mirage and Sandstorm is that Mirage wants apps to be written in OCaml based on a custom API whereas Sandstorm builds of Linux. Any server app that runs on Linux should be easy to port into Sandstorm, but it seems like a lot of things will need to be written from scratch on Mirage. In fact it appears Mirage intends to replace the kernel entirely. I can certainly see the appeal of dumping legacy baggage, but it's going to take quite a bit of work to reach the point of practicality.
  - amirmc 12y ago
    You can look through the ACM article to get a good overview [1]. Yes, it's taken a lot of work to get to this point and there's more to go. In order to enable a fully decentralised system, we need to understand and tackle the basic problems that all developers face, which is more involved than just Mirage (as described in my comment above). All the code is also fully open source under a permissive license [2]. [1] http://queue.acm.org/detail.cfm?id=2566628 http://queue.acm.org/detail.cfm?id=2566628 [2] https://github.com/mirage/ https://github.com/mirage/
pjkundert 12y ago
https://sandstorm.io https://sandstorm.io ?
JPKab 12y ago
What a great idea. Good luck with this sir! Please keep us all updated on your progress.
tn13 12y ago
Users don't care about. Not worth doing.
- oh_sigh 12y ago
  Your comment wasn't worth doing. Neither was mine.
hqm 12y ago
There was an MIT startup, Permabit, that tried to make a cloud based storage system which backed your own local disk. One idea was that your local storage is faster so you should use it for local things, and offline computation, but that if a shadow copy was being backed to the cloud in encrypted fashion. If your local storage fails, you just replace it, and the data dribbles back in from the cloud to your new local device. They also had some hack whereby large common files (like app executables) where compressed using their hash code or something, but that is not the important idea in my opinion. I think having that transparent local/cloud duality is one piece of the puzzle for a service which is both fast and local and also cloud based. You shouldn't have to care where your data lives. The other pieces of the puzzle are how to grant fine grained shared access without confusing users and introducing security holes.
tracker1 12y ago
I was actually thinking something like this for dropbox would be a no brainer.. like a cloudmail client for all your accounts, or an igoogle/ireader service that's hosted and uses your dropbox for its' data.
ForHackernews 12y ago
Interesting. How does this overlap/conflict with efforts like Diaspora or OwnCloud?
rpdillon 12y ago
This sounds quite similar to what the (now more-or-less defunct) Singly was trying to accomplish. Having talked to a former employee about it at some length, the TL;DR is that 1) the technology and ideas are cool and 2) uptake was disappointing. That said, execution matters more than ideas when it comes to success.

by amirmc 12y ago

michaelmior 12y ago

Do users really care about this? My guess is that 99% don't and don't want to deal with these details. The project sounds cool and like something the HN crowd might want, but my intuition is that most users won't understand the implications. It also adds an extra step in the data storage process within the app, which increases friction and seems like it would be hard to get developers to adopt. All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems.

amirmc 12y ago
Yes, users care about this but like most things you have to segment the market appropriately. I meet more and more people who are wary of putting all their trust/data/etc into only one provider, be that Google, Apple or anyone else. It's not just a privacy thing but a growing awareness of the sheer dependency on that one company. (Aside: I have to say FB is being pretty clever by buying up apps and letting them stay 'independent' - users don't notice that as much and reminds me of FMCG.). We don't need most users in the beginning. Just those that are technically savvy enough to understand the landscape and the way the world is headed. From there we can develop more stuff and eventually get it into a state ready for the masses. For example, I can run my personal website as a unikernel [1], which is not suitable for mom-and-pop but great for anyone currently using any static site generators. As more libraries get released I can build on that an eventually run more of my infrastructure this way (my prototypical examples are mail, contacts and calendars). In terms of the friction for developers, we simply have to make the libraries and tooling fun and easy to use. With well-designed systems it's not necessarily any more friction that what developers have to do now (i.e build their own backend). > "All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems." We're working on it. [1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/ http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li...
- josh2600 12y ago
  I thought like this once. Futurists have a tendency to imagine a world of changed human behavior and it's compelling to do so. The reality is that the future rarely arrives as sweeping change, but rather as metaphor and specialization. Whereas you can imagine others adopting new patterns of behavior because you understand the underlying reasons why such behavior is reasonable, the metaphor through which you explain this change is not readily understood. Why, as a User do I want this? If the answer is control and privacy, you might be barking up the wrong tree (time and again we've shown that those are not things consumers want or are willing to pay for). If you want to drive dynamic change in the world, you have to change the underlying structure of complicated systems while steadfastly avoiding changes in user behavior. It turns out this is quite hard. I applaud your efforts but encourage you to avoid the rabbit hole of endless specialization and to improve the marketing metaphor/rhetoric.
  - walterbell 12y ago
    Rhetoric from a16z in a thread of 50+ tweets: "Bitcoin, billions of phones, and NSA eavesdropping are among trends that may eventually result in cloud successor: data-local computing" https://twitter.com/balajis/status/487403585445978112 https://twitter.com/balajis/status/487403585445978112
  - quadrangle 12y ago
    > time and again we've shown that those are not things consumers want or are willing to pay for CITATION please. I think we have never seen such things. We've not had studies that control for all other factors and then conclude that users don't want control and privacy. I think your claim is flat out wrong. You're taking the fact that most users won't sacrifice by using a lower-quality, obscure thing for privacy and control, and leaping to the baseless conclusion that nobody cares about these things. I think you're completely wrong.
    - josh2600 12y ago
      There have not been any case studies that have empirically demonstrated this to be true. If that is your basis for evaluating my statements then I cannot support my claim in a manner that you would find satisfactory. I would however, anecdotally highlight the progression of social services from places of privacy to places of publicity. One could argue that sites like "The Well" were the original social networks but they did not grow (whether by choice is of course a matter for debate). Over time, each successive social startup defaulted to a more open stance and scaled an order of magnitude in users with each step. The trend suggests that all things being equal, users will say they want privacy but want the features publishing and syndication provides (voting purely with their registrations and advertising value). I agree that privacy has value, but I would argue that the anecdotal evidence suggests that users want less private services (and yes I'm aware that correlation is not causation, but I think this observation is apt). This is not law or dogma but rather a thought and I'm sorry if it was communicated poorly previously.
mynegation 12y ago
I do, but, granted, I am a typical HN reader, not a typical user. I want my data either to be encrypted by the key that only I have, before being sent to the online storage service, and/or being sent to the computer that I physically control (NAS at my home). The reasoning that users do not want to maintain that infrastructure is a very valid one. However, it is like having a car vs. using public transportation. Most car owners do not maintain their car themselves, but at least you can remove your personal items from the trunk before you drive it to the mechanic. Now the question is does this analogy extends to the advantages of using your own car vs public transportation. My guess is that need for privacy could be one big driver, but the company that can make home computing infrastructure an item of conspicuous consumption will be a big winner. Think Apple or Sonos, but in terms of what comes next (Augmented Reality, Voice HCI etc).
rektide 12y ago
Users won't care, until someone else builds on the liberated data of your app.
api 12y ago
The problem isn't that users don't care, but that there's no revenue model. As a result none of these decentralized networking or platform companies can get funding. Without funding, they don't have the resources to build truly compelling products. You can prototype something in your spare time, but to polish something to be truly competitive outside enthusiast circles you need to dedicate serious time to it. Even if users did care there is no mechanism for them to pay for it, so it doesn't happen.

ebiester 12y ago

The big thing is each individual user would then be responsible for maintaining their own individual cloud. Otherwise, they'll just outsource that out and we're no better than before. How do you get grandma on her own personal cloud?

rstupek 12y ago
You probably don't. You also don't get grandma to run her own power plant, telephone switching network, etc. As long as her "stuff" is portable, it really doesn't matter who provides the service
- mtbcoder 12y ago
  If her "stuff" is that portable and the service provider has at least some kind of read access, how would one prevent the service provider from simply keeping a mirror of Grandma's data on their own servers?

rschmitty 12y ago

> A different approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application. User Dependency Injection!

CaveTech 12y ago

Isn't that what App.net was supposed to be? By most standards that project was a huge failure. It's hard to proposition a user to start managing their own silo.

api 12y ago
App.net was centrally hosted and centrally controlled, so it wasn't really all that different from its competitors. It also had execution issues.

johnchristopher 12y ago

Wouldn't Dropbox Apps, even though it isn't open source, be a good start for your requirements ?

tdaltonc 12y ago

I'm really intrigued by this model of internet infrastructure. Would you mind helping me understand the relevant difference between what you, Sandstorm[1], and OpenPDS[2] are doing? [1] https://sandstorm.io/ https://sandstorm.io/ [2] http://openpds.media.mit.edu/ http://openpds.media.mit.edu/

walterbell 12y ago
OpenPDS = US gov-defined policy (NSTIC) for access to personal data Sandstorm = packaging & separation of web apps from policy-managed data Nymote & Mirage = UK systems research (OS, storage) on strong enforcement of claims similar but not limited to those being made in the above two projects
kentonv 12y ago
(Sandstorm author here.) Based on a brief skim of the Mirage docs, it looks like a really big difference between Mirage and Sandstorm is that Mirage wants apps to be written in OCaml based on a custom API whereas Sandstorm builds of Linux. Any server app that runs on Linux should be easy to port into Sandstorm, but it seems like a lot of things will need to be written from scratch on Mirage. In fact it appears Mirage intends to replace the kernel entirely. I can certainly see the appeal of dumping legacy baggage, but it's going to take quite a bit of work to reach the point of practicality.
- amirmc 12y ago
  You can look through the ACM article to get a good overview [1]. Yes, it's taken a lot of work to get to this point and there's more to go. In order to enable a fully decentralised system, we need to understand and tackle the basic problems that all developers face, which is more involved than just Mirage (as described in my comment above). All the code is also fully open source under a permissive license [2]. [1] http://queue.acm.org/detail.cfm?id=2566628 http://queue.acm.org/detail.cfm?id=2566628 [2] https://github.com/mirage/ https://github.com/mirage/

pjkundert 12y ago

https://sandstorm.io https://sandstorm.io ?

JPKab 12y ago

What a great idea. Good luck with this sir! Please keep us all updated on your progress.

tn13 12y ago

Users don't care about. Not worth doing.

oh_sigh 12y ago
Your comment wasn't worth doing. Neither was mine.

hqm 12y ago

There was an MIT startup, Permabit, that tried to make a cloud based storage system which backed your own local disk. One idea was that your local storage is faster so you should use it for local things, and offline computation, but that if a shadow copy was being backed to the cloud in encrypted fashion. If your local storage fails, you just replace it, and the data dribbles back in from the cloud to your new local device. They also had some hack whereby large common files (like app executables) where compressed using their hash code or something, but that is not the important idea in my opinion. I think having that transparent local/cloud duality is one piece of the puzzle for a service which is both fast and local and also cloud based. You shouldn't have to care where your data lives. The other pieces of the puzzle are how to grant fine grained shared access without confusing users and introducing security holes.

tracker1 12y ago

I was actually thinking something like this for dropbox would be a no brainer.. like a cloudmail client for all your accounts, or an igoogle/ireader service that's hosted and uses your dropbox for its' data.

ForHackernews 12y ago

Interesting. How does this overlap/conflict with efforts like Diaspora or OwnCloud?

rpdillon 12y ago

This sounds quite similar to what the (now more-or-less defunct) Singly was trying to accomplish. Having talked to a former employee about it at some length, the TL;DR is that 1) the technology and ideas are cool and 2) uptake was disappointing. That said, execution matters more than ideas when it comes to success.