4 ms·

Things like Sandstorm are really important as they allow people options without becoming a full-blown sysadmin. Just look at how many crowd funding ideas are ba

by amirmc 12y ago

Things like Sandstorm are really important as they allow people options without becoming a full-blown sysadmin. Just look at how many crowd funding ideas are based on 'personal cloud' concepts. However, we also need to work on the fundamental problems to make it easier to build decentralised products in the first place (not everything is a web-app). Namely, how such apps are built, how they store/sync data, and how we deal with identity. The current tools simply aren't designed for the world we're heading towards, so we need to re-evaluate our assumptions. On top of this is the need for business models that don't rely on mass data collection (eg advertising) -- we can't rely on everything being open source but the underlying infrastructure must be. There are many ways forward and the particular approach I'm taking is based on unikernels and creating a modern stack to deal with the above issues directly. There's more info at http://amirchaudhry.com/brewing-miso-to-serve-nymote/ http://amirchaudhry.com/brewing-miso-to-serve-nymote/ If anyone happens to be at FOSDEM this weekend I'd be happy to chat about these ideas in person.

jackweirdy 12y ago
Will you be talking about Mirage/other bits of your work at FOSDEM, or are you “just” attending?
- amirmc 12y ago
  Not speaking but doing a demo at the Xen booth on Saturday (likely 1pm-3pm but email me to be sure). The repo has details about the demo if you're interested (though it's probably also a spoiler) https://github.com/amirmc/fosdemo https://github.com/amirmc/fosdemo
paulproteus 12y ago
I'm at FOSDEM! I'm on #fosdem and #sandstorm on freenode -- my nick is paulproteus. Also I work on+for Sandstorm.
mempko 12y ago
I love the idea of unikernels! I took a much high level approach for p2p apps with Fire★ (http://firestr.com http://firestr.com). Where it is GUI centric and has a built in app editor (all application code is viewable/editable). My only concern with the unikernel approach is that you can end up with a system where the code is not viewable and editable. Have you thought about that concern?
lewisl9029 12y ago
In terms of storage, here's something I've been working with for one of my current projects: http://remotestorage.io/ http://remotestorage.io/ Open protocol for synchronized personal storage. It uses a decentralized model where users provide and pay for their own storage. Could be game-changing if it takes off.
- nileshtrivedi 12y ago
  There is also https://tent.io/ https://tent.io/ It seemed quite promising to me but there haven't been updates from the team lately.
  - seagreen 12y ago
    They got into YC with Flynn (https://flynn.io/ https://flynn.io/) which kept them busy for a while, but Tent 0.4 development is starting to pick up again.
- kentonv 12y ago
  remoteStorage is an elegant solution in a lot of ways, but there are some problems: * Apps written in this style have a very hard time being real-time collaborative. Maybe this will be solved by WebRTC, but it's a lot easier with a server component. (Sandstorm has several real-time collaborative apps, like EtherPad, EtherCalc, and Wave. Currently you can share these apps with other users by copy/pasting the URL; the sharing model will get more sophisticated eventually). * You need to set up a storage server for the apps to use, or use one of the big providers that like to do data mining. You then need to connect the app to the correct server, which is extra busy-work. * The permission model of these storage servers isn't terribly sophisticated. Often you'll end up granting the app broader permissions that you really want. * There is, of course, nothing stopping the app from storing data to other places as well, nor is there any isolation between separate documents opened by the same app. (On Sandstorm, each document is a separate instance of the app in a separate sandbox which cannot communicate with the outside world without permission.) Edit: Missed that you said you're working on RemoteStorage. Probably would have phrased differently if I noticed. Wasn't meant to be an attack. Sorry! (We actually like remoteStorage apps a lot in that they are often easily ported to Sandstorm. :) )
greggman 12y ago
I know this is going to sound ridiculous and maybe this is the wrong topic. I'm at a spot where I wish there was a closer to turn key framework for my project. I wrote 3 interdependent web servers, have them running on all the same virtual machine on digital ocean with varnish. Now I need to upgrade ubuntu and I don't want them to go down. So, apparently I need at least 3 machines if not more. One to run varnish or something similar, to direct to the other 2. Some way to bleed people over to one of the 2 machines. Once everyone is on one machine I can upgrade the unused machine. After that I can bleed them all back and then upgrade the other. Is this too small a niche? Is the answer I should have used "google cloud"? It just seems like each step is so much work. I certainly learned a lot (vagrant, ansible, and other stuff) although all of that knowledge will be probably obsolete before I need to do this again. Is there something I should have looked at? Sandstorm seems one level down. Like it's more like a replacement for the old lamp/cpanel isps.
- e12e 12y ago
  In your scenario, you need "just" two machines - and really only one running most of the time. You have varnish+appservers running one one vm now. What you need is one more vm, running the same stack. Depending on how you handle sessions this could be as easy as: 0: reduce ttl of dns to ~60s 1: bring up vm2==vm1 2: upgrade vm2 2.1: test vm2 2.2: shutdown vm2, take DO snapshot 2.3: instanciate vm/droplet of vm2 3: point/change dns from vm1 to vm2 4: wait for traffic to die on vm1 5: shutdown vm1 6: possibly increase ttl in dns again Next round, same procedure. This is not ready for HA -- easiest way there is running 2 vms like now, but with haproxy in front (so the vm has ha-varnish-3appservers), along with heart-beat and a shared ip. I don't thing DO supports that -- and it is probably not worth the hassle if you can live with dns changeover-time==downtime (say, minutes modulus wrongly configured dns caches). With a couple of dedicated ips, you can usually get away with vm2 taking vm1s ip (and vice-versa) -- but again that depends on your session set-up and/or if dropping sessions are ok. With sessions in some kind of replicated cache/db (eg: redis, mysql) -- you could probably set vm2 to slave over/replicate sessions, then do the dns [ed: or ip] switch[over].
  - greggman 12y ago

by amirmc 12y ago

jackweirdy 12y ago

Will you be talking about Mirage/other bits of your work at FOSDEM, or are you “just” attending?

amirmc 12y ago
Not speaking but doing a demo at the Xen booth on Saturday (likely 1pm-3pm but email me to be sure). The repo has details about the demo if you're interested (though it's probably also a spoiler) https://github.com/amirmc/fosdemo https://github.com/amirmc/fosdemo

paulproteus 12y ago

I'm at FOSDEM! I'm on #fosdem and #sandstorm on freenode -- my nick is paulproteus. Also I work on+for Sandstorm.

mempko 12y ago

I love the idea of unikernels! I took a much high level approach for p2p apps with Fire★ (http://firestr.com http://firestr.com). Where it is GUI centric and has a built in app editor (all application code is viewable/editable). My only concern with the unikernel approach is that you can end up with a system where the code is not viewable and editable. Have you thought about that concern?

lewisl9029 12y ago

In terms of storage, here's something I've been working with for one of my current projects: http://remotestorage.io/ http://remotestorage.io/ Open protocol for synchronized personal storage. It uses a decentralized model where users provide and pay for their own storage. Could be game-changing if it takes off.

nileshtrivedi 12y ago
There is also https://tent.io/ https://tent.io/ It seemed quite promising to me but there haven't been updates from the team lately.
- seagreen 12y ago
  They got into YC with Flynn (https://flynn.io/ https://flynn.io/) which kept them busy for a while, but Tent 0.4 development is starting to pick up again.
kentonv 12y ago
remoteStorage is an elegant solution in a lot of ways, but there are some problems: * Apps written in this style have a very hard time being real-time collaborative. Maybe this will be solved by WebRTC, but it's a lot easier with a server component. (Sandstorm has several real-time collaborative apps, like EtherPad, EtherCalc, and Wave. Currently you can share these apps with other users by copy/pasting the URL; the sharing model will get more sophisticated eventually). * You need to set up a storage server for the apps to use, or use one of the big providers that like to do data mining. You then need to connect the app to the correct server, which is extra busy-work. * The permission model of these storage servers isn't terribly sophisticated. Often you'll end up granting the app broader permissions that you really want. * There is, of course, nothing stopping the app from storing data to other places as well, nor is there any isolation between separate documents opened by the same app. (On Sandstorm, each document is a separate instance of the app in a separate sandbox which cannot communicate with the outside world without permission.) Edit: Missed that you said you're working on RemoteStorage. Probably would have phrased differently if I noticed. Wasn't meant to be an attack. Sorry! (We actually like remoteStorage apps a lot in that they are often easily ported to Sandstorm. :) )

greggman 12y ago

I know this is going to sound ridiculous and maybe this is the wrong topic. I'm at a spot where I wish there was a closer to turn key framework for my project. I wrote 3 interdependent web servers, have them running on all the same virtual machine on digital ocean with varnish. Now I need to upgrade ubuntu and I don't want them to go down. So, apparently I need at least 3 machines if not more. One to run varnish or something similar, to direct to the other 2. Some way to bleed people over to one of the 2 machines. Once everyone is on one machine I can upgrade the unused machine. After that I can bleed them all back and then upgrade the other. Is this too small a niche? Is the answer I should have used "google cloud"? It just seems like each step is so much work. I certainly learned a lot (vagrant, ansible, and other stuff) although all of that knowledge will be probably obsolete before I need to do this again. Is there something I should have looked at? Sandstorm seems one level down. Like it's more like a replacement for the old lamp/cpanel isps.

e12e 12y ago
In your scenario, you need "just" two machines - and really only one running most of the time. You have varnish+appservers running one one vm now. What you need is one more vm, running the same stack. Depending on how you handle sessions this could be as easy as: 0: reduce ttl of dns to ~60s 1: bring up vm2==vm1 2: upgrade vm2 2.1: test vm2 2.2: shutdown vm2, take DO snapshot 2.3: instanciate vm/droplet of vm2 3: point/change dns from vm1 to vm2 4: wait for traffic to die on vm1 5: shutdown vm1 6: possibly increase ttl in dns again Next round, same procedure. This is not ready for HA -- easiest way there is running 2 vms like now, but with haproxy in front (so the vm has ha-varnish-3appservers), along with heart-beat and a shared ip. I don't thing DO supports that -- and it is probably not worth the hassle if you can live with dns changeover-time==downtime (say, minutes modulus wrongly configured dns caches). With a couple of dedicated ips, you can usually get away with vm2 taking vm1s ip (and vice-versa) -- but again that depends on your session set-up and/or if dropping sessions are ok. With sessions in some kind of replicated cache/db (eg: redis, mysql) -- you could probably set vm2 to slave over/replicate sessions, then do the dns [ed: or ip] switch[over].
- greggman 12y ago