4 ms·

Hi! I'm the founder / tech lead of Sandstorm. I wrote that "How it Works" page (https://sandstorm.io/how-it-works https://sandstorm.io/how-it-works) and designe

by kentonv 10y ago

Hi! I'm the founder / tech lead of Sandstorm. I wrote that "How it Works" page (https://sandstorm.io/how-it-works https://sandstorm.io/how-it-works) and designed the architecture it is describing. Before Sandstorm, I worked at Google on infrastructure for eight years. As you may know, Google has been doing containers and microservices for over a decade. What I describe on the page is largely based on that experience. > Plenty of precedent there. I've personally never observed a production (non-research) server architecture which containerizes and sandboxes application servers on a per-document level. That is, say, with Etherpad on Sandstorm, every document you create is in a separate container isolated from the others. I would love it if you could name some specific examples that work this way, as I'd very might like to see what others have done.

solipsism 10y ago
I get the grain-per-document thing. Sounds interesting for at least a subset of applications, and disagree with the grandparent that it's common practice. First question that comes to mind is how the data source is sharded like that. You say in another post that you don't mandate a tech stack. So let's say I want to store my data in MySQL. Are you saying each document literally has its own private database instance? I have many more questions, but they will depend on your answer to that question. :)
- kentonv 10y ago
  > Are you saying each document literally has its own private database instance? Yes. Each "grain" (our catch-all term; could mean "an Etherpad document" or "a Gitlab repo" or "a Wekan board", etc.) gets a slice of filesystem to do whatever it wants with. Of course, MySQL is totally overkill for this task. We generally encourage people to use sqlite instead. But there are a couple apps that do actually run MySQL in the container, mostly because switching to sqlite would have been too much work. There are also several apps that use Mongo, but it turns out Mongo 3's WiredTiger engine is reasonably efficient for small data sets, so it works out. The model definitely is friendlier to some kinds of apps than others. Productivity apps with clear data boundaries -- Etherpad (document editor), Wekan (Trello clone), Rocket.Chat (Slack clone), Gitlab, etc. -- work very well, and are our main focus. "Big data processing" apps are not well-suited to this model. We're happy to cede those, as there are tons of people doing great work in that domain.
  - simplehuman 10y ago
    In the case of Rocket.Chat, is each channel a grain or is the entire chat a grain? It appears to me it should be a grain but I am not sure.
    - kentonv 10y ago
      Currently, the whole Rocket.Chat instance is a grain. However, it would be "more Sandstorm-y" if each channel were a separate grain. For that to work well, though, we need some UI features to emulate what Rocket.Chat already does internally: * The ability for grains to highlight themselves in the sidebar when there is activity or when you've been mentioned, so that you know to check them. This API actually just launched last week! So now the apps are being updated to use it. * Some way to create "private message" grains between two people. We haven't done this yet but we have some ideas. Soon, I hope to see Rocket.Chat add a "single-channel grain" mode where its own sidebar is removed and it relies on Sandstorm's sidebar. This would provide better security for channels which should only be accessible to a subset of people. Probably, the multi-channel mode will stick around too, for cases where you want the same people to have access to many channels.
- Nullabillity 10y ago
  > Are you saying each document literally has its own private database instance? Yup. On the other hand, those containers are garbage-collected when nobody is viewing them.
  - kentonv 10y ago
    To be clear: The container shuts down, but the storage stays. Since the data set is small, apps start back up quickly on-demand. Moreover, the storage for app assets (code, binaries, images, etc.) is shared read-only among all instances, so the marginal storage usage of each grain is only the per-grain data, which is usually very small.
andrewstuart2 10y ago
First, apologies for my snippy knee-jerk comment. I should have probably taken more time to compose my thoughts. And congratulations on the platform's popularity (at least as far as I can gather from GitHub stars). It seems that it definitely hits a sweet spot for many people as far as ease-of-use. Additionally, I think it integrates a lot of great ideas, some in combinations that thus far are not quite mainstream. I think what bothered me is actually best described by the current top comment. "How it works," too, seems like mostly marketing hype, and in my personal experience many of its technical claims regarding novelty or distinction are inaccurate and come across as sanctimonious. As you may guess from my mentioning Android above, that's the most mainstream system that I think obtains the described level of sandboxing while allowing highly-fine-grained access control when the data is to leave a sandbox. For example, if I share a photo from one app to another, it does not gain access to all photos but rather only the one I selected. The same-origin policy also springs to mind as a great example of application and data sandboxing by default. Sandstorm does have very fine-grained defaults, which I think is the best, and of course capabilities are the more feasible AuthZ primitive for distributed system [1]. All in all, I think we have many of the same opinions, and I've seen the success of similar models several times in the past. This is probably a good deal of why I find it frustrating and disingenuous when they're described as novel. [1] https://youtu.be/2htZv45lvLM?list=PL-XXv-cvA_iBDyz-ba4yDskqMDY6A1w_c&t=2650 https://youtu.be/2htZv45lvLM?list=PL-XXv-cvA_iBDyz-ba4yDskqM...
- kentonv 10y ago
  Android sandboxes apps, but it does not sandbox documents within apps. For example, when I use the gmail app to connect to both personal and work email, there is nothing stopping an exploit that lands in my personal inbox from reading all my work email. Similarly, same-origin policy generally divides apps (in the form of domains), but not documents -- if I have two Google Docs, and someone XSS's one of them, they can read and infect the other. Of course, on a phone (or a single user's browser), you usually have only one user, which makes it less clear why apps should be internally partitioned. On the server side, the traditional model is multi-tenancy, where all of Google's billions of users are potentially hitting the same server containers (modulo geography-based load-balancing). We definitely don't claim that the capability model is new, and I'm happy to acknowledge that many research systems based on a strong capability model implement something like Sandstorm's fine-grained containerization. Mark Miller, one of the (probably, "the") foremost researchers on capability-based security is an advisor to the Sandstorm project -- you'll see him listed on the Team page. Marc Steigler -- who I believe invented the word "Powerbox" -- is also a friend, as are Tyler Close (Waterken), Norm Hardy (KeyKOS), Alan Karp, etc. The thing is, though, CapDesk and KeyKOS never made it beyond being research systems -- proving a point, but not widely used in the real world. Also, they predated the "cloud services" era. I would still like to know if there are other examples of server-side fine-grained containerization that I'm missing.

by kentonv 10y ago

solipsism 10y ago

I get the grain-per-document thing. Sounds interesting for at least a subset of applications, and disagree with the grandparent that it's common practice. First question that comes to mind is how the data source is sharded like that. You say in another post that you don't mandate a tech stack. So let's say I want to store my data in MySQL. Are you saying each document literally has its own private database instance? I have many more questions, but they will depend on your answer to that question. :)

kentonv 10y ago
> Are you saying each document literally has its own private database instance? Yes. Each "grain" (our catch-all term; could mean "an Etherpad document" or "a Gitlab repo" or "a Wekan board", etc.) gets a slice of filesystem to do whatever it wants with. Of course, MySQL is totally overkill for this task. We generally encourage people to use sqlite instead. But there are a couple apps that do actually run MySQL in the container, mostly because switching to sqlite would have been too much work. There are also several apps that use Mongo, but it turns out Mongo 3's WiredTiger engine is reasonably efficient for small data sets, so it works out. The model definitely is friendlier to some kinds of apps than others. Productivity apps with clear data boundaries -- Etherpad (document editor), Wekan (Trello clone), Rocket.Chat (Slack clone), Gitlab, etc. -- work very well, and are our main focus. "Big data processing" apps are not well-suited to this model. We're happy to cede those, as there are tons of people doing great work in that domain.
- simplehuman 10y ago
  In the case of Rocket.Chat, is each channel a grain or is the entire chat a grain? It appears to me it should be a grain but I am not sure.
  - kentonv 10y ago
    Currently, the whole Rocket.Chat instance is a grain. However, it would be "more Sandstorm-y" if each channel were a separate grain. For that to work well, though, we need some UI features to emulate what Rocket.Chat already does internally: * The ability for grains to highlight themselves in the sidebar when there is activity or when you've been mentioned, so that you know to check them. This API actually just launched last week! So now the apps are being updated to use it. * Some way to create "private message" grains between two people. We haven't done this yet but we have some ideas. Soon, I hope to see Rocket.Chat add a "single-channel grain" mode where its own sidebar is removed and it relies on Sandstorm's sidebar. This would provide better security for channels which should only be accessible to a subset of people. Probably, the multi-channel mode will stick around too, for cases where you want the same people to have access to many channels.
Nullabillity 10y ago
> Are you saying each document literally has its own private database instance? Yup. On the other hand, those containers are garbage-collected when nobody is viewing them.
- kentonv 10y ago
  To be clear: The container shuts down, but the storage stays. Since the data set is small, apps start back up quickly on-demand. Moreover, the storage for app assets (code, binaries, images, etc.) is shared read-only among all instances, so the marginal storage usage of each grain is only the per-grain data, which is usually very small.

andrewstuart2 10y ago

First, apologies for my snippy knee-jerk comment. I should have probably taken more time to compose my thoughts. And congratulations on the platform's popularity (at least as far as I can gather from GitHub stars). It seems that it definitely hits a sweet spot for many people as far as ease-of-use. Additionally, I think it integrates a lot of great ideas, some in combinations that thus far are not quite mainstream. I think what bothered me is actually best described by the current top comment. "How it works," too, seems like mostly marketing hype, and in my personal experience many of its technical claims regarding novelty or distinction are inaccurate and come across as sanctimonious. As you may guess from my mentioning Android above, that's the most mainstream system that I think obtains the described level of sandboxing while allowing highly-fine-grained access control when the data is to leave a sandbox. For example, if I share a photo from one app to another, it does not gain access to all photos but rather only the one I selected. The same-origin policy also springs to mind as a great example of application and data sandboxing by default. Sandstorm does have very fine-grained defaults, which I think is the best, and of course capabilities are the more feasible AuthZ primitive for distributed system [1]. All in all, I think we have many of the same opinions, and I've seen the success of similar models several times in the past. This is probably a good deal of why I find it frustrating and disingenuous when they're described as novel. [1] https://youtu.be/2htZv45lvLM?list=PL-XXv-cvA_iBDyz-ba4yDskqMDY6A1w_c&t=2650 https://youtu.be/2htZv45lvLM?list=PL-XXv-cvA_iBDyz-ba4yDskqM...

kentonv 10y ago
Android sandboxes apps, but it does not sandbox documents within apps. For example, when I use the gmail app to connect to both personal and work email, there is nothing stopping an exploit that lands in my personal inbox from reading all my work email. Similarly, same-origin policy generally divides apps (in the form of domains), but not documents -- if I have two Google Docs, and someone XSS's one of them, they can read and infect the other. Of course, on a phone (or a single user's browser), you usually have only one user, which makes it less clear why apps should be internally partitioned. On the server side, the traditional model is multi-tenancy, where all of Google's billions of users are potentially hitting the same server containers (modulo geography-based load-balancing). We definitely don't claim that the capability model is new, and I'm happy to acknowledge that many research systems based on a strong capability model implement something like Sandstorm's fine-grained containerization. Mark Miller, one of the (probably, "the") foremost researchers on capability-based security is an advisor to the Sandstorm project -- you'll see him listed on the Team page. Marc Steigler -- who I believe invented the word "Powerbox" -- is also a friend, as are Tyler Close (Waterken), Norm Hardy (KeyKOS), Alan Karp, etc. The thing is, though, CapDesk and KeyKOS never made it beyond being research systems -- proving a point, but not widely used in the real world. Also, they predated the "cloud services" era. I would still like to know if there are other examples of server-side fine-grained containerization that I'm missing.