2 ms·

While having each grain implement a search API would be an elegant approach, it probably isn't practical. With the fine-grained model, we expect users to have h

by kentonv 11y ago

While having each grain implement a search API would be an elegant approach, it probably isn't practical. With the fine-grained model, we expect users to have hundreds, maybe thousands of grains, and possibly access to (and therefore searchability over) thousands more. Just replicating the query would probably be too expensive. Also keep in mind that a grain normally isn't running unless it is actively being used, and so such a search query would need to start up thousands of grains all at once. On the other hand, having a single central search index is a pretty big security liability. A bug in search would potentially give you read access to the contents of all documents on the server. Moreover, fine-grained encryption breaks down: since everyone can use the search index, it would need to be encrypted using a key that everyone has access to. Instead, each user should have a separate search index covering their own private data. This search index itself should run as a grain, to get the same protections / encryption as other grains. The search index grain should not be allowed to hold capabilities to any other grain. Communications between the search index and the grains being indexed should be one-way. The only two-way communication is the search query API, which should usually be accessible only to the user. As an (important) optimization, grains that are public or accessible to a large number of users should be indexed in shared indexes, not per-user.

static_noise 11y ago
We were thinking about two/three different things what the grain does: 1) My idea was that the grain exposes selected data to a search indexer. 2) Your idea was that the grain performs the search itself. I see why 2) would not work very well in the current implementation. But 1) should work well and make it possible to only expose selected data. (Also with central/user specific search.) Both ideas could by combined via per-grain-search-grains which only hold the local index and are accessed via a central search interface. The problem, however, remains, that searchable data is exposed data. To me it seems useful that some applications can have fine-grained control about what to expose. (Think of List of Establishments where each has a List of Customers with Credit Card information. You might make one or two things searchable but not everything.)

by kentonv 11y ago

static_noise 11y ago

We were thinking about two/three different things what the grain does: 1) My idea was that the grain exposes selected data to a search indexer. 2) Your idea was that the grain performs the search itself. I see why 2) would not work very well in the current implementation. But 1) should work well and make it possible to only expose selected data. (Also with central/user specific search.) Both ideas could by combined via per-grain-search-grains which only hold the local index and are accessed via a central search interface. The problem, however, remains, that searchable data is exposed data. To me it seems useful that some applications can have fine-grained control about what to expose. (Think of List of Establishments where each has a List of Customers with Credit Card information. You might make one or two things searchable but not everything.)