3 ms·
Big data sets in general composes badly, you have to do the filter+sort+pagination as close to the data as possible. It has to be embedded in the query protocol
by Too 10y ago
Big data sets in general composes badly, you have to do the filter+sort+pagination as close to the data as possible. It has to be embedded in the query protocol, you don't want to transfer one billion entries over the network, just to later filter out 5 of them and then sort.
- Someone 10y agoThat depends on the expressive power of your protocol. For example, a JVM program running on a client could send a _program_ to a JVM program running on a server to run it there, with the server using the security features of the JVM to enforce that that program is well-behaved by limiting what interfaces it can call, how long it can run, etc. With that in place, a service could provide a call that produces the data and takes client-provided Java code fragments that filter and sort the result, and use hotspot to do optimizations across the stack. (SQL, in both theory, can accept query fragments to insert into queries, too, and that can be done in practice, too, by concatenation of query fragments, but it always feels like a hack, and typically doesn't enforce security)