3 ms·
I journal computer ideas and the ideas from database engineering are yet to percolate everywhere, especially to the desktop environment. Why is every company bu
by samsquire 4y ago
I journal computer ideas and the ideas from database engineering are yet to percolate everywhere, especially to the desktop environment. Why is every company building frontends and backends when the CRUD problem could be solved properly once and for all and reused everywhere? We did the same for communication and kernels with Linux, Windows and BSD, and BSD sockets which is shared by practically everybody. Your React frontend is legacy and shall be rewritten in 5 years. But BSD sockets or the Linux kernel doesn't get rewritten everyday.
Rather than writing hand rolled code for querying data structures and manipulating them as Linux does, we can define queries that retrieve data structures in in the shape we're looking for.
To put this simply, this is extremely high level, and the idea that data layout, data structure and algorithm can be unaggregated for cache locality and performance and developer experience. We can form materialized views on top of other materialized views and calculate the most efficient retrieval and storage format based on the structure of the data.
I suspect a materialized view, as in the data structures of the Linux kernel is more efficient than materializing a join at runtime.
One of my ideas is "ideas4 9. Query for data structure", https://github.com/samsquire/ideas4#9-query-for-data-structure https://github.com/samsquire/ideas4#9-query-for-data-structu... which is the idea we should be capable of querying to retrieve data structure in the shape we want. The shape of the data lends itself to solving certain kinds of problems.
An ideas3 is "Query database" https://github.com/samsquire/ideas3#17-query-database https://github.com/samsquire/ideas3#17-query-database, we persist queries as we persist data and use them to optimise data storage format. Rather than using a cost based optimiser to optimise how to retrieve data in the same uniform structure, we optimise how to store data itself. For example, for write heavy loads, we use LSM trees, we persist columnular when a query uses every field of every row and so on.
I also had the idea # 10. in ideas4 to persist data access patterns directly and optimise that. https://github.com/samsquire/ideas4#10-access-pattern-serialization https://github.com/samsquire/ideas4#10-access-pattern-serial...