4 ms·

> Persistence is Orthogonal to the Data Model, ... I have some experience with a custom data runtime where the persistence is orthogonal to the data model, wit

> Persistence is Orthogonal to the Data Model, ... I have some experience with a custom data runtime where the persistence is orthogonal to the data model, with silhouettes reminiscent of the described solutions in many of the features of my system, including multiple orthogonal/model-agnostic persistence backends, automatic data synchronisation, persistable executions, automatable schema changes, automatic reactivity. This direction can indeed bring about great savings in various parts of development; however, it seems to me that more subtlety than indicated in the post is required. The programmer must be provided with ergonomic means to give denotations for things like when and where to persist, in order to reduce data movement, and to keep the system performant (this does not violate orthogonality; we may specify e.g. to persist at the logical location, say, in the cloud, without having to specify the physical persistence). For instance, considering the case of schema changes, unless the system bundles its language inside the database, for performance sake, to perform such changes in an "Orthogonal Persistence" system external to the database would take an completely disproportionate amount of time relative to using SQL in the database. The data runtime I work with uses the idea of lenses (where valid lenses would necessarily be reversible) to allow for coherent, undoable schema changes, but I still resort to SQL for regular (eager) migrations (the lenses system for schema changes can still be useful for migrations applied lazily).

jerf 3y ago
Or, to put it another like, like visual programming, like "programming languages should be able to wear syntaxes like themes", like "there ought to be some sort of nocode type solution with all the power of conventional programming but easy enough for anyone to pick up", there are reasons why this is not how all programming works already. Good ones and big ones. And none of those reasons are that nobody has had the idea before or put work into implementing it. If you want to succeed with an approach like this, you're going to need to understand them. To be honest, such experience as I've had with automated persistence has generally actually strongly convinced me of the opposite, that it is a positive good that we do not get persistence everywhere. Consider the understanding that we get from functional programming that state is generally dangerous and to be carefully managed. Pervasive persistence fights hard against that careful management. Now state is not just in your program up until the OS process is terminated, but it's all permanently and automatically persisted. You get a huge new class of bugs involving path dependence on what bits of code were running across what bits of state when, and who ran which versions, and you hit them all the time, and they are nightmares to debug. At least when the program has the courtesy to completely cease existing and leave some particular concrete bit of state behind for the future, and then run through your code to load it back from that location, you have boundaries, and procedures for minimization and reconstruction. I actually shy away from too much automated persistence, and also have a very skeptical eye on the ever-present promise of memory that is as fast as RAM but persists like SSDs... I rather expect the computing world will discover that "rebooting" is not just a crutch, but actually a pretty fundamental and useful tool. However much in theory your software should never need it, in practice it's just too useful. That said, best of luck to those jousting with this windmill. I'm not saying don't joust, people in general probably don't joust enough, I'm just saying, learn the history of why this hasn't worked before and learn the challenges. Success is at the very least more likely if one learns from the previous efforts.
- dTal 3y ago
  Offtopic perhaps, but I am interested in reading an explanation of the good, big reasons why not "programming languages should be able to wear syntaxes like themes". Racket seems quite an interesting counterpoint, and I've never heard it argued as fundamentally flawed.
  - jerf 3y ago
    The people who think syntax should be worn like themes do not realize how deeply syntax interacts with a language. It isn't a skin. Your homework, should you choose to do it, because of course I can't make you and don't really care :), is to write a "Haskell theme" for Python. The theme must retain *args and **kwargs capabilities, as well as all other Python capabilities, though those two things will be one of the first major issues you hit. On the flip side, write a Python theme for Haskell. (This one based on my own stabs at it doesn't have such problems with the capabilities of the language, but it sure does take all of Haskell's elegance and wrap it in the grace of an elephant with a sprained leg.) Also, bear in mind my claim is not that it can't be done. My claim is more like, nobody would want to use the resulting language with the resulting "skin". Languages are not all the same. Even dynamic versus static imperative languages don't really "skin" very well; compare idiomatic Ruby ActiveRecord-based code with Rust code. The differences are not just skin-deep.

jerf 3y ago

Or, to put it another like, like visual programming, like "programming languages should be able to wear syntaxes like themes", like "there ought to be some sort of nocode type solution with all the power of conventional programming but easy enough for anyone to pick up", there are reasons why this is not how all programming works already. Good ones and big ones. And none of those reasons are that nobody has had the idea before or put work into implementing it. If you want to succeed with an approach like this, you're going to need to understand them. To be honest, such experience as I've had with automated persistence has generally actually strongly convinced me of the opposite, that it is a positive good that we do not get persistence everywhere. Consider the understanding that we get from functional programming that state is generally dangerous and to be carefully managed. Pervasive persistence fights hard against that careful management. Now state is not just in your program up until the OS process is terminated, but it's all permanently and automatically persisted. You get a huge new class of bugs involving path dependence on what bits of code were running across what bits of state when, and who ran which versions, and you hit them all the time, and they are nightmares to debug. At least when the program has the courtesy to completely cease existing and leave some particular concrete bit of state behind for the future, and then run through your code to load it back from that location, you have boundaries, and procedures for minimization and reconstruction. I actually shy away from too much automated persistence, and also have a very skeptical eye on the ever-present promise of memory that is as fast as RAM but persists like SSDs... I rather expect the computing world will discover that "rebooting" is not just a crutch, but actually a pretty fundamental and useful tool. However much in theory your software should never need it, in practice it's just too useful. That said, best of luck to those jousting with this windmill. I'm not saying don't joust, people in general probably don't joust enough, I'm just saying, learn the history of why this hasn't worked before and learn the challenges. Success is at the very least more likely if one learns from the previous efforts.

dTal 3y ago
Offtopic perhaps, but I am interested in reading an explanation of the good, big reasons why not "programming languages should be able to wear syntaxes like themes". Racket seems quite an interesting counterpoint, and I've never heard it argued as fundamentally flawed.
- jerf 3y ago
  The people who think syntax should be worn like themes do not realize how deeply syntax interacts with a language. It isn't a skin. Your homework, should you choose to do it, because of course I can't make you and don't really care :), is to write a "Haskell theme" for Python. The theme must retain *args and **kwargs capabilities, as well as all other Python capabilities, though those two things will be one of the first major issues you hit. On the flip side, write a Python theme for Haskell. (This one based on my own stabs at it doesn't have such problems with the capabilities of the language, but it sure does take all of Haskell's elegance and wrap it in the grace of an elephant with a sprained leg.) Also, bear in mind my claim is not that it can't be done. My claim is more like, nobody would want to use the resulting language with the resulting "skin". Languages are not all the same. Even dynamic versus static imperative languages don't really "skin" very well; compare idiomatic Ruby ActiveRecord-based code with Rust code. The differences are not just skin-deep.