7 ms·

> How do you find a program to run if there are no directories? How do you save stuff, if there's nowhere to save it to? How do you compile code, when there is

> How do you find a program to run if there are no directories? How do you save stuff, if there's nowhere to save it to? How do you compile code, when there is no way to #include one file into another because there are no files, and where does the resulting binary go? The term for this is orthogonal persistence. Lots of people have thought about this before. In terms of Lisp OSes, Squeak (Smalltalk), and the long-dead TUNES project[1]. You don't need special memory to do this, but I imagine it could help. There actually is a modern mainstream OS that does this already. It's called iOS. No, I'm not joking. It effectively provides the illusion of persisted state which is good enough for the user to believe it. No more saving files. Apps just do it. Automatically. Android does the same. Both also provide the illusion of always-on apps. You click a button and apps, typically, load so fast you don't notice they are loading from RAM or flash storage. It's incredibly seamless and precisely what people had in mind 20 years ago. [1] http://tunes.org/cliki/orthogonal_20persistence.html http://tunes.org/cliki/orthogonal_20persistence.html

icodestuff 4y ago
Developers don't see that though. It only looks like that on iOS to users. Behind the scenes, apps are still written using files. Hell, Apple eventually had to add its own "Files" app because storing things and finding them again was getting complicated.
duskwuff 4y ago
Another OS that did something like this, IIRC, was early versions of PalmOS. There was literally no distinction between storage and memory -- everything was stored in battery-backed RAM (except the OS, which was in ROM).
- zootboy 4y ago
  It was the same deal with some Windows Mobile 2003 (and presumably other) devices. Except it was Windows, so of course it had to have a weird interface. There was a slider in the control panel that let you change the allocation of RAM between "memory" and "storage". But the system would auto-adjust this allocation, so the slider's effect was temporary. It was quite strange. See the first screenshot on this page: https://www.pocketpcfaq.com/faqs/5.0/memory_management.htm https://www.pocketpcfaq.com/faqs/5.0/memory_management.htm
  - iggldiggl 4y ago
    I remember you could somewhere (Regular settings? Or perhaps some hidden registry-tweak?) turn off the automatic memory allocation, so I moved as much software and documents as possible to the internal flash storage (still relatively tiny) respectively an external memory card and then moved the slider fully over to the left in order to have as much program memory as possible.
- Brian_K_White 4y ago
  That also describes a TRS-80 Model 100. It still calls files files, but a more meaningful all-ram distinction besides the simple fact everything is in battery backed ram, is that in most cases when a file is used, even for editing not just reading, it is used in-place, not copied from a storage area to a working area. This is a 1984 device with 32k of ram and 32k of rom. Partly this is because of the tiny amount of available ram, none to waste on copies. Partly this is because the cpu doesn't have a relative-jump instruction, so binaries are not easily relocatable. So binaries get compiled to run at a specific static address, sometimes modified at install-time to stack up next to whatever else you already have installed, and that's just where that binary lives forever after that whether it's currently being executed or not. So that flat model is actually driven by a couple of severe limitations. The "files" are really only files for display in the main menu and when exporting like saving to tape or disk. While on-device, the files are really just areas of ram. They're always there at whatever address they're at, whether they're being used right now or not. For instance there are such things as BASIC programs that read data stored in comments to do things the language doesn't normally provide for, because the comments are ultimately nothing but a predictable address in memory like any other.
  - duskwuff 4y ago
    > This is a 1984 device with 32k of ram and 32k of rom. The original Palm Pilot wasn't all that much larger. Despite being released 12 years later, it only had 128 KB RAM and 512 KB ROM. The PalmOS memory model was unusual. Rather than trying to explain it all here, there's a decent summary at: https://www.fuw.edu.pl/~michalj/palmos/Memory.html https://www.fuw.edu.pl/~michalj/palmos/Memory.html
silon42 4y ago
As a user, I don't like the fact that apps now own my data.
- alpaca128 4y ago
  That's the typical difference in expectations between users. Some users don't even want to think about what the computer is doing and want to avoid all the complexity, others don't want the computer to do any thinking for them because they might be aware of all the ways things could go wrong, and part of that is transparency and data ownership/control. I'm definitely in the second group, whenever I see an app just saying "click this button to magically do the thing" I immediately worry about where the files are stored and how I'd back them up and how I could reverse the changes when (not if) something goes wrong.
- wolpoli 4y ago
  I remember how Windows 95 was said to be document centric. It's now 2022 and we don't have documents to speak of.
  - solardev 4y ago
    Maybe to your point, I don't even understand what "document" means in this context. Can you explain?
    - BlueTemplar 4y ago
      https://invisibleup.com/articles/34/ https://invisibleup.com/articles/34/ > What Windows 95 wanted to do was to create a document-centric workflow where the application was out of the user's way and it only provided the means to let the user manipulate files. > It needed to somehow take this file-centric workflow and bolt it onto application-centric DOS.
atoav 4y ago
So you are saving files, but not showing them to the user? Sounds like progress.
- Gravyness 4y ago
  Don't worry, the apps know where the files are when you need them. And you only need them when you are using the app, after all you will own nothing and you will be... wait now that I think about it, you do have the files somewhere on your hard drive, right? so they're technically actually yours, right? my mistake, yeah so nothing to worry here. And there are some rare exceptions of those apps that uploads your stuff to the cloud for security and for easy backups, such a great technology because if you ever lose your computer you can just buy another one and the 'files' will be magically there (as long as you keep paying the app subscription)! Hopefully the author also implemented the app in a way that can be used offline. Btw you will need the newest version of the computer because your apps updated so that's also great because new features are good! Your files are basically your creation, you can't just leave them behind.
iainmerrick 4y ago
Orthogonal persistence was a big research area at my university when I was studying CS. The pitch was that it would be like garbage collection for your persistent data: just like GC means you never have to worry about manually deallocating stuff, orthogonal persistence would mean you never have to worry about manually serialising stuff. I was skeptical because I could never figure out how they proposed to manage code update -- if your data is always live, how do you safely modify the code? And I still haven't seen a good answer to that! Although I haven't studied this area in a long time, so maybe it has been solved? I'd be interested to read any links people have about this. I think of those environments like Squeak as a kind of "object soup", where everything is in one place, rather than rigidly separated into source code, compiled binary, data schema, serialised data, live server, etc. I thought back when I first heard of orthogonal persistence, and still think now, that that rigid separation of concerns is essential for maintainability. It's good to have carefully managed source control, and it's good to have a carefully managed data schema. Have any Squeak-like environments been really successfully in large-scale systems or user-facing apps? I'd say infrastructure as code has been a much more important innovation than orthogonal persistence; and it takes exactly the opposite approach, of explicitly serialising live stuff (your active deployment) into flat files. Edit to add: maybe another way to put it is, the promise of orthogonal persistence is that you never need to reboot your computer. But that's a bad idea, because as we know, sometimes rebooting is (sadly) the only way to fix a problem.
- imtringued 4y ago
  There is no good upgrade path. Database migrations with existing data are a pain in the ass. Schema upgrades of capnproto are a pain in the ass, etc. Separating working memory from persistent memory has obvious benefits in terms of mental compartmentalization. Rebooting is good because it is like a fire drill, if you don't do it regularly, the real thing will happen one day and you won't be prepared. It is the same with backups, you want to replay them regularly. Of course that doesn't inherently invalidate the idea, it just means it isn't as useful as was previously thought.
xtracto 4y ago
Aaaah, and this is why we need Steve Jobs, or someone with his power, guts, vision. I think this is the kind of technology that Steve Jobs would have pushed for the new iPhone/iPod replacement. Similarly to how he pushed the "miniature mechanical disks" while the majority of Mp3 players were struggling with 4/8gb flash memory. I wish that whoever leads The Pixel would adopt this memory... shit, even RaspBerry or some other "underdog" could create an Android phone that used this instead of RAM/Flash.
lproven 4y ago
[Article author here] Excellent point, thank you. I was aware of it, but I should have used the correct term, you're right.

icodestuff 4y ago

Developers don't see that though. It only looks like that on iOS to users. Behind the scenes, apps are still written using files. Hell, Apple eventually had to add its own "Files" app because storing things and finding them again was getting complicated.

duskwuff 4y ago

Another OS that did something like this, IIRC, was early versions of PalmOS. There was literally no distinction between storage and memory -- everything was stored in battery-backed RAM (except the OS, which was in ROM).

zootboy 4y ago
It was the same deal with some Windows Mobile 2003 (and presumably other) devices. Except it was Windows, so of course it had to have a weird interface. There was a slider in the control panel that let you change the allocation of RAM between "memory" and "storage". But the system would auto-adjust this allocation, so the slider's effect was temporary. It was quite strange. See the first screenshot on this page: https://www.pocketpcfaq.com/faqs/5.0/memory_management.htm https://www.pocketpcfaq.com/faqs/5.0/memory_management.htm
- iggldiggl 4y ago
  I remember you could somewhere (Regular settings? Or perhaps some hidden registry-tweak?) turn off the automatic memory allocation, so I moved as much software and documents as possible to the internal flash storage (still relatively tiny) respectively an external memory card and then moved the slider fully over to the left in order to have as much program memory as possible.
Brian_K_White 4y ago
That also describes a TRS-80 Model 100. It still calls files files, but a more meaningful all-ram distinction besides the simple fact everything is in battery backed ram, is that in most cases when a file is used, even for editing not just reading, it is used in-place, not copied from a storage area to a working area. This is a 1984 device with 32k of ram and 32k of rom. Partly this is because of the tiny amount of available ram, none to waste on copies. Partly this is because the cpu doesn't have a relative-jump instruction, so binaries are not easily relocatable. So binaries get compiled to run at a specific static address, sometimes modified at install-time to stack up next to whatever else you already have installed, and that's just where that binary lives forever after that whether it's currently being executed or not. So that flat model is actually driven by a couple of severe limitations. The "files" are really only files for display in the main menu and when exporting like saving to tape or disk. While on-device, the files are really just areas of ram. They're always there at whatever address they're at, whether they're being used right now or not. For instance there are such things as BASIC programs that read data stored in comments to do things the language doesn't normally provide for, because the comments are ultimately nothing but a predictable address in memory like any other.
- duskwuff 4y ago
  > This is a 1984 device with 32k of ram and 32k of rom. The original Palm Pilot wasn't all that much larger. Despite being released 12 years later, it only had 128 KB RAM and 512 KB ROM. The PalmOS memory model was unusual. Rather than trying to explain it all here, there's a decent summary at: https://www.fuw.edu.pl/~michalj/palmos/Memory.html https://www.fuw.edu.pl/~michalj/palmos/Memory.html

silon42 4y ago

As a user, I don't like the fact that apps now own my data.

alpaca128 4y ago
That's the typical difference in expectations between users. Some users don't even want to think about what the computer is doing and want to avoid all the complexity, others don't want the computer to do any thinking for them because they might be aware of all the ways things could go wrong, and part of that is transparency and data ownership/control. I'm definitely in the second group, whenever I see an app just saying "click this button to magically do the thing" I immediately worry about where the files are stored and how I'd back them up and how I could reverse the changes when (not if) something goes wrong.
wolpoli 4y ago
I remember how Windows 95 was said to be document centric. It's now 2022 and we don't have documents to speak of.
- solardev 4y ago
  Maybe to your point, I don't even understand what "document" means in this context. Can you explain?
  - BlueTemplar 4y ago
    https://invisibleup.com/articles/34/ https://invisibleup.com/articles/34/ > What Windows 95 wanted to do was to create a document-centric workflow where the application was out of the user's way and it only provided the means to let the user manipulate files. > It needed to somehow take this file-centric workflow and bolt it onto application-centric DOS.

atoav 4y ago

So you are saving files, but not showing them to the user? Sounds like progress.

Gravyness 4y ago
Don't worry, the apps know where the files are when you need them. And you only need them when you are using the app, after all you will own nothing and you will be... wait now that I think about it, you do have the files somewhere on your hard drive, right? so they're technically actually yours, right? my mistake, yeah so nothing to worry here. And there are some rare exceptions of those apps that uploads your stuff to the cloud for security and for easy backups, such a great technology because if you ever lose your computer you can just buy another one and the 'files' will be magically there (as long as you keep paying the app subscription)! Hopefully the author also implemented the app in a way that can be used offline. Btw you will need the newest version of the computer because your apps updated so that's also great because new features are good! Your files are basically your creation, you can't just leave them behind.

iainmerrick 4y ago

Orthogonal persistence was a big research area at my university when I was studying CS. The pitch was that it would be like garbage collection for your persistent data: just like GC means you never have to worry about manually deallocating stuff, orthogonal persistence would mean you never have to worry about manually serialising stuff. I was skeptical because I could never figure out how they proposed to manage code update -- if your data is always live, how do you safely modify the code? And I still haven't seen a good answer to that! Although I haven't studied this area in a long time, so maybe it has been solved? I'd be interested to read any links people have about this. I think of those environments like Squeak as a kind of "object soup", where everything is in one place, rather than rigidly separated into source code, compiled binary, data schema, serialised data, live server, etc. I thought back when I first heard of orthogonal persistence, and still think now, that that rigid separation of concerns is essential for maintainability. It's good to have carefully managed source control, and it's good to have a carefully managed data schema. Have any Squeak-like environments been really successfully in large-scale systems or user-facing apps? I'd say infrastructure as code has been a much more important innovation than orthogonal persistence; and it takes exactly the opposite approach, of explicitly serialising live stuff (your active deployment) into flat files. Edit to add: maybe another way to put it is, the promise of orthogonal persistence is that you never need to reboot your computer. But that's a bad idea, because as we know, sometimes rebooting is (sadly) the only way to fix a problem.

imtringued 4y ago
There is no good upgrade path. Database migrations with existing data are a pain in the ass. Schema upgrades of capnproto are a pain in the ass, etc. Separating working memory from persistent memory has obvious benefits in terms of mental compartmentalization. Rebooting is good because it is like a fire drill, if you don't do it regularly, the real thing will happen one day and you won't be prepared. It is the same with backups, you want to replay them regularly. Of course that doesn't inherently invalidate the idea, it just means it isn't as useful as was previously thought.

xtracto 4y ago

Aaaah, and this is why we need Steve Jobs, or someone with his power, guts, vision. I think this is the kind of technology that Steve Jobs would have pushed for the new iPhone/iPod replacement. Similarly to how he pushed the "miniature mechanical disks" while the majority of Mp3 players were struggling with 4/8gb flash memory. I wish that whoever leads The Pixel would adopt this memory... shit, even RaspBerry or some other "underdog" could create an Android phone that used this instead of RAM/Flash.

lproven 4y ago

[Article author here] Excellent point, thank you. I was aware of it, but I should have used the correct term, you're right.