3 ms·

The copied vs. mapped memory situation is the only deal breaker in this experiment. Otherwise, file format unification would be a big step forward. The PE/COFF

The copied vs. mapped memory situation is the only deal breaker in this experiment. Otherwise, file format unification would be a big step forward. The PE/COFF executable file format used by Windows (and some older Unix systems) is a relational database as well. The same goes to .NET assembly format - it's a relational database too. The wheel gets reinvented over and over again.

stillpointlab 1mo ago
I was thinking that one could just write a loader that goes from SELF->ELF on load. You then pay the cost on startup. Probably not worth it for one-shot programs like ls/cat/etc, but for long running programs and daemons it might be workable.
- garganzol 1mo ago
  Another option is to adapt a database engine to support page-aligned blobs, which is probably not that hard to achieve.
  - stillpointlab 1mo ago
    Of course there are many options, but the benefit of the SQLite approach is the existing tooling, support, etc. Like others have said, one could just write a SQL interface over a set of virtual tables derived from the actual elf (if one just wanted a CRUD-like interface to a binary). All of the benefits of SQLite disappear once one diverges from the format in any way. At which point it would be better to ask: "what is the best first-principles implementation of this idea?", instead of "what is the minimal change to SQLite to achieve this specific narrow goal?"
  - setheron 1mo ago
    I might look at this next; with Nix it's easy to explore and rebuild seamlessly ;P
    - hedgehog 1mo ago
      I didn't understand the core issue, is it that due to non-alignment of the on-disk data you have to do a copy at load time, but if you could guarantee alignment of the actual blob content then you could use it directly?
      - garganzol 1mo ago
        Exactly, a guaranteed alignment and continuity of a blob makes it memory-mappable allowing to avoid copying the data back and forth.
        hedgehog 1mo ago
        On a brief background read it looks like another issue is that blobs are not necessarily stored in contiguous regions of the file. Maybe there's some clever way to combine a VFS shim and virtual tables to store page aligned blobs in a region of the same file as the main database.

stillpointlab 1mo ago

I was thinking that one could just write a loader that goes from SELF->ELF on load. You then pay the cost on startup. Probably not worth it for one-shot programs like ls/cat/etc, but for long running programs and daemons it might be workable.

garganzol 1mo ago
Another option is to adapt a database engine to support page-aligned blobs, which is probably not that hard to achieve.
- stillpointlab 1mo ago
  Of course there are many options, but the benefit of the SQLite approach is the existing tooling, support, etc. Like others have said, one could just write a SQL interface over a set of virtual tables derived from the actual elf (if one just wanted a CRUD-like interface to a binary). All of the benefits of SQLite disappear once one diverges from the format in any way. At which point it would be better to ask: "what is the best first-principles implementation of this idea?", instead of "what is the minimal change to SQLite to achieve this specific narrow goal?"
- setheron 1mo ago
  I might look at this next; with Nix it's easy to explore and rebuild seamlessly ;P
  - hedgehog 1mo ago
    I didn't understand the core issue, is it that due to non-alignment of the on-disk data you have to do a copy at load time, but if you could guarantee alignment of the actual blob content then you could use it directly?
    - garganzol 1mo ago
      Exactly, a guaranteed alignment and continuity of a blob makes it memory-mappable allowing to avoid copying the data back and forth.
      - hedgehog 1mo ago
        On a brief background read it looks like another issue is that blobs are not necessarily stored in contiguous regions of the file. Maybe there's some clever way to combine a VFS shim and virtual tables to store page aligned blobs in a region of the same file as the main database.