6 ms·

I wonder how much of a factor the "One DB, one file" concept is in SQLite's success. I always loved that an SQLite DB is just a single file. People say "Who ca

by mg 2y ago

I wonder how much of a factor the "One DB, one file" concept is in SQLite's success. I always loved that an SQLite DB is just a single file. People say "Who cares how many files the DB is?". But in practice, I always found it to be very convenient. It makes the concept of a project easy to grasp when the data store is simply a file. I recently realized that a Django project can also be done in a single file. That made me like Django even more. I have the feeling that this type of logic and simplicty is indeed a factor in the long term success of a software project.

TeMPOraL 2y ago
I think it's the same reason Windows feels easier than Linux to manage software in, or why Docker has the extremely fast and wide adoption it enjoys, or why so many of us hate working with modern web stack. Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide). Docker, obviously, because it eliminates all the bullshit of installing software - again, especially painful on Linux systems - and gives you a single file that drives it all. The platform itself may be system-wide, but the thing you care about: individual software - is contained (literally) so it doesn't spill out. Easy to reason about. Modern web stack is a mess of tooling, some of which system-wide (so people will reach for Docker to contain it all!), with huge version churn and lots of moving pieces. Source code of your app doesn't feel like a major component of it; it's useless and unparseable without chains of finicky tooling. That's in contrast to "vanilla JS", old-school experience, where source files were the only thing that mattered. No transpilation, no build chains. You could fire a site up straight from your hard drive, or FTP/SCP it to a file host, and It Would Just Work. Same with SQLite. To this day I don't like RDBMSes, because they're a system-level platform designed for admins, while all I care about is the RDB part. My database. SQLite maps conceptually to how I think about it - one database, one file (+/- WAL). One well-defined place my data lives, that I can move around and between machines using regular file operation tools. Etc. Files over apps, always. EDIT: This also makes me very eager to try doing something with RedBean[0] - a webserver in a single Actually Portable Executable[1] that's also a ZIP file into which you put all the data you want it to serve. A turn-key website in a single file! -- [0] - https://redbean.dev/ https://redbean.dev/ [1] - https://justine.lol/ape.html https://justine.lol/ape.html
- pjc50 2y ago
  > Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide). This hasn't always been true. The portable executables are a nice user convenience for certain apps, but by no means all, and Windows is the platform for which "DLL hell" was coined. They even got briefly worse with the "GAC" for .NET Framework, although it rapidly became apparent what the problem with that was. These days they really want you to use .appx and ship via the Microsoft Store, so of course nobody does that and all Windows software is (a) nice single executables (b) sprawling DLL monsters (c) javascript in a box (Teams!) or (d) games. Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF.
  - TeMPOraL 2y ago
    > Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF. Isn't this already done better by jart's APE I mentioned earlier[0]? Though I imagine you could statically link SQLite to your APE and stuff a .sqlite DB into the ZIP file at the end, to use that for static storage. Or read-write storage if you also link zlib (though I'm not sure if you can portably overwrite your own executable on the hard drive). -- [0] - https://justine.lol/ape.html https://justine.lol/ape.html
    - pjc50 2y ago
      > I'm not sure if you can portably overwrite your own executable on the hard drive You definitely cannot do this on Windows.
      - jasomill 2y ago
        No, but you can 1. Write the new executable. 2. Rename the running executable. 3. Rename the new executable to the original name of the running executable. 4. Launch the new executable. 5. Exit the original executable. 6. Arrange for the new executable to delete the renamed original executable once the original has exited. This is admittedly a bit fiddly, requires care if you need to preserve attributes and ACLs of the original file, and has obvious issues in scenarios where multiple instances of the original executable may be running concurrently, but it does work. You also have to be careful to consider the ramifications of a system crash or snapshot backup in the middle of this process: ensuring the new executable is safely flushed to disk before it replaces the original is, under normal circumstances[1], easy (FlushFileBuffers); ensuring that one of the two executables always exists under the original name is hard (the Win32 ReplaceFiles API sounds like it should help here, but it doesn't). The cleanest solution to the latter problem I've come up with is to use the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations registry key as a backstop to ensure the move (3) happens, if necessary, as soon as the crashed/restored system is booted, but this requires local admin rights (or wanton disregard for system security, i.e., changing the ACL on this registry key to allow ordinary users to write to it, and therefore any file on the system on every reboot). [1] Here "normal circumstances" exclude cases where hardware lies[2] or users explicitly disable buffer flushing[3]. [2] https://devblogs.microsoft.com/oldnewthing/20100909-00/?p=12913 https://devblogs.microsoft.com/oldnewthing/20100909-00/?p=12... https://devblogs.microsoft.com/oldnewthing/20170510-00/?p=95505 https://devblogs.microsoft.com/oldnewthing/20170510-00/?p=95... [3] https://devblogs.microsoft.com/oldnewthing/20130416-00/?p=4643 https://devblogs.microsoft.com/oldnewthing/20130416-00/?p=46...
fud101 2y ago
Django in one file? How?
- adparadox 2y ago
  https://coltrane.readthedocs.io/en/latest/#other-minimal-django-projects https://coltrane.readthedocs.io/en/latest/#other-minimal-dja... has a list of a few options, but the most recent is https://github.com/pauloxnet/uDjango https://github.com/pauloxnet/uDjango.
- mg 2y ago
  You just need a wsgi.py file: import os import django from django.core.wsgi import get_wsgi_application os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi') application = get_wsgi_application() ROOT_URLCONF = 'mysite.wsgi' SECRET_KEY = 'hello' def index(request): return django.http.HttpResponse('This is the homepage') def cats(request): return django.http.HttpResponse('This is the cats page') urlpatterns = [ django.urls.path('', index), django.urls.path('cats', cats), ] Put that file in /var/www/mysite/mysite/wsgi.py, point your webserver to that file and you are good to go. In Apache you do it like this: ServerName mysite.local WSGIPythonPath /var/www/mysite <VirtualHost *:80> WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py <Directory /var/www/mysite/mysite> <Files wsgi.py> Require all granted </Files> </Directory> </VirtualHost>
  - pjc50 2y ago
    > import django Not really in one file, is it?
    - TeMPOraL 2y ago
      If you somehow managed to shove Python and Django into https://redbean.dev/2.0.html https://redbean.dev/2.0.html, then it might just work as bona fide one file.
    - mg 2y ago
influx 2y ago
This is one of the reasons I love Go, it can not only build to a single statically linked library, but you can embed files your app needs like html templates into that same binary. Deployment is copying that binary to where it needs to go.

by mg 2y ago

TeMPOraL 2y ago

I think it's the same reason Windows feels easier than Linux to manage software in, or why Docker has the extremely fast and wide adoption it enjoys, or why so many of us hate working with modern web stack. Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide). Docker, obviously, because it eliminates all the bullshit of installing software - again, especially painful on Linux systems - and gives you a single file that drives it all. The platform itself may be system-wide, but the thing you care about: individual software - is contained (literally) so it doesn't spill out. Easy to reason about. Modern web stack is a mess of tooling, some of which system-wide (so people will reach for Docker to contain it all!), with huge version churn and lots of moving pieces. Source code of your app doesn't feel like a major component of it; it's useless and unparseable without chains of finicky tooling. That's in contrast to "vanilla JS", old-school experience, where source files were the only thing that mattered. No transpilation, no build chains. You could fire a site up straight from your hard drive, or FTP/SCP it to a file host, and It Would Just Work. Same with SQLite. To this day I don't like RDBMSes, because they're a system-level platform designed for admins, while all I care about is the RDB part. My database. SQLite maps conceptually to how I think about it - one database, one file (+/- WAL). One well-defined place my data lives, that I can move around and between machines using regular file operation tools. Etc. Files over apps, always. EDIT: This also makes me very eager to try doing something with RedBean[0] - a webserver in a single Actually Portable Executable[1] that's also a ZIP file into which you put all the data you want it to serve. A turn-key website in a single file! -- [0] - https://redbean.dev/ https://redbean.dev/ [1] - https://justine.lol/ape.html https://justine.lol/ape.html

pjc50 2y ago
> Windows software is often enough "portable executables" that can run off the folder they were unzipped into, and historically, even installed apps could often enough be copy-pasted from their installation folder into another machine and Just Work (it was helpful, in particular, that all the DLL files lived next to the executable instead of being installed system-wide). This hasn't always been true. The portable executables are a nice user convenience for certain apps, but by no means all, and Windows is the platform for which "DLL hell" was coined. They even got briefly worse with the "GAC" for .NET Framework, although it rapidly became apparent what the problem with that was. These days they really want you to use .appx and ship via the Microsoft Store, so of course nobody does that and all Windows software is (a) nice single executables (b) sprawling DLL monsters (c) javascript in a box (Teams!) or (d) games. Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF.
- TeMPOraL 2y ago
  > Now I'm wondering if SQLite could be a viable executable format to unify PE and COFF. Isn't this already done better by jart's APE I mentioned earlier[0]? Though I imagine you could statically link SQLite to your APE and stuff a .sqlite DB into the ZIP file at the end, to use that for static storage. Or read-write storage if you also link zlib (though I'm not sure if you can portably overwrite your own executable on the hard drive). -- [0] - https://justine.lol/ape.html https://justine.lol/ape.html
  - pjc50 2y ago
    > I'm not sure if you can portably overwrite your own executable on the hard drive You definitely cannot do this on Windows.
    - jasomill 2y ago
      No, but you can 1. Write the new executable. 2. Rename the running executable. 3. Rename the new executable to the original name of the running executable. 4. Launch the new executable. 5. Exit the original executable. 6. Arrange for the new executable to delete the renamed original executable once the original has exited. This is admittedly a bit fiddly, requires care if you need to preserve attributes and ACLs of the original file, and has obvious issues in scenarios where multiple instances of the original executable may be running concurrently, but it does work. You also have to be careful to consider the ramifications of a system crash or snapshot backup in the middle of this process: ensuring the new executable is safely flushed to disk before it replaces the original is, under normal circumstances[1], easy (FlushFileBuffers); ensuring that one of the two executables always exists under the original name is hard (the Win32 ReplaceFiles API sounds like it should help here, but it doesn't). The cleanest solution to the latter problem I've come up with is to use the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations registry key as a backstop to ensure the move (3) happens, if necessary, as soon as the crashed/restored system is booted, but this requires local admin rights (or wanton disregard for system security, i.e., changing the ACL on this registry key to allow ordinary users to write to it, and therefore any file on the system on every reboot). [1] Here "normal circumstances" exclude cases where hardware lies[2] or users explicitly disable buffer flushing[3]. [2] https://devblogs.microsoft.com/oldnewthing/20100909-00/?p=12913 https://devblogs.microsoft.com/oldnewthing/20100909-00/?p=12... https://devblogs.microsoft.com/oldnewthing/20170510-00/?p=95505 https://devblogs.microsoft.com/oldnewthing/20170510-00/?p=95... [3] https://devblogs.microsoft.com/oldnewthing/20130416-00/?p=4643 https://devblogs.microsoft.com/oldnewthing/20130416-00/?p=46...

fud101 2y ago

Django in one file? How?

adparadox 2y ago
https://coltrane.readthedocs.io/en/latest/#other-minimal-django-projects https://coltrane.readthedocs.io/en/latest/#other-minimal-dja... has a list of a few options, but the most recent is https://github.com/pauloxnet/uDjango https://github.com/pauloxnet/uDjango.
mg 2y ago
You just need a wsgi.py file: import os import django from django.core.wsgi import get_wsgi_application os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi') application = get_wsgi_application() ROOT_URLCONF = 'mysite.wsgi' SECRET_KEY = 'hello' def index(request): return django.http.HttpResponse('This is the homepage') def cats(request): return django.http.HttpResponse('This is the cats page') urlpatterns = [ django.urls.path('', index), django.urls.path('cats', cats), ] Put that file in /var/www/mysite/mysite/wsgi.py, point your webserver to that file and you are good to go. In Apache you do it like this: ServerName mysite.local WSGIPythonPath /var/www/mysite <VirtualHost *:80> WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py <Directory /var/www/mysite/mysite> <Files wsgi.py> Require all granted </Files> </Directory> </VirtualHost>
- pjc50 2y ago
  > import django Not really in one file, is it?
  - TeMPOraL 2y ago
    If you somehow managed to shove Python and Django into https://redbean.dev/2.0.html https://redbean.dev/2.0.html, then it might just work as bona fide one file.
  - mg 2y ago

influx 2y ago

This is one of the reasons I love Go, it can not only build to a single statically linked library, but you can embed files your app needs like html templates into that same binary. Deployment is copying that binary to where it needs to go.