7 ms·

Slight alternative: let smart people who design rock-solid software handle serialization for you, use SQLite to get the benefits of files-on-disk, portability,

by z3ugma 5y ago

Slight alternative: let smart people who design rock-solid software handle serialization for you, use SQLite to get the benefits of files-on-disk, portability, and make it easier if you use different datatypes in your scripts.

nemetroid 5y ago
I think SQLite fits the bill described in the post. It's included in Python's standard library, stores data on the file system, and doesn't require an external process.
- phkahler 5y ago
  But it IS an additional dependency.
  - BiteCode_dev 5y ago
    If you have cPython, you have sqlite. How is that an additional dependency?
    - WillDaSilva 5y ago
      Not all distributions of CPython 3 come with sqlite, as it's an optional feature that is only included when the sqlite library is found at compile time.
      - BiteCode_dev 5y ago
        The official installers for windows, linux, the app stores and the red hat and debian repo all include it. At this point, this is nitpicking. Tkinter I would understand, but not sqlite. Even django doesn't work without it.
        WillDaSilva 5y ago
        The most common scenario in which you may be without the sqlite package is when you build Python from source, which is fairly common nowadays thanks to tools like pyenv. Also the FreeBSD Python package doesn't come with sqlite by default.
        BiteCode_dev 5y ago
        Freebsd and compiling is everything but common. Probably 1 coder for 100000. If you do that, you are assumed to have removed the safety belt and know what you are doing. I know we are on HN, but you have to remember half the python coders are not even dev. They are geographers, mathematicians, biologists, students, teachers, data analysts. And of course they are on mac or windows. Even I, and I started to code in python 2.4, never have to compile python. I do it for fun from time to time, but I don't use that professionnally. If you use a platform where sqlite is not there, you are an exeception in a niche of minority. I drive my car assuming there will be breaks on it. It can be missing, but it's not a great assumption.
        im3w1l 5y ago
        pyenv as he mentioned is a common tool. That it happens to compile under the hood is an implementation detail that users may not even be aware off.
        BiteCode_dev 5y ago
        > pyenv as he mentioned is a common tool. Pyenv is not a "common tool". It's used by a very small minority of devs since: - it's used only by dev, and half of the python coders are not devs - it works only on unix. Most people install python on windows (https://discuss.python.org/t/python-download-stats-for-may-2020/4454 https://discuss.python.org/t/python-download-stats-for-may-2..., and that doesn't take in consideration anaconda or the appstore). Unless you count the obscure fork used by even less people. - And yet on mac, if they don't use the official installer, they use mostly brew, not pyenv. On Linux, the officials repos or things like deadnsnake or epel. And of course docker. Pyenv is a tiny fractions of that, because any alternative solution is better known and easier. - hard to use because it compiles things, fails easily in non obvious ways and assume you manage the path correctly, so it's really, really, popular only with people that want to deal with that. In fact the setups instructions are huge: https://github.com/pyenv/pyenv#how-it-works https://github.com/pyenv/pyenv#how-it-works - that's just what we see in the field. I go to a lot of different shops every years because I'm a freelancers, and I encountered pyenv twice in 10 years. > That it happens to compile under the hood is an implementation detail that users may not even be aware off. Of course not. The setup docs themself tell you to "Install Python build dependencies before attempting to install a new Python version.". So not only you have to be aware of it, but you must be capable of setting up a compilation environment, and if you fail to do so, it will not work. pyenv is not standard python. It's not "a common tool". It's a tool for a niche of specialists with domain specific knowledge.
  - deleted 5y ago
    [deleted]
KronisLV 5y ago
This really reminds me of an article that i read a while ago, called "SQLite as an application file format", which seemed to express some ideas like that: https://www.sqlite.org/draft/appfileformat.html https://www.sqlite.org/draft/appfileformat.html

by z3ugma 5y ago

nemetroid 5y ago

I think SQLite fits the bill described in the post. It's included in Python's standard library, stores data on the file system, and doesn't require an external process.

phkahler 5y ago
But it IS an additional dependency.
- BiteCode_dev 5y ago
  If you have cPython, you have sqlite. How is that an additional dependency?
  - WillDaSilva 5y ago
    Not all distributions of CPython 3 come with sqlite, as it's an optional feature that is only included when the sqlite library is found at compile time.
    - BiteCode_dev 5y ago
      The official installers for windows, linux, the app stores and the red hat and debian repo all include it. At this point, this is nitpicking. Tkinter I would understand, but not sqlite. Even django doesn't work without it.
      - WillDaSilva 5y ago
        The most common scenario in which you may be without the sqlite package is when you build Python from source, which is fairly common nowadays thanks to tools like pyenv. Also the FreeBSD Python package doesn't come with sqlite by default.
        BiteCode_dev 5y ago
        Freebsd and compiling is everything but common. Probably 1 coder for 100000. If you do that, you are assumed to have removed the safety belt and know what you are doing. I know we are on HN, but you have to remember half the python coders are not even dev. They are geographers, mathematicians, biologists, students, teachers, data analysts. And of course they are on mac or windows. Even I, and I started to code in python 2.4, never have to compile python. I do it for fun from time to time, but I don't use that professionnally. If you use a platform where sqlite is not there, you are an exeception in a niche of minority. I drive my car assuming there will be breaks on it. It can be missing, but it's not a great assumption.
        im3w1l 5y ago
        pyenv as he mentioned is a common tool. That it happens to compile under the hood is an implementation detail that users may not even be aware off.
        BiteCode_dev 5y ago
        > pyenv as he mentioned is a common tool. Pyenv is not a "common tool". It's used by a very small minority of devs since: - it's used only by dev, and half of the python coders are not devs - it works only on unix. Most people install python on windows (https://discuss.python.org/t/python-download-stats-for-may-2020/4454 https://discuss.python.org/t/python-download-stats-for-may-2..., and that doesn't take in consideration anaconda or the appstore). Unless you count the obscure fork used by even less people. - And yet on mac, if they don't use the official installer, they use mostly brew, not pyenv. On Linux, the officials repos or things like deadnsnake or epel. And of course docker. Pyenv is a tiny fractions of that, because any alternative solution is better known and easier. - hard to use because it compiles things, fails easily in non obvious ways and assume you manage the path correctly, so it's really, really, popular only with people that want to deal with that. In fact the setups instructions are huge: https://github.com/pyenv/pyenv#how-it-works https://github.com/pyenv/pyenv#how-it-works - that's just what we see in the field. I go to a lot of different shops every years because I'm a freelancers, and I encountered pyenv twice in 10 years. > That it happens to compile under the hood is an implementation detail that users may not even be aware off. Of course not. The setup docs themself tell you to "Install Python build dependencies before attempting to install a new Python version.". So not only you have to be aware of it, but you must be capable of setting up a compilation environment, and if you fail to do so, it will not work. pyenv is not standard python. It's not "a common tool". It's a tool for a niche of specialists with domain specific knowledge.
- deleted 5y ago
  [deleted]

KronisLV 5y ago

This really reminds me of an article that i read a while ago, called "SQLite as an application file format", which seemed to express some ideas like that: https://www.sqlite.org/draft/appfileformat.html https://www.sqlite.org/draft/appfileformat.html