3 ms·

Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different

Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. A single person can develop and release extremely high quality software, and as long as it meets the needs of the users (it's not missing a lot of features that a taking a long time to deliver), a single person in absolute control and writing all the code is probably a benefit in keeping it high quality and with less bugs. It may not follow that the same can be said a few years from now, or even a few months from now, since the bus factor of that project is one, and if "bus events" includes "I don't want to work on that anymore but nobody else knows it well at all" then for some users that's a problem (and for others not so much). On situation isn't necessarily better or worse than the other (and it's probably something in-between anyway), it really just depends on the project and the audience it's intended for. That audience might be somewhat self-selected by the style of development though.

arka2147483647 4y ago
> Extremely well written and maintained and high quality as of now ... As I see it, the quality comes from the fixed need, tight scope and small professional team Fixed need -> Somebody always needs a database Smaller scope -> Less features, less code, less bugs Professional team -> No newbies around to break things Now, to be sure, this kind of development model can only work for some projects.
- dathinab 4y ago
  Also tooling focused on their dev flow. In difference to the article implying that they use out-date project tooling they don't(). They VCS isn't out-date but in some way more modern then git, it's just focused on dev flows similar to theirs to a point where using typical git dev flows will not work well. Similar not using GitHub is a more then right decisions for how the project is manage, github is too much focused on open contribution projects. (): You could argue they use "not modern" tools like C and similar to do modern things like fuzz testing. But the articles author clearly puts a focus on the project tooling highlighting git/GitHub so reading it as implying that their VCS is "not modern" i.e. outdated i.e. bad seem very reasonable IMHO.
diffxx 4y ago
Fair points, although the bus factor is more like 3 or 4 for SQLite as far as I know. The question though is that if the entire team vanished from the face of the earth, what would the impact be? My guess is that either SQLite would be good enough as is for 99% of use cases and it wouldn't need much development apart from maybe some minor platform specific use cases or, if new functionality truly is needed, then it would be better for a new team to rewrite a similar program from scratch using SQLite as more of a POC than as a blueprint.
- kbenson 4y ago
  I was speaking less to the SQLite situation specifically and more to the general idea of "Could it be that this is precisely because they don't use "modern tools" and accept outside contributions?" and how I think teams that are very small and not very accepting of outside help/influence might affect that. To that end I purposefully compares extremes, and tried to allude to the fact that most situations fall between those extremes in some way. SQLite is more towards one end than the other, but it's obviously not a single developer releasing binaries to the world, which is about as far to that extreme as you can go. The other end would probably be something like Debian. That's not to say either of those situations have to be horrible at what the other excels at. That singular person could have set things in place such that all their code and the history of it gets released on their death, and Debian obviously has a working process for releasing a high quality distribution.
  - Quekid5 4y ago
    AFAIUI the company behind SQLite (Hipp & co.) have basically endless funding. Not unlimited, just a good enough budget and not likely to end soon. That's also a big factor.
- chasil 4y ago
  SQLite is supported until 2050, and will likely outlast many other platforms if this goal is attained. I hope the bus factor is high enough to reach the goal. "Every machine-code branch instruction is tested in both directions. Multiple times. On multiple platforms and with multiple compilers. This helps make the code robust for future migrations. The intense testing also means that new developers can make experimental enhancements to SQLite and, assuming legacy tests all pass, be reasonably sure that the enhancement does not break legacy." https://www.sqlite.org/lts.html https://www.sqlite.org/lts.html
RcouF1uZ4gsC 4y ago
> Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. I think in this area SQLite has most other open source software beat. SQLite is used in the Airbus A350 and has support contracts for the life of the airframe. https://mobile.twitter.com/copiousfreetime/status/675834543304081409 https://mobile.twitter.com/copiousfreetime/status/6758345433... Other places I have see that they have support contracts through 2050. This is far more future support than any open source software or likely most commercial software.
cryptonector 4y ago
> Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. SQLite has both.
- kbenson 4y ago
  And so does the Linux kernel. There are numerous cases of successes and failures at both ends of that spectrum. My point wasn't to imply that you can only pick one, but that in some cases choices to maximize one aspect can negatively affect the other if care is not taken, and depending on audience high quality released software is not the only thing under consideration in open source projects. Keeping the developer group small and being extremely selective of what outside code or ideas are allowed might bring benefits in quality, but if not carefully considered could yield long term problems that ultimately harm a project.
  - cryptonector 4y ago
    > > > Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. > [...], but if not carefully considered could yield long term problems that ultimately harm a project. SQLite has a successful consortium and a successful business model, namely: leveraging a proprietary test suite to keep a monopoly on SQLite development that then drives consortium membership and, therefore, consortium success, which then funds the SQLite team. This has worked for a long time now. It will not work forever. It should work for at least the foreseeable future. If it fails it will be either because a fork somehow overcomes the lack of access to the SQLite proprietary test suite, or because a competitor written in a better programming language arises and quickly gains momentum and usage, and/or because the SQLite Consortium members abandon the project.
    - diffxx 4y ago
      Very good points. The proprietary test suite is clearly the (open) secret to SQLite's success. It seems to me that it isn't even entirely accurate to describe SQLite as written in C when the vast majority of its code is probably written in TCL that none of us have seen. It's more like C is just how they represent the virtual machine which is described and specified by its tests. The virtual machine exists outside of any particular programming language but C is the most convenient implementation language to meet their cross platform distribution goals. If someone did want to carve into SQLite's embedded db monopoly, it would take years to develop a comparable test suite. This seems possible, particularly if they develop a more expressive language for expressing the solutions to the types of problems that we use SQLite for. Who would fund this work though when SQLite works as well as it does?
tablespoon 4y ago
> Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. Software that's truly "extremely well written and maintained and high quality as of now" has the option of a plan like: > "At the time of my death, it is my intention that the then-current versions of TEX and METAFONT be forever left unchanged, except that the final version numbers to be reported in the “banner” lines of the programs should become [pi and e] respectively. From that moment on, all “bugs” will be permanent “features.” (http://www.ntg.nl/maps/05/34.pdf http://www.ntg.nl/maps/05/34.pdf) If your software needs perpetual maintenance, that's a good sign that it's probably not that high quality.
- mschuster91 4y ago
  > If your software needs perpetual maintenance, that's a good sign that it's probably not that high quality The problem in a lot of cases is not the software per se, but changing environments. Windows upholds backwards compatibility to a ridiculous degree (you can still run a lot of Win95 era games or business software on Win10), macOS tends to do major overhauls of subsystems every five-ish years that require sometimes substantial changes (e.g. they completely killed off 32-bit app support), but the Linux space is hell. Anything that needs special kernel modules has no guarantees at all unless the driver is mainlined into the official kernel tree (which can be ridiculously hard to achieve). Userspace is bleh (if you're willing to stick to CLI and statically linking everything sans libc) to horrible (for anything involving GUI or heaven forbid games, or when linking to other libraries dynamically). The worst of all offenders however is the entire NodeJS environment. The words "backwards compatibility" simply do not exist in that world, so if you want even a chance at keeping up with security updates you have an awful lot of churn work simply because stuff breaks left and right at each "npm update".
  - parminya 4y ago
    You say nothing seriously false, but perhaps depending on things like NodeJS just inherently means your software is going to be poor quality. If that is true, then both you and the PP are probably right. I tend to think software quality will be higher if you depend on a third-party collection (such as so-called Linux distributions) than a second-party aggregation (such as NPM).
cryptonector 4y ago
> It may not follow that the same can be said a few years from now, or even a few months from now, since the bus factor of that project is one, and if "bus events" includes "I don't want to work on that anymore but nobody else knows it well at all" then for some users that's a problem (and for others not so much). You may have been speaking generally, and you'd be right, but specifically the bus factor of the SQLite team and the SQLite Consortium is larger than 1, and they could hire more team members if need be. If and when the SQLite team is no longer able to keep the project moving forwards, then I think we'd see one or more forks or rewrites or competitors take over SQLite's market share.
- kbenson 4y ago
  Yes, I was speaking generally. Specific development models have advantages and disadvantages, but those can often be countered by non-development model actions taken to limit those disadvantages. For example, and extremely open development model is likely prone to more bugs and quality problems, as well as a harder to read and work in code base. There are steps to combat that, such as style guides and automatic style converters, numerous reviewers that can go through code to fund bugs and make suggestions for better quality, etc. It's not so much that one model over the other will have those problems I've mentioned for each, as much as I think those are common things those projects should be cognizant of and take steps to combat. As you noted elsewhere, it sounds like SQLite has done a lot that mitigates what I see as the inherent disadvantages of their development model, which is laudable. At the same time I doubt the average SQLite developer is as easily and quickly replaced as the average Linux kernel contributor, even if there are specific kernel developers which would be hard to deal with their loss. Sometimes all you can do is mitigate the harm of a problem, not remove it entirely.
  - cryptonector 4y ago
    I dunno, I've seen new team members jump onto projects and thrive where most other members have a decade or two of experience. Once you've been around the block dealing with large and complex codebases, picking up a new one gets easier, so I'm not at all worried about the SQLite bus factor. I agree that much larger projects like Linux can have much larger communities to draw leadership from, but I think SQLite is fine. Once upon a time I wanted to make a contribution to SQLite, and I tried to negotiate making it, but it was quite an uphill battle. On the other hand, I found the codebase itself quite approachable.

arka2147483647 4y ago

> Extremely well written and maintained and high quality as of now ... As I see it, the quality comes from the fixed need, tight scope and small professional team Fixed need -> Somebody always needs a database Smaller scope -> Less features, less code, less bugs Professional team -> No newbies around to break things Now, to be sure, this kind of development model can only work for some projects.

dathinab 4y ago
Also tooling focused on their dev flow. In difference to the article implying that they use out-date project tooling they don't(). They VCS isn't out-date but in some way more modern then git, it's just focused on dev flows similar to theirs to a point where using typical git dev flows will not work well. Similar not using GitHub is a more then right decisions for how the project is manage, github is too much focused on open contribution projects. (): You could argue they use "not modern" tools like C and similar to do modern things like fuzz testing. But the articles author clearly puts a focus on the project tooling highlighting git/GitHub so reading it as implying that their VCS is "not modern" i.e. outdated i.e. bad seem very reasonable IMHO.

diffxx 4y ago

Fair points, although the bus factor is more like 3 or 4 for SQLite as far as I know. The question though is that if the entire team vanished from the face of the earth, what would the impact be? My guess is that either SQLite would be good enough as is for 99% of use cases and it wouldn't need much development apart from maybe some minor platform specific use cases or, if new functionality truly is needed, then it would be better for a new team to rewrite a similar program from scratch using SQLite as more of a POC than as a blueprint.

kbenson 4y ago
I was speaking less to the SQLite situation specifically and more to the general idea of "Could it be that this is precisely because they don't use "modern tools" and accept outside contributions?" and how I think teams that are very small and not very accepting of outside help/influence might affect that. To that end I purposefully compares extremes, and tried to allude to the fact that most situations fall between those extremes in some way. SQLite is more towards one end than the other, but it's obviously not a single developer releasing binaries to the world, which is about as far to that extreme as you can go. The other end would probably be something like Debian. That's not to say either of those situations have to be horrible at what the other excels at. That singular person could have set things in place such that all their code and the history of it gets released on their death, and Debian obviously has a working process for releasing a high quality distribution.
- Quekid5 4y ago
  AFAIUI the company behind SQLite (Hipp & co.) have basically endless funding. Not unlimited, just a good enough budget and not likely to end soon. That's also a big factor.
chasil 4y ago
SQLite is supported until 2050, and will likely outlast many other platforms if this goal is attained. I hope the bus factor is high enough to reach the goal. "Every machine-code branch instruction is tested in both directions. Multiple times. On multiple platforms and with multiple compilers. This helps make the code robust for future migrations. The intense testing also means that new developers can make experimental enhancements to SQLite and, assuming legacy tests all pass, be reasonably sure that the enhancement does not break legacy." https://www.sqlite.org/lts.html https://www.sqlite.org/lts.html

RcouF1uZ4gsC 4y ago

> Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. I think in this area SQLite has most other open source software beat. SQLite is used in the Airbus A350 and has support contracts for the life of the airframe. https://mobile.twitter.com/copiousfreetime/status/675834543304081409 https://mobile.twitter.com/copiousfreetime/status/6758345433... Other places I have see that they have support contracts through 2050. This is far more future support than any open source software or likely most commercial software.

cryptonector 4y ago

kbenson 4y ago
And so does the Linux kernel. There are numerous cases of successes and failures at both ends of that spectrum. My point wasn't to imply that you can only pick one, but that in some cases choices to maximize one aspect can negatively affect the other if care is not taken, and depending on audience high quality released software is not the only thing under consideration in open source projects. Keeping the developer group small and being extremely selective of what outside code or ideas are allowed might bring benefits in quality, but if not carefully considered could yield long term problems that ultimately harm a project.
- cryptonector 4y ago
  > > > Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. > [...], but if not carefully considered could yield long term problems that ultimately harm a project. SQLite has a successful consortium and a successful business model, namely: leveraging a proprietary test suite to keep a monopoly on SQLite development that then drives consortium membership and, therefore, consortium success, which then funds the SQLite team. This has worked for a long time now. It will not work forever. It should work for at least the foreseeable future. If it fails it will be either because a fork somehow overcomes the lack of access to the SQLite proprietary test suite, or because a competitor written in a better programming language arises and quickly gains momentum and usage, and/or because the SQLite Consortium members abandon the project.
  - diffxx 4y ago
    Very good points. The proprietary test suite is clearly the (open) secret to SQLite's success. It seems to me that it isn't even entirely accurate to describe SQLite as written in C when the vast majority of its code is probably written in TCL that none of us have seen. It's more like C is just how they represent the virtual machine which is described and specified by its tests. The virtual machine exists outside of any particular programming language but C is the most convenient implementation language to meet their cross platform distribution goals. If someone did want to carve into SQLite's embedded db monopoly, it would take years to develop a comparable test suite. This seems possible, particularly if they develop a more expressive language for expressing the solutions to the types of problems that we use SQLite for. Who would fund this work though when SQLite works as well as it does?

tablespoon 4y ago

> Extremely well written and maintained and high quality as of now and having a plan to make sure that can continue in the future are sometimes entirely different things with needs that oppose each other. Software that's truly "extremely well written and maintained and high quality as of now" has the option of a plan like: > "At the time of my death, it is my intention that the then-current versions of TEX and METAFONT be forever left unchanged, except that the final version numbers to be reported in the “banner” lines of the programs should become [pi and e] respectively. From that moment on, all “bugs” will be permanent “features.” (http://www.ntg.nl/maps/05/34.pdf http://www.ntg.nl/maps/05/34.pdf) If your software needs perpetual maintenance, that's a good sign that it's probably not that high quality.

mschuster91 4y ago
> If your software needs perpetual maintenance, that's a good sign that it's probably not that high quality The problem in a lot of cases is not the software per se, but changing environments. Windows upholds backwards compatibility to a ridiculous degree (you can still run a lot of Win95 era games or business software on Win10), macOS tends to do major overhauls of subsystems every five-ish years that require sometimes substantial changes (e.g. they completely killed off 32-bit app support), but the Linux space is hell. Anything that needs special kernel modules has no guarantees at all unless the driver is mainlined into the official kernel tree (which can be ridiculously hard to achieve). Userspace is bleh (if you're willing to stick to CLI and statically linking everything sans libc) to horrible (for anything involving GUI or heaven forbid games, or when linking to other libraries dynamically). The worst of all offenders however is the entire NodeJS environment. The words "backwards compatibility" simply do not exist in that world, so if you want even a chance at keeping up with security updates you have an awful lot of churn work simply because stuff breaks left and right at each "npm update".
- parminya 4y ago
  You say nothing seriously false, but perhaps depending on things like NodeJS just inherently means your software is going to be poor quality. If that is true, then both you and the PP are probably right. I tend to think software quality will be higher if you depend on a third-party collection (such as so-called Linux distributions) than a second-party aggregation (such as NPM).

cryptonector 4y ago

> It may not follow that the same can be said a few years from now, or even a few months from now, since the bus factor of that project is one, and if "bus events" includes "I don't want to work on that anymore but nobody else knows it well at all" then for some users that's a problem (and for others not so much). You may have been speaking generally, and you'd be right, but specifically the bus factor of the SQLite team and the SQLite Consortium is larger than 1, and they could hire more team members if need be. If and when the SQLite team is no longer able to keep the project moving forwards, then I think we'd see one or more forks or rewrites or competitors take over SQLite's market share.

kbenson 4y ago
Yes, I was speaking generally. Specific development models have advantages and disadvantages, but those can often be countered by non-development model actions taken to limit those disadvantages. For example, and extremely open development model is likely prone to more bugs and quality problems, as well as a harder to read and work in code base. There are steps to combat that, such as style guides and automatic style converters, numerous reviewers that can go through code to fund bugs and make suggestions for better quality, etc. It's not so much that one model over the other will have those problems I've mentioned for each, as much as I think those are common things those projects should be cognizant of and take steps to combat. As you noted elsewhere, it sounds like SQLite has done a lot that mitigates what I see as the inherent disadvantages of their development model, which is laudable. At the same time I doubt the average SQLite developer is as easily and quickly replaced as the average Linux kernel contributor, even if there are specific kernel developers which would be hard to deal with their loss. Sometimes all you can do is mitigate the harm of a problem, not remove it entirely.
- cryptonector 4y ago
  I dunno, I've seen new team members jump onto projects and thrive where most other members have a decade or two of experience. Once you've been around the block dealing with large and complex codebases, picking up a new one gets easier, so I'm not at all worried about the SQLite bus factor. I agree that much larger projects like Linux can have much larger communities to draw leadership from, but I think SQLite is fine. Once upon a time I wanted to make a contribution to SQLite, and I tried to negotiate making it, but it was quite an uphill battle. On the other hand, I found the codebase itself quite approachable.