9 ms·

> We don't look at this difference as a workaround in Fossil for autosync, but instead as a test-first philosophical difference: fossil commit is a commitment.

> We don't look at this difference as a workaround in Fossil for autosync, but instead as a test-first philosophical difference: fossil commit is a commitment. When every commit is pushed to the parent repo by default, it encourages a working style in which every commit is tested first. It encourages thinking before acting. We believe this is an inherently good thing. I wonder how this works with tests that cannot be run locally (or ones that I'm not willing to sit through for the whole 1 hour the test suite runs for?)? Eager to hear from Fossil practicioners on this. The ideals do sound very good - but they remind me of the difference between an academic and a practicioner. One gets things done while the other advances the state of the field. Depends on what you like.

hprotagonist 5y ago
> The ideals do sound very good - but they remind me of the difference between an academic and a practicioner. Well, fossil is notably used for development of SQLite, which is about as practitioner-centric as it gets.
- hashhar 5y ago
  Didn't know that. Thanks a lot for sharing this info. So I can take a look at some prior work of how it gets used in large projects and make sense of it.
wyoung2 5y ago
> tests that cannot be run locally Fossil just got a "patch" command to help with cases like that: https://fossil-scm.org/home/doc/trunk/www/patchcmd.md https://fossil-scm.org/home/doc/trunk/www/patchcmd.md Previously, the standard solution was to commit work with potential cross-platform issues to a short-lived experimental branch, so you can check out the branch on all your test machines and try it there before merging it down to trunk. > I'm not willing to sit through for the whole 1 hour the test suite runs for Hour-long test suite runs are a problem in and of themselves. That sets the shortest period of time your development organization can react to a test failure. Look to any application of control theory — feedback systems, process control, OODA loops... — for what happens when you have a slow-reacting feedback loop. One of those "guys who wrote the book" on Continuous Delivery covered the problem well here: https://www.davefarley.net/?p=218 https://www.davefarley.net/?p=218 You either need multiple levels of testing so you get some feedback quickly, or you need some sort of parallel build system that breaks the tests up so that the whole set completes in a reasonable amount of time. Without that, you're going to have people either sitting around waiting on test runners or skipping the tests entirely. > One gets things done Effort and resources spent on a massively parallel CD buildbot sounds like a good way to get things done to me. Computers are cheap; developer time is expensive.
- hashhar 5y ago
  Awesome, for some reason I couldn't think of the workaround of committing to a temporary branch. > Hour-long test suite runs are a problem in and of themselves. I'm not sure I agree. Let's say I have a data integration system that has 70 modules. Each module have some integration tests and some baseline benchmarks. At the time of development I'm only concerned with the 1 or 2 modules I change whose tests can finish in < 5m easily. But when I push the change I want to be sure nothing else broke. Maybe because the change was in a base module used as a library by some other modules. Maybe I forgot to run tests on 2 of the dependent modules. So running the entire suite on merges (and optionally on PRs if required) is a necessity in my opinion. Thanks for the link though - I'll take a look. > You either need multiple levels of testing so you get some feedback quickly, or you need some sort > of parallel build system that breaks the tests up so that the whole set completes in a reasonable > amount of time. Without that, you're going to have people either sitting around waiting on test > runners or skipping the tests entirely. 100% agreed. With most of these points you can now ignore my previous paragraph. By a 1 hour test suite I mean the wall-clock time of a single build-test cycle on CI runners. i.e. to run ~130 test suites in parallel against various containerised databases and some DBaaS (like maybe Redshift, BigQuery etc.) takes 1 hour where each individual module takes 5 to 10 mins. Some may take longer depending on the depth of things you are testing. > Effort and resources spent on a massively parallel CD buildbot sounds like a good > way to get things done to me. Computers are cheap; developer time is expensive. 100% agreed there again. In my opinion the organisations that treat CI improvements as technical debt and not as first class citizens of the software development flow are doomed to move slowly and have a hard time getting things done.
mpweiher 5y ago
> I wonder how this works with tests that cannot be run locally (or ones that I'm not willing to sit through for the whole 1 hour the test suite runs for?)? Create local, quick-running proxies for those tests. I realise that this seems glib, but it is workable and the correct approach in the vast majority of cases, including the vast majority of those who claim that it's not possible for them.
- hashhar 5y ago
  Agreed. It's useful either way to have some sort of "smoke-test" version of the entire test suite so that you can have reasonable confidence that things are working and if needed you can still run the full in-depth suite. Thanks.

hprotagonist 5y ago

> The ideals do sound very good - but they remind me of the difference between an academic and a practicioner. Well, fossil is notably used for development of SQLite, which is about as practitioner-centric as it gets.

hashhar 5y ago
Didn't know that. Thanks a lot for sharing this info. So I can take a look at some prior work of how it gets used in large projects and make sense of it.

wyoung2 5y ago

> tests that cannot be run locally Fossil just got a "patch" command to help with cases like that: https://fossil-scm.org/home/doc/trunk/www/patchcmd.md https://fossil-scm.org/home/doc/trunk/www/patchcmd.md Previously, the standard solution was to commit work with potential cross-platform issues to a short-lived experimental branch, so you can check out the branch on all your test machines and try it there before merging it down to trunk. > I'm not willing to sit through for the whole 1 hour the test suite runs for Hour-long test suite runs are a problem in and of themselves. That sets the shortest period of time your development organization can react to a test failure. Look to any application of control theory — feedback systems, process control, OODA loops... — for what happens when you have a slow-reacting feedback loop. One of those "guys who wrote the book" on Continuous Delivery covered the problem well here: https://www.davefarley.net/?p=218 https://www.davefarley.net/?p=218 You either need multiple levels of testing so you get some feedback quickly, or you need some sort of parallel build system that breaks the tests up so that the whole set completes in a reasonable amount of time. Without that, you're going to have people either sitting around waiting on test runners or skipping the tests entirely. > One gets things done Effort and resources spent on a massively parallel CD buildbot sounds like a good way to get things done to me. Computers are cheap; developer time is expensive.

hashhar 5y ago
Awesome, for some reason I couldn't think of the workaround of committing to a temporary branch. > Hour-long test suite runs are a problem in and of themselves. I'm not sure I agree. Let's say I have a data integration system that has 70 modules. Each module have some integration tests and some baseline benchmarks. At the time of development I'm only concerned with the 1 or 2 modules I change whose tests can finish in < 5m easily. But when I push the change I want to be sure nothing else broke. Maybe because the change was in a base module used as a library by some other modules. Maybe I forgot to run tests on 2 of the dependent modules. So running the entire suite on merges (and optionally on PRs if required) is a necessity in my opinion. Thanks for the link though - I'll take a look. > You either need multiple levels of testing so you get some feedback quickly, or you need some sort > of parallel build system that breaks the tests up so that the whole set completes in a reasonable > amount of time. Without that, you're going to have people either sitting around waiting on test > runners or skipping the tests entirely. 100% agreed. With most of these points you can now ignore my previous paragraph. By a 1 hour test suite I mean the wall-clock time of a single build-test cycle on CI runners. i.e. to run ~130 test suites in parallel against various containerised databases and some DBaaS (like maybe Redshift, BigQuery etc.) takes 1 hour where each individual module takes 5 to 10 mins. Some may take longer depending on the depth of things you are testing. > Effort and resources spent on a massively parallel CD buildbot sounds like a good > way to get things done to me. Computers are cheap; developer time is expensive. 100% agreed there again. In my opinion the organisations that treat CI improvements as technical debt and not as first class citizens of the software development flow are doomed to move slowly and have a hard time getting things done.

mpweiher 5y ago

> I wonder how this works with tests that cannot be run locally (or ones that I'm not willing to sit through for the whole 1 hour the test suite runs for?)? Create local, quick-running proxies for those tests. I realise that this seems glib, but it is workable and the correct approach in the vast majority of cases, including the vast majority of those who claim that it's not possible for them.

hashhar 5y ago
Agreed. It's useful either way to have some sort of "smoke-test" version of the entire test suite so that you can have reasonable confidence that things are working and if needed you can still run the full in-depth suite. Thanks.