3 ms·

You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to b

You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to be generated once, ever. So for example if someone publishes a library, no new build step would ever be required for consumers of that library. Even with CI/CD there would be no build step because the very act of merging one's code into another codebase operates on canonical representations and so they never need to be regenerated.

lib-dev 4y ago
Could this be done in other languages too?
- brabel 4y ago
  kind of. For example, in Java, build tools will normally cache the hash of the source file that generated a class file. That class file is a cached version of the source, basically. IF you could keep both the class file and the hashes of the source around, you could avoid ever re-compiling the source unless you've changed it. This is just incremental compilation, in essence. However, you'll need quite a lot of tooling around this simple idea: where to store the class files and the hashes, should you commit it into source control, how to distribute the cache, can this be made multi-platform so it works on all developers' machines? All solvable, but Unison already does all that, and more, with its single tool, the `ucm`.
- pharmakom 4y ago
  This is one of things Bazel tries to do.
- sparkie 4y ago
  An issue with many languages is the presence of mutually dependant parts of code, because how do you generate a content-hash for something whose constituent part's content-hashes depend on it? Merkle trees can be generalized to DAGs, but not directed graphs with cycles. As far as I understand, Unison's syntax has been designed specifically to avoid such cycles, although it supports recursive functions and inductive data types which refer to themselves such as lists. A self-cycle can be handled, but I'm not sure if, and if so how, it would support mutually recursive functions or data types. I suspect they're not supported for the reason that content-hashing becomes non-trivial. If you wanted to generalize the solution to arbitrary languages, you would need to somehow sever the cycles, which is likely to require a specific solution for each language, or come up with another solution to content-hashing which does not have the problem of Merkle-DAGs. Either way, you are looking at something non-trivial compared to having a language which simply forbids the cycles.
  - fwip 4y ago
    Unison does allow mutually recursive functions. https://www.unisonweb.org/docs/faq/#how-does-hashing-work-for-mutually-recursive-definitions https://www.unisonweb.org/docs/faq/#how-does-hashing-work-fo...
raggi 4y ago
Not once ever. Once per toolchain version and unique argument set. Perhaps the toolchain starts by attempting to never offer arguments, and perhaps it has lofty goals to release on day 1 with a perfect compilation model. All of my experience suggests these constraints are untenable, so ever is somewhat of an embellishment. The model also precludes cross-unit optimizations so there's a performance upper bound on compilation model that leaves it in the middle tier of performance without the model being broken (given things we've seen before).
goodpoint 4y ago
> You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to be generated once, ever. That's not very different from using ccache. Claiming that there are no builds is simply not true. > So for example if someone publishes a library, no new build step would ever be required for consumers of that library. Does that require trusting compiled blobs from 3rd parties? That could make it trivial to inject backdoors.

lib-dev 4y ago

Could this be done in other languages too?

brabel 4y ago
kind of. For example, in Java, build tools will normally cache the hash of the source file that generated a class file. That class file is a cached version of the source, basically. IF you could keep both the class file and the hashes of the source around, you could avoid ever re-compiling the source unless you've changed it. This is just incremental compilation, in essence. However, you'll need quite a lot of tooling around this simple idea: where to store the class files and the hashes, should you commit it into source control, how to distribute the cache, can this be made multi-platform so it works on all developers' machines? All solvable, but Unison already does all that, and more, with its single tool, the `ucm`.
pharmakom 4y ago
This is one of things Bazel tries to do.
sparkie 4y ago
An issue with many languages is the presence of mutually dependant parts of code, because how do you generate a content-hash for something whose constituent part's content-hashes depend on it? Merkle trees can be generalized to DAGs, but not directed graphs with cycles. As far as I understand, Unison's syntax has been designed specifically to avoid such cycles, although it supports recursive functions and inductive data types which refer to themselves such as lists. A self-cycle can be handled, but I'm not sure if, and if so how, it would support mutually recursive functions or data types. I suspect they're not supported for the reason that content-hashing becomes non-trivial. If you wanted to generalize the solution to arbitrary languages, you would need to somehow sever the cycles, which is likely to require a specific solution for each language, or come up with another solution to content-hashing which does not have the problem of Merkle-DAGs. Either way, you are looking at something non-trivial compared to having a language which simply forbids the cycles.
- fwip 4y ago
  Unison does allow mutually recursive functions. https://www.unisonweb.org/docs/faq/#how-does-hashing-work-for-mutually-recursive-definitions https://www.unisonweb.org/docs/faq/#how-does-hashing-work-fo...

raggi 4y ago

Not once ever. Once per toolchain version and unique argument set. Perhaps the toolchain starts by attempting to never offer arguments, and perhaps it has lofty goals to release on day 1 with a perfect compilation model. All of my experience suggests these constraints are untenable, so ever is somewhat of an embellishment. The model also precludes cross-unit optimizations so there's a performance upper bound on compilation model that leaves it in the middle tier of performance without the model being broken (given things we've seen before).

goodpoint 4y ago

> You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to be generated once, ever. That's not very different from using ccache. Claiming that there are no builds is simply not true. > So for example if someone publishes a library, no new build step would ever be required for consumers of that library. Does that require trusting compiled blobs from 3rd parties? That could make it trivial to inject backdoors.