6 ms·

> Unison does away with builds. Completely! > It has a canonical representation for all code, Isn't generating that canonical representation the build step?

> Unison does away with builds. Completely! > It has a canonical representation for all code, Isn't generating that canonical representation the build step? > The important thing is that now, I can actually compile and run this program Now I'm confused whether there is or isn't a build step.

naikrovek 4y ago
JavaScript doesn't have a build step either. sounds like it is interpreted and you reference methods by hash instead of by name. sorta?
- __MatrixMan__ 4y ago
  I'm not an expert at JS internals, but I'm under the impression that if you call a function ten times, its code has to be interpreted ten times, so you get ten, temporary, abstract syntax trees. Such is life in interpreter-land--the code is authoritative, the instructions are ephemeral. My impression of Unison is that if you want to call a function ten times, you just need to evaluate that code into an AST once. Once you have the hash of that AST, you can delete the code and still call the function 9 more times without having to bother with building the tree each time. I don't know what to call it, but I don't think it's interpreted because you don't need the code to run it (except the first time, which is how you know find the hash of the function).
  - mrtweetyhack 4y ago
    [dead]
  - goto11 4y ago
    > I'm not an expert at JS internals, but I'm under the impression that if you call a function ten times, its code has to be interpreted ten times, so you get ten, temporary, abstract syntax trees. This is not the case even in an interpreted language. You only need to build an AST once, at parse time. In JavaScript the AST is then compiled into byte code or machine code depending on the implementation.
    - __MatrixMan__ 4y ago
      Sorry, let me rephrase. In JS, if 1 developer and 9 users with a total of 10 separate browser instances all call the same function, the code will be parsed 10 times. In Unison, the AST is generated by the developer, cached, and the 9 users can call it by hash (assuming they have access to that database where the AST lives) without reparsing the code.
      - goto11 4y ago
        So the AST is transferred over the wire? How is it serialized?
        zawodnaya 4y ago
        It’s a binary-encoded merkle tree
        __MatrixMan__ 4y ago
        It's in a sqlite database. Unison doesn't prescribe how those are passed around, but since everything is content addressed there are never any name collisions. You can always merge two of them into one without having to resolve conflicts and without breaking anything. Doing so just expands your codebase to include more code than you're currently calling. The way I intend to use it is that users periodically assemble the codebases from all of the developers that they care about and merge them into a single codebase. This can be very latency tolerant (like, sneak-a-thumb-drive-across-the-border levels of latency tolerant) because it's not like you're waiting around for transport every time you want to do something, just every time you want to upgrade. Then, when you decide to use the new version of something (the developer would communicate a new hash to you), you just start calling the new hash and now you're using the new software. If something goes wrong, you just go back to calling the old hash. No implicit trust (e.g. no need for SSL, DNS, package names), atomic upgrade/downgrades, no merge conflicts.
- osigurdson 4y ago
  I think the compiled form is associated with the hash of the source. It can’t be machine code though.
  - zawodnaya 4y ago
    It’s bytecode
    - osigurdson 4y ago
      Does it require a jit then or more like llvm?
- brabel 4y ago
  JS doesn't type check, generate an AST, or even minifies anything until the user runs the code. If you just write JS in notepad then immediately send it over to your users, then yeah, there's no build in JS. But if your idea of writing software involves: * type checking, linting and otherwise ensuring properties of the code before even running it. * testing code before deploying it. * turning it from text into a representation that's more compact and safer to run. * generating documentation. Then you'll definitely need a build, and a very complex one for that matter, in JS world. Have you ever seen a JS project without any sort of build in a professional context? And you'll need a big IDE to help you make sense of anything because JS doesn't even have a "compiler" or "helper tool" that'll give you any help to navigate and understand the code base.
  - zdragnar 4y ago
    > Have you ever seen a JS project without any sort of build in a professional context? Imagine a world that pre-dates typescript, node, webpack, browserify, gulp, grunt. You were lucky if you had a server framework that concatenated files for you, otherwise you were stuck with 6000+ line "global.js" files. Then you had to fix merge conflicts in that file because SVN. Whenever someone complains that frontend JS is too much, I just think back to how it used to be.
xtagon 4y ago
I had the same confusion and re-read the article. Best I can understand, what they're trying to say is that it could be used without a build step in the sense that you could just manipulate the database's AST-like structure directly instead of compiling code, and it would be basically incrementally "compiling" each unit as you edit instead of interpreting a code file. But of course that's not a very ergonomic way to do it so there is a language on top, that they used as an example to show off the language. I can see this being useful for implementing alternative editing interfaces besides just writing code textually. For example, flow-based visual programming or something like that. Adding and connecting nodes in a visual interface would each be small updates to the database, and then there would never be a "build step" because the database you've been editing is already the AST that could either be interpreted or compiled. So it only really does away with the parser, but that doesn't seem revolutionary to me, other languages can already do that. I might be missing something still.
dwohnitmok 4y ago
You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to be generated once, ever. So for example if someone publishes a library, no new build step would ever be required for consumers of that library. Even with CI/CD there would be no build step because the very act of merging one's code into another codebase operates on canonical representations and so they never need to be regenerated.
- lib-dev 4y ago
  Could this be done in other languages too?
  - brabel 4y ago
    kind of. For example, in Java, build tools will normally cache the hash of the source file that generated a class file. That class file is a cached version of the source, basically. IF you could keep both the class file and the hashes of the source around, you could avoid ever re-compiling the source unless you've changed it. This is just incremental compilation, in essence. However, you'll need quite a lot of tooling around this simple idea: where to store the class files and the hashes, should you commit it into source control, how to distribute the cache, can this be made multi-platform so it works on all developers' machines? All solvable, but Unison already does all that, and more, with its single tool, the `ucm`.
  - pharmakom 4y ago
    This is one of things Bazel tries to do.
  - sparkie 4y ago
    An issue with many languages is the presence of mutually dependant parts of code, because how do you generate a content-hash for something whose constituent part's content-hashes depend on it? Merkle trees can be generalized to DAGs, but not directed graphs with cycles. As far as I understand, Unison's syntax has been designed specifically to avoid such cycles, although it supports recursive functions and inductive data types which refer to themselves such as lists. A self-cycle can be handled, but I'm not sure if, and if so how, it would support mutually recursive functions or data types. I suspect they're not supported for the reason that content-hashing becomes non-trivial. If you wanted to generalize the solution to arbitrary languages, you would need to somehow sever the cycles, which is likely to require a specific solution for each language, or come up with another solution to content-hashing which does not have the problem of Merkle-DAGs. Either way, you are looking at something non-trivial compared to having a language which simply forbids the cycles.

naikrovek 4y ago

JavaScript doesn't have a build step either. sounds like it is interpreted and you reference methods by hash instead of by name. sorta?

__MatrixMan__ 4y ago
I'm not an expert at JS internals, but I'm under the impression that if you call a function ten times, its code has to be interpreted ten times, so you get ten, temporary, abstract syntax trees. Such is life in interpreter-land--the code is authoritative, the instructions are ephemeral. My impression of Unison is that if you want to call a function ten times, you just need to evaluate that code into an AST once. Once you have the hash of that AST, you can delete the code and still call the function 9 more times without having to bother with building the tree each time. I don't know what to call it, but I don't think it's interpreted because you don't need the code to run it (except the first time, which is how you know find the hash of the function).
- mrtweetyhack 4y ago
  [dead]
- goto11 4y ago
  > I'm not an expert at JS internals, but I'm under the impression that if you call a function ten times, its code has to be interpreted ten times, so you get ten, temporary, abstract syntax trees. This is not the case even in an interpreted language. You only need to build an AST once, at parse time. In JavaScript the AST is then compiled into byte code or machine code depending on the implementation.
  - __MatrixMan__ 4y ago
    Sorry, let me rephrase. In JS, if 1 developer and 9 users with a total of 10 separate browser instances all call the same function, the code will be parsed 10 times. In Unison, the AST is generated by the developer, cached, and the 9 users can call it by hash (assuming they have access to that database where the AST lives) without reparsing the code.
    - goto11 4y ago
      So the AST is transferred over the wire? How is it serialized?
      - zawodnaya 4y ago
        It’s a binary-encoded merkle tree
      - __MatrixMan__ 4y ago
        It's in a sqlite database. Unison doesn't prescribe how those are passed around, but since everything is content addressed there are never any name collisions. You can always merge two of them into one without having to resolve conflicts and without breaking anything. Doing so just expands your codebase to include more code than you're currently calling. The way I intend to use it is that users periodically assemble the codebases from all of the developers that they care about and merge them into a single codebase. This can be very latency tolerant (like, sneak-a-thumb-drive-across-the-border levels of latency tolerant) because it's not like you're waiting around for transport every time you want to do something, just every time you want to upgrade. Then, when you decide to use the new version of something (the developer would communicate a new hash to you), you just start calling the new hash and now you're using the new software. If something goes wrong, you just go back to calling the old hash. No implicit trust (e.g. no need for SSL, DNS, package names), atomic upgrade/downgrades, no merge conflicts.
osigurdson 4y ago
I think the compiled form is associated with the hash of the source. It can’t be machine code though.
- zawodnaya 4y ago
  It’s bytecode
  - osigurdson 4y ago
    Does it require a jit then or more like llvm?
brabel 4y ago
JS doesn't type check, generate an AST, or even minifies anything until the user runs the code. If you just write JS in notepad then immediately send it over to your users, then yeah, there's no build in JS. But if your idea of writing software involves: * type checking, linting and otherwise ensuring properties of the code before even running it. * testing code before deploying it. * turning it from text into a representation that's more compact and safer to run. * generating documentation. Then you'll definitely need a build, and a very complex one for that matter, in JS world. Have you ever seen a JS project without any sort of build in a professional context? And you'll need a big IDE to help you make sense of anything because JS doesn't even have a "compiler" or "helper tool" that'll give you any help to navigate and understand the code base.
- zdragnar 4y ago
  > Have you ever seen a JS project without any sort of build in a professional context? Imagine a world that pre-dates typescript, node, webpack, browserify, gulp, grunt. You were lucky if you had a server framework that concatenated files for you, otherwise you were stuck with 6000+ line "global.js" files. Then you had to fix merge conflicts in that file because SVN. Whenever someone complains that frontend JS is too much, I just think back to how it used to be.

xtagon 4y ago

I had the same confusion and re-read the article. Best I can understand, what they're trying to say is that it could be used without a build step in the sense that you could just manipulate the database's AST-like structure directly instead of compiling code, and it would be basically incrementally "compiling" each unit as you edit instead of interpreting a code file. But of course that's not a very ergonomic way to do it so there is a language on top, that they used as an example to show off the language. I can see this being useful for implementing alternative editing interfaces besides just writing code textually. For example, flow-based visual programming or something like that. Adding and connecting nodes in a visual interface would each be small updates to the database, and then there would never be a "build step" because the database you've been editing is already the AST that could either be interpreted or compiled. So it only really does away with the parser, but that doesn't seem revolutionary to me, other languages can already do that. I might be missing something still.

dwohnitmok 4y ago

You can think of it as an incremental build step but the crucial key that Unison's content addressing gives you is that canonical representation only needs to be generated once, ever. So for example if someone publishes a library, no new build step would ever be required for consumers of that library. Even with CI/CD there would be no build step because the very act of merging one's code into another codebase operates on canonical representations and so they never need to be regenerated.

lib-dev 4y ago
Could this be done in other languages too?
- brabel 4y ago
  kind of. For example, in Java, build tools will normally cache the hash of the source file that generated a class file. That class file is a cached version of the source, basically. IF you could keep both the class file and the hashes of the source around, you could avoid ever re-compiling the source unless you've changed it. This is just incremental compilation, in essence. However, you'll need quite a lot of tooling around this simple idea: where to store the class files and the hashes, should you commit it into source control, how to distribute the cache, can this be made multi-platform so it works on all developers' machines? All solvable, but Unison already does all that, and more, with its single tool, the `ucm`.
- pharmakom 4y ago
  This is one of things Bazel tries to do.
- sparkie 4y ago
  An issue with many languages is the presence of mutually dependant parts of code, because how do you generate a content-hash for something whose constituent part's content-hashes depend on it? Merkle trees can be generalized to DAGs, but not directed graphs with cycles. As far as I understand, Unison's syntax has been designed specifically to avoid such cycles, although it supports recursive functions and inductive data types which refer to themselves such as lists. A self-cycle can be handled, but I'm not sure if, and if so how, it would support mutually recursive functions or data types. I suspect they're not supported for the reason that content-hashing becomes non-trivial. If you wanted to generalize the solution to arbitrary languages, you would need to somehow sever the cycles, which is likely to require a specific solution for each language, or come up with another solution to content-hashing which does not have the problem of Merkle-DAGs. Either way, you are looking at something non-trivial compared to having a language which simply forbids the cycles.