12 ms·
The Unison language – a new approach to Distributed programming
- gauddasa 4y agoNo description on the home page, the documentation, Github page ... mentions that this is a Haskell-like language inspired by Haskell. No credit to Haskell while the code base is 99.6% Haskell!
- tr33house 4y agohow is this different from apache beam? also, any orgs using this at scale?
- adamgordonbell 4y agoIt's a programming language where the file representation isn't text of the code but ASTs and you can call things distributedly by a hash of that AST. So I guess its different from apache beam in all of the ways.
- halfmatthalfcat 4y agoI had to dig for the distributed part but it's outlined here: https://www.unison-lang.org/articles/distributed-datasets/ https://www.unison-lang.org/articles/distributed-datasets/
- revskill 4y ago"Each Unison definition is identified by a hash of its syntax tree." [0] I remember many years ago, i had the same idea on how to correctly version a dependency. [0]: https://www.unison-lang.org/learn/the-big-idea/ https://www.unison-lang.org/learn/the-big-idea/
- thefaux 4y agoDoes Unison have stack traces? Using the hash of the ast as the only identifier seems like it would lose some useful runtime debugging information.
- iamevn 4y agoWhile hashes are the primary identifier, things also generally have names associated with them. (Unless you deliberately remove the names that is.)
- galaxyLogic 4y agoSounds oddly much like git. Git-commits are identified by hashes. And git commits "are" (versions of) programs. So what does Unison have that git does not?
- infogulch 4y agoMostly, it heavily normalizes code, to the point where a name change will not register as a change to the program.
- stewoconnor 4y agoYes, we have stack traces. You might see some hashes in the middle of the stack traces, but whenever we do have a name, we'll show you the name. So if you are calling into an anonymous clousure, but functions in your call stack will typically have names available
- anthropodie 4y agoDiscussion from - a year ago https://news.ycombinator.com/item?id=27652677 https://news.ycombinator.com/item?id=27652677 - 8 years ago https://news.ycombinator.com/item?id=9512955 https://news.ycombinator.com/item?id=9512955
- dang 4y agoThanks! Macroexpanded: Unison Programming Language - https://news.ycombinator.com/item?id=27652677 https://news.ycombinator.com/item?id=27652677 - June 2021 (131 comments) Unison: A Content-Addressable Programming Language - https://news.ycombinator.com/item?id=22156370 https://news.ycombinator.com/item?id=22156370 - Jan 2020 (12 comments) The Unison language - https://news.ycombinator.com/item?id=22009912 https://news.ycombinator.com/item?id=22009912 - Jan 2020 (141 comments) Unison – A statically-typed purely functional language - https://news.ycombinator.com/item?id=20807997 https://news.ycombinator.com/item?id=20807997 - Aug 2019 (25 comments) Unison Language March Update - https://news.ycombinator.com/item?id=19528189 https://news.ycombinator.com/item?id=19528189 - March 2019 (1 comment) Unison: a next-generation programming platform - https://news.ycombinator.com/item?id=9512955 https://news.ycombinator.com/item?id=9512955 - May 2015 (128 comments)
- 6gvONxR4sf7o 4y agoI’m excited for ideas like this to become mainstream. Today’s approaches to heterogeneous and distributed computing are how I imagine single core computing was 40 years ago. You have to manually manage practically everything. Instead, let the compiler or interpreter or whatever figure out where to actually run it (CPU vs ALU vs GPU vs remote machine #42), what to keep in what part of cache (L1 vs L2 vs RAM vs disk vs S3), etc.
- junon 4y agoWe have, though. Spark, for example, does this in just about every language. It's been around for ages, is liberally licensed, and deployed at scale in thousands of enterprises.
- cmollis 4y agospark is great for distributed computation.. also has about a million config switches and is generally kind of 'bulky'. EMR makes management a lot easier, but you still have to fiddle with num executors, memory, etc. But it has been 'through the wars' and is generally pretty solid on some pretty large data sets. Once you get it conf'd it's pretty good. The best part is just writing the scala code to run the job.. admittedly, it would be great to use something a bit lighter for certain workloads.
- __MatrixMan__ 4y agoSpark still references functions by name though, right? So if a peer says "call this function" I have to trust both that peer AND whatever system resolves the name to some cpu instructions. Plus you're significantly limited in how much you can memoize if you can have different versions of the "same" function.
- zawodnaya 4y agoWell, Spark does allow you to accomplish distributed workloads for certain forms of computation. But it's limited to those forms of computations (streaming, map-reduce). It also has a large operational footprint. It's also lamentable that distributed code that uses Spark looks nothing like its non-distributed counterpart. Something very much like Spark map-reduce can be implemented in ~100 lines of Unison code: https://www.unison-lang.org/articles/distributed-datasets/ https://www.unison-lang.org/articles/distributed-datasets/ Some videos on Unison's capabilities over and above Spark: Distributed programming overview: https://www.youtube.com/watch?v=ZhoxQGzFhV8 https://www.youtube.com/watch?v=ZhoxQGzFhV8 Collaborative data structures (CRDTs): https://www.youtube.com/watch?v=xc4V2WhGMy4 https://www.youtube.com/watch?v=xc4V2WhGMy4 Distributed data types: https://www.youtube.com/watch?v=rOO2gtkoZ3M https://www.youtube.com/watch?v=rOO2gtkoZ3M Distributed global optimization with genetic algorithms: https://www.youtube.com/watch?v=qNShVqSbQJM https://www.youtube.com/watch?v=qNShVqSbQJM
- politician 4y ago- Function definitions stored in a content-addressed database. - Dependency management handled the same way the Nix handles it. - Some kind of object storage system that uses content-addressable structures as the schema. - Hyperlinked codebase. - Human-readable function names as (essentially) git tags. These ideas are all pretty nice. A dedicated IDE for this language would be a lot of fun to work with. The debugging story likewise seems like it will be pretty solid. I'm not sold on the zero config storage layer: basic object retrieval is different than schema prepared for query performance. I'd like to learn more about the concurrency and synchronization story.
- klabb3 4y agoI think that content addressability works really well for code, at the bottom layer. But in order to be practical, you typically need: Naming and resolvers, in order to be human friendly. This isn't easy to get right, but we have a lot of prior art in dependency management systems. Persistence layers with GC, like cache and db. You're gonna want to fetch and prefetch in ways that are quite advanced. You don't want to be be blocked in a critical section by network fetching the leftpad function.
- brabel 4y agoI think they are making a mistake that's common in this sort of project: trying too many new things at once! They already have a very innovative way of managing source code, with a database of definitions that keeps the hash of the syntax tree instead of actual source. That's a very neat idea that solves many problems (read their docs to understand why). But instead of developing that well enough so that it works with source control tools, IDEs, can be deployed easily and painlessly on existing infrastructure... no, they decided to ALSO solve distributed computing, a really, really complex space with a pretty crowded space of solutions... and seem to be focusing on that now instead of the "original" ideas. Looks like a huge issue with scope creep to me... unless they are kind of pivoting to distributed computing now only because the original ideas were not attractive enough for people to embrace it, but I have not heard of anything like that, everyone seems to be pretty vibed by those things.
- dack 4y agoTo be fair, for a while they were ALSO working on their own graphical source editor that allowed for type-correct transformations and assisted refactorings. They put that on the back burner specifically because they are trying to focus on fewer things :) I think the distributed computing problem is pretty related once you have "content-addressable" source code. Agreed that it's a lot of work but I hope it pans out!
- renox 4y agoI disagree, there's no real relationship between 'content addressable' source code and distributed computing. Also I don't think that you need to create a new language to have 'content addressable' source code distribution.. Creating yet another language ensure that this will get nowhere, too bad.
- nerdponx 4y agoMy impression is that the original use case was distributed computing, and the content-addressable stuff took on a life of its own after that.
- 1letterunixname 4y agoIs this the same language from the how to create a programming language book?
- mgax 4y agoI LOVE the idea but the language itself is so ugly I don’t want to learn it. It looks really ugly. Sorry.
- zawodnaya 4y agoWhat's an example of a pretty language?
- jjtheblunt 4y agoScheme! :)
- zawodnaya 4y agoGood news: the surface syntax of Unison is totally arbitrary and can in practice be swapped out. There's currently only one (Haskell-like) syntax, but in future I imagine there will be others. So you can imagine on https://share.unison-lang.org https://share.unison-lang.org a little dropdown that lets you select what syntax you want to see the code in.
- IshKebab 4y agoI really hate the language but I have to say Python does at least look quite nice.
- tasuki 4y agoYes it does look nice, but I wonder how you'd do eg pattern matching on ADTs...
- optymizer 4y agoI don't think this language is ugly and it is indeed hard to say which language is objectively beautiful, but I personally have always disliked syntax that uses the ' operator for some reason. It's small - it looks like a piece of dirt on the screen, which makes it hard to read. I imagine the same arguments could apply to the '.' or ',' operators, though ' tends to appear around whitespace, while ./, around characters, so there's at least some contextual information around them.
- samsquire 4y agoThis project hashing of sourcecode (or AST) in the interpreter is a really powerful idea I plan to use it for a slightly different purpose. I want to implement an interpreter that is multithreaded similar to Java or Erlang that can send objects between shared memory without marshalling or copying. I had a talk with someone on HN https://news.ycombinator.com/item?id=32907523 https://news.ycombinator.com/item?id=32907523 about python's Global Interpreter Lock and we talked about how objects are marshalled between subinterpreters due to object identity. The identity of an object is defined at creation time as the hash of that object. If the hash of the object was the sourcecode, two interpreters could load the same Object hierarchy and send data by hash reference. I wrote a multithreaded interpreter that uses message passing to send integers and program counters to jump to code in other threads This is at https://GitHub.com/samsquire/multiversion-concurrency-control https://GitHub.com/samsquire/multiversion-concurrency-contro...
- nuphonenuacc 4y agoIirc pony langs messages could work like that.
- toolslive 4y agoNil nove sub sole ? It reeks of COOLs (Concurrent Object Oriented Languages). Here's an example of the mid 90s: https://distrinet.cs.kuleuven.be/projects/CORRELATE/ https://distrinet.cs.kuleuven.be/projects/CORRELATE/
- nerdponx 4y agoPerhaps not, but plenty of good ideas kick around for years or decades before someone finally figures out how to put them together right.
- klabb3 4y agoObject oriented? Unison looks pretty functional to me. And I don't see the content addressability, which is arguably the most important part. Anyway, you'll very rarely see completely novel ideas in this space. Good combinations, compromises and applications matters a lot. Look at your favorite applied Merkle tree tool.
- ModernMech 4y ago> Good combinations, compromises and applications matter a lot An understatement — I’d say when it comes to language design, this is pretty much the whole game right here.
- jrumbut 4y agoThat is the perfect programming language landing page. The Hello World example introduces one of their concepts that you might not see every day, then they show a little algorithm, then a practical "stuff you need to get work done" example. I got an immediate sense that the language has some familiar "ML family" type features (like F# or Scala) but also some distinctive aspects.
- tomcam 4y agoOops I posted something similar to this. Agree completely.
- muricula 4y agoHow does the runtime manage different levels of trust between clients? How does the language grapple with code injection vulnerabilities? If I want to be able to receive data from an untrusted entity, but not code, how do I make sure they're not submitting code to my server to unpack and execute?
- zawodnaya 4y agoYou can simply expose an HTTP endpoint that receives the data. You wouldn't want to expose the internode protocol endpoint to the internet. That said, it wouldn't be as bad as it sounds. Unison is a purely functional language, so if you don't explicitly provide the ability to e.g. do arbitrary I/O, then other nodes will not be able to send you code that does I/O. It will not type-check.
- stewoconnor 4y agoYes, we additionally have functions builtin to the runtime that let you evaluate a term before you evaluate it to make sure it doesn't call any "forbidden" functions. So in our cloud runtime, we blacklist EVERY IO function, then we can in our cloud runtime give you back the ability to do, for example, http requests, but not any other network IO. We won't let you open arbitrary files, but we'll provide ephemeral / persistent block storage through some other runtime ability. This could also be used to do something like blacklist functions with known security vulnerabilities to catch people that aren't applying their patches!
- muricula 4y agoRe the http endpoint, that's well and good, it just requires serialization and deserialization to a different protocol at the endpoint, which the docs led me to believe you wanted to avoid. There is probably an opportunity here to do interesting work around authenticating and validating computations from remote clients. A word of caution -- javascript engines routinely get hacked, and once attackers can execute native code in the engine process they can call syscalls and have all of the rights of the underlying process. It may be useful to have some form of sandboxing of the language runtime for clients exposed to the internet or intranet. Additionally, Java & Ruby web servers routinely suffer from code injection when deserializing objects, which it seems like your language may be prone to as well.
- Serow225 4y agoDoes anyone have a comparison with Darklang/ecosystem and it's goals? Some of the things mentioned sound similar, at a high level.
- ingomaro 4y ago>Other tools try to recover structure from text; Unison stores code in a database Just like VisualAge in the last century. Sign me up, that was great stuff.
- zawodnaya 4y agoSmalltalk had a lot of good ideas!
- kybernetikos 4y agoI loved it, except that it seemed to periodically corrupt the source code database which was an absolute nightmare.
- alexashka 4y agoThis strikes me as a compiler and IDE feature, rather than a reason to have a separate programming language. What does having a separate language give us as opposed to taking say, Kotlin, and having a compiler that stores the AST/whatever in a database and does all the interesting goodies? Or is that the end goal, but we're using a basic language to test it out and work out all the quirks before writing a compiler for existing languages?
- dwohnitmok 4y agoHow do Unison abilities handle non-commutative abilities (i.e. abilities where the order in which one applies handlers matters). Does it just assume that abilities are commutative? Or rely on the programmer to make sure that handlers are applied in an order that makes sense?
- zawodnaya 4y agoThe latter.
- stewoconnor 4y agoIt is up to the code handling the abilities to decide in which order to handle the abilities (or to handle them all at once). I can't think of any cases where it would make a difference in which order they were handled, however. Can you? I think perhaps it might in the case where abilities themselves were able to make requests of other abilities, but that's not something allowed by our type system currently
- dwohnitmok 4y ago> I can't think of any cases where it would make a difference in which order they were handled, however. Can you? Presumably someAction : '{Choose, Abort} a if it's handled by `Choose.toList` and then `Abort.toOptional` in that order you end up with `Optional [a]` whereas if you do in the other order you have `[Optional a]` right? N.B. the reason this is theoretically important is that `someAction` may be written with the assumption of e.g. certain short-circuiting behavior in mind and the "wrong" order of handlers might cause different short-circuiting behavior. In other words there's no consistent semantic interpretation you can assign to `someAction` even if you establish certain invariants that your abilities and ability handlers individually satisfy, since the global configuration of your ability handler changes what `someAction` means. I think the jury is still out on whether this is a practical issue for any language that doesn't try to focus too hard on code having formal semantics (which is most real-world languages). I can definitely craft "real-looking" code that would be buggy depending on the order of handlers, but I'm not personally sure how much of a problem that actually would be for people familiar with the issue.
- tomcam 4y agoOn a tangential note--do you like this home page as much as I do? It really draws me in as a programmer. I like all the code examples up front, and their mission statement ("A new approach to distributed programming/No more writing encoders and decoders at every network boundary") seems to be quite clear. To me one of the best landing pages I've run across.
- dboreham 4y agoAn answer to the question "what useful thing has been build with Haskell"?
- flylikeabanana 4y agoHere's a couple more that I've found useful over the years: - pandoc: convert (almost) any document format to (almost) any other document format - The Elm Compiler: Possibly the most widely used statically typed pure frontend language - XMonad: A tiling window manager for linux that I enjoyed using even without knowing much Haskell - Purescript: the other widely-used (for an FP language) statically typed pure compile-to-javascript language
- cochne 4y agoI really like the idea of Unison, but unfortunately when I went to try it out, it was much slower than even Python. I hope they can make it performant!
- stewoconnor 4y agoIn one of our most recent blog posts we talked about the progress we are making on just-in-time native compilation: https://www.unison-lang.org/blog/jit-announce/ https://www.unison-lang.org/blog/jit-announce/ We are expecting to be a monumental speedup for us. We have some promising results so far, but we haven't yet ported all of the runtime.
- FpUser 4y ago' - is used to denote a delayed computation. This is hard to see / easy to miss. What it this trend to abbreviate everything down to a shortest. What is wrong with delayed keyword for example
- rlmark 4y ago(Unison dev here) I hear you! :) `'` is easy to miss so we added the `do` keyword, and at least a few of us want to substitute `delay` as the keyword. Fortunately, we should be able to do that programmatically at the codebase level if the time comes for that, so folks won't have to hunt for their single quotes to change it everywhere.
- bigcat12345678 4y agoSanjay already had a somewhat similar approach in managing the scheduling of concurrent RPCs, i.e., RPCs are enqueued, and a scheduler looks at the RPCs and figure out windowed batching order to optimize for the overall execution time (here the concurrent RPCs are interdependent).
- bhawks 4y agoThat sounds like a slightly different problem. Unison looks like any function can transparently cross network boundaries versus the efficient scheduling and dispatching of concurrent rpcs.
- mikewarot 4y agoCan someone please ELI 5 for me how to get started? I downloaded and ran it, it created folders NOT where I told it to, and started.... but no command I type seems to result in anything other than an error message. How can I get it to add 5+5, without using an external editor?
- iamevn 4y agoYou're not going to just be able to write code into ucm (unison code manager) because it's not a repl. You'll need some sort of text editor to actually write the code it will then add to the codebase. https://www.unison-lang.org/learn/quickstart/ https://www.unison-lang.org/learn/quickstart/ If this quick start tutorial doesn't work for you then there's a bug on unison's end.
- huqedato 4y agoIt smells like Haskell. Perhaps more human-friendly. Nice.