14 ms·

Clojure.spec – Rationale and Overview

• nikolay 10y ago
  I'm waiting for the day somebody with write a Clojure transpiler for a non-Lispy syntax! Although I love Lisp (I used to use it 20+ years ago daily), but the year is 2016!
  • ambulancechaser 10y ago
    How could you implement macros in any sane way without s-exps? You need an AST and the hoops involved if you didn't have something like lisp would be terrible. Imagine running a macro over python or C code.
    • ebiester 10y ago
      Elixir has an interesting way of approaching the problem. http://elixir-lang.org/getting-started/meta/macros.html http://elixir-lang.org/getting-started/meta/macros.html
      • Gonzih 10y ago
        But this introduces additional ast format. Something in between simplicity of the syntax and complexity of the real ast. In lisp like languages you are working with syntactic forms from the language. Seems like extra overhead without any gain.
        • ebiester 10y ago
          I would personally rather optimize for the convenience of reading the language than writing the macro, but I understand the tradeoff. (And though I came to S-exprs early in my learning, algol-style still is easier for me to parse and read.)
          • fnordsensei 10y ago
            And this is precisely the point of divergence, as "readable" understandably means different things to different people. Personally I find Clojure very readable, but of course I don't mean that in some kind of objective sense of the word.
        • kimi 10y ago
          Coming from a Clojure background, "extra overhead without any gain" is more or less the definition of Elixir.
          • fnordsensei 10y ago
            Isn't the Erlang VM and ecosystem sort of the selling point of Elixir? Not that I'm particularly well versed in the specific strengths and weaknesses of that platform.
            • LeonidasXIV 10y ago
              Yes, but you can have that with Erlang as well and I think this is exactly what grandparent was alluding to.
          • hackerboos 10y ago
            Runtime overhead possibly. Developer productivity is likely increased. Hard to measure that though.
  • konradb 10y ago
    What syntax do you feel would be more expressive and as versatile, while not losing macros?
    • nikolay 10y ago
      Well, Rust has macros [0], too! [0]: https://doc.rust-lang.org/book/macros.html https://doc.rust-lang.org/book/macros.html
      • nickik 10y ago
        Rust has tons of problems with macros and they are still not fully in the language (last I checked). Rust macros actually do go back to Lisp. Specifically the Dylan language, a Lisp once envisioned as the equivalent of modern Objective C by Apple. Their are a number of papers from that period that you can read. Macro systems are not all the same, they don't all have the same power. Clojure uses a very powerful form that is not as safe. Their is a lot of modern research in Racket. A extremely powerful macro in a language with syntax was specified but never implemented by David Moon, you can find it here: http://users.rcn.com/david-moon/PLOT/ http://users.rcn.com/david-moon/PLOT/
        • steveklabnik 10y ago
          "Macros" can be an overloaded term in Rust. Specifically, "macros" mean "macro_rules!", which is very much in the language. Compiler extensions, which are sometimes called "procedural macros", are still unstable.
  • bad_user 10y ago
    Not sure what the year has to do with anything, but most things in computer science were invented in the sixties and the seventies or even earlier and since then we've only been refining them. In terms of programming languages, the theoretical breakthroughs have been in terms of static typing, even though note that even the Hindley-Milner type system has been first described in 1969. In other words, strange thing to complain about a syntax that has stood the test of time. After all, most mainstream languages (C, C++, Simula, Pascal, Java, etc) are Algol-based, a language that first appeared in 1958. And no, you can't have LISP without its syntax, because that's what's giving it power. The "code is data" mantra only works because Lisp is homoiconic, because it basically has no meaningful syntax other than syntax for describing its primitive data-structures, which makes its macros first class and its "eval" natural. But in the LISP family Clojure is actually quite modern. It runs on both the JVM and Javascript engines. It does have extra syntax for describing sets and maps. Its data-structures are immutable and it encourages immutability to a far larger degree than other LISPs. But it also has very good interoperability with the underlying platform.
    • nikolay 10y ago
      By the way, it's now "Lisp", not "LISP". I love the original Lisp, because it was simple Syntax was so suitable for AI and treating code as data, but Clojure is not as simple syntax as Lisp! So, it's both complicated and unusual to the nowaday developer!
      • deleted 10y ago
        [deleted]
    • xearl 10y ago
      > And no, you can't have LISP without its syntax, because that's what's giving it power. The "code is data" mantra only works because Lisp is homoiconic No. Homoiconicity is nice for "code is data" and macros, bot not necessary. Being able to reify and generate ASTs is sufficient. See for example Dylan, Terra/Lua, Scala LMS, MetaML, or Julia's or Rust's macros.
      • 10y ago
  • ungzd 10y ago
    Usually people do the opposite — write transpiler from lispy syntax to some language's syntax. For example: lfe, hy, to some extent clojurescript.
  • willismichael 10y ago
    For those who find all of those parenthesis off-putting: http://clochure.org/ http://clochure.org/
    • mannykannot 10y ago
      I suspect that if you think parentheses are the problem, they are not the problem.
      • nikolay 10y ago
        Yeah, I actually love the parentheses!
• drcode 10y ago
  > spec has been designed from the ground up to directly support generative testing via test.check. When you use spec you get generative tests for free. Great, I've been holding out on digging into the test.check generator syntax... looks like my laziness paid off!
  • puredanger 10y ago
    It's still worth digging in! There are times when you will want to write your own generators.
• deleted 10y ago
  [deleted]
• pyritschard 10y ago
  This is an important move for the Clojure language. As with all dynamic languages, and exarcerberated by the fact that it encourages relying on data first - as opposed to say, classes in other languages - Clojure puts the onus on the developer to catch errors in data. The clojure community as a whole embraced prismatic's schema as a way to provide occurence typing for data. Since library authors often tend to reduce their dependency surface, most libraries do not ship with it though. With the inclusion of this in clojure core, it will now be possible to provide occurence typing at the edge of functions instead of tracking malformed input deep inside apps. The racket-type contract notation is also a very welcome change from other similar approaches in my opinion. [1]: https://github.com/plumatic/schema https://github.com/plumatic/schema
  • gshayban 10y ago
    Agreed that this is a big step, but what is being included in core is very distinct from Prismatic Schema. For example the way keysets vs values are handled is intentionally different from schema. The rationale is a good read.
    • pyritschard 10y ago
      Yes, I've read and see the difference from schema. The huge advantage with this, is that even though the approach is different, it supersedes schema's abilities and it will be a require away for every library and application author.
    • jrcii 10y ago
      > The rationale is a good read. Hickey's design choices consistently appear to be exceptionally well reasoned.
      • solipsism 10y ago
        It's because of the hammock time.
  • pyritschard 10y ago
• ComNik 10y ago
  Clojure, with its dabbling in schemas and rule systems / logic programming (as a substitute for conditional statements) and thanks to its great tools (like figwheel or devcards), could really be establishing a cheaper, "as-good" alternative to static typing and full formal verification. Instead of encoding constraints as type signatures, the Clojure folks (true to character) encode them in data. In my eyes a very interesting, pragmatic trade-off between expressiveness and automatic verifiability.
  • llamaz 10y ago
    > dabbling in schemas and rule systems / logic programming (as a substitute for conditional statements That sounds interesting. Could you elaborate or post a link?
    • mhluongo 10y ago
      Quick refs to logic programming and pattern matching in Clojure. core.logic https://github.com/clojure/core.logic https://github.com/clojure/core.logic core.match https://github.com/clojure/core.match https://github.com/clojure/core.match
    • untothebreach 10y ago
      I assume @ComNik is referring to core.schema and core.logic[1] here, respectively? (if not, sorry @ComNik) 1: https://clojure.github.io/core.logic/ https://clojure.github.io/core.logic/
    • ComNik 10y ago
      As a side note in his talk "Simple Made Easy" (https://www.infoq.com/presentations/Simple-Made-Easy https://www.infoq.com/presentations/Simple-Made-Easy, around minute 42) Rich Hickey mentions, that conditional statements are complex, because they spread (business-)logic throughout the program. As a simpler (in the Hickey-sense) alternative, he lists rule systems and logic programming. For example, keeping parts of the business logic ("What do we consider an 'active' user?", "When do we notify a user?", etc...) as datalog expressions, maybe even storing them in a database, specifies them all in a single place. This helps to ensure consistency throughout the program. One could even give access to these specifications to a client, who can then customise the application directly in logic, instead of chasing throughout the whole code base. Basically everyone involved agrees on a common language of predicates explicitly, instead of informally in database queries, UI, application code, etc... But Hickey also notes that this thinking is pretty "cutting-edge" and probably not yet terribly practical.
• sdegutis 10y ago
  > Defining specifications of every subset/union/intersection, and then redundantly stating the semantic of each key is both an antipattern and unworkable in the most dynamic cases. This is the reason I haven't jumped on the Schema bandwagon. Surprised to see I'm not the only one who had reservations about it. > Finally, in all languages, dynamic or not, tests are essential to quality. Too many critical properties are not captured by common type systems. Glad to see some acknowledgement of the need for at least some tests from Rich, who previously critized TDD, saying that you don't keep a car on the road by banging into the guardrails, and advocated H(ammock)DD. > But manual testing has a very low effectiveness/effort ratio. Property-based, generative testing, as implemented for Clojure in test.check, has proved to be far more powerful than manually written tests. (Assuming by manual testing he means manually writing automated tests,) I can't help but wonder if he came to that conclusion by writing the wrong tests and, naturally, finding them unhelpful. > Enable and start a dialog about semantic change and compatibility Okay, slightly rant-y tangent, but out of all the fluff phrases and buzzwords I've ever heard, "start a dialog" is probably the worst offender, and I wish it was wiped from the English language. > I hope you find spec useful and powerful. Looks useful so far. But where's the Github repo? Or is it coming in the next version of Clojure? Or, like, how/when can we use it?
  • puredanger 10y ago
    Bits will be coming into the Clojure repo today and we'll have an alpha release in the next couple days along with a lot more info.
  • hellofunk 10y ago
    >of all the fluff phrases You're forgetting "it is worth noting", "it is important to note," "it should be noted that," "I should note," and their many derivatives which serve no purpose and are nothing but filler. [0] [0] https://en.wikipedia.org/wiki/Wikipedia:It_should_be_noted https://en.wikipedia.org/wiki/Wikipedia:It_should_be_noted
  • nickik 10y ago
    Rich was never against testing, he was against TEST DRIVEN development, and thats what TDD stands for.
• hellofunk 10y ago
  >The basic idea is that specs are nothing more than a logical composition of predicates. This reminds me of this technique: https://github.com/astoeckley/Eat-Static#going-further https://github.com/astoeckley/Eat-Static#going-further
• minikomi 10y ago
  I enjoy using racket and contracts make it quite enjoyable to use - error messages in particular are much improved by their presence. Looking forward to this!
• jcadam 10y ago
  Excellent. This definitely looks to be an improvement over schema, which I use but don't particularly like. Though, I'm fairly new to Clojure (about a month or so), so it may just be my inexperience :)
• noelwelsh 10y ago
  It's unfortunate that Rich Hickey (assume he's the author) can't write this without trolling about type systems. It's pretty clear to anyone who's read the literature, or has thought for a few moments, that contract systems provide different guarantees to type systems, so no need to poke at type systems throughout the Guidelines.
  • erichmond 10y ago
    I think you're "seeing what you want to" in his comments. One of the reasons I appreciate clojure and the community so much is that it's one of the few that doesn't actually troll things. If anything, the community tends to give really well thought out reasoning around why they make the decisions they are making. When making an argument, sometimes it's important to explain why something is different, or why you've made the choice you made. Just because you explain why your idea is different doesn't make it trolling.
  • wtetzner 10y ago
    Yes, especially since there's no reason you can't have contracts in a language with a type system.
  • llamaz 10y ago
    > or has thought for a few moments As is obvious even to a child...
  • joncampbelldev 10y ago
    I don't think he's trolling, he's proposing spec due to issues he personally sees in type systems whilst acknowledging that clojure without types or spec or whatever is definitely missing "something" ... it's just that something doesn't have to be types. Given his comments here he seems well aware that not everyone shares his views on type systems and that is fine: https://groups.google.com/forum/#!msg/clojure/DUKeo7sT4qA/TU0Q7CIbsTIJ https://groups.google.com/forum/#!msg/clojure/DUKeo7sT4qA/TU...
  • bad_user 10y ago
    For one it goes with the territory. When designing a language you need to emphasize its strong points and Clojure's strong point is its (sane) dynamic nature and they are playing to its strengths for example in solutions such as their Transducers. I don't view it as trolling. The mentality and style in dynamic languages are simply different. Once you add static typing, you break that mentality, replacing it with something different. Not inferior, just different. The patterns, the solutions end up looking different. This is why static typing added to a LISP like Clojure can only be optional and probably less powerful than what you can get with something like Haskell. Or in other words, type systems are not practical for Clojure. There are attempts, like core.typed. Such solutions don't work very well because as said, adding types changes what you're allowed to do, even if those types are optional. So he's speaking primarily in the context of Clojure. Also, poking fun at your competition is good marketing. People love it :-P
• thesorrow 10y ago
  This is awesome. Is it available for clojurescript too ?
  • MikeOfAu 10y ago
    From Alex Miller via slack: > will be in cljs as well, but not on first release > most of the code should work as is, most of the tricky bits are around vars
• junke 10y ago
  Those regular expressions for sequences seem related to "Efficient dynamic type checking of heterogeneous sequences", or maybe some other prior work I don't know about. See https://github.com/jimka2001/regular-type-expression https://github.com/jimka2001/regular-type-expression and http://www.lrde.epita.fr/dload/papers/newton.16.rte.report.pdf http://www.lrde.epita.fr/dload/papers/newton.16.rte.report.p...
• jraines 10y ago
  Could someone explain what the paragraphs under "Map specs should be of keysets only" are getting at? I'm trying to map it onto my experience with Clojure and plumatic/schema, but nothing is clicking.
  • xmlblog 10y ago
    If you have a function that accepts a map of a certain shape and want to write a specification for that map, you should only be specifying the required/optional keys in that spec. You should not be forced to also specify the values for those keys. When you are (as in Schema) then you wind up with redundancies when you have a slightly different map structure that also includes one or more (but not all) of the same keys with the same semantic meaning.
  • puredanger 10y ago
    Generally in most of the existing libs like Schema you specify that a "person entity" is a map that has particular attributes that have particular properties and a "company entity" is a map that has other attributes (some of which might overlap). In spec, you specify that a first-name attribute is a string and an email attribute matches some regex. Those are specifications about the attributes and can be used across many entity types. A person entity spec is then defined as a map that contains a set of required or optional attributes. A company entity spec is a different map with different attributes, some of which might be shared with a person (like email address). This is in many ways a shift in perspective, allowing you to start defining and reusing attribute specs to a much greater degree. The use of structural systems with mathematical properties (regular expressions for sequence structure and sets for map keys) turns out to have many useful effects in computing error sets, etc.
    • zcp 10y ago
      thanks for this excellent explanation; this provides just about all the justification I need. looking forward to toying around with this.
    • greghendershott 10y ago
      The division -- between predicates for attribute values vs. specs for maps/seqs -- is interesting because AFAICT Datomic schemas ~= the former, while leaving the latter to your app? (Although I guess :db.cardinality/many straddles that line.)
• lispm 10y ago
  Luckily Common Lisp already has ways to do that: Functions have defined keyword argument lists and can have type declarations which for example a compiler like SBCL can partially check at compile-time. Common Lisp uses for anything more complex than lists of keyword value combinations CLOS classes, which form a multiple inheritance hierarchy and types for slots. That way I can look into a debugger at runtime and see what functions are actually supposed to receive and what these collects of keyword/value maps actually mean (since they are CLOS classes). Having simple lists as maps was popular with assoc lists in the 70s - basically the same as hash tables, but slower. Then lots of dynamic object systems were developed which were in essence dynamic key/value sets. Later various object systems appeard, with CLOS as a local maximum. Where Rich Hickey points to RDF, there is a lot prior art in Lisp as well -> Frame systems / description logics / .... http://wilbur-rdf.sourceforge.net/docs/ivanhoe.html http://wilbur-rdf.sourceforge.net/docs/ivanhoe.html http://franz.com/agraph/racer/krss-spec.pdf http://franz.com/agraph/racer/krss-spec.pdf and zillion others CLIM, the Common Lisp interface manager, uses 'presentation types' to specify user level data structures...
• DigitalJack 10y ago
  If you are new to using a snapshot of clojure (like me), you can find some helpful information here: http://dev.clojure.org/display/community/Maven+Settings+and+Repositories http://dev.clojure.org/display/community/Maven+Settings+and+... tl;dr In a Leiningen project.clj file: :repositories {"sonatype-oss-public" "https://oss.sonatype.org/content/groups/public/"} Also, in that lein project file, reference clojure as [org.clojure/clojure "1.9.0-master-SNAPSHOT"] (1.9.0 being the dev head right now)
  • drewr 10y ago
    This won't work with Cider btw. I had to change it to 1.9.0-SNAPSHOT in pom.xml before a mvn install. error in process filter: version-to-list: Invalid version syntax: '1.9.0-master-SNAPSHOT' error in process filter: Invalid version syntax: '1.9.0-master-SNAPSHOT'
    • DigitalJack 10y ago
      Interesting. It works for me, but I'm using a dev snapshot of cider too. Having -master- in there is pretty non standard, but I was poking in oss.sonatype.org and that's what the jar looked like in there. Maybe I goofed. There is a little bit of magic that happens with maven and snapshots (magic to me anyway).
• xmlblog 10y ago
  Incidentally, the source is here: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj https://github.com/clojure/clojure/blob/master/src/clj/cloju...
• georgewsinger 10y ago
  I'm confused what a function spec would look like using fdef (etc). Can someone write an example? If in Haskell we have a type signature f :: Type1 -> Type2 ...what will a clojure function spec roughly look like?
  • puredanger 10y ago
    (spec/fdef clojure.core/range :args (spec/alt :infinite (spec/cat) :range-0 (spec/cat :end number?) :range (spec/cat :start number? :end number?) :step (spec/cat :start number? :end number? :step number?)) :ret #(instance? clojure.lang.Seqable %))```
    • georgewsinger 10y ago
      Interesting. I don't know what to think. It's a lot to read. But in some ways adds more information than types (you can make reference to values). Maybe you could make it more concise with more definitions? For example: (def UnboundedInterval (spec/cat)) (def PositiveUnboundedInterval (spec/cat :end number?)) (def ClosedInterval (spec/cat :start number? :end number?)) (def StepClosedInterval (spec/cat :start number? :end number? :step number?)) (def SeqableInstance #(instance? clojure.lang.Seqable %)) And then: (spec/fdef clojure.core/range :args (spec/alt :infinite UnboundedInterval :range-0 PositiveUnboundedInterval :range ClosedInterval :step StepClosedInterval :ret SeqableInstance) Perhaps this would be non-idiomatic, and perhaps I'm trying to make this into something that looks like a static type annotation when it isn't.
      • hcarvalhoalves 10y ago
        Looks sane, specially because you can (s/and ...) the specs so you should have reusable specs in some common namespace.
      • catnaroek 10y ago
        > (you can make reference to values) Dependent types let you do just that as well. While theoretical computer scientists for the most part focus on dependently typed systems capable of expressing and proving any mathematical proposition (which obviously results in increased complexity), there exist more pragmatic variants of dependent types aimed at solving the everyday needs of programmers, with a minimum of annotation burden, see: http://hackage.haskell.org/package/liquidhaskell http://hackage.haskell.org/package/liquidhaskell
• elwell 10y ago
  This site could really benefit from a custom stylesheet for printing. Having to delete lots of nodes with devtools just to get a printable doc.
  • puredanger 10y ago
    Noted. Won't get to it anytime soon though. https://github.com/clojure/clojure-site/issues/99 https://github.com/clojure/clojure-site/issues/99
• georgewsinger 10y ago
  Will there be performance hits to using this?
• bvladimir 10y ago
  Bothered your spouse might be having an affair? Clear your doubts by Contacting: whitewiz007@programmer.net to check your spouse's sincerity You will be happy or shocked, and for your other hacking needs such as; emails, facebook,social media, CHANGING COLLEGE GRADES, clearing bad records, keeping tabs on employees or getting any classified information
• macmac 10y ago
  clojure.spec is available in 1.9.1/alpha1 via [org.clojure/clojure "1.9.0-alpha1"] in your project.clj. There is an excellent guide available here: http://clojure.org/guides/spec http://clojure.org/guides/spec