6 ms·

So Clojure and CLJS are amazing. If I were to pick a Lisp for a real world project, Clojure is the obvious pick. The thing is, I'm curious to hear from people w

So Clojure and CLJS are amazing. If I were to pick a Lisp for a real world project, Clojure is the obvious pick. The thing is, I'm curious to hear from people who DO use Lisp in the real world. I would characterize my experience with it as 1: Really fun. 2: Productive on a small scale due to the tooling, interop, libraries, and community being innovative and awesome, and 3: Infinitely flexible, because you know .. it's a Lisp. Development becomes like doodling on a piece of paper, like a stream of consciousness. But I ended up choosing Scala for a recent, fairly large, project of mine and I'm convinced it was the right decision. Without a static type system I would have simply lost a handle on the complexity of the project. And this is while working alone (the case where Lisp traditionally has the upper hand). A few questions for people who use Lisp in production. No need to answer all, just things I'm curious about: * During refactoring, do you feel more flexible, or less? I can see both sides to the argument. On one hand, static types are more rigid, and getting something to compile during a refactoring can be a pain. Often for the right reasons, but often it's due to some annoying technicality in the type system. But once you get there, "if it compiles, it runs" is a cliche that is pretty accurate in my experience. On the other hand, with a dynamic type system, you can slowly evolve a code base and see incremental progress during a refactoring, but I can never get over the dreadful feeling of "what did I forget?". Has clojure.spec worked out in terms finding bugs during refactoring/maintenance and general QA? * Do you make use of the Lispy-ness (Homoionicity, macros, code is data, etc ...) or is it generally an anti-pattern to use these language features in daily development? * Do you wish you had static types? If so, have you tried gradual typing like Typed Clojure? And if you don't miss types, why? I remember someone saying of Typed Clojure "We went to Clojure to get away from types!". And I'm still trying to understand the mindset. I often have the feeling that I'm not smart enough to use Lisp. If I had 2x the brain power, I would be able to use macros to conjure up some black magic in 100 lines what would take a mortal 5k in a lesser language. But even if I manage such a feat, I couldn't imagine bringing on another developer to that codebase.

wooby 8y ago
At a Clojure meetup I attended a couple years ago, a long-time Java programmer asked "how do you manage huge projects without static types?" The best answer I could think of was, "don't manage huge projects." Instead, divide projects as early as possible in development into supporting libraries, each capable of being independently tested and maintained. In my experience, applications developed with this attitude coalesce themselves into a small ecosystem of mostly generic library code with a domain-specific apex "application" entrypoint to bind them. In an application factored like this, type checking isn't something you want as much anymore, since the constituent libraries interface with one another at a level above. They present and consume "public" (intra-application) APIs, the behavior of which are defined not just by types but by at least documentation and tests. The thing that static types let you do -- have huge amounts of code, woven together with structural dependencies -- is no longer a thing you need or want. It would seem bad, actually, since what you'd have is an inscrutable mass only a machine can understand. I haven't worked enough with large non-Clojure codebases to say whether or not this approach is superior, but I can say that it scales. Another nice thing about it in a business setting is that supporting libraries lend themselves to open-sourcing, and a relationship with an open source community is a good thing for a company's development efforts/hiring/contracting in my experience. To answer your specific questions: * I don't think I write more unit tests (I should probably write more tests in every language I use) * I don't have any experience with clojure.spec * All of the features of Lisp work together to make it awesome, including macros. In particular, in the context of an "ecosystem of libraries" approach, macros are a fantastic tool for exporting a DSL from a supporting library. They're also use for DSLs in the apex code, where application-specific behavior can be defined in terms of a DSL. * I don't really wish I had static types - I love Lisp the way it is. But I have found various languages for structural checking useful with certain kinds of sub-program, like receiving data from the outside world, and for writing compilers. If you write a compiler as a series of transformations, it's nice if the shape of the code between transformations is well-defined somehow. You're definitely smart enough to use Lisp. I think the key to success is not necessarily smarts, but attentiveness to modularity or cognizance of the difference between "library" and "application" types of code. * There's a Paul Graham essay related to this idea: http://www.paulgraham.com/progbot.html http://www.paulgraham.com/progbot.html * I think this approach is applicable not just to Lisp, but maybe to any dynamic language with a sufficiently rich set of generic collections and with the concepts of a "module" or "package".
- lopatin 8y ago
  Thanks for the insightful answer! Building an ecosystem of libraries to do most of the heavy lifting is generally what I strive for as well. It's certainly the goal, and I'll usually end up there for a given project. Perhaps one thing that makes me fond of static types is that, more often than not, I don't actually know how or what to modularize until after the project is nearly built in the first place. And I use types not only as a QA tool, but also as a tool for reasoning. The feedback loop between me encoding logic in types, and the IDE providing me with errors, is a big part of how I understand what I'm building, as I'm building it. I imagine that Clojure development on the REPL can actually serve the same purpose. I think the idea of bottom-up programming also resonates with me, though I tend to find it hard to use as a general approach to programming. In my mind, it's similar to the development of DSLs, which I do fairly infrequently, and with care. There is an undeniable elegance to it ... find a set of primitive operations that can be composed into larger ones. My development style usually goes in either direction: Either super generalized, for which I'll take the bottom-up DSL approach, or the more typical case where it's more approachable for me to be explicit with types/code.
  - FPGAhacker 8y ago
    Myself, I sort of do both bottom up and top down programming. Usually I start at the top and write out high level code as if everything I needed magically existed. That helps me think about what I want and how I think I'll use it and that helps and clues me in to what low level stuff I need to build. Then I start to think about how to build those things and that usually goes bottom up. I do this iteratively. When I get stuck I go back to high level top down, figure out what I'm missing, go build that, and repeat.
    - AnimalMuppet 8y ago
      That's called Ping-pong Design, and is a perfectly legitimate approach.
  - yogthos 8y ago
    My approach to modularizing from the start is to do bottom up development, and to treat each namespace as a little library. Ultimately, pretty much all problems can be viewed as data transformation pipelines, and those naturally map to independent components doing different kinds of transformations. I find that the APIs tend to form naturally around domain boundaries as you move data from one context to another. The REPL is indeed a major component of doing the Clojure workflow where you literally run each function as you're writing it. This approach also encourages refactoring as you go since you can make changes and see the results immediately.
blandflakes 8y ago
Clojure is a great language. I've used both Clojure and Scala in my personal and professional life. * In refactoring, I almost always make mistakes in Clojure that Scala will catch. I feel empowered rather than limited by the Scala compiler 9 times out of 10. * Homoiconicity makes me feel great, but I find I don't actually miss it (directly) in Scala. That said, Scala has a complex grammar, and I miss the overall consistency of s-expressions. * I always miss static types. I'd love to try the new spec improvements in Clojure 1.10; debugging mistakes prior to that are a nightmare. I don't need static types to be happy, but I do need real tooling support when things go wrong. Clojure has a lot of lifting power and I would like to keep using it. I find that once I get past type errors, programs work very well. Why would I spend time getting past those if I don't have to?
- FPGAhacker 8y ago
  Yeah, I agree with this, other than not having used Scala. Static typing communicates intent to the user of your functions, they help you think about the problem before going all-in. Rich presented some new ideas for where he wants to take Spec, but I didn't think anything very new made it into 1.10 regarding spec. Personally I use clojure whenever I can. I can quickly build up a project using the fast and loose data types, and when my conception of the project solidifies, I convert over to records and protocols. That's usually enough to get 80% of the way there. The free and loose ad-hoc data structures get me across the finish line, and it's there that Spec comes into play.
  - blandflakes 8y ago
    Yeah I think I was thinking of the error message improvements rather than any changes that are specific to Spec in 1.10.
dizzystar 8y ago
I've worked on numerous Clojure projects, both personal and professional. I started most my learning on Lisp, so I'm probably not the most qualified to discuss this outside of the thought that many people come to Lisp and Clojure with approaches that aren't entirely compatible with Lisp. I know my thoughts are going to be controversial, so take it for what it's worth. During refactoring, do you feel more flexible, or less? I can see both sides to the argument. On one hand, static types are more rigid, and getting something to compile during a refactoring can be a pain. Often for the right reasons, but often it's due to some annoying technicality in the type system. But once you get there, "if it compiles, it runs" is a cliche that is pretty accurate in my experience. On the other hand, with a dynamic type system, you can slowly evolve a code base and see incremental progress during a refactoring, but I can never get over the dreadful feeling of "what did I forget?". Has clojure.spec worked out in terms finding bugs during refactoring/maintenance and general QA? I think there is good and bad to this, but in my experience, using a lot of types seems to be an anti-pattern. Since most web-based projects really only deal with strings, basic numbers, lists and dictionaries, I really don't see the entire benefit, even in strongly-typed languages. I work with typed languages as well and generally feel ambivalent about types. A bad refactor is a bad refactor and bad code is bad code no matter what language you are using. If you can't understand what the code is doing, don't touch it until you can test against it or test against what it's really supposed to be doing. Types don't alter this fact. Do you make use of the Lispy-ness (Homoionicity, macros, code is data, etc ...) or is it generally an anti-pattern to use these language features in daily development? Homoiconicity is the very definition of Lisp, so there is no way to work around that. Macros are generally eschewed in Lisp. It's glaringly obvious when and if you need to use a macro, and honestly, most people can go their entire career without pushing a macro into production. Code is data is pretty hand-wavey in my opinion. All it's really saying is that, if you call a function with a value, you get a value back. There is no mental quantum leap needed for this. Do you wish you had static types? If so, have you tried gradual typing like Typed Clojure? And if you don't miss types, why? I remember someone saying of Typed Clojure "We went to Clojure to get away from types!". And I'm still trying to understand the mindset. I don't think the community has any strong for-or-against opinions on types. In general, it's bad practice to use multiple-type functions. The perspective that's generally embraced is that each function does exactly one thing, which is why you see so many Lisp projects with many short functions, and if you pay attention, the functions aren't doing a lot of acrobatics based on possible types. What the community doesn't like is someone coming along and demanding something that isn't compatible. This is a little more nuanced, but it boils down to: "Add types if you want me to use it" or "Create a RoR system for me to use it." The community would rather you take the time to embrace Lisp and really try to understand what it is attempting to do before pushing your judgment on it. I often have the feeling that I'm not smart enough to use Lisp. If I had 2x the brain power, I would be able to use macros to conjure up some black magic in 100 lines what would take a mortal 5k in a lesser language. But even if I manage such a feat, I couldn't imagine bringing on another developer to that codebase. Anyone can use Lisp. Most of what you do is very simple. The entirety of Clojure is written on a single webpage. https://clojure.org/api/cheatsheet https://clojure.org/api/cheatsheet No one is building up systems with loads of macros and wild abstractions that cause a 5KLOC code base to shrink to 100 lines. This just isn't possible, and anyone that claims it is has no understanding of what a macro actually does.
dmichulke 8y ago
I programmed Java for years before I started using Clojure so I will make that distinction here to answer your questions > During refactoring, do you feel more flexible, or less? Both. More flexible because in Java you have 5x more code (was worse before lambdas). Less because I sometimes look up the code and don't know (and can't immediately infer) the types anymore. Usually I have a handful of integration tests that show me immediately when there is a problem but I wish there was a way to show the inferred types in my IDE. The problem is though usually only a big one for the one big state where you have lots of nested maps, vectors, arrays and so on and only if you modify the structure of this state. For smaller things, fixes are fast and easy, even though they still take 10 minutes while they could take 5. I feel it's kind of the price I pay to use Clojure but then again there's probably a reasonable solution to it - Clojure infers the types anyway, so why not show them somehow. I use clojure.spec partly, often when I need to throw useful error messages (e.g., a user facing REST API). It's a little bit tedious but it's always a good decision in retrospect. > Do you make use of the Lispy-ness? Yes and no. on the positive side: I don't ever need to look up the syntax of anything (I still make mistakes in Java switch for example) and using case and multi-methods I can really interpret quickly a DSL in a web server (e.g., binary arithmetic operations on OHLC bar time series) and I wouldn't want to implement that (safely) in another language. > Do you wish you had static types? If so, have you tried gradual typing like Typed Clojure? I tried for a brief two days gradual typing and didn't get it. I wish I had static typing exactly when refactoring or debugging. > I often have the feeling that I'm not smart enough to use Lisp. Oh, I'd even say the opposite, i.e., I am smarter because I use Lisp. I know Sapir-Whorf is "deprecated" but I feel I am thinking faster because I am thinking in much less convoluted structures compared to when I programmed in Java.
FPGAhacker 8y ago
I know this isn't answering a question you asked, but next time you feel like diving into clojure, focus on Records and Protocols. I think you'll get some relief from the free fall feeling.
weavie 8y ago
I use Common Lisp rather than Clojure at work. - During refactoring, do you feel more flexible, or less? A lot of it depends on how the code has been written. There are definitely parts of our code base that I really dread going near and there have been plenty of times when I have broken stuff by making a change without fully understanding the code. It is quite easy to write incomprehensible lisp code. With Clojure being more pure and opinionated things are probably not quite as hairy as it can get in CL. But if you write your code well CL can be fairly easy to refactor. Generally preferring to use CLOS (the object system) rather that just property lists and hash tables will give you some typing information that can make life a lot easier. Avoid nested lists as data structures - there is nothing worse than code that does stuff like `(dosomething (cadar list) (cddar list)` - Do you make use of the Lispy-ness? Macros tend to only be used for writing framework code. There is generally no need to use them when writing business logic. The aspect of Lisp that is often not mentioned is the iterative development. When code is data it becomes possible to evaluate code against a running system. This is, in my opinion, the nicest feature of Lisp. No compile times, you can change code on the fly, evaluate and test functions as you are writing them. Development becomes a real conversation with the compiler. You can even connect into the live server if needed to debug problems. - Do you wish you had static types? Yes.
badsavage 8y ago
We've used Clojure in production for years. 1. "Has clojure.spec worked out in terms finding bugs during refactoring/maintenance and general QA?" - Spec worked out as a much more useful library than we expected. You could describe any data structures with it, then validate, generate, regenerate it. I think spec is a must have for any Clojure programmer from 2018. 2. "Do you make use of the Lispy-ness" Macros are super useful sometimes, but in general we try to avoid them, because we prefer to solve everything with just pure functions. Homoiconicity and the "code is data" aspect are must haves in any programming language after you realised their power. 3. "is it generally an anti-pattern to use these language features in daily development?" macros are feel like cheating sometimes, and we fear the consequences, but homiconicity is useful for us on a daily basis. 4. "Do you wish you had static types?" We prefer to create our own "types" up to complex data structures. A built-in type system seems like a downgrade from here. 5. "I often have the feeling that I'm not smart enough to use Lisp." I think you are just scared from the unknown, Clojure is an alien technology compared to most of the currently popular languages. Any kids could learn programming in Clojure due to its simplicity and the REPL driven workflow. Its an interactive experience and very likeable.
- pvinis 8y ago
  Can you elaborate a bit on your own "types"? I am very interested and intrigued.
  - badsavage 8y ago
    Yes, I could show you a simple example: Let's create our own 78 digits long string type: (def hex-digit (set "0123456789")) (defn hex-digit? [x] (contains? hex-digit x)) (defn hex-str? [s] (every? hex-digit? (seq s))) (s/def ::hash (s/and hex-str? #(= (count %) 78))) ::hash is now registered, now we go to the REPL: (s/valid? ::hash "23179372") ;;=> false (s/valid? ::hash "nope93721907914920047210715459933122004671648400678953445710500236944435987060") ;;=> false (s/valid? ::hash "231793721907914920047210715459933122004671648400678953445710500236944435987060") ;;=> true That's cool, but we want to generate too: (defn hex-str-gen [n] (let [digit (g/elements hex-digit)] (g/fmap clojure.string/join (g/vector digit n)))) (s/def ::hash (s/with-gen (s/and hex-str? #(= (count %) 78)) #(hex-str-gen 78))) We extended our ::hash spec with a generator function, so we can do this now: (g/generate (s/gen ::hash)) ;;=> "310584715385467847758653938894742415543975445356609397864862925839413265904779" (+ Pro tip: you could spec functions too) Learn more about spec here: https://clojure.org/guides/spec https://clojure.org/guides/spec
aerique 8y ago
"Not smart enough to use Lisp." I hear that more often and I do not understand it. I chose Common Lisp (CL) because it is an easy language not because it's hard (I would have chosen Haskell or C++ then). I'm not sure I can answer your questions. While I do use CL at work for smaller jobs I've only used it for bigger personal / solo projects, so I'm not sure that counts. About refactoring: it's not a thing that comes up very much. Not in the sense that it was a thing like when writing enterprise Java. Maybe because I try to keep my code composable I don't have to change it in many places? But this isn't limited to Common Lisp for personal projects: also bigger projects with multiple persons at work in C++, Python, etc. refactoring isn't such a big thing that I'd use it as an argument when chosing programming languages. Also once your code settles down, when you're going from the exploratory phase to the "let's make this ready for maintenance"-phase, CL has the option to annotate functions with types. Not at the level of Haskell but close enough to catch most things. That's what I like about CL: it's very practical and multi-paradigm. It doesn't force into a specific paradigm like object oriented or whatever fad comes next. I never have that feeling "what did I forget?". I'm sure I do forget things, but by building up code function by function and testing those functions one gets a pretty good feel for the code base. BUT, I've never worked on CL projects with multiple people, so I get where you come from. I love being able to use macros and the Lisp syntax is one of its main appeals to me. It is so simple and straight-forward. Algol-like languages with their curly braces, brackets, statement-ending, etc. etc. feel like such a chore. I try not to overuse macros, in fact I rarely use them but they're there when I need them and make the language very powerful and, if used correctly, more readable and easier to understand. "Code is data" … Do you mean something like this?: http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html http://www.moserware.com/2008/04/towards-moores-law-software... I guess I've been using Lisp long enough that I don't quite get what it means but I use it automatically. "Do you wish you had static types?" — Not really, like I said one can add garnish to the CL code to get something that is good enough, but even then I rarely use it. However, I only use it in mission-critical stuff and I don't write much of that.
- lmm 8y ago
  > Maybe because I try to keep my code composable I don't have to change it in many places? But this isn't limited to Common Lisp for personal projects: also bigger projects with multiple persons at work in C++, Python, etc. refactoring isn't such a big thing that I'd use it as an argument when chosing programming languages. I find refactoring is the most important part of programming; as the problem (or one's understanding of it) evolves, the code that describes it needs to evolve to match. When I worked in Python I found that I hit a wall where as my codebase got bigger, changes to my model took longer and longer to apply, because I had no way of reliably knowing what was coupled to what else; types solved that. (I like lightweight syntax but not to the extent of giving up infix operators/methods)
lkrubner 8y ago
"Without a static type system I would have simply lost a handle on the complexity of the project." A hardcore Clojure developer will probably tell you that Spec is, in some sense, even better than having a static type system. I haven't used Spec, so I can not say. But I do find, when I'm writing Clojure code, I write pre-conditions on every function, so every function has a contract. This goes a bit beyond what you can get from static type checking, in the sense that I'm testing all of my assumptions, including key names in maps, or at least, this is less work. There is also the advantage of testing against real world data, instead of mock data. I've had the problem, when using Java, that all the unit tests passed, but the app died in the real world, because the real world data (perhaps from a 3rd party API) was different from our mock data. In such a case you need run time contracts to help find the problem, and static types offer no help (depends on the project really, but the more your app interacts with 3rd parties, the less useful static types are). I'm not proposing this as The One True Way, but I would propose this as the Really Easy Way to write sane code. I don't have to think about it. It's like writing a comment for the function, but it gets enforced and can not possibly go out of date without giving an error. It documents everything the function is expecting. An example: (defn push-sentence [current-database-connection permanent-id map-of-sentence] {:pre [ (not (nil? permanent-id)) (not (nil? map-of-sentence)) (map? map-of-sentence) (:sentence map-of-sentence) (:belongs-to map-of-sentence) ]} (mc/update current-database-connection "reports" {:permanent-id permanent-id} {operators/$push {:sentences map-of-sentence}} {:upsert true})) (defn push-transcript-processed [current-database-connection transcript] {:pre [ (not (nil? transcript)) (map? transcript) (:permanent-id transcript) ]} (mc/update current-database-connection "reports" {:permanent-id (:permanent-id transcript)} {operators/$set {:processed :true}} {:upsert true})) This also gives me good error messages which makes debugging easy. For instance, if this fails: (not (nil? map-of-sentence)) I get an error about it being nil, whereas if this fails: (:belongs-to map-of-sentence) I get an error about the missing key. This helps me narrow down the errors very fast. I keep the functions short, which feels natural in Clojure. Again, this isnt' the One True Way. Realy Clojure programmers will tell you to use Spec, or maybe Typed Clojure. But I will say, heavy use of pre-conditions gives me the right balance between the contract enforcement I want, while still giving me the flexibility of a dynamic language. I like Java, but often the verbose quality of a static typed language is just not something I need.
yogthos 8y ago
My team has been using Clojure for close to a decade now. We have apps that have been in production for many years, and are still under active development. - refatoring I don't really find it to be problematic with dynamic typing. I tend to rely on specification tests and spec to test functionality at the API level. If the API behaves according to the specification, then the code can be said to be working as intended. - homoiconicity Absolutely, my team gave a talk about one of our projects recently https://www.youtube.com/watch?v=aXBe6hoi-Mw https://www.youtube.com/watch?v=aXBe6hoi-Mw The project heavily relies on homoiconicity as the business logic for each form is encoded using EDN and stored in the database. - static types I've primarily worked with statically typed languages prior to Clojure. I've worked with Java for close to a decade, then used Haskell for about a year, and some Scala. My experience is that dynamic typing is problematic in imperative/OO languages. One problem is that the data is mutable, and you pass things around by reference. Even if you knew the shape of the data originally, there's no way to tell whether it's been changed elsewhere via side effects. What I find to be of highest importance is the ability to reason about parts of the application in isolation, and types don't provide much help in that regard. When you have shared mutable state, it becomes impossible to track it in your head as application size grows. Knowing the types of the data does not reduce the complexity of understanding how different parts of the application affect its overall state. My experience is that immutability plays a far bigger role than types in addressing this problem. Immutability as the default makes it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don't need to track types across your entire application, and you're able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components. The other problem is that OO encourages proliferation of classes in your code. Each class is its own DSL with an ad hoc API, and knowing the behavior of one class doesn't tell you anything about that of the next. Keeping track of that quickly gets out of hand. With Clojure, you're working with a common set of data structures and using standard library functions to manipulate them. REPL driven development also plays a big role in the workflow. Any code I write, I evaluate in the REPL straight from the editor. The REPL has the full application state, so I have access to things like database connections, queues, etc. I can even connect to the REPL in production. So, say I'm writing a function to get some data from the database, I'll write the code, and run it to see exactly the shape of the data that I have. Then I might write a function to transform it, and so on. At each step I know exactly what my data is and what my code is doing.
hderms 8y ago
Scala is a great language for large software projects. The only downside really is compilation speed/tooling.
freshhawk 8y ago
I feel more flexible, clojure.spec helps a lot and is a wonderful tool to add to your reflection/parsing/validation/testing toolboxes. The REPL driven development and immutability defaults means it isn't saved in a file until in runs. Clojure idioms put those black magic macros or even complex ones in the "last resort" category. It is very much focused on representing problems as data then composing general functions to work with them. That is ranked as more elegant than the macro driven dsl style and similar ones. They are still used and are very useful. I find types are one thing among many, all affecting each other, when it comes to how productive I am and how much I enjoy it. Clojure is unreasonably effective and spec solved the class of problem that I would have liked to have gradual types for so I don't miss them no. Gradual typing is cool though, I'm sure I'd make use of it.

wooby 8y ago

At a Clojure meetup I attended a couple years ago, a long-time Java programmer asked "how do you manage huge projects without static types?" The best answer I could think of was, "don't manage huge projects." Instead, divide projects as early as possible in development into supporting libraries, each capable of being independently tested and maintained. In my experience, applications developed with this attitude coalesce themselves into a small ecosystem of mostly generic library code with a domain-specific apex "application" entrypoint to bind them. In an application factored like this, type checking isn't something you want as much anymore, since the constituent libraries interface with one another at a level above. They present and consume "public" (intra-application) APIs, the behavior of which are defined not just by types but by at least documentation and tests. The thing that static types let you do -- have huge amounts of code, woven together with structural dependencies -- is no longer a thing you need or want. It would seem bad, actually, since what you'd have is an inscrutable mass only a machine can understand. I haven't worked enough with large non-Clojure codebases to say whether or not this approach is superior, but I can say that it scales. Another nice thing about it in a business setting is that supporting libraries lend themselves to open-sourcing, and a relationship with an open source community is a good thing for a company's development efforts/hiring/contracting in my experience. To answer your specific questions: * I don't think I write more unit tests (I should probably write more tests in every language I use) * I don't have any experience with clojure.spec * All of the features of Lisp work together to make it awesome, including macros. In particular, in the context of an "ecosystem of libraries" approach, macros are a fantastic tool for exporting a DSL from a supporting library. They're also use for DSLs in the apex code, where application-specific behavior can be defined in terms of a DSL. * I don't really wish I had static types - I love Lisp the way it is. But I have found various languages for structural checking useful with certain kinds of sub-program, like receiving data from the outside world, and for writing compilers. If you write a compiler as a series of transformations, it's nice if the shape of the code between transformations is well-defined somehow. You're definitely smart enough to use Lisp. I think the key to success is not necessarily smarts, but attentiveness to modularity or cognizance of the difference between "library" and "application" types of code. * There's a Paul Graham essay related to this idea: http://www.paulgraham.com/progbot.html http://www.paulgraham.com/progbot.html * I think this approach is applicable not just to Lisp, but maybe to any dynamic language with a sufficiently rich set of generic collections and with the concepts of a "module" or "package".

lopatin 8y ago
Thanks for the insightful answer! Building an ecosystem of libraries to do most of the heavy lifting is generally what I strive for as well. It's certainly the goal, and I'll usually end up there for a given project. Perhaps one thing that makes me fond of static types is that, more often than not, I don't actually know how or what to modularize until after the project is nearly built in the first place. And I use types not only as a QA tool, but also as a tool for reasoning. The feedback loop between me encoding logic in types, and the IDE providing me with errors, is a big part of how I understand what I'm building, as I'm building it. I imagine that Clojure development on the REPL can actually serve the same purpose. I think the idea of bottom-up programming also resonates with me, though I tend to find it hard to use as a general approach to programming. In my mind, it's similar to the development of DSLs, which I do fairly infrequently, and with care. There is an undeniable elegance to it ... find a set of primitive operations that can be composed into larger ones. My development style usually goes in either direction: Either super generalized, for which I'll take the bottom-up DSL approach, or the more typical case where it's more approachable for me to be explicit with types/code.
- FPGAhacker 8y ago
  Myself, I sort of do both bottom up and top down programming. Usually I start at the top and write out high level code as if everything I needed magically existed. That helps me think about what I want and how I think I'll use it and that helps and clues me in to what low level stuff I need to build. Then I start to think about how to build those things and that usually goes bottom up. I do this iteratively. When I get stuck I go back to high level top down, figure out what I'm missing, go build that, and repeat.
  - AnimalMuppet 8y ago
    That's called Ping-pong Design, and is a perfectly legitimate approach.
- yogthos 8y ago
  My approach to modularizing from the start is to do bottom up development, and to treat each namespace as a little library. Ultimately, pretty much all problems can be viewed as data transformation pipelines, and those naturally map to independent components doing different kinds of transformations. I find that the APIs tend to form naturally around domain boundaries as you move data from one context to another. The REPL is indeed a major component of doing the Clojure workflow where you literally run each function as you're writing it. This approach also encourages refactoring as you go since you can make changes and see the results immediately.

blandflakes 8y ago

Clojure is a great language. I've used both Clojure and Scala in my personal and professional life. * In refactoring, I almost always make mistakes in Clojure that Scala will catch. I feel empowered rather than limited by the Scala compiler 9 times out of 10. * Homoiconicity makes me feel great, but I find I don't actually miss it (directly) in Scala. That said, Scala has a complex grammar, and I miss the overall consistency of s-expressions. * I always miss static types. I'd love to try the new spec improvements in Clojure 1.10; debugging mistakes prior to that are a nightmare. I don't need static types to be happy, but I do need real tooling support when things go wrong. Clojure has a lot of lifting power and I would like to keep using it. I find that once I get past type errors, programs work very well. Why would I spend time getting past those if I don't have to?

FPGAhacker 8y ago
Yeah, I agree with this, other than not having used Scala. Static typing communicates intent to the user of your functions, they help you think about the problem before going all-in. Rich presented some new ideas for where he wants to take Spec, but I didn't think anything very new made it into 1.10 regarding spec. Personally I use clojure whenever I can. I can quickly build up a project using the fast and loose data types, and when my conception of the project solidifies, I convert over to records and protocols. That's usually enough to get 80% of the way there. The free and loose ad-hoc data structures get me across the finish line, and it's there that Spec comes into play.
- blandflakes 8y ago
  Yeah I think I was thinking of the error message improvements rather than any changes that are specific to Spec in 1.10.

dizzystar 8y ago

I've worked on numerous Clojure projects, both personal and professional. I started most my learning on Lisp, so I'm probably not the most qualified to discuss this outside of the thought that many people come to Lisp and Clojure with approaches that aren't entirely compatible with Lisp. I know my thoughts are going to be controversial, so take it for what it's worth. During refactoring, do you feel more flexible, or less? I can see both sides to the argument. On one hand, static types are more rigid, and getting something to compile during a refactoring can be a pain. Often for the right reasons, but often it's due to some annoying technicality in the type system. But once you get there, "if it compiles, it runs" is a cliche that is pretty accurate in my experience. On the other hand, with a dynamic type system, you can slowly evolve a code base and see incremental progress during a refactoring, but I can never get over the dreadful feeling of "what did I forget?". Has clojure.spec worked out in terms finding bugs during refactoring/maintenance and general QA? I think there is good and bad to this, but in my experience, using a lot of types seems to be an anti-pattern. Since most web-based projects really only deal with strings, basic numbers, lists and dictionaries, I really don't see the entire benefit, even in strongly-typed languages. I work with typed languages as well and generally feel ambivalent about types. A bad refactor is a bad refactor and bad code is bad code no matter what language you are using. If you can't understand what the code is doing, don't touch it until you can test against it or test against what it's really supposed to be doing. Types don't alter this fact. Do you make use of the Lispy-ness (Homoionicity, macros, code is data, etc ...) or is it generally an anti-pattern to use these language features in daily development? Homoiconicity is the very definition of Lisp, so there is no way to work around that. Macros are generally eschewed in Lisp. It's glaringly obvious when and if you need to use a macro, and honestly, most people can go their entire career without pushing a macro into production. Code is data is pretty hand-wavey in my opinion. All it's really saying is that, if you call a function with a value, you get a value back. There is no mental quantum leap needed for this. Do you wish you had static types? If so, have you tried gradual typing like Typed Clojure? And if you don't miss types, why? I remember someone saying of Typed Clojure "We went to Clojure to get away from types!". And I'm still trying to understand the mindset. I don't think the community has any strong for-or-against opinions on types. In general, it's bad practice to use multiple-type functions. The perspective that's generally embraced is that each function does exactly one thing, which is why you see so many Lisp projects with many short functions, and if you pay attention, the functions aren't doing a lot of acrobatics based on possible types. What the community doesn't like is someone coming along and demanding something that isn't compatible. This is a little more nuanced, but it boils down to: "Add types if you want me to use it" or "Create a RoR system for me to use it." The community would rather you take the time to embrace Lisp and really try to understand what it is attempting to do before pushing your judgment on it. I often have the feeling that I'm not smart enough to use Lisp. If I had 2x the brain power, I would be able to use macros to conjure up some black magic in 100 lines what would take a mortal 5k in a lesser language. But even if I manage such a feat, I couldn't imagine bringing on another developer to that codebase. Anyone can use Lisp. Most of what you do is very simple. The entirety of Clojure is written on a single webpage. https://clojure.org/api/cheatsheet https://clojure.org/api/cheatsheet No one is building up systems with loads of macros and wild abstractions that cause a 5KLOC code base to shrink to 100 lines. This just isn't possible, and anyone that claims it is has no understanding of what a macro actually does.

dmichulke 8y ago

I programmed Java for years before I started using Clojure so I will make that distinction here to answer your questions > During refactoring, do you feel more flexible, or less? Both. More flexible because in Java you have 5x more code (was worse before lambdas). Less because I sometimes look up the code and don't know (and can't immediately infer) the types anymore. Usually I have a handful of integration tests that show me immediately when there is a problem but I wish there was a way to show the inferred types in my IDE. The problem is though usually only a big one for the one big state where you have lots of nested maps, vectors, arrays and so on and only if you modify the structure of this state. For smaller things, fixes are fast and easy, even though they still take 10 minutes while they could take 5. I feel it's kind of the price I pay to use Clojure but then again there's probably a reasonable solution to it - Clojure infers the types anyway, so why not show them somehow. I use clojure.spec partly, often when I need to throw useful error messages (e.g., a user facing REST API). It's a little bit tedious but it's always a good decision in retrospect. > Do you make use of the Lispy-ness? Yes and no. on the positive side: I don't ever need to look up the syntax of anything (I still make mistakes in Java switch for example) and using case and multi-methods I can really interpret quickly a DSL in a web server (e.g., binary arithmetic operations on OHLC bar time series) and I wouldn't want to implement that (safely) in another language. > Do you wish you had static types? If so, have you tried gradual typing like Typed Clojure? I tried for a brief two days gradual typing and didn't get it. I wish I had static typing exactly when refactoring or debugging. > I often have the feeling that I'm not smart enough to use Lisp. Oh, I'd even say the opposite, i.e., I am smarter because I use Lisp. I know Sapir-Whorf is "deprecated" but I feel I am thinking faster because I am thinking in much less convoluted structures compared to when I programmed in Java.

FPGAhacker 8y ago

I know this isn't answering a question you asked, but next time you feel like diving into clojure, focus on Records and Protocols. I think you'll get some relief from the free fall feeling.

weavie 8y ago

I use Common Lisp rather than Clojure at work. - During refactoring, do you feel more flexible, or less? A lot of it depends on how the code has been written. There are definitely parts of our code base that I really dread going near and there have been plenty of times when I have broken stuff by making a change without fully understanding the code. It is quite easy to write incomprehensible lisp code. With Clojure being more pure and opinionated things are probably not quite as hairy as it can get in CL. But if you write your code well CL can be fairly easy to refactor. Generally preferring to use CLOS (the object system) rather that just property lists and hash tables will give you some typing information that can make life a lot easier. Avoid nested lists as data structures - there is nothing worse than code that does stuff like `(dosomething (cadar list) (cddar list)` - Do you make use of the Lispy-ness? Macros tend to only be used for writing framework code. There is generally no need to use them when writing business logic. The aspect of Lisp that is often not mentioned is the iterative development. When code is data it becomes possible to evaluate code against a running system. This is, in my opinion, the nicest feature of Lisp. No compile times, you can change code on the fly, evaluate and test functions as you are writing them. Development becomes a real conversation with the compiler. You can even connect into the live server if needed to debug problems. - Do you wish you had static types? Yes.

badsavage 8y ago

We've used Clojure in production for years. 1. "Has clojure.spec worked out in terms finding bugs during refactoring/maintenance and general QA?" - Spec worked out as a much more useful library than we expected. You could describe any data structures with it, then validate, generate, regenerate it. I think spec is a must have for any Clojure programmer from 2018. 2. "Do you make use of the Lispy-ness" Macros are super useful sometimes, but in general we try to avoid them, because we prefer to solve everything with just pure functions. Homoiconicity and the "code is data" aspect are must haves in any programming language after you realised their power. 3. "is it generally an anti-pattern to use these language features in daily development?" macros are feel like cheating sometimes, and we fear the consequences, but homiconicity is useful for us on a daily basis. 4. "Do you wish you had static types?" We prefer to create our own "types" up to complex data structures. A built-in type system seems like a downgrade from here. 5. "I often have the feeling that I'm not smart enough to use Lisp." I think you are just scared from the unknown, Clojure is an alien technology compared to most of the currently popular languages. Any kids could learn programming in Clojure due to its simplicity and the REPL driven workflow. Its an interactive experience and very likeable.

pvinis 8y ago
Can you elaborate a bit on your own "types"? I am very interested and intrigued.
- badsavage 8y ago
  Yes, I could show you a simple example: Let's create our own 78 digits long string type: (def hex-digit (set "0123456789")) (defn hex-digit? [x] (contains? hex-digit x)) (defn hex-str? [s] (every? hex-digit? (seq s))) (s/def ::hash (s/and hex-str? #(= (count %) 78))) ::hash is now registered, now we go to the REPL: (s/valid? ::hash "23179372") ;;=> false (s/valid? ::hash "nope93721907914920047210715459933122004671648400678953445710500236944435987060") ;;=> false (s/valid? ::hash "231793721907914920047210715459933122004671648400678953445710500236944435987060") ;;=> true That's cool, but we want to generate too: (defn hex-str-gen [n] (let [digit (g/elements hex-digit)] (g/fmap clojure.string/join (g/vector digit n)))) (s/def ::hash (s/with-gen (s/and hex-str? #(= (count %) 78)) #(hex-str-gen 78))) We extended our ::hash spec with a generator function, so we can do this now: (g/generate (s/gen ::hash)) ;;=> "310584715385467847758653938894742415543975445356609397864862925839413265904779" (+ Pro tip: you could spec functions too) Learn more about spec here: https://clojure.org/guides/spec https://clojure.org/guides/spec

aerique 8y ago

"Not smart enough to use Lisp." I hear that more often and I do not understand it. I chose Common Lisp (CL) because it is an easy language not because it's hard (I would have chosen Haskell or C++ then). I'm not sure I can answer your questions. While I do use CL at work for smaller jobs I've only used it for bigger personal / solo projects, so I'm not sure that counts. About refactoring: it's not a thing that comes up very much. Not in the sense that it was a thing like when writing enterprise Java. Maybe because I try to keep my code composable I don't have to change it in many places? But this isn't limited to Common Lisp for personal projects: also bigger projects with multiple persons at work in C++, Python, etc. refactoring isn't such a big thing that I'd use it as an argument when chosing programming languages. Also once your code settles down, when you're going from the exploratory phase to the "let's make this ready for maintenance"-phase, CL has the option to annotate functions with types. Not at the level of Haskell but close enough to catch most things. That's what I like about CL: it's very practical and multi-paradigm. It doesn't force into a specific paradigm like object oriented or whatever fad comes next. I never have that feeling "what did I forget?". I'm sure I do forget things, but by building up code function by function and testing those functions one gets a pretty good feel for the code base. BUT, I've never worked on CL projects with multiple people, so I get where you come from. I love being able to use macros and the Lisp syntax is one of its main appeals to me. It is so simple and straight-forward. Algol-like languages with their curly braces, brackets, statement-ending, etc. etc. feel like such a chore. I try not to overuse macros, in fact I rarely use them but they're there when I need them and make the language very powerful and, if used correctly, more readable and easier to understand. "Code is data" … Do you mean something like this?: http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html http://www.moserware.com/2008/04/towards-moores-law-software... I guess I've been using Lisp long enough that I don't quite get what it means but I use it automatically. "Do you wish you had static types?" — Not really, like I said one can add garnish to the CL code to get something that is good enough, but even then I rarely use it. However, I only use it in mission-critical stuff and I don't write much of that.

lmm 8y ago
> Maybe because I try to keep my code composable I don't have to change it in many places? But this isn't limited to Common Lisp for personal projects: also bigger projects with multiple persons at work in C++, Python, etc. refactoring isn't such a big thing that I'd use it as an argument when chosing programming languages. I find refactoring is the most important part of programming; as the problem (or one's understanding of it) evolves, the code that describes it needs to evolve to match. When I worked in Python I found that I hit a wall where as my codebase got bigger, changes to my model took longer and longer to apply, because I had no way of reliably knowing what was coupled to what else; types solved that. (I like lightweight syntax but not to the extent of giving up infix operators/methods)

lkrubner 8y ago

"Without a static type system I would have simply lost a handle on the complexity of the project." A hardcore Clojure developer will probably tell you that Spec is, in some sense, even better than having a static type system. I haven't used Spec, so I can not say. But I do find, when I'm writing Clojure code, I write pre-conditions on every function, so every function has a contract. This goes a bit beyond what you can get from static type checking, in the sense that I'm testing all of my assumptions, including key names in maps, or at least, this is less work. There is also the advantage of testing against real world data, instead of mock data. I've had the problem, when using Java, that all the unit tests passed, but the app died in the real world, because the real world data (perhaps from a 3rd party API) was different from our mock data. In such a case you need run time contracts to help find the problem, and static types offer no help (depends on the project really, but the more your app interacts with 3rd parties, the less useful static types are). I'm not proposing this as The One True Way, but I would propose this as the Really Easy Way to write sane code. I don't have to think about it. It's like writing a comment for the function, but it gets enforced and can not possibly go out of date without giving an error. It documents everything the function is expecting. An example: (defn push-sentence [current-database-connection permanent-id map-of-sentence] {:pre [ (not (nil? permanent-id)) (not (nil? map-of-sentence)) (map? map-of-sentence) (:sentence map-of-sentence) (:belongs-to map-of-sentence) ]} (mc/update current-database-connection "reports" {:permanent-id permanent-id} {operators/$push {:sentences map-of-sentence}} {:upsert true})) (defn push-transcript-processed [current-database-connection transcript] {:pre [ (not (nil? transcript)) (map? transcript) (:permanent-id transcript) ]} (mc/update current-database-connection "reports" {:permanent-id (:permanent-id transcript)} {operators/$set {:processed :true}} {:upsert true})) This also gives me good error messages which makes debugging easy. For instance, if this fails: (not (nil? map-of-sentence)) I get an error about it being nil, whereas if this fails: (:belongs-to map-of-sentence) I get an error about the missing key. This helps me narrow down the errors very fast. I keep the functions short, which feels natural in Clojure. Again, this isnt' the One True Way. Realy Clojure programmers will tell you to use Spec, or maybe Typed Clojure. But I will say, heavy use of pre-conditions gives me the right balance between the contract enforcement I want, while still giving me the flexibility of a dynamic language. I like Java, but often the verbose quality of a static typed language is just not something I need.

yogthos 8y ago

My team has been using Clojure for close to a decade now. We have apps that have been in production for many years, and are still under active development. - refatoring I don't really find it to be problematic with dynamic typing. I tend to rely on specification tests and spec to test functionality at the API level. If the API behaves according to the specification, then the code can be said to be working as intended. - homoiconicity Absolutely, my team gave a talk about one of our projects recently https://www.youtube.com/watch?v=aXBe6hoi-Mw https://www.youtube.com/watch?v=aXBe6hoi-Mw The project heavily relies on homoiconicity as the business logic for each form is encoded using EDN and stored in the database. - static types I've primarily worked with statically typed languages prior to Clojure. I've worked with Java for close to a decade, then used Haskell for about a year, and some Scala. My experience is that dynamic typing is problematic in imperative/OO languages. One problem is that the data is mutable, and you pass things around by reference. Even if you knew the shape of the data originally, there's no way to tell whether it's been changed elsewhere via side effects. What I find to be of highest importance is the ability to reason about parts of the application in isolation, and types don't provide much help in that regard. When you have shared mutable state, it becomes impossible to track it in your head as application size grows. Knowing the types of the data does not reduce the complexity of understanding how different parts of the application affect its overall state. My experience is that immutability plays a far bigger role than types in addressing this problem. Immutability as the default makes it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don't need to track types across your entire application, and you're able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components. The other problem is that OO encourages proliferation of classes in your code. Each class is its own DSL with an ad hoc API, and knowing the behavior of one class doesn't tell you anything about that of the next. Keeping track of that quickly gets out of hand. With Clojure, you're working with a common set of data structures and using standard library functions to manipulate them. REPL driven development also plays a big role in the workflow. Any code I write, I evaluate in the REPL straight from the editor. The REPL has the full application state, so I have access to things like database connections, queues, etc. I can even connect to the REPL in production. So, say I'm writing a function to get some data from the database, I'll write the code, and run it to see exactly the shape of the data that I have. Then I might write a function to transform it, and so on. At each step I know exactly what my data is and what my code is doing.

hderms 8y ago

Scala is a great language for large software projects. The only downside really is compilation speed/tooling.

freshhawk 8y ago

I feel more flexible, clojure.spec helps a lot and is a wonderful tool to add to your reflection/parsing/validation/testing toolboxes. The REPL driven development and immutability defaults means it isn't saved in a file until in runs. Clojure idioms put those black magic macros or even complex ones in the "last resort" category. It is very much focused on representing problems as data then composing general functions to work with them. That is ranked as more elegant than the macro driven dsl style and similar ones. They are still used and are very useful. I find types are one thing among many, all affecting each other, when it comes to how productive I am and how much I enjoy it. Clojure is unreasonably effective and spec solved the class of problem that I would have liked to have gradual types for so I don't miss them no. Gradual typing is cool though, I'm sure I'd make use of it.