6 ms·

At a Clojure meetup I attended a couple years ago, a long-time Java programmer asked "how do you manage huge projects without static types?" The best answer I

by wooby 8y ago

At a Clojure meetup I attended a couple years ago, a long-time Java programmer asked "how do you manage huge projects without static types?" The best answer I could think of was, "don't manage huge projects." Instead, divide projects as early as possible in development into supporting libraries, each capable of being independently tested and maintained. In my experience, applications developed with this attitude coalesce themselves into a small ecosystem of mostly generic library code with a domain-specific apex "application" entrypoint to bind them. In an application factored like this, type checking isn't something you want as much anymore, since the constituent libraries interface with one another at a level above. They present and consume "public" (intra-application) APIs, the behavior of which are defined not just by types but by at least documentation and tests. The thing that static types let you do -- have huge amounts of code, woven together with structural dependencies -- is no longer a thing you need or want. It would seem bad, actually, since what you'd have is an inscrutable mass only a machine can understand. I haven't worked enough with large non-Clojure codebases to say whether or not this approach is superior, but I can say that it scales. Another nice thing about it in a business setting is that supporting libraries lend themselves to open-sourcing, and a relationship with an open source community is a good thing for a company's development efforts/hiring/contracting in my experience. To answer your specific questions: * I don't think I write more unit tests (I should probably write more tests in every language I use) * I don't have any experience with clojure.spec * All of the features of Lisp work together to make it awesome, including macros. In particular, in the context of an "ecosystem of libraries" approach, macros are a fantastic tool for exporting a DSL from a supporting library. They're also use for DSLs in the apex code, where application-specific behavior can be defined in terms of a DSL. * I don't really wish I had static types - I love Lisp the way it is. But I have found various languages for structural checking useful with certain kinds of sub-program, like receiving data from the outside world, and for writing compilers. If you write a compiler as a series of transformations, it's nice if the shape of the code between transformations is well-defined somehow. You're definitely smart enough to use Lisp. I think the key to success is not necessarily smarts, but attentiveness to modularity or cognizance of the difference between "library" and "application" types of code. * There's a Paul Graham essay related to this idea: http://www.paulgraham.com/progbot.html http://www.paulgraham.com/progbot.html * I think this approach is applicable not just to Lisp, but maybe to any dynamic language with a sufficiently rich set of generic collections and with the concepts of a "module" or "package".

lopatin 8y ago
Thanks for the insightful answer! Building an ecosystem of libraries to do most of the heavy lifting is generally what I strive for as well. It's certainly the goal, and I'll usually end up there for a given project. Perhaps one thing that makes me fond of static types is that, more often than not, I don't actually know how or what to modularize until after the project is nearly built in the first place. And I use types not only as a QA tool, but also as a tool for reasoning. The feedback loop between me encoding logic in types, and the IDE providing me with errors, is a big part of how I understand what I'm building, as I'm building it. I imagine that Clojure development on the REPL can actually serve the same purpose. I think the idea of bottom-up programming also resonates with me, though I tend to find it hard to use as a general approach to programming. In my mind, it's similar to the development of DSLs, which I do fairly infrequently, and with care. There is an undeniable elegance to it ... find a set of primitive operations that can be composed into larger ones. My development style usually goes in either direction: Either super generalized, for which I'll take the bottom-up DSL approach, or the more typical case where it's more approachable for me to be explicit with types/code.
- FPGAhacker 8y ago
  Myself, I sort of do both bottom up and top down programming. Usually I start at the top and write out high level code as if everything I needed magically existed. That helps me think about what I want and how I think I'll use it and that helps and clues me in to what low level stuff I need to build. Then I start to think about how to build those things and that usually goes bottom up. I do this iteratively. When I get stuck I go back to high level top down, figure out what I'm missing, go build that, and repeat.
  - AnimalMuppet 8y ago
    That's called Ping-pong Design, and is a perfectly legitimate approach.
- yogthos 8y ago
  My approach to modularizing from the start is to do bottom up development, and to treat each namespace as a little library. Ultimately, pretty much all problems can be viewed as data transformation pipelines, and those naturally map to independent components doing different kinds of transformations. I find that the APIs tend to form naturally around domain boundaries as you move data from one context to another. The REPL is indeed a major component of doing the Clojure workflow where you literally run each function as you're writing it. This approach also encourages refactoring as you go since you can make changes and see the results immediately.

by wooby 8y ago

lopatin 8y ago

Thanks for the insightful answer! Building an ecosystem of libraries to do most of the heavy lifting is generally what I strive for as well. It's certainly the goal, and I'll usually end up there for a given project. Perhaps one thing that makes me fond of static types is that, more often than not, I don't actually know how or what to modularize until after the project is nearly built in the first place. And I use types not only as a QA tool, but also as a tool for reasoning. The feedback loop between me encoding logic in types, and the IDE providing me with errors, is a big part of how I understand what I'm building, as I'm building it. I imagine that Clojure development on the REPL can actually serve the same purpose. I think the idea of bottom-up programming also resonates with me, though I tend to find it hard to use as a general approach to programming. In my mind, it's similar to the development of DSLs, which I do fairly infrequently, and with care. There is an undeniable elegance to it ... find a set of primitive operations that can be composed into larger ones. My development style usually goes in either direction: Either super generalized, for which I'll take the bottom-up DSL approach, or the more typical case where it's more approachable for me to be explicit with types/code.

FPGAhacker 8y ago
Myself, I sort of do both bottom up and top down programming. Usually I start at the top and write out high level code as if everything I needed magically existed. That helps me think about what I want and how I think I'll use it and that helps and clues me in to what low level stuff I need to build. Then I start to think about how to build those things and that usually goes bottom up. I do this iteratively. When I get stuck I go back to high level top down, figure out what I'm missing, go build that, and repeat.
- AnimalMuppet 8y ago
  That's called Ping-pong Design, and is a perfectly legitimate approach.
yogthos 8y ago
My approach to modularizing from the start is to do bottom up development, and to treat each namespace as a little library. Ultimately, pretty much all problems can be viewed as data transformation pipelines, and those naturally map to independent components doing different kinds of transformations. I find that the APIs tend to form naturally around domain boundaries as you move data from one context to another. The REPL is indeed a major component of doing the Clojure workflow where you literally run each function as you're writing it. This approach also encourages refactoring as you go since you can make changes and see the results immediately.