5 ms·

From what I have seen in this article, and hn thread, and every time in general when there is a discussion about clojure or dynamic typed languages, you are onl

From what I have seen in this article, and hn thread, and every time in general when there is a discussion about clojure or dynamic typed languages, you are only speaking about tiny examples and even more tiny projects. I can’t really imagine myself working on a million line project written in clojure or another language without any type safety. Happy to be wrong, but I’d like to hear the experiences of people actually working on this kind of big projects rather than pointing to toy projects less than 10k lines maintained by tens of people... From my experience dynamic typing can be awesome for small prototypes, but to be production ready you need to spend much more energy for testing. More on the topic I can say that I don’t really see anything compelling that you can’t easily do in F# with a lot more safety in this article...

deleted 8y ago
[deleted]
cageface 8y ago
I haven't worked on any significant Clojure codebases but I have worked in large codebases in other dynamic languages and it's a nightmare. Maintaining a Rails app, for example, after it gets beyond a certain size is just punishing.
- pankajdoharey 8y ago
  Its a nightmare to modify a moderately large Rails App.
- christophilus 8y ago
  The problem with Rails isn’t so much its dynamic nature as much as it is how everything is implicit, and it’s really hard to track down where certain functionality is happening. I doubt Clojure would be so bad, as it’s explicit. Immutability also helps.
bunderbunder 8y ago
The article does address this, though maybe not authoritatively, under the "Refactoring" header. > I’ve felt this pain a few times and the only conclusions I’ve drawn are to use clojure.spec for non-trivial code, especially around domain boundaries, and try even harder to solve very small problems in ways that can compose to solve larger ones. This seems reasonable to me. A lot of where refactoring gets painful is when you have bits of code being reused across wide swaths of the program. Keeping modules small limits the opportunity for that to happen, and makes the scope of impact for a change easier to conceptualize without leaning on a compiler. The problem I see there is, it seems to be exceedingly rare that you can get a whole team of developers to be so disciplined - and all it takes is one lapse in discipline for it all to fall apart.
sankyo 8y ago
> I can’t really imagine myself working on a million line project Any project of this size is too big and is better off being broken up. You need tests no matter what kind of typing you are using. The best thing about Clojure is that it is immutable by default. Start writing pure functions that transform you data step by step until you send it on its way with a side effect. I don’t miss types much at all - immutable maps lists, sets and pure functions get the job done and it is fun to program. Static typing fans hardly ever acknowledge how difficult the types make it to read the code. Too much focus on that one time when someone made a typo and there was no unit test. This has been argued infintum, nobody is changing their mind. WTF am I doing here?
- xrd 8y ago
  This. "Static typing fans hardly ever acknowledge how difficult the types make it to read the code. Too much focus on that one time when someone made a typo and there was no unit test." Well said.
  - tigershark 8y ago
    Same question to you, what do you find difficult to read in the code that I posted in the previous comment?
    - xrd 8y ago
      Hey, sorry, I meant to post my comment into the parent comment of yours. I didnt mean to state your code was difficult to read, I was thinking of all the Java code I've seen over the years...
    - xrd 8y ago
      And, point well made about not needing to know the types but still getting compile assistance because of the types.
- tigershark 8y ago
  How is this F# code difficult to read? let add n1 n2 = n1 + n2 Where do you spend time reading the types? Though it’s fully typed and if you pass an int and a string or some type for which the + operator is not defined you’ll get a compile time error.
dustingetz 8y ago
Million loc systems are a symptom of cultural disease, they are not something to celebrate. Take a step back and seek the root cause of the problem. The whole point of Java was to let companies throw bodies at a problem and not have the result segfault or get hacked. It succeeded, and now all those bodies are generating all those lines, just as intended. Clojure shows us, I think, a different way. One in which a small team with a small amount of money can out perform a billionaire's body shop.
- mikekchar 8y ago
  Not to be disagreeable (because I think I agree with your main point, even though I've never used Clojure, or any similar language, in anything but toy programs), but we were writing million line pojects in C++ long before Java came out ;-) There were really 2 main points to java (as I understand it). The first was to have a statically typed language that had a relatively fast runtime that could easily be ported to many platforms. Indeed, the killer application for Java, IMHO, was the idea of implementing that run time in hardware (and it's a shame that Sun never really pushed that angle very hard). The other main point of Java was to codify a set of "best practices" as a kind of "defence against the bad programmers". The intent was to reduce the difficulties of hiring programmers by making the programming language less expressive (compared to C++, for instance). IMHO, this largely failed as programmers are creative and will always find ways to be expressive. So on one had we got C++ template crack and on the other hand we had abstract factory factories. Basically, though, I don't think there was ever a real intent to allow Java (the language) to make it easy to scale development to larger teams, or larger code bases. The success that Java had in getting into large enterprises just meant that the normal Java user was interested in that, because it's the environment that they were working in. The idea is attractive to enterprise people because they already have a lot of people and they want to get a lot done in a short period of time. Most managers (and many influential programmers) do not understand the loss of productivity that accompanies communication overhead from having larger teams. The large code bases are the necessary result of programmers trying to isolate themselves from the hundreds or even thousands of other programmers that they are working with. By building complex boundaries, they can reduce the communication complexity at the cost of increasing the program complexity. This is usually a good trade off if you are stuck having to work with an insane number of people. BTW, in case you think that it is only non-technical people who fall into this trap, I will provocatively mention 2 words: micro services. The way micro services are usually designed and implemented, they are usually the very definition of premature subsystem decomposition. This premature decomposition is chosen so as to allow a few people to "lock in" design decisions and reduce the need to communicate with a whole bunch of people who might ruin their architectural vision. Basically, same problem, same result. Technical people (who didn't live through the CORBA years) drink it up like the yummy, yummy cool aide it is, though ;-)
jwr 8y ago
I try not to get too deep into these kinds of discussions, so I'll just say that first, I'm pretty happy with a fairly large system written in Clojure and ClojureScript. Second, given Clojure's expressiveness, I find it difficult to imagine a system with a million lines of code.
Jach 8y ago
Not a personal experience, but more of an easy to remember existence proof: https://www.ptc.com/en/products/cad/elements-direct/modeling https://www.ptc.com/en/products/cad/elements-direct/modeling is made of several million lines of Common Lisp and has been developed over decades. I find the counter-arguments to your discomfort in this thread very strange. Some big problems just require big solutions. A language might indeed be best suited for smaller problems, but that doesn't mean that bigger problems are a "cultural disease". For Clojure in particular, I recall Hickey saying something to the extent that for the problems where Java is successful, Clojure should be successful too. Maybe this rhetoric has changed, but I think it's also related to people's experience that what takes 750k lines of Java takes under 1000 in Clojure, for some problems, so you get this bias that 750k lines of Clojure is "wrong". No, it's just that the equivalent Java would be a factor of 10-100 bigger. If a language can support modularity, it should be able to scale. More at addressing your discomfort, I'd be curious to know what you mean by "language without any type safety". Dynamic typing isn't the same thing as untyped, and some dynamic languages (like Lisp) support a notion of compile time that lets you have type warnings, optimization, etc. that you might be used to from static compilers... as far as testing goes, the big testing pushes in industry have come from people dealing with giant C++ and Java projects more than the dynamic language crowds, because more energy for testing is needed in general. Whether dynamic languages need more or less of it could be interesting, but because of the variations in what we call "static languages" and what we call "dynamic languages" it might not be useful to compare things so broadly. Instead we could compare language against language, and take into account usual dev practices. Having a REPL to leverage while developing, testing, and debugging changes things in a big way, for instance.
- tigershark 8y ago
  I don’t really, really think that clojure is 750 times more dense than java. Probably F# can approach between 5 and 10 times more density than C# in the right conditions. The 750 times less is simply false in my opinion. Any strong proof of your extremely strong declaration?
  - Jach 8y ago
    You're right that Clojure isn't 750 times more dense than Java in even the 99% case, I'm not trying to claim that. I probably should have used something more conservative to make my point (that dynamic languages will usually result in significantly less code for the same problems and that might itself cause suspicion of large projects entirely by dynamic language fans rather than take on the question directly of can their language scale to projects millions of lines big). The particular project I was thinking of in this case is Hibernate ORM, which somehow managed to weigh around 750k lines last I remember. It's a bit of a facetious comparison, I'll admit, to point to a few Clojure projects I know of that seem to solve the same core problem better in around 1.5k lines. Even if you agreed that they did, I think we'd both agree it was because of language features apart from dynamic typing (your own experience of two static languages differing by as much as 10x implies as much). When the problem domain is related to symbolic computing and metaprogramming, languages that support symbols and macros are going to have a huge advantage, especially when you start generating code you would have written by hand otherwise.
lispm 8y ago
In the 80s people were already developing multi-100kloc programs in Lisp. The operating system of the MIT Lisp Machine grew to something like 500kloc Lisp. Later versions (end 80s) of it were more like 1.5MLOC of Lisp. Common Lisp was designed to be a successor of such a Lisp and was supposed to support both complex and/or large applications. Code density can be a bit higher than C++ (say, factor 3) for larger programs. In the end 80s / early 90s Evans&Sutherland developed a high-end 3d design software in mostly Lisp - the Conceptual Design and Rendering System (CDRS) - many of the cars at that time were designed with it (from Ford, Jaguar, German brands, ...). A designer work environment did cost most than a million USD per seat - including a high-end graphics system from E&S. The 3d software was written in atleast 150kloc C++ and 400kloc Lisp. Other CAD systems written in Lisp should have been much larger like iCAD or the design system from PTC (years ago to be said at 7MLOC Lisp). iCAD for example was used at Boeing for the development of turbines, wings, internal stuff - it's even said that there was a complete model of some Boeing in iCAD - which used kind of an object-oriented language on top of Lisp for the parametric construction of technical things. Similar applications and languages still exist - for example GenDL: https://gitlab.common-lisp.net/gendl/gendl https://gitlab.common-lisp.net/gendl/gendl There are a bunch of things which makes this possible: Lisp compilers with various compile-time warnings, extensive runtime error handling facilities, object-oriented programming, good development environments, ...
- pjmlp 8y ago
  > There are a bunch of things which makes this possible: Lisp compilers with various compile-time warnings, extensive runtime error handling facilities, object-oriented programming, good development environments, ... I guess one issue might be that most developers that get introduced to Lisp, do so in primitive ways. So they never get to learn about those toolings. Even with Clojure, not everyone is willing to pay for Cursive.

deleted 8y ago

[deleted]

cageface 8y ago

I haven't worked on any significant Clojure codebases but I have worked in large codebases in other dynamic languages and it's a nightmare. Maintaining a Rails app, for example, after it gets beyond a certain size is just punishing.

pankajdoharey 8y ago
Its a nightmare to modify a moderately large Rails App.
christophilus 8y ago
The problem with Rails isn’t so much its dynamic nature as much as it is how everything is implicit, and it’s really hard to track down where certain functionality is happening. I doubt Clojure would be so bad, as it’s explicit. Immutability also helps.

bunderbunder 8y ago

The article does address this, though maybe not authoritatively, under the "Refactoring" header. > I’ve felt this pain a few times and the only conclusions I’ve drawn are to use clojure.spec for non-trivial code, especially around domain boundaries, and try even harder to solve very small problems in ways that can compose to solve larger ones. This seems reasonable to me. A lot of where refactoring gets painful is when you have bits of code being reused across wide swaths of the program. Keeping modules small limits the opportunity for that to happen, and makes the scope of impact for a change easier to conceptualize without leaning on a compiler. The problem I see there is, it seems to be exceedingly rare that you can get a whole team of developers to be so disciplined - and all it takes is one lapse in discipline for it all to fall apart.

sankyo 8y ago

> I can’t really imagine myself working on a million line project Any project of this size is too big and is better off being broken up. You need tests no matter what kind of typing you are using. The best thing about Clojure is that it is immutable by default. Start writing pure functions that transform you data step by step until you send it on its way with a side effect. I don’t miss types much at all - immutable maps lists, sets and pure functions get the job done and it is fun to program. Static typing fans hardly ever acknowledge how difficult the types make it to read the code. Too much focus on that one time when someone made a typo and there was no unit test. This has been argued infintum, nobody is changing their mind. WTF am I doing here?

xrd 8y ago
This. "Static typing fans hardly ever acknowledge how difficult the types make it to read the code. Too much focus on that one time when someone made a typo and there was no unit test." Well said.
- tigershark 8y ago
  Same question to you, what do you find difficult to read in the code that I posted in the previous comment?
  - xrd 8y ago
    Hey, sorry, I meant to post my comment into the parent comment of yours. I didnt mean to state your code was difficult to read, I was thinking of all the Java code I've seen over the years...
  - xrd 8y ago
    And, point well made about not needing to know the types but still getting compile assistance because of the types.
tigershark 8y ago
How is this F# code difficult to read? let add n1 n2 = n1 + n2 Where do you spend time reading the types? Though it’s fully typed and if you pass an int and a string or some type for which the + operator is not defined you’ll get a compile time error.

dustingetz 8y ago

Million loc systems are a symptom of cultural disease, they are not something to celebrate. Take a step back and seek the root cause of the problem. The whole point of Java was to let companies throw bodies at a problem and not have the result segfault or get hacked. It succeeded, and now all those bodies are generating all those lines, just as intended. Clojure shows us, I think, a different way. One in which a small team with a small amount of money can out perform a billionaire's body shop.

mikekchar 8y ago
Not to be disagreeable (because I think I agree with your main point, even though I've never used Clojure, or any similar language, in anything but toy programs), but we were writing million line pojects in C++ long before Java came out ;-) There were really 2 main points to java (as I understand it). The first was to have a statically typed language that had a relatively fast runtime that could easily be ported to many platforms. Indeed, the killer application for Java, IMHO, was the idea of implementing that run time in hardware (and it's a shame that Sun never really pushed that angle very hard). The other main point of Java was to codify a set of "best practices" as a kind of "defence against the bad programmers". The intent was to reduce the difficulties of hiring programmers by making the programming language less expressive (compared to C++, for instance). IMHO, this largely failed as programmers are creative and will always find ways to be expressive. So on one had we got C++ template crack and on the other hand we had abstract factory factories. Basically, though, I don't think there was ever a real intent to allow Java (the language) to make it easy to scale development to larger teams, or larger code bases. The success that Java had in getting into large enterprises just meant that the normal Java user was interested in that, because it's the environment that they were working in. The idea is attractive to enterprise people because they already have a lot of people and they want to get a lot done in a short period of time. Most managers (and many influential programmers) do not understand the loss of productivity that accompanies communication overhead from having larger teams. The large code bases are the necessary result of programmers trying to isolate themselves from the hundreds or even thousands of other programmers that they are working with. By building complex boundaries, they can reduce the communication complexity at the cost of increasing the program complexity. This is usually a good trade off if you are stuck having to work with an insane number of people. BTW, in case you think that it is only non-technical people who fall into this trap, I will provocatively mention 2 words: micro services. The way micro services are usually designed and implemented, they are usually the very definition of premature subsystem decomposition. This premature decomposition is chosen so as to allow a few people to "lock in" design decisions and reduce the need to communicate with a whole bunch of people who might ruin their architectural vision. Basically, same problem, same result. Technical people (who didn't live through the CORBA years) drink it up like the yummy, yummy cool aide it is, though ;-)

jwr 8y ago

I try not to get too deep into these kinds of discussions, so I'll just say that first, I'm pretty happy with a fairly large system written in Clojure and ClojureScript. Second, given Clojure's expressiveness, I find it difficult to imagine a system with a million lines of code.

Jach 8y ago

Not a personal experience, but more of an easy to remember existence proof: https://www.ptc.com/en/products/cad/elements-direct/modeling https://www.ptc.com/en/products/cad/elements-direct/modeling is made of several million lines of Common Lisp and has been developed over decades. I find the counter-arguments to your discomfort in this thread very strange. Some big problems just require big solutions. A language might indeed be best suited for smaller problems, but that doesn't mean that bigger problems are a "cultural disease". For Clojure in particular, I recall Hickey saying something to the extent that for the problems where Java is successful, Clojure should be successful too. Maybe this rhetoric has changed, but I think it's also related to people's experience that what takes 750k lines of Java takes under 1000 in Clojure, for some problems, so you get this bias that 750k lines of Clojure is "wrong". No, it's just that the equivalent Java would be a factor of 10-100 bigger. If a language can support modularity, it should be able to scale. More at addressing your discomfort, I'd be curious to know what you mean by "language without any type safety". Dynamic typing isn't the same thing as untyped, and some dynamic languages (like Lisp) support a notion of compile time that lets you have type warnings, optimization, etc. that you might be used to from static compilers... as far as testing goes, the big testing pushes in industry have come from people dealing with giant C++ and Java projects more than the dynamic language crowds, because more energy for testing is needed in general. Whether dynamic languages need more or less of it could be interesting, but because of the variations in what we call "static languages" and what we call "dynamic languages" it might not be useful to compare things so broadly. Instead we could compare language against language, and take into account usual dev practices. Having a REPL to leverage while developing, testing, and debugging changes things in a big way, for instance.

tigershark 8y ago
I don’t really, really think that clojure is 750 times more dense than java. Probably F# can approach between 5 and 10 times more density than C# in the right conditions. The 750 times less is simply false in my opinion. Any strong proof of your extremely strong declaration?
- Jach 8y ago
  You're right that Clojure isn't 750 times more dense than Java in even the 99% case, I'm not trying to claim that. I probably should have used something more conservative to make my point (that dynamic languages will usually result in significantly less code for the same problems and that might itself cause suspicion of large projects entirely by dynamic language fans rather than take on the question directly of can their language scale to projects millions of lines big). The particular project I was thinking of in this case is Hibernate ORM, which somehow managed to weigh around 750k lines last I remember. It's a bit of a facetious comparison, I'll admit, to point to a few Clojure projects I know of that seem to solve the same core problem better in around 1.5k lines. Even if you agreed that they did, I think we'd both agree it was because of language features apart from dynamic typing (your own experience of two static languages differing by as much as 10x implies as much). When the problem domain is related to symbolic computing and metaprogramming, languages that support symbols and macros are going to have a huge advantage, especially when you start generating code you would have written by hand otherwise.

lispm 8y ago

In the 80s people were already developing multi-100kloc programs in Lisp. The operating system of the MIT Lisp Machine grew to something like 500kloc Lisp. Later versions (end 80s) of it were more like 1.5MLOC of Lisp. Common Lisp was designed to be a successor of such a Lisp and was supposed to support both complex and/or large applications. Code density can be a bit higher than C++ (say, factor 3) for larger programs. In the end 80s / early 90s Evans&Sutherland developed a high-end 3d design software in mostly Lisp - the Conceptual Design and Rendering System (CDRS) - many of the cars at that time were designed with it (from Ford, Jaguar, German brands, ...). A designer work environment did cost most than a million USD per seat - including a high-end graphics system from E&S. The 3d software was written in atleast 150kloc C++ and 400kloc Lisp. Other CAD systems written in Lisp should have been much larger like iCAD or the design system from PTC (years ago to be said at 7MLOC Lisp). iCAD for example was used at Boeing for the development of turbines, wings, internal stuff - it's even said that there was a complete model of some Boeing in iCAD - which used kind of an object-oriented language on top of Lisp for the parametric construction of technical things. Similar applications and languages still exist - for example GenDL: https://gitlab.common-lisp.net/gendl/gendl https://gitlab.common-lisp.net/gendl/gendl There are a bunch of things which makes this possible: Lisp compilers with various compile-time warnings, extensive runtime error handling facilities, object-oriented programming, good development environments, ...

pjmlp 8y ago
> There are a bunch of things which makes this possible: Lisp compilers with various compile-time warnings, extensive runtime error handling facilities, object-oriented programming, good development environments, ... I guess one issue might be that most developers that get introduced to Lisp, do so in primitive ways. So they never get to learn about those toolings. Even with Clojure, not everyone is willing to pay for Cursive.